Preparing ONNX for use with TwinCAT Machine Learning Server

Generate interface descriptions for the PLC

To be able to use an ONNX with the FB_MlSvrPrediction in the TwinCAT PLC, interface information is required. These are generated by the TwinCAT Machine Learning Model Manager.

Information on the supported ONNX Opset and restrictions can be found here: ONNX Support.

Preparing ONNX for use with TwinCAT Machine Learning Server 1:

JSON file

The JSON file contains metadata required by the function block FB_MlSvrPrediction. Users can also add their own metadata to the JSON file via the Custom Attributes. The JSON file is loaded by the function block, see Configure method. Both the JSON file and the ONNX file must be available for loading on the runtime PC, see Make model description files available on the Server Device.

PlcOpenXml

The PlcOpenXml contains the PLC type description of the input and output nodes. The automatically generated input/output structures (DUTs) reflect the input/output definition of the ONNX file provided by you. Therefore, make sure that you use meaningful names for the input and output nodes in your ONNX.

The use of these created data types is recommended.

Sample of name generation in the PlcOpenXml:

Preparing ONNX for use with TwinCAT Machine Learning Server 2:

File name: lemon_model.onnx

Name Input node: input.1

Name Output node: 367

Two DUTs of type STRUCT are generated with the names ST_lemon_modelInput and ST_lemon_modelOutput. The data area elements are named according to the input and output nodes; in the example above, these are in_input1 and out_367. Characters not permitted in the PLC are automatically removed. If an AI model has several input or output nodes, these are each displayed as an element of the STRUCT.

Sample of the generated input data types

TYPE ST_lemon_modelInput :
STRUCT
   in_input1 : ARRAY[0..0,0..2,0..223,0..223] OF REAL;
END_STRUCT
END_TYPE

The names of the elements of the STRUCT can be adjusted if required, but not their order in the STRUCT, nor the size of the ARRAY.

Import of a PlcOpenXml in TwinCAT 3

The generated PlcOpenXml can be transferred to the PLC project in the PLC by right-clicking on a folder (e. g. DUTs) via the "Import PLCopenXML" field.

Preparing ONNX for use with TwinCAT Machine Learning Server 3:

Notice

It is recommended to run the machine learning server using generated input and output model types

Note that the use of the input/output model types defined in the supplied PlcOpen file is recommended, but not mandatory. The data must be sorted correctly and have the correct size.