F_VN_DetectCodesNeuralNetwork

Detect various code types, such as barcodes, QR codes, Data Matrix codes and DotCodes within the provided image using a trained neural network model based on eModelType. The detected code regions and types (if supported) are sorted by their model confidence, from highest to lowest.
Can use available TwinCAT Job Tasks for executing parallel code regions.

Syntax

Definition:

FUNCTION F_VN_DetectCodesNeuralNetwork : HRESULT
VAR_INPUT
    ipSrcImage    : ITcVnImage;
    ipCodeRegions : Reference To  ITcVnContainer;
    ipCodeTypes   : Reference To  ITcVnContainer;
    eModelType    : ETcVnCodeDetectionModel;
    nCodeType     : ULINT;
    hrPrev        : HRESULT;
END_VAR

Inputs

Name	Type	Description
ipSrcImage	ITcVnImage	Source image (USINT elements, 1 or 3 channels).The function automatically normalizes and converts the input to the format required by the selected eModelType
ipCodeRegions	Reference To ITcVnContainer	Returns the code positions as upright rectangles or rotated rectangles based on eModelType (ContainerType_Vector_TcVnRectangle_DINT or ContainerType_Vector_TcVnRotatedRectangle; Non-zero interface pointers are reused.)
ipCodeTypes	Reference To ITcVnContainer	Returns the code type (ETcVnCodeType) of each detected code (ContainerType_Vector_ULINT; Non-zero interface pointers are reused), set to 0 if not required.
eModelType	ETcVnCodeDetectionModel	Type of the neural network model for code detection. Different neural network models are available, including models specialized for 1D codes, 2D codes, or all code types.
nCodeType	ULINT	Specifies the code type(s) to detect. Multiple values from the ETcVnCodeType enumeration can be combined by performing a bitwise OR operation.
hrPrev	HRESULT	HRESULT indicating the result of previous operations (If SUCCEEDED(hrPrev) equals false, no operation is executed.)

Return value

HRESULT

Further information

The F_VN_DetectCodesNeuralNetwork function uses neural networks to localize 1D and 2D codes in the input image and classify their type. In contrast to the classic code reading functions, no decoding of the code content takes place here. The F_VN_DetectCodesNeuralNetwork is used for pre-localization, with the neural network models providing probabilistic detection results in the form of bounding boxes and the code types.

These code regions can be decoded in the next step using the corresponding code reading functions. Due to the specified region, the execution time is reduced as the localization of the code is restricted to a small image area. This also enables the direct use of code reading functions that require a predefined ROI.

Parameter

ipSrcImage

The input image ipSrcImage must be a 1-channel or 3-channel image of the type USINT (8-bit).

Depending on the model selected, the image is automatically converted internally. The specific input format and size of the models is listed in the description of ETcVnCodeDetectionModel. In order to reduce the internal conversion steps and thus also the processing time required, it is recommended to transfer the image with the same number of channels if it already exists in the form before the call.

In general, it should be ensured that the aspect ratio of the input image is as similar as possible to that of the model. Otherwise, the internal scaling may result in distortions that impair the detection quality.

ipCodeRegions

The parameter ipCodeRegions returns the code regions localized in the input image as containers of the type ContainerType_Vector_TcVnRectangle_DINT or ContainerType_Vector_TcVnRotatedRectangle. Which container type is used depends on the transferred eModelType. The entries in the list are sorted in descending order of confidence, starting with the highest probability.

ipCodeTypes

The parameter ipCodeTypes returns the classified types of the recognized codes in the same order as ipCodeRegions. The return container is of type ContainerType_Vector_ULINT and contains values of the ETcVnCodeType enum. Which code types are returned depends on the recognition scope of the model used.

eModelType

The parameter eModelType of type ETcVnCodeDetectionModel specifies the neural network model for code recognition. The choice of model is based on two dimensions: the scope of recognition (which code types are supported) and the input resolution (accuracy vs. speed).

Model type	Scope of recognition
TCVN_CDM_DETECT_1_x	1D and 2D codes with classification: `TCVN_CT_1D` `TCVN_CT_2D` `TCVN_CT_QR` `TCVN_CT_DATAMATRIX` `TCVN_CT_DOTCODE`
TCVN_CDM_DETECT_2_x	Only 1D codes (`TCVN_CT_1D`)
TCVN_CDM_DETECT_3_x	Only 2D codes (`TCVN_CT_2D`)

Each of the above model types is available in three resolution levels. The suffix (_1, _2, _3) defines the size to which the image is scaled internally:

..._1: Small image size (fastest processing, for applications with large codes or high performance requirements)
..._2: Average image size (balanced ratio between speed and accuracy. Recommended for most applications)
..._3: Large image size (highest resolution, best recognition of small or distant codes, highest processing time)

The choice of model influences both the detection speed and the accuracy. The models with a reduced recognition scope, such as those for 1D or 2D codes only, offer higher performance.

nCodeType

The parameter nCodeType is used as a filter to restrict detection to certain code types (ETcVnCodeType). To detect all codes supported by the model, the value TCVN_CT_ANY can be used. Alternatively, specific types can be combined using a OR link (e.g. TCVN_CT_QR_CODE OR TCVN_CT_DATAMATRIX). Please note that this filter only applies to the recognition scope of the selected model eModelType. If the filter does not match the recognition scope, a 0x70E HRESULT is returned.

Application

The prerequisite for execution is the one-time initialization of the models to be used using the function block FB_VN_InitializeFunction.

The following code segment demonstrates how to call the F_VN_DetectCodesNeuralNetwork function to detect 1D and 2D codes simultaneously:

hr := F_VN_DetectCodesNeuralNetwork(
        ipSrcImage      := ipImageIn,
        ipCodeRegions   := ipCodeRegions,
        ipCodeTypes     := ipCodeTypeResult,
        eModelType      := TCVN_CDM_DETECT_1_2,
        nCodeType       := ,
        hrPrev          := hr);

The detected regions can then be set as ROI and decoded using a suitable code reading function. To ensure reliable decoding, it is recommended that the areas found are enlarged in advance using F_VN_ResizeRegion_TcVnRectangle_DINT. This ensures that the quiet zones required for decoding are included in the image section.

	Notes on performance The execution of the models benefits considerably from CPU parallelization. It is therefore recommended to execute the PLC code on isolated CPU cores and assign these to dedicated job tasks.

Examples

Required License

TC3 Vision Code Reading