Images

Querying an image interface of a Vision device

In many cases images are created with a camera or by loading an image file. The following function blocks are available for this purpose:

• FB_VN_SimpleCameraControl
  Can be connected to a GigE Vision camera instance or a File Source instance.
• FB_VN_GevCameraControl
  Is connected to a GigE Vision camera instance.
• FB_VN_FileSourceControl
  Is connected to a File Source instance.

Depending on the setting of the Vision device, a startup process or an image trigger may be necessary. Only the querying of the interface pointer in the PLC, via which access is gained to the image in the memory, is displayed here:

VAR
    hr: HRESULT;
    fbVisionDevice: FB_VN_SimpleCameraControl;
END_VAR

hr := fbVisionDevice.GetCurrentImage(ipImageIn);
IF SUCCEEDED(hr) AND ipImageIn <> 0 THEN
    // Process image…
END_IF

Image formats

The two most commonly used image formats are mono and RGB. In the camera, the transferred image format is set under the PixelFormat tab. In order to reduce the amount of data and thus be able to realize higher transfer rates, there are also packed or filtered image formats, such as the Bayer format. Since these transmitted image formats cannot be used or do not correspond to the original image due to the filter, the formats should be converted directly after receipt.

• In the case of packed images, the conversion happens internally when you call GetCurrentImage() and you get the unpacked image directly.
• In all other cases, like with the Bayer format, the image should be converted first with the F_VN_ConvertColorSpace. After that, the input image is no longer needed and can be discarded.

This point is especially important to note when using functions that interpolate image information, such as F_VN_CompensateLensDistortion, F_VN_WarpAffine, F_VN_RotateImageExp or such as the functions for calibration that are disturbed by the filter image. Therefore, it is generally recommended to first read the image into the PLC, convert it there, and then apply all subsequent functions to the original image.

Creating images

There are situations where an image is needed that has not been captured by a camera or loaded from the file system. For example, a color image might be created to display results. The function F_VN_CreateImage can be used for this purpose by specifying the image size, pixel type and number of channels. The pixel type TCVN_ET_USINT selected here corresponds to a usual image with 8 bits per channel. Please note that the memory required for the image is only allocated, but not initialized. Therefore the image can be set to a uniform color with the function F_VN_SetPixels.

Maximum image size The maximum image size that can be created, received, or processed is limited by the number of 2^20 pixels for rows or columns, or by the total size of 2^30 pixels for an image, whichever comes first.

PROGRAM MAIN
VAR
    ipImage     :   ITcVnImage;
    aColorBlack :   TcVnVector4_LREAL := [0, 0, 0];
    hr          :   HRESULT;
END_VAR

hr := F_VN_CreateImage(
    ipImage     :=  ipImage,
    nWidth      :=  640,
    nHeight     :=  480,
    ePixelType  :=  TCVN_ET_USINT, 
    nChannelNum :=  3,
    hrPrev      :=  hr
);
hr := F_VN_SetPixels(ipImage, aColorBlack, hr);

Alternatively, an image can be created from existing image data using the F_VN_CreateImageFromArray function.

PROGRAM MAIN
VAR
    aImageData  :   ARRAY [0..479, 0..639] OF USINT;
    ipImage     :   ITcVnImage; 
    hr          :   HRESULT;
END_VAR

hr := F_VN_CreateImageFromArray(
    pData       :=  ADR(aImageData),
    ipImage     :=  ipImage,
    nWidth      :=  640,
    nHeight     :=  480,
    ePixelType  :=  TCVN_ET_USINT, 
    nChannelNum :=  1,
    hrPrev      :=  hr
);

Accessing image data

The functions F_VN_GetPixel and F_VN_SetPixel can be used to access individual pixels of an image. This method is easy to use. However, it has a relatively long runtime when applied repeatedly to many pixels. In this case, direct access to the image data array is more appropriate. A pointer for a complete row of pixels is fetched and the array operator is used to access the individual pixels:

PROGRAM MAIN
VAR
    ipImage     :   ITcVnImage;
    nHeight     :   UDINT;
    nWidth      :   UDINT;
    y           :   UDINT;
    x           :   UDINT;
    // Pointer type must match pixel type of ipImage!
    pRow        :   POINTER TO USINT;
    nPixelValue :   USINT;
END_VAR

// Create ipImage here or set to an existing image

// Determine width and height of image
hr := F_VN_GetImageWidth(ipImage, nWidth, hr);
hr := F_VN_GetImageHeight(ipImage, nHeight, hr);

// Iterate every pixel of the image
FOR y := 0 TO nHeight - 1 DO
    IF hr = S_OK THEN
        hr := ipImage.GetRowPointer(y, ADR(pRow));
        IF hr = S_OK THEN
            FOR x := 0 TO nWidth - 1 DO
                // Access pixel via pRow[x]
                nPixelValue := pRow[x]; // e.g.
                pRow[x] := nPixelValue; // e.g.
            END_FOR
        END_IF
        // It’s important to release the pointer. Otherwise there will be memory leaks.
        hr := ipImage.ReleaseRowPointer(ADR(pRow)); 
    END_IF
END_FOR

Displayable images

In order to be able to transfer and display an image by ADS (e.g. in the ADS Image Watch), it must first be converted into a displayable image of the type ITcVnDisplayableImage.

The strict distinction between displaying and processing images is made for the following reason: Image transmission via ADS is not synchronous with the PLC cycle. Consequently, image processing functions could be performed at the same time as image transfer. In order to avoid memory conflicts during image transfer, it is therefore necessary to separate the memory areas from the images to be displayed and processed.

The two interfaces ITcVnImage and ITcVnDisplayableImage prevent image access for unwanted purposes (either processing or transferring). The following functions are available for conversion into a displayable image (ITcVnDisplayableImage):

• F_VN_CopyIntoDisplayableImage
• F_VN_TransformIntoDisplayableImage
• F_VN_TransformIntoDisplayableImageExp

The F_VN_CopyIntoDisplayableImage function creates a deep copy of the image data and makes it available as a displayable image. It should always be used if the image to be displayed is to be used in the following program sequence:

// Processing, e.g.:
hr := F_VN_Threshold(ipImage, ipImage, 128, 255, TCVN_TT_BINARY, hr);

// Show intermediate processing step
hr := F_VN_CopyIntoDisplayableImage(ipImage, ipImageDisp, hr);

// Further processing, e.g.:
hr := F_VN_FindContours(ipImage, ipContours, hr);

The function F_VN_TransformIntoDisplayableImage uses the existing image data and releases the original interface pointer of the image. This means that the image data can no longer be processed and are fully available for image transmission. Since no copy of the image data is created, the execution time of this function is much shorter and no additional memory is used. However, this function only works if there is no other pointer to the image data. Otherwise the error code E_INCOMPATIBLE (16#70E) is returned. This can occur among other things if the Ads Communicator object is activated (e.g. stream recording), as this continues to reference the image internally for a short time. This function is therefore mainly useful for the end of the processing chain in order, for example, to display an already generated result image.

// Put last results on image, e.g.:
hr := F_VN_DrawContours(ipContours, -1, ipImageRes, aColorRed, 5, hr);

// Finally transform result image into a displayable image
hr := F_VN_TransformIntoDisplayableImage(ipImageRes, ipImageResDisp, hr);

In addition, the function F_VN_TransformIntoDisplayableImageExp offers the option to create a deep copy of the original image on demand. Thus, the resource-saving transform variant is used if possible, while the copy variant is used if necessary. A corresponding use case can occur if the AdsCommunitor object of a GigE Vision Camera instance is activated to save images as a stream to the file system. In addition to the pointer in the PLC, there is another image data pointer in the AdsCommunicator object. If this is not yet enabled in the PLC at the time the image is used, the image must be copied for display.

hr := fbCamera.GetCurrentImage(ipImageIn);
hr := F_VN_CopyImage(ipImageIn, ipImageWork, hr);

// Reliably show original image and try to reuse image data to avoid deep copy
hr := F_VN_TransformIntoDisplayableImageExp(
    ipSrcImage      :=  ipImageIn,
    ipDestImage     :=  ipImageInDisp,
    bAllowDeepCopy  :=  TRUE,
    hrPrev          :=  hr
);

// Manipulate ipImageWork during processing here ...

Memory limitation for displayable images Images that are displayed in the development environment (ADS Image Watch or camera assistants) are located in the same RAM area as the respective Visual Studio instance. If you now wish to view a very large number of images or images of a large size, bottlenecks may occur due to the maximum available memory per instance. As the images are retrieved from the PLC for display in the ADS Image Watch (ADS communication) and this exchange takes place via the router memory, sufficient memory space must be available at this point.

Meta information

Meta information for an image can be obtained with the function F_VN_GetImageInfo. As a result of the function, a structure of the type TcVnImageInfo is returned. The information it contains can be used, for example, to ensure that an image has the expected format in order to be converted:

hr := F_VN_GetImageInfo(ipImageIn, stImageInfo, hr);
IF stImageInfo.stPixelFormat.nChannels = 3 THEN
    hr := F_VN_ConvertColorSpace(ipImageIn, ipImageIn, TCVN_CST_RGB_TO_GRAY, hr);
END_IF