Match Contours 1vsN (manual shapes)
In this sample, contours are compared with reference contours using the function F_VN_MatchContours1vsN. Where
- the reference contours are idealized, manually created polygons
- and the other contours are extracted from an image.
Explanation
The function F_VN_MatchContours compares two contours used in the sample based on the Hu moments. The function F_VN_MatchContours1vsN represents an extension that can be used to compare a contour simultaneously with several other contours. In this sample, this is used to classify workpieces according to predefined reference shapes.
The reference shapes are idealized, manually created polygons of an isosceles triangle, a rectangle/square and a trapezoid. Since only the corner points were specified for the polygons, the reference shapes differ greatly from the test contours in the number of descriptive contour points. Furthermore, they differ in position, orientation, size and the degree of interference within the contour.
The function F_VN_DetectBlobs is used to search for contours with a certain minimum size in order to extract the shape of the components from the image. The contours that are found are then used for the described matching.
Input parameters
In addition to the contours to be compared, only a parameter hast to be passed to the function that describes the calculation method used to determine the dissimilarity from the Hu moments (eComparisonMethod of type ETcVnContoursMatchComparisonMethod). The method TCVN_CMCM_CONTOURS_MATCH_I1 calculates the sum of the differences in the reciprocal values of the individual characteristics, while the method TCVN_CMCM_CONTOURS_MATCH_I2 calculates the sum of the differences in the pure characteristics. Unlike the first two methods, the third variant TCVN_CMCM_CONTOURS_MATCH_I3 only calculates the maximum difference between the individual characteristics. Which of the three methods is best suited for comparing two contours depends on the use case.
Variables
 | Data type only in this sample The CustomTcVnArray4_Point2_DINT data type is not included in TwinCAT Vision, but was created explicitly for this sample. The implementation can be found in the download of this sample. |
// Indexes representing the different shapes
TRIANGLE : USINT := 0;
RECTANGLE : USINT := 1;
TRAPEZOID : USINT := 2;
// Images
ipImageIn : ITcVnImage;
ipImageInDisp : ITcVnDisplayableImage;
ipImageRes : ITcVnImage;
ipImageResDisp : ITcVnDisplayableImage;
// Contours
ipContourList : ITcVnContainer;
ipContour : ITcVnContainer;
ipIterator : ITcVnForwardIterator;
aShapeArrays : ARRAY [0..2] OF CustomTcVnArray4_Point2_DINT :=
[[[0,0],[40,0],[20, LREAL_TO_DINT(SQRT(1200))], [0,0]], // Triangle
[[0,0],[40,0],[40,40],[0,40]], // Rectangle
[[0,0],[40,0],[60,22],[20,22]]]; // Trapezoid;
aShapes : ARRAY[0..2] OF ITcVnContainer;
ipShapes : ITcVnContainer;
// Matching
ipMatchIndexes : ITcVnContainer;
ipDissimilarities : ITcVnContainer;
aMatchIndexes : ARRAY [0..2] OF ULINT;
aDissimilarities : ARRAY [0..2] OF LREAL;
// Parameters
stBlobParams : TcVnParamsBlobDetection;
fThreshold : REAL := 170;
fMinArea : REAL := 10000;
fMaxDissimilarity : LREAL := 0.02;
eComparisonMethod : ETcVnContoursMatchComparisonMethod := TCVN_CMCM_CONTOURS_MATCH_I3;
// drawing
aColors : ARRAY[0..2] OF TcVnVector4_LREAL :=
[[0, 175, 0, 0], [0, 0, 255, 0], [255, 0, 0, 0]]; // green, blue, red
aColorWhite : TcVnVector4_LREAL := [255, 255, 255];
aColorBlack : TcVnVector4_LREAL := [0, 0, 0];
aTexts : ARRAY[0..2] OF STRING := ['Triangle', 'Rectangle', 'Trapezoid'];
sText : STRING(255);
nTopLeftX : UDINT;
nTopLeftY : UDINT;
// Miscellaneous
aPixelValue : TcVnVector4_LREAL;
I : USINT;
stBoundingRectangle : TcVnRectangle_UDINT;
aOffsets : ARRAY [0..2] OF TcVnPoint := [[20, 50], [20, 100], [20, 160]];
hr : HRESULT;
Code
// Fill manually defined shapes into one container
hr := F_VN_CreateContainer(ipShapes, ContainerType_Vector_Vector_TcVnPoint2_DINT, 0, hr);
FOR i:=TRIANGLE TO TRAPEZOID DO
hr := F_VN_CreateContainerFromArray(ADR(aShapeArrays[i]), aShapes[i], ContainerType_Vector_TcVnPoint2_DINT, 4, hr);
hr := F_VN_InsertIntoContainer_ITcVnContainer(aShapes[i], ipShapes, i, hr);
END_FOR
// Prepare result image
hr := F_VN_ConvertColorSpace(ipImageIn, ipImageRes, TCVN_CST_GRAY_TO_RGB, hr);
// Check if background is light or dark
hr := F_VN_GetPixel(ipImageIn, aPixelValue, 50, 50, hr);
IF SUCCEEDED(hr) AND_THEN aPixelValue[0] < 128 THEN
stBlobParams.eThresholdType := TCVN_TT_BINARY;
ELSE
stBlobParams.eThresholdType := TCVN_TT_BINARY_INV;
END_IF
// Find contours in image
stBlobParams.bFilterByArea := TRUE;
stBlobParams.fMinArea := fMinArea;
stBlobParams.fMinThreshold := fThreshold;
hr := F_VN_DetectBlobs(ipImageIn, ipContourList, stBlobParams, hr);
// Iterate through all found contours
hr := F_VN_GetForwardIterator(ipContourList, ipIterator, hr);
WHILE hr = S_OK AND_THEN ipIterator.CheckIfEnd() <> S_OK DO
hr := F_VN_GetContainer(ipIterator, ipContour, hr);
hr := F_VN_IncrementIterator(ipIterator, hr);
// Match the current contour with the reference shapes
hr := F_VN_MatchContours1vsN(
ipRefContour:= ipContour,
ipContours:= ipShapes,
ipMatchIndexes:= ipMatchIndexes,
ipDissimilarities:= ipDissimilarities,
fDissimilarityThreshold:= 100,
eComparisonMethod:= eComparisonMethod,
hrPrev:= hr
);
hr := F_VN_ExportContainer(ipMatchIndexes, ADR(aMatchIndexes), SIZEOF(aMatchIndexes), hr);
hr := F_VN_ExportContainer(ipDissimilarities, ADR(aDissimilarities), SIZEOF(aDissimilarities), hr);
// Draw matching results
IF aDissimilarities[0] < fMaxDissimilarity THEN
// Calculate position of object
hr := F_VN_UprightBoundingRectangle(ipContour, stBoundingRectangle, hr);
nTopLeftX := LREAL_TO_UDINT(stBoundingRectangle.nX + 30);
nTopLeftY := LREAL_TO_UDINT(stBoundingRectangle.nY + (stBoundingRectangle.nHeight / 2));
// Draw matching result
hr := F_VN_DrawContours(ipContour, -1, ipImageRes, aColors[aMatchIndexes[0]], 5, hr);
hr := F_VN_PutTextExp(aTexts[aMatchIndexes[0]], ipImageRes, nTopLeftX, nTopLeftY, TCVN_FT_HERSHEY_SIMPLEX, 0.8, aColors[aMatchIndexes[0]], 2, TCVN_LT_8_CONNECTED, FALSE, hr);
hr := F_VN_DrawRectangle(nTopLeftX, nTopLeftY + 5, nTopLeftX + 200, nTopLeftY + 85, ipImageRes, aColorBlack, -1, hr);
// Draw all dissimilarity values
FOR i:=0 TO 2 DO
sText := CONCAT(LEFT(aTexts[aMatchIndexes[i]], 4), CONCAT(' ', REAL_TO_STRING(LREAL_TO_REAL(aDissimilarities[i]))));
hr := F_VN_PutTextExp(sText, ipImageRes, nTopLeftX + 5, nTopLeftY + 30 + 20*i, TCVN_FT_HERSHEY_SIMPLEX, 0.6, aColorWhite, 1, TCVN_LT_8_CONNECTED, FALSE, hr);
END_FOR
END_IF
END_WHILE
// Draw reference shapes
FOR i:=TRIANGLE TO TRAPEZOID DO
hr := F_VN_DrawContoursExp(aShapes[i], -1, ipImageRes, aColors[i], 2, TCVN_LT_8_CONNECTED, 0, 0, aOffsets[i], hr);
END_FOR
hr := F_VN_TransformIntoDisplayableImage(ipImageRes, ipImageResDisp, hr);
hr := F_VN_TransformIntoDisplayableImage(ipImageIn, ipImageInDisp, hr);
Results
The result image shows the reference shapes (left border) and the found contours whose dissimilarity relating to all reference shapes does not exceed a maximum value. Furthermore, a visualization indicates which reference shape the contour resembles most closely. A corresponding description text (including dissimilarities for each reference shape) is superimposed over the respective contour. For the parameters used in this sample, the result for the MatchContours1.bmp image looks as follows:
In each case the lowest value indicates the correct reference contour. If the parameter eComparisonMethod is changed from TCVN_CMCM_CONTOURS_MATCH_I3 to TCVN_CMCM_CONTOURS_MATCH_I2, the results change considerably. This also means that the parameter for the maximum accepted dissimilarity must be adjusted:
The new values allow a much less clear assignment to corresponding reference shapes, and the round component is incorrectly assigned to the rectangle reference shape. This calculation method is therefore obviously less suitable for this sample.