Calibrate and Transform Images to Correct for Image Distortion

In its deployment environment, the In-Sight vision system may capture images that exhibit distortions induced by certain setup conditions, such as lens and perspective distortions. Some lens may produce barrel or pincushion radial distortion patterns which cause lines which appear straight in the real-world to appear as curved lines in the image. Perspective distortion is induced when an In-Sight vision system cannot be mounted perpendicular to the inspection area. In these situations, the features in the image may appear stretched or oblong, depending on their location within the field of view, resulting in inaccurate results.

The In-Sight vision system contains a set of functions to compensate and correct for these types of non-linear and perspective distortions: the Calibrate functions.

The Calibrate functions are used to "calibrate" the In-Sight vision system's optics, a process which serves two purposes:

  • The In-Sight vision system will compensate and correct optical distortions ("transforming" the image) and return highly accurate measurements. For more information, see Calibration Accuracy.
  • The In-Sight vision system converts pixel units into real-world measurement units, such as millimeters or inches.

Calibrating an In-Sight vision system enables you to more accurately locate objects at the edges of images, and provides greater mounting flexibility, including the ability to mount the In-Sight vision system up to 45° from the inspection surface. For more information, see Calibration Guidelines.

Example Overview

This topic illustrates how to use the CalibrateGrid and TransformImage functions to calibrate and transform an image to correct for lens and perspective distortion. This example also demonstrates how to optimize the performance of other In-Sight Vision Tools, specifically the TrainPatMaxPattern, FindPatMaxPatterns and FindCircle functions, by referencing the calibrated and transformed output.

Application

Locate a gasket in the field of view, using its shape as a pattern, for a robotic pick-and-place operation. After locating the part, and reporting its location (in millimeters), the radius of the largest hole in the gasket should be measured (in millimeters) to verify that the part is within the specified tolerance. The physical requirements of the deployment environment require that the In-Sight vision system be mounted at a 30° angle from the inspection surface, introducing perspective distortion.

Load the Sample Application and Images

In a real-world application, you would be required to mount the In-Sight vision system, connect it and acquire images of your part or object. For the purposes of this example, a sample job and images are provided to simulate the application.

To begin, you will need to load the sample calibration job and its accompanying images.

  1. Log on to your local emulator or In-Sight vision system.
  2. Select Open Job from the File Menu, and from the Open dialog, navigate to the Calibration folder of your In-Sight Explorer Spreadsheet Sample Jobs directory (for Windows 7/Windows 10: C:\Users\Public\Public Documents\Cognex\In-Sight\In-Sight Explorer x.x.x\Sample Jobs\Spreadsheet\Calibration), and choose the "CalibGrid_TransformImage.job" file.
  3. Once the job has been loaded, you will need to load the corresponding sample images. Select Record/Playback Options from the Image Menu.
  4. The Record/Playback Options dialog should automatically open with the Playback tab presented; if not, click on the Playback tab.
  5. From the Playback Folder section, press the Browse button to launch the Browse for Folder dialog. Navigate to the Sample Jobs > Spreadsheet directory, and select the Calibration folder. Press the OK button to continue.
  6. The Playback Folder should now display the directory that you specified, and the Image Count indicates the number of images in the directory. Press the OK button in the Record/Playback Options dialog to continue.

Calibrate the Image

For In-Sight vision systems, calibration refers to the process of "mapping" the image area by establishing a relationship between features in the image and known, real-world measurement units. This process corrects for linear and non-linear distortions, as well as creating a 2D transformation between the image's pixel units and real-world measurement units.

There are three different functions, with varying degrees of accuracy, that are used to calibrate an In-Sight vision system: Calibrate, CalibrateAdvanced and CalibrateGrid.

Note: The Calibrate and CalibrateAdvanced functions rely on four to nine user-specified points, respectively, in the image to relate the real-world measurements of the objects being inspected. These functions only compensate for linear and non-linear distortions.

The CalibrateGrid function creates a "map" of the image area by acquiring an image of a grid pattern of either dots or checkerboard squares. The function then relates the found features of the grid (the dots or intersections of squares) to the user-specified grid spacing of the grid pattern. This process creates a highly accurate computation that can then be used to transform distorted features into their correct shape.

Configure the Calibrate Grid function

  1. Select the cell containing the Calib data structure returned by the CalibrateGrid function, and double-click the cell to open the CalibrateGrid dialog.

  2. Click on Setup in the navigational pane of the CalibrateGrid dialog to open the CalibrateGrid Setup dialog. The Grid Type parameter is set to "Dots, with Fiducial"; the Grid Spacing parameter is set to 25; the Grid Units parameter is set to Millimeters; and the Number of Poses parameter is set to 1.

    Note: If your part or object being inspected requires a large field of view, or if the image is highly distorted, consider using multiple poses to increase the calibration accuracy.
  3. Select Pose from the navigational pane to acquire the image used for the calibration. For this application, press the From File button and load the CalibrationGrid.bmp file from the Sample Jobs > Spreadsheet directory.
  4. Once the image has been loaded, the CalibrateGrid function will automatically process the image and report the number of feature points found. Press the Calibrate button to continue.
  5. The calibration data is then contained and stored in the Calib data structure. This structure contains the data needed to both correct for distortion and convert pixels to real-world units, and can now be referenced by other In-Sight Vision Tools.

Transform the Image

The next step in the process is to add the TransformImage function to create an undistorted image that can be referenced by the In-Sight Vision Tools. The CalibrateGrid function returns a Calib data structure, which contains the mathematical computations used to create the image "map." The TransformImage function utilizes this Calib data structure to create an Image data structure that applies the corrective computations, returning an undistorted image.

Configure the Transform Image Function

  1. Select the cell containing the Image data structure returned by the TransformImage function, and open its property sheet.
  2. The Calibration parameter is configured as an absolute cell reference to the Calib data structure returned by the CalibrateGrid function, and all of the other parameters are left at their default settings.
  3. Press the OK button to close the TransformImage property sheet.
  4. Single-click on cell containing the Image data structure returned by the TransformImage function to view the undistorted image. This structure corrects for the perspective distortion, creating a circular shape versus an oval shape, for example.
  5. When a TransformImage function is created, an ExtractCalibration function is also automatically created and inserted into the spreadsheet. The ExtractCalibration function returns a Calib data structure which contains the calibration between the transformed image's pixel values and the real-world units specified in the CalibrateGrid function.

Configure the Vision Tools and Convert the Results

In this application, the gasket may appear at random positions within the field of view, so it must first be located within the image, then inspected to verify its correct assembly before being picked up by a robot. This requires that the location of the gasket be returned in millimeters, and the radius of the largest hole in the gasket also be returned in millimeters.

Before the image was calibrated and transformed, the shape of the gasket and the largest hole would vary, depending on the location of the gasket in the field of view, and the results were returned in pixels. The CalibrateGrid and TransformImage functions were used to correct the distortion effects and return calibrated results, and the In-Sight Vision Tools are now configured to reference the undistorted and calibrated image they return.

Set up the Inspection

To set up the Vision Tools, the functions must be installed and linked together to perform their inspections. This application requires the use of a fixture, due to the object's varied locations within the field of view, from image-to-image. Our fixture will also serve another purpose — its location data will be passed to a robot.

Note: Use the Job Audit toolbar to view the various dependencies between the In-Sight Vision Tools.

Configure the TrainPatMaxPattern Function

The TrainPatMaxPattern function produces the model pattern that is referenced by the FindPatMaxPatterns function. The shape of the gasket will be used as the model.

  1. Using the Record/Playback toolbar, load image Gasket_01.bmp.
  2. Select the cell containing the Patterns data structure returned by the TrainPatMaxPattern function, and open its property sheet.
  3. The Image parameter is set as an absolute cell reference to the Image data structure output by the TransformImage function. This ensures that the TrainPatMaxPattern function always references the undistorted image.
  4. Enter interactive graphics mode to adjust the region of interest of the Pattern Region parameter by pressing the Edit Graphic button on the property sheet toolbar. Configure the region of interest so it tightly encompasses the shape of the gasket. Double-click the region to accept the changes and exit interactive graphics mode.
  5. Press the OK button to close the TrainPatMaxPattern function property sheet.
  6. The model pattern has now been established.

Configure the FindPatMaxPatterns Function

The FindPatMaxPatterns function is used to locate the part within the field of view, based on the model pattern established by the TrainPatMaxPattern function.

  1. Select the cell containing the Patterns data structure returned by the FindPatMaxPatterns function, and open its property sheet.
  2. The Image parameter is set as an absolute cell reference to the Image data structure output by the TransformImage function. This ensures that the FindPatMaxPatterns function always references the undistorted image.
  3. The FindRegion parameter is maximized to encompass the entire image, because the gasket may appear anywhere within the image.
  4. The Pattern parameter is set as an absolute cell reference to the Patterns data structure returned by the TrainPatMaxPattern function.
  5. The Outside Region parameter is set to 15, because parts of the gasket may appear outside of the image area.
  6. The Angle Start and Angle End parameters have been adjusted to -90 and 90, respectively, to account for the rotation of the gasket within the image.
  7. All of the other parameters are left at their default settings.
  8. Press the OK button to close the FindPatMaxPatterns function property sheet.

Configure the FindCircle Function

The FindCircle function is used to determine the radius of the largest hole in the gasket.

  1. Select the cell containing the Edges data structure returned by the FindCircle function, and open its property sheet.
  2. The Image parameter is set as an absolute cell reference to the Image data structure output by the TransformImage function. This ensures that the FindCircle function always references the undistorted image.
  3. The Fixture parameter is set as an absolute reference to the Row, Col and Theta outputs of the FindPatMaxPatterns function. This ensures that the Annulus of the FindCircle function is moved in relation to the found pattern returned by the FindPatMaxPatterns function.
  4. Enter interactive graphics mode to adjust the region of interest of the Annulus parameter by pressing the Edit Graphic button on the property sheet toolbar. Position the annulus over the largest hole of the gasket and configure the region of interest so that the outer radius is slightly larger than the largest hole of the gasket, while the inner radius is about half the size of the outer radius. Double-click the region to accept the changes and exit interactive graphics mode.
  5. The Polarity parameter is set to white-to-black; the annulus search region is inspected from the inner radius to the outer radius, and the gasket is "black" while the background is "white." Specifying the edge polarity helps to speed the search.
  6. The Find By parameter is set to largest circle, also in an attempt to speed the search.
  7. All of the other parameters are left at their default settings.
  8. Press the OK button to close the FindCircle function property sheet.

Convert the Results

Both the FindPatMaxPatterns and FindCircle functions return their results in pixels, but the robot and inspection require the results in millimeters. In addition, because the gasket may appear in the field of view in random positions and orientations, its necessary to use a fixture to accurately position the search regions of the In-Sight Vision Tools in the image, which requires that the results be returned in pixels.

For example, the FindCircle function requires the output of the FindPatMaxPatterns function results in pixels: the Row, Col, and Angle values returned by the FindPatMaxPatterns function are being used as a Fixture input for the FindCircle function, indicating where in the image to position its Region of Interest (ROI).

To solve for this, the TransPatternsToWorld and TransEdgesToWorld functions will be used to convert the pixel results of the FindPatMatPatterns and FindCircle functions to real-world units, millimeters. The values returned by the TransPatternsToWorld and TransEdgesToWorld functions will then be output, and the FindCircle function will still be able to utilize the pixel values returned by the FindPatMaxPatterns function as a fixture.

Note: If a fixture was not required, the CalibrateImage function could be utilized to convert the pixel results of the tools to real-world units.

Configure the TransPatternsToWorld Function

The TransPatternsToWorld function converts the pixel results returned by the FindPatMaxPatterns function into real-world measurement units.

  1. Select the cell containing the Patterns data structure returned by the TransPatternsToWorld function, and open its property sheet.
  2. The Calib parameter is set as an absolute cell reference to the calibration data contained in the Calib data structure returned by the ExtractCalibration function. The ExtractCalibration function contains the calibration between the transformed image's pixel units and the real-world units specified in the CalibrateGrid function.
  3. The Patterns parameter is set as an absolute cell reference to the Patterns data structure returned by the FindPatMaxPatterns function.
  4. The Number to Convert parameter is left at its default setting.
  5. Press the OK button to close the TransPatternsToWorld function property sheet.
  6. The results returned by the TransPatternsToWorld function should now appear in their real-world units, in this case, millimeters.

Configure the TransEdgesToWorld Function

The TransEdgesToWorld function converts the pixel results returned by the FindCircle function into real-world measurement units.

  1. Select the cell containing the Edges data structure returned by the TransEdgesToWorld function, and open its property sheet.
  2. The Calib parameter is set as an absolute cell reference to the calibration data contained in the Calib data structure returned by the ExtractCalibration function. The ExtractCalibration function contains the calibration between the transformed image's pixel units and the real-world units specified in the CalibrateGrid function.
  3. The Edges parameter is set as an absolute reference to the Edges data structure returned by the FindCircle function.
  4. The Number to Convert parameter is set to 1.
  5. Press the OK button to close the TransEdgesToWorld function property sheet.
  6. The results returned by the TransEdgesToWorld function should now appear in their real-world units, in this case, millimeters.

Test the Application

To get a better understanding of the effects of perspective distortion:

  1. Open the TrainPatMaxPattern, FindPatMaxPatterns and FindCircle functions, and set their Image parameter to A0, to view the image in its uncorrected format.
  2. Adjust the model pattern of the TrainPatMaxPattern function to the distorted image.
  3. Using the Record/Playback toolbar, cycle through the images and note the difficulty in locating the pattern and measuring the size of the hole in the gasket.
  4. Reset the functions' Image parameter back to the Image data structure returned by the TransformImage function, and note that the tools successfully perform on each image.

Inspection Results Example

The following table illustrates the effectiveness of the CalibrateImage and TransformImage functions in correcting for perspective distortion.

Inspection Results Distorted Image Undistorted Image
Gasket_00.bmp (Trained Pattern Image)
Pattern Score 99.997 99.997
Radius (MM) 16.666 20.573
Gasket_01.bmp
  57.27 99.954
18.109 20.694
Gasket_02.bmp
  54.576 99.954
18.302 20.723
Gasket_03.bmp
  0 99.997
#ERR 20.765
Gasket_04.bmp
  86.539 99.912
17.615 20.641
Gasket_05.bmp
  66.928 99.997
18.047 20.669
Gasket_06.bmp
  0 98.815
#ERR 20.507
Gasket_07.bmp
  57.133 99.826
18.267 20.770
Gasket_08.bmp
  0 98.070
#ERR 20.633
Gasket_09.bmp
  0 99.933
#ERR 20.777
 
Average Radius (MM) 17.834 20.664
Standard Deviation 0.622 0.089

Copyright © 1999 - 2019  |  Cognex Corporation  |  All Rights Reserved.

In-Sight® Explorer Help  |  2019 October 02  |  Revision: 5.8.0.1