Train Neural Network

After all views are correctly labeled and split into the training images and the test images, neural network training is performed in the following general way:

1. Configure tool parameters and start the training. Train the tool by pressing the Brain icon. Each image in the image set that is being used for training (defined in the Training Set dialog) is sampled across its full extent, using the specified Feature Size when the architecture is Focused.
2. The resulting samples are provided as input to the VisionPro Deep Learning deep neural network.
3. For each sample, the neural network produces a specific response (depending on the tool type), and this response is compared with the image labeling associated with the sample's location in the training image.
4. The internal weights within the network are repeatedly adjusted as the samples are processed and reprocessed. The network training system continually adjusts the network weights with a goal of reducing the error (difference or discrepancy) between the network's response and the labeling provided by the user.
5. This overall process is repeated many times, until every sample from every training image has been included at least the number of times specified by he Epoch Count parameter.

 

 

1. The sampling region.

2. The user-drawn, labeled defect region.

3. The neural network.

4. The response by the network.

5. The iterative process of adjusting the weights to reduce the discrepancy (in other words, error) between the labeled defect (in yellow) and the network response (in blue).

 

The specific characteristics of the neural network training vary somewhat depending on the type of tool being trained. The goal for network training of the Red Analyze tool in Supervised Mode is to reduce the spatial discrepancy between the defect labeling and the detected defects. For Red Analyze Focused Supervised, the network is trained to locate and identify defect regions within an image. The labeling that you perform for the Red Analyze tool in Supervised Mode labels all of the defect pixels in the labeled image. For a given sampling region within the image, the goal of training is for the network to correctly identify the defect pixels as defects.

 

• Sampling Region and Sampling Parameter

  Red Analyze Focused Supervised samples pixels with the sampler that is tied to a sampling region defined by the users with Sampling Parameters and thus it requires configuring the sampling parameters to determine a sampling region before start its training. If a sampling region does not include any defect pixels, then the network should produce no response.

• Training with Validation

  Red Analyze Focused Supervised does not support the training with the validation scheme and so it doesn't have the validation data set nor require configuring the validation set in Tool Parameters. The purpose of adopting the validation data is to validate each trained neural network and with the results from this validation data for selecting the most performing and stable neural network with the given training set.

 

Configure Tool Parameters

After a tool has been added, the tool's parameters can be accessed to fine-tune the tool's performance prior to training, and how the tool will process images during runtime operation. The VisionPro Deep Learning tool parameters adjust the how the neural network model is trained, and also how the tool processes statistical results.For the majority of applications, the most common Tool Parameters to be adjusted are the following:

​

• Feature Size
• Training Set
• Perturbation
• Sampling Density

 

 

Architecture Parameter

The Architecture parameters selects the type of neural network model that will be used. This option is useful when you want to get more accurate results, at the expense of increased training and processing times. The High Detail and High Detail Quick architecture setting configures the tool to consider the entire image equally, while the Focused architecture setting is selective, focusing on the parts of the image with useful information. Due to this focus, the network can miss information, especially when the image has important details everywhere.

 

Network Model Parameter

The Network Model parameter allows you to reduce the size of the trained tool network, which will reduce the amount of time required for processing. The default value (Large) retains the standard tool behavior and performance. The two other values, Small and Medium, provide progressively faster run time execution.

Selecting a value other than Large might result in different tool results. These differences are typically minor, however, when using this option, you will need to validate tool performance using your own labeled data.

 

Sampling Parameters

Features are the pixels in your image data you are interested in and, at the same time, the pixels that are critical for solving your machine vision problem and achieving your specific goal. For example, the features are the normal/defect pixels in a view for a Red Analyze tool.

For Focused Mode tools, you need to specify in detail how your features are like as much as possible to effectively teach your tool about them. This specification is achieved by configuring up Sampling Parameters. A Focused tool uses a feature sampler which samples pixel information from a view, and by configuring up Sampling Parameters, you tell this feature sampler about the properties of the features that should be sampled or not.

 

The Sampling tool parameters control the way that images are sampled during training and processing.

Parameter	Description
Feature Size	Specifies the typical feature diameter, in pixels. The Feature Size parameter is graphically displayed in the lower left of the image, and can be graphically resized within the image to set a more accurate size. Feature size strongly influences processing time (n2), in other words,, a Feature Size of 100 is 100 times faster than a size of 10, while a Feature Size less than 15 usually does not yield good results. When setting the Feature Size, consider the following in regards to Processing time (Ptime): Note: The tool will actually see an area five times larger than the Feature Size setting. However, the level of detail will be much higher in the center of the feature, as opposed to the periphery.
Color	Specifies the number of color channels to use when sampling the image. When set to 1, color images will be converted to greyscale. Treats the image as greyscale Two channels images (spectral images, gray+alpha) BGR image. BGR(A) image Note: If the image is RGB(A) it will be converted to grayscale. For computational efficiency (memory allocation, transfer, file save, color conversion,… ) always prefer the correct number of channels. The VisionPro Deep Learning tools use the BGR channel order. If your application relies on color images, only use the minimal number of required color channels possible, and only send images that already have the correct number of channels to avoid conversion. This is because: Number of Image Channels Number of Training Channels Description 1 (greyscale) 1 The correct setting for a greyscale image. 1 2, 3, 4 Most likely, this will result in a training error. 2 1 The tool will use only the first channel. 2 2 The tool will use the full pixel information. 2 3, 4 This will result in a training error. 3 (BGR) 1 This will result in a BGR to greyscale conversion. 3 2 The tool will only use the first two channels (in other words, B and G). 3 3 The tool will use the full pixel information. 3 4 Most likely, this will result in a training error. 4 (BGRA) 1 This will result in a BGRA to greyscale conversion. 4 2 The tool will only use the first two channels (in other words, B and G). 4 3 The tool will only use the first three channels (in other words, B, G and R). 4 4 The tool will use the full pixel information.
Border Type	Specifies how pixels on the outside of the image are sampled. Tip: Adding an additional mask at the boundaries of the image highly reduces the false detection rate. Black: Fills the outside of the image with a solid color, black. Replicate: Fills the outside of the image with the last pixel.
Masking Mode	Specifies how a mask will be applied to the sampled image. A mask is used to limit areas of the image processed by the tool. Note: Masks can be set after training, however, setting them prior to training will help the learning phase. Transparent: Samples are only collected within the unmasked parts of the image, but data from the Context Region is also collected from the areas that are masked. This ensures that features or defects that are at the boundary of the mask or ROI generate the same response as features or defects that are centrally located. Mask: The mask is used to ignore the areas which are masked. All pixels that are masked are set to 0, which prevents data from the masked parts of the image from being considered during training or runtime. However, it also alters the response of the tool to defects or features near the mask boundary. This setting also works to focus the tool toward the center of the ROI. Overlay: The mask is added to the sampled image as an additional color channel.
Centered	This parameter can be used when your inspection contains objects that have a circular symmetry in respect to the center of the view, such that the resulting learned model becomes simpler because the object can be virtually unrolled.

 

Sampling Parameter Details: How Features are Sampled

Green Classify in Focused mode, Red Analyze in Focused mode, Blue Locate and Blue Read tools do not sample the input images uniformly (although the image sampling does cover the entire image extent). During training, the tools use a special technique to selectively sample at a higher rate those parts of the images that are determined to be more likely to contribute additional information to the network.

Because the network training is performed using both information within the sample region and contextual information from around the sample region, the tool can be strongly influenced by samples collected at the edge of an image. If you are using a view within an image, then the context information for samples collected at the edge of the view will use pixels from outside of the view for context data.

1	Feature Size
2	Sample Region
3	Context Region

If the sample is at the edge of the image itself, the tool will generate synthetic pixels for use as context. You can control the specific method used for this through the use of masks, borders, and sample color channels (via the Border Type and Color , respectively).

The tools also allow you to provide masks for use during sampling. This allows you to explicitly exclude parts of the image from training, although even masked regions are considered as context, depending on the Masking Mode parameter setting.

Finally, if you are using color images (or any image with multiple planes or channels), you can explicitly specify which channels are sampled. Using multiple channels has a minor impact on training and processing time, but it can allow the tools to work more accurately in cases where color provides important information in the image.

 

Sampling Parameter Details: Feature Sampling and Feature Size

Green Classify in Focused mode, Red Analyze in Focused mode, Blue Locate and Blue Read tools analyze images based on the Feature Size that you specify. The feature size, which is based in pixels, functions as a hint to the tool about the expected size of "meaningful" or "distinctive" features in the input images. The best method for selecting a feature size is to examine the input images as if you were a human inspector. Note the features in the image that you would use to characterize the image as good or bad, to identify a defect or problem, or to determine where something was and what it was.

For example, if you were attempting to classify pictures of airplanes based on the number of engines, the feature size would be based on the approximate size of an airplane engine.

During both training and runtime, the tool will collect samples from the image that correspond to the pixels within a subregion of the image, as well as contextual information around that region. The contextual region is approximately five times the feature size.

1	Feature Size
2	Sampling Region
3	Context Region

 

Specifying a feature size is a subjective judgment, although there are a few specific guidelines to follow. For the Red Analyze tool (in either Unsupervised Mode or Supervised Mode), the feature size should approximate the size of a typical defect. For elongated defects, such as streaks, smears, scratches and tears, the feature size should approximate the width of the defect.

1	Defect
2	Feature Size

 

Training Parameters

The Training tool parameters control the training process. If any changes are made to the Training tool parameters after the tool has been trained, it will invalidate the training and necessitate that the tool be retrained.

Parameters	Description
Training Set	Launches the Training Set dialog, which is used to specify the sample sets and percentage of labeled images to randomly select as training samples for the neural network each time a new training is initiated.
Epoch Count	Specifies the number of optimization iterations done during training. This setting can be reduced when your application has limited complexity, or if a lower quality model may be useful when testing different parameter settings. An epoch is the term for passing your entire Training Set into the neural network. The tools typically need to see the training image set data somewhere on the order of 50 times (and is in the Fine Tuning area), which is the default setting, and is typically sufficient for most standard applications. Choosing to use fewer epochs could result in the neural network being stuck in learning, or unable to accurately solve the problem, while too many epochs could result in overfitting (aka over-training) the results (in other words, it will learn only the trained images, and anything outside of the trained images will be considered invalid). It is important to train the network to the point where it is generalized off of your training image set. If you increase the epochs too much, you run the risk of over-training and overfitting the training image set data. Tip: If your training image set contains a large number of images, or the size of the images in your training image set are large compared to the Feature Size, you may need to increase the Epoch Count setting. There is a risk of overfitting if you have a very small training image set. If you have a large training image set, and your network is addressing all of the samples in the set, the network is much more general than if you have a smaller training image set and the network is keyed into that small sample set.
Low Precision	Specifies whether or not the tool will create a low precision model after training. The low precision model is useful for applications where speed optimization is a priority during runtime operation. There is a likelihood that results will be different between the two modes of operation. When enabled, there will be a white lightning bolt icon added to the tool's icon. Note: If the tool has been previously trained prior to checking the check box, you will be prompted to retrain the tool. If you clear the check box after enabling the low precision model, you will also need to retrain the tool. The low precision model requires the Standard or Advanced VisionPro Deep Learning Licenses, and also requires a GPU with CUDA Compute Capability 6.1 or higher (for more information, visit the NVIDIA website).

 

Training Parameter Details: Epoch Count

Epoch Count parameter lets you control how much network refinement is performed. As described in the Neural Network Training topic, the training process repeatedly processes input samples through the network, compares the network result with the user-supplied labeling, then adjusts the network weights with a goal of reducing this error. Because of the large number of network nodes (and thus weights), this process can be repeated almost indefinitely, with each iteration resulting in a progressively smaller improvement in the error. Increasing the Epoch Count parameter setting increases the number of iterations of training that are performed. This will reduce the network error on the training images at the cost of requiring more time for training.

It is important to keep in mind, however, that the goal for training the network is to perform accurately on all images, not just those used for training. As the epoch count is increased, the network will tend to experience overfitting (Terminology), where the error on untrained images increases at the same time that the error on trained images decreases. For this reason, you should carefully monitor the network performance on all images as you adjust the epoch count. You have to choose an optimal Epoch for your dataset because its optimal value is different by dataset, particularly the statistical diversity of your dataset.

 

Perturbation Parameters

The VisionPro Deep Learning neural network can only be trained to learn the features in images that it actually sees. In an ideal world, your training image set would include a representative set of images that expressed all of the normal image and part variations. In most cases, however, training needs to be performed with an unrepresentative image set. In particular, image sets are often collected over a short period of time, so normal part and lighting variations over time, as well as changes or adjustments to the optical or extrinsic characteristics of the camera are not reflected.

The VisionPro Deep Learning training system allows you to augment the image set by specifying the types of appearance variation that you expect during operation, through the use of the Perturbation parameters, such as the following:

• Luminance
• Contrast
• Rotation
• Scale

 

The Perturbation parameters allow the VisionPro Deep Learning tools to artificially generate images to be trained on, improving results for applications with high amounts of variance. These parameters are common across all of the tools. The Perturbation parameters can also be combined. This allows for the generation of more complex images by using the parameters separately, as well as in conjunction.

 

For each perturbation type, you can specify the range for the perturbation.

Parameter	Description
Rotation	Specifies the possible orientations, defining a piece-wise uniform distribution from which a perturbing rotation angle is drawn (if set to [0°, 0°], no rotational perturbation will be used). For example, with the following settings, [-10°, 10°] or [170°, 190°], the part can be detected between -10 and 10 degrees, or between 170 and 190 degrees. These types of settings are particularly useful if your part can be orientated with a regular orientation, or upside down with a small variation in its angle, but never in between.
Scale	Specifies the normal distribution around 1 from which a perturbing scaling factor is drawn (if set to 0%, it will not be applied). For example, a Scale setting of 100%, provides images half the original size to double the original (0.5x to 2.0x the original). Original = Possible Perturbations =
Aspect-Ratio	Specifies the normal distribution around 1 from which a perturbing aspect-ratio factor is drawn (if set to 0%, it will not be applied).
Shear	Specifies the normal distribution around 1 from which a perturbing shearing factor is drawn (if set to 0%, it will not be applied).
Flip	Specifies whether to randomly flip the image during sampling in the horizontal, vertical or both directions. This is useful if the object or surface exhibits the corresponding symmetries. This setting can help to highly increase the quantity of training data. Horizontal Vertical Both
Luminance	Specifies the normal distribution around 1 from which a perturbing luminance factor is drawn (if set to 0%, it will not be applied).
Contrast	Specifies the normal distribution around 1 from which a perturbing contrast factor is drawn (if set to 0%, it will not be applied).
Invert Contrast	Specifies whether or not to randomly invert the contrast in order to simulate contrast reversal in the training samples. Note: The Invert Contrast parameter is only available when Expert Mode is enabled. This is enabled via the Help menu.

 

Recover Last Parameters: Restore Parameters

Restore Parameter button is designed for the easy turning back of tool parameter values to the values that you chose in the last training task. It remembers all values in Tool Parameters used in the last training session. So, if you changed any of its values and now want to revert this change, you can click it to roll back to the tool parameter values which are used in the last training. Note that it is disabled when the tool has never been trained or there were no changes from the initial set of tool parameter values.

The following steps explain how Restore Parameter works:

1. Restore Parameter button is always disabled when the current tool was not trained.

2. Once the current tool is trained, the checkpoint of parameter rollback is set to the values in Tool Parameters of the last training session. At this point, if you change any value in Tool Parameters, the button is enabled.

3. Click Restore Parameter button and it reverts the changed value to the value of the checkpoint.

4. If you train the current tool again, with some changes in Tool Parameters, then the checkpoint of parameter rollback is updated to the changed parameter values. Again, the button is disabled unless you make another change for the values in Tool Parameters.

5. If you make another change and click Restore Parameter button again, it reverts the changed value into the value of the updated checkpoint.

Note that if you re-process a trained tool after changing the values in Processing Parameters, the checkpoint of parameter rollback is not updated, and thus Restore Parameter remains enabled. The checkpoint is updated only after the training of a tool is completed.

Disabled

Enabled

 

Control Neural Network Training

The training of Red Analyze Focused Supervised can be controlled by configuring the tool parameters and the training set.

 

Training Set

The largest single determinant affecting the network training phase is the composition of the Training Set. The best method for controlling the network training phase is to construct a proper training set for your tool. In this way, you can separate images/views into categories that allow you to determine if your tool is generalizing your images/views properly.