Train Neural Network

After all views are correctly labeled and split into the training images and the test images, neural network training is performed in the following general way:

1. Configure tool parameters and start the training. Train the tool by pressing the Brain icon. Each image in the image set that is being used for training (defined in the Training Set dialog) is sampled across its full extent, using the specified Feature Size when the architecture is Focused.
2. The resulting samples are provided as input to the VisionPro Deep Learning deep neural network.
3. For each sample, the neural network produces a specific response (depending on the tool type), and this response is compared with the image labeling associated with the sample's location in the training image.
4. The internal weights within the network are repeatedly adjusted as the samples are processed and reprocessed. The network training system continually adjusts the network weights with a goal of reducing the error (difference or discrepancy) between the network's response and the labeling provided by the user.
5. This overall process is repeated many times, until every sample from every training image has been included at least the number of times specified by he Epoch Count parameter.

 

 

1. The sampling region.

2. The user-drawn, labeled defect region.

3. The neural network.

4. The response by the network.

5. The iterative process of adjusting the weights to reduce the discrepancy (in other words, error) between the labeled defect (in yellow) and the network response (in blue).

 

The specific characteristics of the neural network training vary somewhat depending on the type of tool being trained. The goal for the Green Classify tool network training is to reduce the number of mismatches between labeled and detected classes. The Green Classify tool is unique among the Deep Learning tools in that it produces a single result for the entire input image. The tool collects image samples in the same way as the other tools, but the samples are pooled during processing and a single result for the entire image is produced.

 

• Sampling Region and Sampling Parameter

  While Green Classify Focused samples pixels with a sampling region defined by the users, Green Classify High Detail and Green Classify High Detail Quick samples from the entire image, so they don't have a sampling region, not requiring the sampling parameters in training.

• Training with Validation

  Green Classify High Detail and Green Classify High Detail Quick shares most aspects in training (and also in processing) as they come from the same architectural origin, but Green Classify High Detail uses the validation data to validate each trained neural network and with these validation results selects the most performing and stable neural network with the given training set while Green Classify High Detail Quick does not. Green Classify High Detail Quick didn't adopt the validation of neural networks in training phase to serve its built-in purpose: faster speed of training than High Detail.

 

Configure Tool Parameters

After a tool has been added, the tool's parameters can be accessed to fine-tune the tool's performance prior to training, and how the tool will process images during runtime operation. The VisionPro Deep Learning tool parameters adjust the how the neural network model is trained, and also how the tool processes statistical results.

When the Green Classify tool's Architecture parameter is set to High Detail Quick mode, the tool is configured to consider the entire image equally, like High Detail mode. This option is useful when you want to get more accurate results, at the expense of increased training and processing times.

However, unlike Green Classify High Detail, you do not need configuring many parameters in Tool Parameters by hand because Green Classify High Detail Quick requires only a few parameters due to its state-of-the-art training algorithm. There are only 2 categories of tool parameters for Green Classify Tool in High Detail Quick mode. You can see more detail information for each parameter.​

 

Architecture Parameter

The Architecture parameters selects the type of neural network model that will be used. This option is useful when you want to get more accurate results, at the expense of increased training and processing times. The High Detail and High Detail Quick architecture setting configures the tool to consider the entire image equally, while the Focused architecture setting is selective, focusing on the parts of the image with useful information. Due to this focus, the network can miss information, especially when the image has important details everywhere.

 

Training Parameters

The Training tool parameters control the training process. If any changes are made to the Training tool parameters after the tool has been trained, it will invalidate the training and necessitate that the tool be retrained.

Parameter	Description
Epoch Count	The number of times to train using all training images. Higher the value more the repetition. The value is from 1 through 100000. Tip: High Detail Quick mode uses different concept of Epoch Count from Focused mode. You’d better use higher epoch count in High Detail Quick mode than in Focused mode with same database.
Training Set	The dataset that is used to create a deep learning model. This means that during the deep learning, only the features of the images included in the train set are extracted to create a deep learning model. You can select training set in Select Training Set dialog.

 

Training Parameter Details: Epoch Count

Epoch Count parameter lets you control how much network refinement is performed. As described in the Neural Network Training topic, the training process repeatedly processes input samples through the network, compares the network result with the user-supplied labeling, then adjusts the network weights with a goal of reducing this error. Because of the large number of network nodes (and thus weights), this process can be repeated almost indefinitely, with each iteration resulting in a progressively smaller improvement in the error. Increasing the Epoch Count parameter setting increases the number of iterations of training that are performed. This will reduce the network error on the training images at the cost of requiring more time for training.

It is important to keep in mind, however, that the goal for training the network is to perform accurately on all images, not just those used for training. As the epoch count is increased, the network will tend to experience overfitting (Terminology), where the error on untrained images increases at the same time that the error on trained images decreases. For this reason, you should carefully monitor the network performance on all images as you adjust the epoch count. You have to choose an optimal Epoch for your dataset because its optimal value is different by dataset, particularly the statistical diversity of your dataset.

 

Recover Last Parameters: Restore Parameters

Restore Parameter button is designed for the easy turning back of tool parameter values to the values that you chose in the last training task. It remembers all values in Tool Parameters used in the last training session. So, if you changed any of its values and now want to revert this change, you can click it to roll back to the tool parameter values which are used in the last training. Note that it is disabled when the tool has never been trained or there were no changes from the initial set of tool parameter values.

The following steps explain how Restore Parameter works:

1. Restore Parameter button is always disabled when the current tool was not trained.

2. Once the current tool is trained, the checkpoint of parameter rollback is set to the values in Tool Parameters of the last training session. At this point, if you change any value in Tool Parameters, the button is enabled.

3. Click Restore Parameter button and it reverts the changed value to the value of the checkpoint.

4. If you train the current tool again, with some changes in Tool Parameters, then the checkpoint of parameter rollback is updated to the changed parameter values. Again, the button is disabled unless you make another change for the values in Tool Parameters.

5. If you make another change and click Restore Parameter button again, it reverts the changed value into the value of the updated checkpoint.

Note that if you re-process a trained tool after changing the values in Processing Parameters, the checkpoint of parameter rollback is not updated, and thus Restore Parameter remains enabled. The checkpoint is updated only after the training of a tool is completed.

Disabled

Enabled

 

Control Neural Network Training

The training of Green Classify High Detail Quick can be controlled by configuring the tool parameters and the training set.

 

Training Set

The largest single determinant affecting the network training phase is the composition of the Training Set. The best method for controlling the network training phase is to construct a proper training set for your tool. In this way, you can separate images/views into categories that allow you to determine if your tool is generalizing your images/views properly.