Filter Grammar

The grammar components of filter expressions within VisionPro Deep Learning are comprised of the following:

Literal values
Built-in variables
Operators
Functions
Regular Expressions

Filter Literal Values

The VisionPro Deep Learning filter grammar supports string and floating point value types. These literal values must be entered, as shown below:

Type	Example	Notes
string	filename = 'RUN00031.bmp' sets ~ 'PlantA'	Strings must be enclosed in single quotation marks, and escaping is not supported.
floating point	feature [score > 0.750] count(feature) = 10	A decimal point is not required.

Type

Example

Notes

string

filename = 'RUN00031.bmp'

sets ~ 'PlantA'

Strings must be enclosed in single quotation marks, and escaping is not supported.

floating point

feature [score > 0.750]

count(feature) = 10

A decimal point is not required.

Filter Variables

The filter grammar includes variables that are defined for views. You use these simply by including them in an expression that evaluates to a Boolean result, such as the following:

width > 1500// This filter returns views with a width greater than 1500.

View-level variables can be one of the following:

Generic, in which case they are available in all views.
Tool-specific, in which case they are only available for specific tool types.

For example, the trained variable is available for any view, while the precision variable is only available for Red Analyze or Blue Locate tool views.

In addition to view-level variables, the grammar also supports tool-specific, axis-level variables that you can use to filter views based on the presence, absence, or values of individual markings, labels, and other view elements. You use axis variables by including them in square brackets, following the name of the axis.

For example, the Blue Locate tool has two axes: feature (for found features) and match (for matched models). If you want to filter views that contain specific features, you would include feature, followed by square brackets containing the axis filter:

feature[x > 10 and x < 900]// This filter returns views containing a feature with an x-position between 10 and 900.

In this case, feature is the axis, and x and y are the axis-level variables.

Note:

For the region and filter axis in the Blue Locate tool, append an exclamation mark ('!') to the axis name to match labels instead of features:

feature![x > 10 and x < 900]// This filter returns views containing a label with an x-position between 10 and 900.

View-level and axis-level variables can be combined in a single expression:

trained and feature[x > 10 and x < 900]// This filter returns trained views containing a feature with an x-position between 10 and 900.

You can also use a path-delimiter ('/') to refer to axis variables in an expression.

feature/x > 10 and feature/y < 900// This filter returns trained views containing at least one feature with an x-position greater than 10, and at least one feature with a y-position less than 900.

Generic View-Level Variables

These variables are used with any view, regardless of the Deep Learning tool.

View-level Variable	Data Type	Description
new	Boolean	Returns all views that were added since the last training.
anomalous	Boolean	Returns samples where the image plane characteristics do not match the tool configuration.
all	Boolean	Returns all the views in the database.
processed	Boolean	Returns views with markings that were generated since the last training.
masked	Boolean	Returns all views that contain a mask.
invalid	Boolean	Returns all views containing invalid labels, such as empty Blue Locate tool features, or overlapping regions in a Red Analyze tool.
trained	Boolean	Returns all views that were included in the training set.
labeled	Boolean	Returns all views that contain a label.
sample_name	String	The name of sample, which may be different if multi-channel samples are being used.
filename	String	Returns the view with that filename.
added	String	Returns the date when the image was added to the workspace.
duration	Floating Point	Returns the last processing time for the view.
index	Floating Point	Returns an index of views in the enumeration of all views, from 0 to N-1. This is useful when you want to select a specific range of views.
view_index	Floating Point
random	Floating Point	Returns a random value. This is useful to shuffle views (when used to sort views) or to select a random set of views. For example, 'random < 0.5' would select half of the views.
width	Floating Point	Returns views that match the specified width value.
height	Floating Point	Returns views that match the specified height value.
difficulty	Floating Point	Returns views that match a unit-less measure of the degree of difficulty to train a view.
sets	String [ ]	Returns all of the views that are included in the specified set. This variable is unique, in that a vector of strings are specified in order to return a result. The following grammar summarizes the semantics of the equality, inequality, and contains operators ('~') when applied to the sets variable: sets = '<set_name>'// This returns true and the associated views if at least one element of a sets is identical to set_name. sets != '<set_name>'// This returns true and the associated views if none of the elements of sets is identical to set_name. sets ~ '<set_name>'// Returns true and the associated views if at least one element of a sets contains the string set_name. Note: You can use the contains operator ('~') to perform partial matching. For example, this syntax can be used: sets ~ "Plant17"// This returns true for views that are members of Plant17, such as "Plant17-LineA" or "Plant17-LineB", but not "Plant18-LineA".

Tool-Specific View-Level Variables

These variables are tied to specific VisionPro Deep Learning tools, along with specific components of those tools. The variables below can be used to filter the result of processing.

Blue Locate and Blue Read Tool View Variables	Data Type	Notes
mismatch	Boolean
recall	Floating Point
precision
f_score
best_match_string	String
Blue Locate and Blue Read Tool - Feature Axis Variables	Data Type	Notes
covered	Boolean
matched	Boolean
score	Floating Point
angle
size
aspect_ratio
width
height
x
y
angle_dist
size_dist
aspect_ratio_dist
dist
x_dist
y_dist
id	String
covered_id	String
Blue Locate and Blue Read Tool - Match Axis Variables	Data Type	Notes
covered	Boolean
score	Floating Point
angle
scale
aspect_ratio
shear
x
y
features
name	String
feature_string	String

Green Classify Tool View Variables	Data Type	Notes
threshold	Floating Point
score
unknown		Denotes an image that is labeled as an unknown (in other words, untrained) class.
mismatch
best_tag	String
validated		Returns only the views that are included in the validation set of Green Classify High Detail.
not validated		Returns only the views that are not included in the validation set of Green Classify High Detail.
Green Classify Tool - Tag Axis Variables	Data Type	Notes
score	Floating Point
name	String

Red Analyze Tool View Variables	Data Type	Notes
good	Boolean	Returns views that are labeled as good. Example: "good and score<=0.2"
negative		Returns the same results as good.
bad		Returns views that are labeled as bad. Example: "bad and score>=0.8"
positive		Returns the same results as bad.
mismatch
recall	Floating Point	Returns the recall as the processing result. Example: "recall >= 0.7"
precision		Returns the precision as the processing result. Example: "precision>= 0.7"
f_score		Returns the f-score as the processing result. Example: "f-score>= 0.75"
T1		Returns the lower threshold value. Example: "T1>= 0.4"
T2		Returns the upper threshold value. Example: "T2>= 0.6"
score		Returns the highest defect probability. Example: "score>= 0.5"
deviation		Returns the same results as score.
validated		Returns only the views that are included in the validation set of Red Analyze High Detail.
not validated		Returns only the views that are not included in the validation set of Red Analyze High Detail.

Filter Grammar Operators

The grammar operators are used to construct expressions that can be evaluated to true or false (for filtering) or to an integer or floating point value (for sorting) or to a vector (for counting).

Note: There are some specific constraints and behaviors associated with the sets variable, which is unique because it is a vector of strings (in other words, all of the sets of which the view is a member).

Operator	Notes	When used with sets
~	String comparison: True if LHS contains RHS. Use 'string' as shorthand for filename ~ 'string'	Returns a second vector of strings in LHS that contain RHS. The returned vector can be evaluated as a Boolean (true if at least one string) or used as a new vector of strings.
in	Floating point within range: True if LHS is within the range specified by RHS.	N/A
and
or
not
<	Natural sort for strings.	Natural sort for strings.
<=
>
>=
=		True if the vector of strings contains at least one element that exactly matches the supplied string. Cannot be used to compare two vectors for equality.
!=		True if at least one element of the vector does not exactly match the supplied string. This is not the same as not(sets = 'string')
( )	Precedence

Filter Grammar Functions

When used without qualification, filter axes (feature, match, region, tag) return lists of items. These lists cannot be tested directly, but they can be supplied to the count() function, the result of which can be tested.

In addition, vectors of string (for example sets) can also be supplied to count(), but not to the other functions.

Additionally, when any axis floating point variable is provided using a path separator instead of a part of an axis filter, it will return a list of values which cannot be tested directly, but can be supplied directly to the min() and max() functions, and those results can be tested.

For example:

count (feature) > 10// Returns views that have more than ten features.
min (feature/score) < 0.95// Returns views where the minimum score for any feature is less than 0.95.
count (sets) > 1// Returns views that are members of more than one set.

Operator	Notes
count()
min()
max()
sum()
avg()
std()

Regular Expressions in Filters

You can apply a regular expression to any filter variable that is a String data type.

You can return whether a regex matches: /<regex>/.test(<variable>)

/DF\d\d\d\d/.test(sample_name)// Returns true if the view's sample name matches the regex DF\d\d\d\d.

You can also use a capture string and compare the captured string to another string: /<regex(capture)>/.extract(<variable>), which returns the captured string.

/DF(\d\d\d\d)/.extract(sample_name) = '1234'// Captures the four digits from the regex and tests against '1234'.

Note:

The examples above use the view-level variable sample_name, which evaluates to a String. If you attempt to use a String data type axis variable, those variables may evaluate to a vector of strings, since there may be multiple instances of the axis variable within a single view. In this case, the regex must be applied within the axis filter, as shown below:

match[/FH\d\d\d\d/.test(feature_string)]// For this expression, the regex is applied for each match's feature_string. If any of the matches' feature_string match the regex, the view will be included.

The following is an example of an invalid regex filter expression:

/FH\d\d\d\d/.test (match/feature_string)// The expression should return true if the view includes a match with a feature_string matching the regex. However, this is invalid, because each view may have multiple matches.
You cannot use the sets variable in a regular expression.

How to Write Filter Expressions

Filter expressions can include view-level filters where the variable refers to the entire view.

filename ~ 'REJECT'// Returns views where filename contains 'REJECT'.
count(feature) > 3// Returns views with more than three features.

Filter expressions can also include axis filters, where the individual marking or labeling for each view is evaluated.

feature [score > 0.5 ]// Returns views with a feature with score greater than 0.5.

You can combine view-level and axis-level filters.

filename ~ 'REJECT' and feature [score > 0.5 ]// Returns views where filename contains 'REJECT' and a feature scoring greater than 0.5.

You can combine filters for multiple axes.

feature [score > 0.5 ] and match [ score > 0.5 ]// Returns views with a feature with score greater than 0.5 and a match with a score greater than 0.5.

You can specify multiple filter conditions for an axis.

feature[score > 0.5 and matched ]// Returns views with a feature with a score greater than 0.5 and the feature is in a matched model.