Latency threshold

The Latency threshold parameter can be used to define the maximum allowable inference latency of the AI model.

This allows the model training to be specifically tailored to the requirements of the subsequent machine or process runtime.

The user specifies the target hardware, the target software, and the maximum allowed runtime for model execution. The TwinCAT Machine Learning Creator takes these constraints into account during model creation and optimizes the model accordingly in terms of runtime and model complexity.

This makes it possible to create AI models that meet both the desired model quality and the required time constraints.

Example:

During a quality control check, a product is recorded by a camera and then sorted out via a rejection station. There are only 10 ms available between image capture and reaching the ejection position. In this case, the inference—including preprocessing and postprocessing—must be completed within this timeframe.

The configured latency threshold allows the AI model to be adapted to this process requirement.

Balance between model quality and latency

The configured latency threshold influences the choice and optimization of the model architecture.

If no latency threshold is specified, there is no restriction on the subsequent inference runtime. This allows the optimization process to make a choice of more complex model architectures, which generally leads to better model metrics, such as accuracy, precision, or recall.

In the engineering process, it is therefore recommended to train initially without a latency threshold. This allows the data set to be iteratively improved and evaluated until a model that is convincing from a technical standpoint is achieved.

Only then should the latency threshold be reduced gradually. This allows us to verify whether sufficient model quality can still be achieved with lower model complexity and reduced inference latency.

This approach allows for a careful balance between model performance and the real-time capability of the eventual application.