System configuration for GPU operation
If you want to run AI models on a GPU in an accelerated manner, the following requirements and configurations must be observed.
Requirements
Use a Beckhoff image for IPCs with an ex factory installed GPU to ensure that all necessary requirements for the operation of the TwinCAT Machine Learning Server are already fulfilled.
If you are operating a third-party PC, your own image or a Beckhoff IPC with a third-party GPU, please refer to the section Setting up an NVIDIA graphics card.
Configuration of the GPU operation mode
You can put NVIDIA® GPUs into so-called TCC mode and thus reserve all of the GPU's computing resources for computing tasks such as AI model inference.
What is the TCC mode?
The NVIDIA® TCC mode (Tesla Compute Cluster) is a special operation mode for NVIDIA® GPUs in which the graphics output is disabled in order to optimize the GPU exclusively for computing tasks.
Note: Many CPUs are equipped with an integrated GPU (iGPU) so that a connected monitor can still be used despite TCC mode being enabled. This applies, for example, to the IPCs of the C6043 series.
Please note that TCC mode is not available for all GPUs. You can find information on the compatibility of your GPU directly from NVIDIA®. All GPUs offered by Beckhoff support the TCC mode.What are the advantages of the TCC mode?
- The entire memory and all computing units are available for AI calculations
- No loss of time due to competing tasks such as rendering or display interrupts.
- Improved response times and reduced jitter
- More usable vRAM (especially important for large models or several models on one GPU)
How do I set the TCC mode?
Open the Windows Command Prompt with administrator rights. Enable the TCC mode with the command
nvidia-smi -g <GPU-Index> -dm 1-g stands for the GPU ID (e.g. 0, 1, 2 …)
-dm 1 means switch on TCC mode
Examples
Enable TCC mode for the first GPU
nvidia-smi -g 0 -dm 1Enable TCC mode for all compatible GPUs
nvidia-smi -dm 1A restart of the system is recommended.