Mapping of the runtime function

This documentation describes how to read data from the Analytics format using the Analytics Data Exchange API.

Installation

The following two components must be installed to use the Analytics Data Exchange API:

  1. Microsoft Visual C++ Redistributable 2019 (x64/ x86) or higher. If this package is missing, errors occur when processing symbols.
  2. TwinCAT 3 ADS

NuGet packages

The Analytics Data Exchange API is delivered as a NuGet package. Three different packages are available:

  1. Beckhoff.TwinCAT.Analytics.DataExchange.Core (.Net-Core 3.1 or .Net-5 and higher)
  2. Beckhoff.TwinCAT.Analytics.DataExchange.Framework (.Net-Framework 4.5.2 or higher)
  3. Beckhoff.TwinCAT.Analytics.DataExchange (multi-target projects)

License

Two licenses are needed to read data from the Analytics format:

  1. "TwinCAT 3 Analytics Runtime" or "TwinCAT 3 Analytics Runtime Base"
  2. "TwinCAT 3 Analytics Control Device Pack" for each parallel read operation.

TwinCAT must be in "RUN" mode in order to use the runtime functionality.

Create and configure a data source:

The first step is to create and configure a Builder in order to read data from the Analytics format.

ISourceFactory factory = new SourceFactory();

IFileSourceBuilder fileBuilder = factory.FromFile(„C:\Data\Demo“);

IMqttLiveSourceBuilder mqttBuilder = factory.FromLiveStream(„127.0.0.1“, 1883);

IStorageProviderSourceBuilder FromStorageProvider(„127.0.0.1“, 1883);

There are currently three different Builders available:

  1. IFileSourceBuilder: data from a file
  2. IMqttLiveSourceBuilder: data from an Analytics Logger data stream
  3. IStorageProviderSourceBuilder: recorded data of an Analytics Storage Provider

The Builder is parameterized via a concatenation of method calls. The following options are available:

Symbols

Description

WithSymbol(…)

Adds an additional symbol for reading. Available symbols can be read via the TwinCAT Target Browser.

WithSymbols(...)

Allows you to add several symbols at the same time.

WithSymbols(pattern, …)

Adds multiple symbols using a regex expression. Additionally, RegexOptions and a SymbolFilter can be specified.

SymbolFilter:

Include Parent: uses the parent symbol even if child symbols apply

IncludeChilds: uses the child symbols even if the parent symbol applies.

IncludeChilds or InlcudeParent must be set

WithSymbol<T>()

Adds a symbol of the generic data type. The section Generic data types contains information about the allowed data types.

 

 

MQTT only:

 

FromTopic(…)

MQTT topic from which the data should be read.

WithStreamFormat(…)

Defines the format of the received data. If nothing is specified, the format will be detected automatically. For automatic detection, the data stream must be online at startup.

Storage Provider and file:

 

From(…)

Start of the data retrieval process.

To(…)

End of the data retrieval process.

StorageProvider only

 

FromStorageProivder(…)

Specifies the storage provider at the broker used to retrieve the data.

FromHistoricalStream(…)

The ProviderId and the MainTopic are read from the specified topic, which otherwise must be passed via the FromStorageProvider method.

FromLayout(…)

Layout of the historical data stream.

FromOriginSystem(…)

SystemId from which the data was originally sent.

FromOriginStreamTopic (…)

Topic via which the data was originally sent.

FromStreamAlias(…)

Alias under which the data stream is recorded and displayed in the TargetBrowser.

The methods FromLayout, FromOriginSystem, FromOriginStreamTopic, FromStreamAlias, are used to identify the correct data set. It is not mandatory to parameterize all of them, but it is recommended to specify as many as possible to avoid confusion of data sets. The easiest way to read this information is to use Target Browser. To do this, drag a symbol of the data set into a text editor (e.g. Notepad ++). You can find the necessary information in the Target Info section in the XML.

Special parameters for MQTT communication are configured using the following methods:

WithTLS(…)

Configures the certificates to use TLS

WithCredentials(...)

User name and password for access to the message broker

WithTcpBufferSize(…)

Sets the size of the buffer for the TCP connection to the message broker. The default is 32 kB. If very large messages are sent, this value can be increased.

WithKeepAlivePeriod(…)

Set the KeepAlive time for the MQTT connection. (the default is 60 s)

WithCommunicationTimeout(…)

Timeout for TCP communication to the message broker. (the default is 3 s)

ToTopic(…) / FromTopic(…)

Topic over which the data is communicated.

The configuration of a data source is completed by calling the build method, which returns the configured data source. The data source provides various methods. If a data source is no longer used, it is necessary to call the Dispose method to release all resources.

InitSymbols()

Initializes the ReadSymbols collection and resolves existing regex expressions.

ReadSamples(...)

Queries all data synchronously in the calling context.

ReadAsync(...)

Queries all data asynchronously. The ReadDelegate is called for each date.

Dispose()

Closes all open MQTT connections or file accesses and releases all resources. The data source can no longer be used after that.

Generic data types:

The Analytics Data Exchange API supports only "unmanaged structs" as generic data types. This results in the following restrictions

  1. Only structures or basic data types may be used within the structures.
  2. Arrays within structures are only possible with "unsafe" code. (Fixed-Array-Size)
  3. Only arrays of basic data types can be used. Arrays of structures are not possible.

Furthermore, only a sequential or explicit layout can be used for the structures (see https://docs.microsoft.com/de-de/dotnet/api/system.runtime.interopservices.structlayoutattribute?view=net-6.0)

  1. An explicit layout should be used when writing data.
  2. The layout must not contain padding (Pack = 1) when writing data
  3. The padding must correspond to the data source when reading data (https://infosys.beckhoff.com/index.php?content=../content/1033/tc3_plc_intro/3539428491.html&id=)

Structured data types:

The Analytics Data Exchange API automatically detects the data types of symbols if no generic data type is specified. If the symbol used is a Structured Data Type, an IStructuredValue is returned. The values of the subsymbols can be queried from the IStructuredValue on the one hand. This can be converted directly into a JSON text on the other hand. The last option is to create an IStructedValueReader to parse through the entire symbol similar to an XMLReader.

void Read(ISample sample)
{
  foreach(var value in sample.Values)
  {
    if(value is IStructuredValue stValue)
    {
      //Values of structured DataTypes without generic DataType could be proccessed here
      var subValue = value[„SubValues1“]; //SubValues1 is a known SubElement of the structured Value

      var jsonString = stValue.ToJsonString(); //Convert the value to JSON

      IStructuredValueReader reader stValue.GetReader(); //Use reader to parse the value iterative
    }
  }
}

Error messages

If errors occur while using the Data Exchange API, they are passed on to the user by exception. The following exceptions are used:

Class name

Superclass

Description

DataExchangeException

Exception

Superclass for all further exceptions within the API.

InvalidSymbolDefinitionException

DataExchangeException

The configured symbols are not correct.

SymbolNotFoundException

InvalidSymbolDefinitionException

A configured symbol could not be found in the data stream.

InvalidSymbolNameException

InvalidSymbolDefinitionException

The entered symbol name is not valid.

InvalidSymbolOffsetException

InvalidSymbolDefinitionException

The entered symbol offset is smaller than 0.

InvalidSymbolByteSizeException

InvalidSymbolDefinitionException

The entered size in bytes is less than or equal to 0.

InvalidSymbolTypeCodeException

InvalidSymbolDefinitionException

The TypeCode entered for a symbol is not valid.

InvalidSymbolRegexPatternException

InvalidSymbolDefinitionException

The entered regex pattern for a symbol is not valid.

InvalidSymbolFilterException

InvalidSymbolDefinitionException

The entered SymbolFilter for a regex pattern is not valid.

DataExchangeLicenseException

DataExchangeException

No valid license for the API could be found or TwinCAT is not in run mode.

DataExchangeTimeoutException

DataExchangeException

A time-out period has been exceeded.

QueueFullException

DataExchangeException

A queue for processing data is full because the processing was too slow.

FileSourceNotFoundException

DataExchangeExeption

No Analytics file could be found in the specified folder.

ValueExtractException

DataExchangeException

An error occurred while reading the values from the data stream. More detailed information is available in the inner exception.

DataExchangeMQTTException

DataExchangeException

An error has occurred with the MQTT connection.

BrokerConnectException

DataExchangeMQTTException

A connection to the broker could not be established.

TopicNotFoundException

DataExchangeMQTTException

The specified topic was not found at the broker.

InvalidMqttStreamFormat

DataExchangeMQTTException

The specified stream format is not correct.

Examples:

Samples of how to use the Analytics Data Exchange API can be found here:

AnalyticsDataExchange_Samples.zip