Cyclical data and time series databases

This document describes the handling of time series and how cyclic data is stored in time series databases.

Database used: InfluxDB
Database type used: TimeSeriesDB

Introduction

Writing of data at regular or cyclic intervals is a common application in control technology. The data should be recorded with high time precision. Since database communication is not real-time capable, it is useful to store data that is measured regularly in a buffer. An array of the data structure can be used for this purpose. The collected data is then sent to the TwinCAT Database Server, where it can be processed without time constraints and subsequently stored in the database.

Time

Each data set that is stored in the database is assigned a timestamp. Together with the tag columns these form a unique ID. If two data sets have the same ID (same timestamp and tag values), the newer data set overwrites the old one.

Example:

 

time

locationname (tag)

temperature (field)

windspeed (field)

1

1581675200630326200

Verl

11.5

6.3

2

1581675200630327200

Verl

10.3

5.2

3

1581675200630328200

Verl

9.8

2.8

4

1581675200630328200

Hamburg

14.2

14.9

4

1581675200630328200

Hamburg

15.6

8.9

Cyclical data and time series databases 1:

Data set no. 4 is overwritten by a new data set, since the ID is identical.

The timestamp in the database is saved by default as UNIX epoch time. With the exception of the user-created insert commands, the timestamps are received and converted as TwinCAT time (number of 100 ns steps since January 1, 1601) in the function blocks of the TwinCAT 3 Database Server. The times are not converted for insert commands.

Database configuration

InfluxDB should be selected in the database configuration. Insert the connection parameters for the required database. If no database is available yet, you can create the database by clicking the "Create" button. Pay attention to your firewall settings or port approvals. A table does not have to be created, since it is automatically created during the first InfluxDB access. InfluxDB does not have a fixed table schema. Columns can also be extended or added later.

Writing cyclic data

This example shows how symbols can be written from the PLC into a time series database with minimal effort.

Declaration:

    State: E_DbLogState;
    bWriting: BOOL; // Set this bool fla to write the data once into the InfluxDB
    
    dbid: UDINT := 1; // Handle to the configured database
    
    QueryOption_TSDB_Insert : T_QueryOptionTimeSeriesDB_Insert; // defines detailed Queryparameter
    fbNoSQLQueryBuilder_TimeSeriesDB : FB_NoSQLQueryBuilder_TimeSeriesDB; // defines database type specific api
    fbNoSqlQueryEvt : FB_NoSQLQueryEvt(sNetID := '', tTimeout := T#15S); // functionblock to execute queries
    
    // databuffer for 1 second with 10 ms time delta
    windTurbineData: ARRAY[1..100] OF WindTurbineData;
    
    // error handling helper values
    TcResult: Tc3_Database.I_TcMessage;
    bError: BOOL;
    sErrorMessage: STRING(255);

    i: INT;
    rand : DRAND;
    nrand: LREAL;

Declaring the data source structure:

In this structure the attributes "TagName" and "FieldName" are used to declare the data fields as tags or fields. By default they are declared as fields. These attributes can also be used if you want the column name in the table to differ from the symbol name in the PLC.

Cyclical data and time series databases 2:

To capture unset data during data analysis, default values outside the value range can be used to detect such data during the analysis.

TYPE WindTurbineData :
STRUCT
    
    {attribute 'TagName' := 'ID'}
    WindTurbineID : STRING(255);
    
    {attribute 'FieldName' := 'Power'}
    Power : LREAL := -1; // [0) [kW]
    
    {attribute 'FieldName' := 'Wind Speed'}
    WindSpeed : LREAL := -1; // [0) [m/s]
    
    {attribute 'FieldName' := 'Wind Direction'}
    WindDirection : LREAL := -1; // [0,360][°]
    
END_STRUCT
END_TYPE
        

(WindTurbineData.tcDUT)

Declaration of the ENUM for the State-Machine:

TYPE E_DbLogState :
(
    idle := 0,
    init,
    writing,
    error
);
END_TYPE

Generating sample data:

        FOR i := 1 TO 100 BY 1 DO
             rand();
             nrand := rand.Num;
            windTurbineData[i].WindDirection := 240.328 + nrand;
            windTurbineData[i].WindSpeed := 7.3292 + nrand;
            windTurbineData[i].Power := 1133.1975 + nrand;
            windTurbineData[i].WindTurbineID := 'Wind Turbine Verl 13';
        END_FOR

CASE State OF
    
    E_DbLogState.idle:
        IF bWriting THEN
            bWriting := FALSE;
            State := E_DbLogState.init;
        END_IF

Preparing the call:

In this case, the array 'windTurbineData' is written into the 'WindMeasurement' table of the database. Therefore the data is read directly from the process image. The data type is specified to read the addresses in the memory. The time of the data sets is automatically generated by specifying the start time and the time interval. The data must be stored correctly in the array. For example, one data set in the array can be used per PLC cycle. It is useful to create a PLC task for this process. In this example the cycle time is 10 ms.

    E_DbLogState.init:
    
        fbNoSQLQueryBuilder_TimeSeriesDB.pQueryOptions := ADR(QueryOption_TSDB_Insert);
        fbNoSQLQueryBuilder_TimeSeriesDB.cbQueryOptions := SIZEOF(QueryOption_TSDB_Insert);    
        
        QueryOption_TSDB_Insert.sTableName := 'WindMeasurement';
        QueryOption_TSDB_Insert.sDataType := 'WindTurbineData';
        QueryOption_TSDB_Insert.pSymbol := ADR(windTurbineData);
        QueryOption_TSDB_Insert.cbSymbol := SIZEOF(windTurbineData);
        QueryOption_TSDB_Insert.nDataCount := 100;
        QueryOption_TSDB_Insert.nStartTimestamp := F_GetSystemTime();
        QueryOption_TSDB_Insert.nCycleTime := 10000; // (in 100 ns)
        State := E_DbLogState.writing;
        

Writing the data:

This call writes the data to the configured database with the corresponding database ID. This can take several cycles, since it is an asynchronous process. If necessary, several storage arrays must be used to ensure that the data is recorded seamlessly without gaps.

    E_DbLogState.writing:    
    
        IF fbNoSqlQueryEvt.Execute(dbid, fbNoSQLQueryBuilder_TimeSeriesDB) THEN
            IF fbNoSqlQueryEvt.bError THEN
                TcResult := fbNoSqlQueryEvt.ipTcResult;                
                State := E_DbLogState.error;
            ELSE
                State := E_DbLogState.idle;
            END_IF
        END_IF
        

Error handling:

Use the Tc3 Eventlogger for error handling

    E_DbLogState.error:
        
        IF TcResult.RequestEventText(1033, sErrorMessage, SIZEOF(sErrorMessage)) THEN
            TcResult.Send(F_GetSystemTime());
            State := E_DbLogState.idle;
            bError := TRUE;
        END_IF
        
        
END_CASE

Download: TF6420_BestPractise_TSDB_Cyclic.zip