Calculation of the measured value

The measured value is calculated in the following steps between reading the ADC and outputting the measured value:

Offset correction

Formula: Calculation of the measured value 1:

Otherwise: Calculation of the measured value 2:

Name	CoE Index	Meaning	Unit
Raw Value X_ADC	0x800E:01 0x9000:01^*)	Output value of the A/D converter	Range: 0..262,144 (18 bits) + sign in the respective active measuring range Unit: [Digits / V_FSV] or [Digits / A_FSV] Example: Instantaneous value = 2¹⁸ / FSV [V or A] * electrical value
B_Z	0x800E:02 0x9000:02^*)	Current ADC Zero value
B_ZCal	0x800F:20 0x9000:03^*)	ADC Zero at the time of calibration
Y_Z	-	Measured value after ADC offset correction

Note B_Z: Contains the measured value of the last ADC conversion with Zero Compensation or (if Zero Compensation Interval = 0) the value from 0x800F:1F.

Filtering

Formula: Calculation of the measured value 3:

Otherwise: Calculation of the measured value 4:

Name	CoE Index	Meaning	Unit
Enable Filter	0x8000:04		-
Y_F	-	Measured value after averaging	-

50/60 Hz filtering

Formula at 60 Hz: Calculation of the measured value 5:

Formula at 50 Hz: Calculation of the measured value 6:

Name	CoE Index	Meaning	Unit
Frequency	0x8000:0A	50/60Hz interference suppression	-
B_F	0x800F:1D	60 Hz offset	Range: 0..262,144 (18 bits) + sign in the respective active measuring range Unit: [Digits / V_FSV] or [Digits / A_FSV] Example: Instantaneous value = 2¹⁸ / FSV [V or A] * electrical value
A_F	0x800F:1E	60 Hz gain
Y₆₀	-	Measured value after adaptation of the 60 Hz integration time	-

Vendor calibration

The previously determined measured value "raw value" is now offset by the vendor calibration values by default so that the channel meets the specified value.

Formula: Calculation of the measured value 7:

Otherwise: Calculation of the measured value 8:

Name	CoE Index	Meaning	Unit
Enable Vendor calibration	0x8000:01	Enables SAI Vendor data 0x800F; enabled by default	-
B_K	0x800F (odd subindices)	Vendor Data: Offset	Range: 0..262,144 (18 bits) + sign in the respective active measuring range Unit: [Digits / V_FSV] or [Digits / A_FSV] Example: Instantaneous value = 2¹⁸ / FSV [V or A] * electrical value
A_K	0x800F (even subindices)	Vendor Data: Gain
Y_H	-	Measured value after vendor calibration	-

User calibration

If necessary, the measured value can be changed on the application side.

The terminal has one user calibration value pair for each measuring range and measuring mode, i.e. 14 pairs in total. To increase the resolution of the gain value, the "+1" (0x4000) is omitted.
As gain values smaller than 1 can also occur, the gain value is provided with an offset of 1/8. For a gain of 1 the gain value must be 0x2000. For gain G = 1 the following applies:

Calculation of the measured value 9:

For other values, "G" must be replaced by the desired gain factor.
Before calibration, the presentation of the values should be set to "right-aligned": CoE index 0x8000:1D (Presentation = 1).

Formula: Calculation of the measured value 10:

Otherwise: Calculation of the measured value 11:

Name	CoE Index	Meaning	Unit
Enable User calibration	0x8000:02	Enables SAI vendor data 0x8002; default disabled	-
B_A	0x8002 (odd subindices)	User Data: Offset	Range: 0..262,144 (18 bits) + sign in the respective active measuring range Unit: [Digits / V_FSV] or [Digits / A_FSV] Example: Instantaneous value = 2¹⁸ / FSV [V or A] * electrical value
A_A	0x8002 (even subindices)	User Data: Gain
Y_A	-	Measured value after user calibration	-

AC system offset

This internal calculation is mandatory in the data stream and cannot be changed or switched off. This is used for linearization with small AC values.

Formela: Calculation of the measured value 12: for

If condition is not met: Calculation of the measured value 14:

Name	CoE Index	Meaning	Unit
X_SO	0x800F:1F	AC system offset	-
Y	-	Measured value after AC system offset	Range: 0..262,144 (18 bits) + sign in the respective active measuring range Unit: [Digits / V_FSV] or [Digits / A_FSV] Example: Instantaneous value = 2¹⁸ / FSV [V or A] * electrical value

Name

CoE Index

Meaning

Unit

X_SO

0x800F:1F

AC system offset

Measured value after AC system offset

Range: 0..262,144 (18 bits) + sign in the respective active measuring range

Unit:

[Digits / V_FSV] or

[Digits / A_FSV]

Example:

Instantaneous value = 2¹⁸ / FSV [V or A] * electrical value

Scaling

During scaling, the previously calculated measured value is converted to the desired output format. The scaling contains the powers of ten selected by the range.
All gain factors must therefore correspond to 1/8 to obtain a total factor of 1. The value 8192 (0x2000) results in a total factor of 1

Formula: Calculation of the measured value 15:

If condition is not met: Calculation of the measured value 16:

Name	CoE Index	Meaning	Unit
Presentation	0x8000:1D	F_Scal	-
Y_Scale	0x6000:11	Measured value after scaling	Presentation e.g. in scaled: 1 bit per 1 µV / 1 µA

Name

CoE Index

Meaning

Unit

Presentation

0x8000:1D

F_Scal

Y_Scale

0x6000:11

Measured value after scaling

Presentation e.g. in scaled:

1 bit per 1 µV / 1 µA

*) from Firmware 06 / Revision -0021