Resolution
The following table shows the resolution for each measuring range.
Measuring range | Measured variable | Resolution |
|---|---|---|
-1 (0xFF/255) | 0 - 0.1 Ω | 11.9 nΩ/Digit |
0 | 100 mΩ - 1 Ω | 119 nΩ/Digit |
1 | 1 Ω - 10 Ω | 1.19 µΩ/Digit |
2 | 10 Ω - 100 Ω | 11.9 µΩ/Digit |
3 | 100 Ω - 1 kΩ | 119 µΩ/Digit |
4 | 1 kΩ - 10 kΩ | 1.19 mΩ/Digit |
5 | 10 kΩ - 100 kΩ | 11.9 mΩ/Digit |
6 | 100 kΩ - 1 MΩ | 119 mΩ/Digit |
7 | 1 MΩ - 10 MΩ | 1.19 Ω/Digit |