Valencell evaluates a lot of wearables in our Biometrics Lab. Last year we completed over 36,000 device tests measuring over 2,000 hours of data collection and validation on biometric wearables. Almost all of this testing uses an electrode-based (ECG) chest strap to validate the performance of the device or devices we’re testing. Most people consider ECG chest straps to be the gold standard for testing biometric wearable devices and for good reason. They are very accurate for measuring heart rate and R-R interval. However, there’s a misconception in the market that needs to be corrected:
The misconception is that ECG chest straps work all of the time. The reality is they are not 100% accurate. Failures of tracking metrics occur just like any other biometric wearable device.
With over a decade of evaluating all kinds of evaluating these devices, we have learned that the amount of mistracking in ECG chest straps is not an insignificant number. To provide a baseline for these devices we set out to quantify their accuracy by measuring two chest straps, the Bluetooth low energy Polar H7 (BLECS) and Polar CX800 watch and chest strap (CSHRM), on the same subjects determine the frequency of data points within +/-5% of each other (data measured once per second).
Figure 1
Each participant completed the standard “Valencell Test”, a single, dynamic, 8-minute treadmill session consisting of standing, walking and running in order to elicit a variety of exercise intensities. Here are the details of the protocol:
Results
It turns out that the two chest straps agree with each other within +/-5% across 92% of the data points. Here are the detailed results:
So even the gold standard has a bad test every now and then, just like every other device. The internal processes of the human body are difficult to measure, particularly during movement and exercise.
If you’re wondering about data you may have seen that shows chest straps are 99-100% accurate compared to an ECG, those data points are likely NOT including data and/or tests where there are obvious errors in the data. In data sets where the chest strap is obviously not tracking to the trained eye one can clearly see when a chest strap is providing inaccurate data. This typically comes in the form of highly erratic data, which are physiologically unlikely in a healthy individual during activity, including highly variable heart rate “jitter” or reported heart rate outside of expected ranges (for example, reported at 0 or greater than 200).
In fact, when we removed the chest strap data that was clearly erroneous from our analysis we obtained results that were 98% accurate:
So what does all this mean?
Nothing is perfect. When looking at the accuracy of heart rate monitoring devices, remember that even the “gold standard” has errors. When you are comparing any device measuring heart rate to a chest strap, 92-94% of the data within +/-5% is the BEST you can expect from that device during activity. If you see results that are higher than that, just understand that the evaluators are very likely not including data where the chest strap is not reading correctly. Remember, if the data looks too good to be true, it probably is.
If you are interested in seeing the details behind this research in a white paper, reach out at info@valencell.com and we’d be happy to send you a copy.