Whether it’s for health, fitness, daily living, or performance training, today’s biometric wearables are becoming a bigger part of our everyday lives. But as the demand continues to grow, there is even a greater importance placed on providing accurate data through sensor technology to prevent garbage-in, garbage-out products from saturating the market.
Establishing benchmarks, clinical validation of wearable sensors, and applying accurate biometrics to enhance user experience are all ways to ensure the production of the highest quality products and keep sensor technology moving forward.
In order to understand how our clinical testing and validation occurs, let’s take a quick look at Valencell’s sensors and how the laboratory operates to ensure accuracy of its products.
The systems approach Valencell uses takes three forms: Benchmark, licensee, and patent. The Benchmark sensor system is our own optical sensors that are used in our ear and wrist wearables. Incorporated with a Benchmark sensor are algorithms to interpret data such as heart rate, VO2, blood pressure, and energy expenditure. These sensors can be purchased by our partners, and the product can then be built according to their device’s specifications. Licensing arrangements can also be made with our partners allowing Valencell sensor patents to be used to develop their own sensors.
When it comes to optomechanics, firmware, algorithms, and the testing and validation process, this work is done in our in-house biometrics laboratory. Each week there are close to 90 trials run. Of those 90, there are 60 hour-long session trials that are centered around validation of our products. This may include new sensor designs or even our partner designs. In addition to these, there are another 30 trials done on longitudinal data that can include training studies that are monitored over a-period-of-time. These are completed on a wide-range of athletes including those with activities levels at each extreme end of the spectrum. Resting trials that are done include those associated with blood pressure, stress monitoring, and sleep studies.
To complete all of this testing, there are five basic positions within our laboratory staff. These include the director, the operations lead, the data lead, the reports lead, and the trials specialist. In addition to these staff members, we also have four to seven individuals monitoring the data collection and ensuring that everything within the lab is running smoothly.
The Importance of Data Accuracy
One key to the function of a successful laboratory is the ability to interpret when sensors are reporting accurately and when the results might be slightly skewed. With all of Valencell’s sensor systems, partners are provided with figures of merit. One of those figures of merit is the quality of signal. This allows for determination if-and-when a signal isn’t great and the likelihood that the data being reported is accurate or not.
This is important because a slight reading error can produce data that is very close to data that is measured accurately, and if overlooked that inaccuracy has the potential to cause problems in the user experience. In an industry that has lacked for many years a clear method of benchmarking sensor accuracy, Valencell takes its testing and accuracy of all sensor technology very seriously.
By holding products to the highest standards, it helps to promote the overall reputation of the industry and ensure it will not be damaged by allowing poor quality sensors to flood the market and erode pricing. This industry awareness of what constitutes a good fitness tracker is what will ultimately prevent the biometric wearable industry from becoming a low-quality, low-value industry.
The good news is, the efforts of the Consumer Technology Association (CTA) has developed a standards committee that aims to monitor and standardize accuracy in health and fitness electronics. One of these standards is CTA-2065, which is physical activity monitoring for heart rate. This standard used at Valencell can be downloaded from the CTA website, and in that standard you will see the importance placed on the number of participants needed for clinical validations, types of participants, the five different protocols needed, and data performance for things like output rate, availability, accuracy, and offset.
In addition to this standard the CTA also has standards for step counting in fitness wearables, sleep and stress monitoring, activity intensity, and mobile health applications, among others.
Biometric Wearable Sensors & Validation
Because there’s such a variety of activities that consumers partake in, there needs to be a substantial form-factor diversity in biometric wearables that use PPG. Placing sensors in helmets for cyclists, in jewelry for lifestyle activities, or in earbuds for general fitness are all considerations that must be made to provide the best overall user experience. Where the sensor is placed and what data you hope to get out of the sensor are also primary factors that need to be determined depending on the use case to provide the best overall experience and accuracy.
For example, if a user is going to be completing upper-body activities like weight training or pullups, a wrist wearable might not be the best choice because of all the changes in blood flow. In this instance, it makes to develop an earbud wearable because the head will be stable, blood flow will be consistent, and interference will be minimal.
In the lab at Valencell, lots of raw data is collected so that algorithms can be run on various metrics like heart rate, steps, and blood pressure. When collecting this data, baseline devices like chest strap HRMs, ECG, video, manual and automatic blood pressures, measured course, and calibrated treadmills are also used.
Other data collecting tools used in all of our validation testing include:
• A wide range of participants (diversity, fitness level, etc.)
• Experienced staff members
• Data collection for commercial and custom applications, including the PerformTek Data Collector app made by Valencell
• Exercise equipment
• Reference and baseline devices
• Collection devices like phones, cameras, programs, and computers
The process for data collection involves multiple areas of consideration as well. Optical noise, skin tone, sensor location, blood perfusion, and crossover problems are all factors that should be taken-into-account when developing a sensor for a wearable. Field trials are conducted to account for the variety of activities of our users, and include sunlight testing outside the lab, resting heart rate tests, high-intensity activities like running intervals, strength circuits, and water testing done at the pool. Statistical analysis is then done to determine bias as a percent error and other correlations as needed to determine accuracy of the results.
Developing use cases that the individual can benefit from and that are simple to follow and can be supported by research are important. Here are a few examples:
• Qualification for activity intensity: The American Heart Association recommends 150 minutes of moderate exercise and 75 minutes of vigorous exercise each week for cardiovascular health. Intensity can be quantified by heart rate data, accelerometer, rating of perceived exertion. This allows to go beyond the counting “10,000 steps”.
• Overall stress status: Variation in heart rate variability (HRV) from day-to-day or week-to-week allows for tracking of overall stress that can be taken from several sources.
• Estimating cardiac age: This can be done by heart rate variability inputs and outputs. Researchers have developed a general empirical index of cardiac age by observing significant correlations between HRV descriptors and age.
• Estimation of human core temperature: Biometric data is accurate enough to provide indication of thermal work. Research demonstrates correlation between temperature and ability for work output.
As with all use cases, accuracy is critical for a rewarding user experience. Validation takes planning, experience, and intuition from a team of experts to achieve the highest standard of biometric wearables. For more information on this topic, check out this presentation from our Biometrics Lab Director, Dr. Chris Eschbach, below and if you have any questions, please contact us at firstname.lastname@example.org