Wearable technology is the number one trend right now in fitness. More than 60% of Americans are interested in some sort of fitness tracker to help them get their step counts, monitor their health, and track the effectiveness of their training.
The device industry, and certainly their marketing departments, have taken note and seem to be endlessly rolling out new features and metrics in a bid to elevate their device above the rest.
As a result, while it took years to perfect heart rate monitors and power meters, the new product cycle seems to have gone from years for things like step counts and heart rate variability, to often months for the newest “must-have” metrics like oxygen saturation, hydration status, continuous glucose monitoring, sleep cycles, and recovery assessments.
All of these measures could prove to be highly valuable and revolutionize our training. They all deserve their time in the spotlight to be tested and proven. But that raises two important questions. First, have these metrics been scientifically validated, and second, even if they are valid, are they valuable? If you’re in the middle of a race, is knowing your core temperature really going to impact how you race?
Here to help us navigate this interesting and increasingly complex subject is renowned physiologist Dr. Stephen Cheung who, as an expert in extreme environments, has been asked to evaluate some of these new products. As a scientist his primary concerns relate to the validity and the reliability of these products.
In this show, we also hear from a host of experts, including Lindsay Golich, a coach at the US Olympic center, Dr. Inigo San Milan, physiologist and coach with pro cycling team UA Emirates, physiologist Dr. Stephen Seiler, and coach Grant Holicky.
So, start tracking on your favorite device, and let’s make you fast!
Episode Transcript
Rob Pickels 00:04
Hey listeners, this is Rob Pickels with Fast Talk. Before I get into the introduction for this episode, I do want to give a quick apology that I recorded this episode remotely and the sound quality left a little bit to be desired. My apologies for that, we’ll do better next time.
Rob Pickels 00:19
With that said, hello and welcome to Fast Talk, your source for the science of endurance performance. I’m your host Rob Pickels here with Coach Connor. Wearable technology is making its way into sports and onto our bodies. Looking at your fellow athletes shows most monitoring their step counts health and training metrics. The device industry and certainly their marketing departments have taken note, professional athletes are paid to prominently display their device deisure and companies race to release new features and create ecosystems that trap consumers. As a result, we are constantly inundated with the latest and greatest in wearable technology with hopes that our purchase improves our performance or experience while lightening our wallet. The question we’re answering today is whether there’s value in metrics like oxygen saturation, hydration status, glucose monitors, sleep cycles, core temperature and recovery assessment. Here to ask two important questions, are these devices valid and reliable is renowned physiologist Dr. Steven Chung, who as an expert in extreme environments has been integral in evaluating new sensors. As always, we’ll round out our conversation with additional experts like Coach Grant Holicky of Forever Endurance. Lindsey Gulledge of USOPC. Neil Henderson of Wahoo Fitness. Frank Overton of Fast Cat Coaching. Indigo San Milan of UAE, Team Emirates and physiologist Dr. Steven Siler. So now that you’ve pushed play, it’s time to push start on recording your data and let’s make you fast.
Rob Pickels 01:56
Pathways from Fast Talk Laboratories are a new way to explore concepts, master skills and solve training challenges. Our new cycling interval training pathway begins with the basics of interval workouts and progresses to more advanced details. How to flawlessly execute interval workouts, which intervals bring which adaptations and how to analyze your interval workout performance. Over 21 articles, interviews, workshops and workouts, our new cycling interval training pathway offers you the chance to master cycling’s most critical and nuanced workout format. See this pathway at fasttalklabs.com.
Trevor Connor 02:42
Well, Dr. Cheung, welcome to the show. Always a pleasure having you join us.
Dr. Stephen Cheung 02:45
It’s always great to be back and chatting science in sports.
Intro to Wearable Technology
Trevor Connor 02:49
This is going to be a fun one. We’re today going to talk about wearables, which I think is a really interesting conversation and we are talking about this right before we started the show. It’s an interesting world that you’re seeing more and more data more and more information being collected. I have one of those Phoenix watches and it’s remarkable the number of things it says it tracks and records for me, but the big question that we have hear that I think is what we’re going to dive into today is how good is that data? How good is this information that it is collecting and even more important than that a lot of these devices are creating algorithms on top of that data. So they might be tracking your heart rate variability, determine your sleep and then they do an assessment of your sleep and then based on that they give you a recovery score and tell you how you’re doing. So there’s all sorts of these algorithms on top of this original data and what we’re going to discuss is how good is that original data and if that data isn’t great, how much can you trust these algorithms that are being built on top of bad data? So that’s kind of the big overview of the conversation day, but we’re gonna go all sorts of places. We’re going to talk about continuous glucose monitoring, we’re going to talk about oxygen saturation, heart rate, you name it there’s a whole bunch that you can now get and put on your wrist.
How Good is the Data that is Collected?
Dr. Stephen Cheung 04:14
Yeah and it’s really neat kind of brave new world that we’re entering in terms of the amount of data that we can get as athletes and that we can get out in the field as opposed to being stuck in a lab tethered to a whole bunch of wires and being in a very controlled environment. It’s very analogous to in the old days, we have biker diameters and we could get power output in the lab and that was about it, but then with the introduction of power meters, that we were able to actually get live data outside in the field and real conditions and this is in some senses where the field is heading to with lots of other biometrics. So whether it’s core temperature, glucose and the hydration status, all of these things. So the potential is really there and we’re hopefully really going to talk about is it able to actually drive our everyday training and improve our performance?
Rob Pickels 05:12
Yeah, I think that we’re in a really unique time right now, right where technology, maybe for the past few years, but technology has really been rapidly expanding, taking off in our capabilities. I think that we’re seeing a lot of startup businesses that are looking to establish their place in the consumer electronics world and I know even for myself when I was back at Pearl Azumi, doing advanced development, I may or may not have been experimenting with some of these technologies, but before we go too far, I do want to say on my wrist, I have an old school automatic, analog watch that self winds and everything, but on the other wrist, I do have a Garmin Vivofit. So I’m covering all of my bases, right now.
Trevor Connor 05:57
Rob uses a watch to tell time, I just don’t get it. Why are you still there? Like using your phone to make phone calls? Why would you do that?
Rob Pickels 06:05
It’s true. I know. Well, that’s what my phone is. It’s the computer in my pocket. It’s not where I tell my time. Yeah.
Validity and Reliability
Trevor Connor 06:12
So we’ll put the references in the show notes on the page, but there was a very interesting 2022 review that I read and they cited a survey in 2019 that said that wearable technology is now the number one fitness trend in the world is what people are most interested in, but what I found particularly interesting, I’m just going to read out of the review here and then hand it off to you Dr. Cheung, as they said ‘to date studies investigating the validity and reliability are sparse, with wide disparity and findings’. So I think those two words validity and reliability are very important and Dr. Cheung, do you want to talk a little bit to why they are and what they mean?
Dr. Stephen Cheung 06:57
Sure, yeah. They’re absolutely critical concepts and I think a lot of times we just throw those words around without really understanding the meaning. So validity or accuracy is another synonym for it, that represents whether a monitor or something actually measures accurately the actual thing that you’re intending to and the analogy I sometimes give is, if you want to know the height of Everest. So accuracy is does it actually reflect the actual altitude of Everest? If you have a watch at the top, does it actually say 1848 meters, if you’re at the top of Everest or does it say 8000 meters? So if it says 8000, then it’s not accurate. So that’s accuracy or validity and then the flip side of that is precision or reliability. So reliability is, if you take 2,3,4 measures, does it tell you the same number each time? So going back to the what’s the altitude of Everest kind of analogy is, it may tell you that it’s 8000 meters at the top and it may be inaccurate, but every time you measure it, it tells you it’s 8000 meters, so it’s very reliable. So it’s reliable, but not accurate in that case. The flip side is you can have a very accurate measurement of the height of Mount Everest, but then you go back and do it again and the next time it says, 8200 meters and the next time it says 8500 meters, so you can’t really trust the value because it is not very reliable or precise. So that’s kind of the difference between accuracy and validity and in a tool that we are using, ideally, we want both. You sometimes may get away with just being reliable and one of the things is a power meter. I mean, ideally, you want it to be accurate, you want it to if you’re actually pushing at 250 Watts, you want it to be staying 250 Watts and you also want it to be reliable each time it’s going to say 250 Watts, but if I had to pick one, in that case, I would rather have an inaccurate, but reliable power meter. I’d rather even if it says 240 Watts instead of 250, if I can guarantee that it’s just 240 Watts and I’m only measuring it to myself and that’s also my only power meter, then it’s fine. It works because I can still have reliable data that I can build a training program on, as opposed to, the worst would be having a non-reliable power meter where again one day, you may be pushing with the same force, but it’s saying 250 Watts and other day it’s 240 and other days is 270 then you can’t really trust it at all.
Trevor Connor 10:00
So this is an argument, I used to have some of the physiology labs where they would bring athletes in to do testing on them and give them their their power zones and they would take them off of their bike and put them on this highly accurate, I think it was EveloTron and my comment is, ‘why are you doing that’ and they go, ‘well, this is accurate’. I go, but if their power meter on their bike is inaccurate, let’s say it’s 30, 40 Watts off, you’re giving them power zones that are gonna be way off when they actually go and do their training. So my argument was always you should be putting them on a trainer on their bike. It might be less accurate power numbers, but the zones you’re gonna give them are going to work for their power meter because generally the power meters have pretty good reliability.
Dr. Stephen Cheung 10:44
Yeah, absolutely and that’s also, this isn’t the focus of today’s episode, but it’s a reminder to zero offset your power meter each time you ride because you want it to be reliable.
Rob Pickels 10:59
Yeah, something a concept that I’d love to bring into this conversation that we’re having about validity and reliability, as we’re talking about the consumer who has to use this device that we’re talking about today, is the concept of is it worthwhile? I could have the world’s most accurate, the world’s most reliable sensor, but I don’t know that necessarily helps me improve my training and so I begin thinking of as we work through the different metrics that we’ll discuss today. Are they worthwhile for my training? Can I actually do anything to improve them and does the improvement of that metric make a difference in my training or not? Sometimes we can be focused on aspects that are just data pulling us away from really what should be the goal, the focus of our training and we might begin chasing numbers that don’t actually help us.
Trevor Connor 11:50
Before we dive into the different metrics, let’s hear from Dr. Inigo San Milan and his thoughts on the explosion of devices and their validity.
How to use Metrics to improve your Performance
Dr. Inigo San Millan 11:58
As you guys know, we’re flooded. The market is flooded with devices, with gadgets, with information with pretty graphs, etc. Many times are not even validated. So that’s where we need to be careful and how to interpret that information and how we then modify training with athletes. Even this hurry variability or this watch’s stuff, they look at multiple parameters. They might tell you they have these algorithms and they tell you, ‘oh, you’re fatigued today’. Whereas in fact, this is what I get from my athletes. They say, ‘hey, my magnanimity brand, but my whatever tells me that I’m fatigued today that I should take a day off and obviously I follow the plan and I did my PR today’. Other days is the other way around. They tell me that I’m super good and I go out there like, ‘I’m really fatigued’. So I’m not saying this happens all the time, obviously, but it happens. So that’s why all this information that we’re getting, including the saturation, which is popular now we have to take it with a grain of salt.
Trevor Connor 12:58
So with that note, let’s cut a shift gears here and talk about some of these metrics. So Rob, you put together a list of some of the things that are directly measured and obviously you have your classics for a couple decades now, we’ve had heart rate monitors and power meters, but you’re starting to see some new ones such as oxygen saturation, heart rhythms. So there’s some devices that can see if you’re in afib. We got heart rate variability and now what’s becoming really popular are glucose monitors. I’m sure we could go with a much larger list. I know my watch is starting to now tell me it’s reading temperature just from my skin, which I don’t put a ton of faith in, but there’s a lot of new metrics that are being measured.
Rob Pickels 13:44
Yeah, I think, Trevor a great place to start is with the device on your wrist, whether it’s a Garmin, a Suunto, a polar watch and something I want to bring up is that in this episode today, we are not reviewing product. This is not buy the Suunto and don’t buy the Garmin, don’t take that. We’re talking big picture here and so whatever I would love to start is that wearable on your wrist, if you’re going to buy one wearable it’s going to be that smartwatch, that smart, athletic watch and that product is integrating heart rate oftentimes from an optical sensor. It could be looking at your oxygen saturation in your periphery. Some of them now are doing heart rhythm detection through electrical analysis, as well as heart rate variability. Some of the running specific watches, they’re looking at running power and so yeah, Stephen, I’d love to start there. I don’t know if any of those metrics is something that just rings a bell off the top of your head, is maybe something that’s very worthwhile or something that’s not worthwhile at all.
How does the Heart Rate Sensor work?
Dr. Stephen Cheung 14:47
Well first off, I think the heart rate sensor is is a great thing to have and it’s the technology for the large part is pretty solid. I think in terms of the light sensor and just recording the variations in the blood flow that based on your heart rate through the skin and there are still some challenges with that technology. I think people with tattoos that it has problem. People with dark skin, I think it also has some problems with. So there are some very kind of situation specific cases where it may not be highly accurate those sensors, but from those optical sensors you can get a ton of information and Trevor already highlighted a lot of them. Everything from the heart rate by itself, to heart rate variability, to oxygen saturation, to sleep patterns and all of that is really built from these, so I think there’s a lot of potential with those. The most solid of those measures would be heart rate because it is in essence, the rawest data, it is not processed through multiple kind of proprietary algorithms to come up with a measure. It’s really just the heart rate by itself, then the next level is heart rate variability, where you’re starting to process the variability between individual heart rates and then you have again, whether it’s oxygen sensing or whether it is sleep patterns and stuff, then it gets grayer the more processed it is. I guess my basic overview of how the technology goes. So it’s like anything, the raw data is the cleanest it’s most accurate and then as soon as you start processing it, then you are putting in a lot of assumptions and one of the challenges with a lot of these units is that the algorithms are proprietary.
Trevor Connor 16:52
So I completely agree with you here and I would say the two that I would that are really tried and tested, which we’ve had decades to perfect them are heart rate and power. I think those have gotten better and you can trust those measurements. That said heart rate, for a long time, it was done with a chest strap, which we know works really well, but you touched on the fact that now we have optical sensors which are on your wrist, which do have some challenges as you brought up. Skin color, sweating, how it’s placed on your wrist. All these things can impact the accuracy of the heart rate information.
Dr. Stephen Cheung 17:34
Yeah, those photo sensors, they literally shine a particular wavelength of light from the watch into the very surface of your skin. So it is not going very deep into your skin it is not measuring your arterial kind of blood flow. It is really measuring just what is happening at the skin and then it is the individual or the very small vessels within your skin, it is measuring the blood flow and it is basing the heart rate on the variability in the flow pattern of the small blood vessels and your skin and as you noted, Trevor, one of the challenges is you need to have the watch tight. You can’t just have it loose and flopping on your wrist. So you need to make sure you have good skin contact, but also not so tight that you are compressing the skin. So those are just important technical things. So you want it snug, but you don’t want it too loose or too tight and then the other challenges you pointed out is that some individuals, especially those with tattoos, with a lot of pigmentation, dark skin, some of those units seem to not work really well with that just because it’s harder to get a good solid image of the blood flow. So those are things to keep in mind when you’re considering a heart rate based on your wrist or on these optical sensors rather than with the chest strap. The chest strap, as you said, works very reliably. It’s very clear what they do and you can get very, very good signals with those. Whereas would I not wear a heart rate strap and just wear a watch to get my heart rate? I’m not sure I’m there.
Rob Pickels 19:26
Yeah, to be honest, I’m in that same boat Dr. Cheung because primarily I cycle and maybe this is a different experience for people who run, but oftentimes it seems like my wrist or my hand position on the bar, whether that is changing the contact of the photo sensor or if it’s maybe occluding blood flow to my hand, my heart rate values really don’t feel very accurate when I’m using a wrist based sensor. I personally I hate to say it because we’re talking about wearables right now. I stick to a traditional chest strap unless I forget it and then I try to rely on my watch as a backup.
Trevor Connor 20:01
Let’s hear from Coach Grant Holicky who sees a lot of value in this new data but raises an important question. Is this data actually useful for athletes?
Is Data on Heart Rate Beneficial for Athletes?
Grant Holicky 20:09
I think what’s interesting about cycling right now is that so much of the stuff that we would do in the lab 20 years ago, is now the lab is every rider that ever goes out on the road. In other words, is this stuff beneficial? Yeah, absolutely. If we could take all this data and analyze it and come up with something that is this unifying theory of what heart rate is supposed to do. A, that’s never gonna happen because everybody’s so individualized. B, we have wearables out there right now that we don’t even know what they necessarily mean. We’re getting great data. It’s unbelievable data, but what does that data mean? I keep coming back to the, the Moxy O2 monitor. I can put that on two different people, the same spot, having them do the exact same workout and I’m gonna get totally inverse relationships. They’re totally inverse numbers, one’s going to have their oxygen levels go down in the calf, one’s going to have their oxygen levels go up in the calf. So until you can understand how to use that specific data with that specific athlete, it’s just noise and I worry a little bit that we’re losing the art of coaching because we have so much of the numbers in the science of coaching and I’m even watching this and this is where I feel like I get in trouble, but I feel like I’m even watching what’s popular right now in coaching and in training plan information, really gearing towards the sports scientists and away from the coaches and while I think the sports scientists have a lot to offer, even sports scientists, like Seiler are going to say the coaches are the ones in the field that are doing this. Our information is something that they have to then put into play somehow some way and they’re going to put it into play different ways and oftentimes, the coaching is ahead of the science because they figured something out in the field of play.
Rob Pickels 22:10
Now, something that as we’re on this heart rate topic, I do believe that the devices and this is a little bit off topic for us, but the devices that are looking for arrhythmias, using contact where you have to reach across with your other hand, maybe touch a certain part on the crown and they’re monitoring the electrical activity in that circuit. I think that they’re of reasonable accuracy, you might not get an EKG worthy tracing, but you are able to get some basic diagnosis for rate rhythm, maybe an atrial fibrillation and so I do think that that is accurate enough, but I will say I think for an athlete, I think it’s a question of whether or not a feature like that is worthwhile, but perhaps if somebody is struggling with something like atrial fibrillation, it’s something that helps them understand what’s happening inside their body.
Trevor Connor 23:02
Interestingly, those are pretty accurate. So the Apple Watch was the first one to have it and that’s getting into the medical territory. So they have to pass certain medical standards to be able to offer it and say that we can diagnose atrial fibrillation. So I saw that in a few of the studies that I read and they were saying this is actually pretty accurate. It has to be because it has the past medical certification.
The Healthcare Industry’s Role in the Development of Wearable Technology
Dr. Stephen Cheung 23:25
That’s a good point you raised there, Trevor, a lot of this technology is driven, really driven by the healthcare industry and it’s very important if we can get good quality diagnostics in patients, not in the hospital or not in doctor’s office and especially if the doctor can see those or the patient can see those. That’s amazing, for diagnostic, for just preventative health care. So just remember though, again a lot of that technology is developed for example, for passive use, it’s not meant for you to be exercising at 180 beats per minute, for example. It is really designed for relatively kind of sedentary individual of average weight and fitness and really in a resting kind of modality or at most light exercise. So the easy offshoot for those is, hey, if we have this medical technology and the prime example is continuous glucose monitors, they’re developed for diabetics and if they’ve been adapted for sports use with continuous glucose monitors now and so you also have to keep in mind where the technology originated from and who was the original target audience. Is it really developed first and foremost for an athletic use with an athletic population or is it really being developed for again, a healthcare setting and then really being trying to be in essence almost shoehorned into an athletic kind of market?
How Research is Conducted in the Field
Trevor Connor 24:54
To continue with that, another theme that I saw on the research that came up multiple times, is you have to be careful even when they do research this, how is the research conducted. So, a lot of these devices, a lot of these measurements actually do pretty well when you’re in a lab. So let’s use that example of the wrist based heart rate, but in a lab, they’re making you keep your arm pretty steady, they’re wiping the sweat off, there’s a lot that they can control that will give you a better measurement. So now you’re seeing some of the more recent studies saying, let’s go out and test this out in the field where your wrist is moving all over the place, where you are sweating and in the field, you’re seeing a lot lower accuracy with these devices.
Dr. Stephen Cheung 25:39
Yeah and that’s absolutely the case. A lot of the tools that I use in my day to day research, they’re great for passive exposure to heat or cold or whatever, but as soon as I get the participant exercising, then all the accuracy goes out the window because of movement artifact because of just tiny, tiny micro slippage in the probe, sweat build up all of these things. So, what works in a kind of a passive sedentary environment may not translate to a athletic situation out in the field.
Wrist-Worn Sensors and Heart Rate Variability
Rob Pickels 26:17
One thing that I’d love to talk about, Trevor, based on your question of the research is, in my research it looks like heart rate variability and whether or not that’s accurate, from a wrist worn photo based sensor, there seems like there’s research that’s like, ‘Yeah, this is spot on’ and other research that’s like, ‘No, man, it’s a joke’. You guys, do you have an opinion on can that wearable device on your wrist accurately measure to a degree that we can have some knowledgeable, actionable information? Can it accurately measure heart rate variability?
Trevor Connor 26:51
Everybody should go and take a look at this 2000. I’ve read a couple reviews, but this 2022 review, I’ll just give you the name quickly. It’s called ‘Wearable Activity Trackers, Advanced Technology or Advanced Marketing’. They actually had a table in there where they gave all these different metrics, star ratings and heart rate variability, they gave it two out of five stars and said ‘more individualization needed for better accuracy’ and definitely said the accuracy better was chest strap, which is what you’d expect.
Dr. Stephen Cheung 27:22
I think that really comes down to individuality of the data. Again, a lot of and we’ll get more into the kind of the challenges of algorithms in general later, but one of the big challenges is that algorithms are based on large populations of people and it may not necessarily represent you as an individual. So also to translate kind of these measures into actionable outcome, you may get great quality data, but it may be kind of giving you recommendations based on a very large cross section of individuals, not taking into account your fitness, your age, height, weight, whatever ethnic background, all of these things and, yeah so there’s also a challenge in that outcome. Even if everything going into the algorithm is perfectly accurate, you may still not get quality, individualized data at the end.
Oxygen Saturation Metric in Wearable Devices
Trevor Connor 28:21
To give you another example Rob, I’ve read an interesting study about Oxygen Saturation, which is becoming really popular in a lot of these wrist-based devices and there had been some earlier studies again, in a lab, where they created a simulated altitude situation and said, ‘Hey, these devices aren’t too bad’. So this particular study, they took 13, healthy ‘lowlanders’, as they put it, who had had a history of having negative reactions to altitude, so getting high altitude pulmonary edema and took him up to 4500 meters. So about 15,000 feet and then compared the one of the more popular wearable devices for oxygen saturation to a gold standard and basically concluded accuracy and reliability is really bad. You just can’t trust this.
Rob Pickels 28:21
Yeah, it’s interesting that they’re even trying to integrate this information I think into wearables. Coming from the medical world, with a pulmonary testing background, even like your fingertip worn a specific device, looking at SPO2 or oxygen saturation is oftentimes inaccurate and I would take the pulse of the individual and see if the pulse is matching up with what the device is saying, to see if I had any reliability of the oxygen saturation number, but I think the bigger issue with oxygen saturation is even if it is accurate, I don’t know what I have to do with that number, to tell you the truth because I’m at sea level it’s going to be high. I go up in altitude it’s going to be low. If I get pneumonia, it’s is going to be low, but I probably know that I have pneumonia for other reasons and I just don’t see that as being an actionable metric. I’m not going to train harder, I’m not going to change what I do tomorrow based on whether or not my oxygen saturation is high or low.
Trevor Connor 29:15
That was actually kind of the point of that study that I read, which is really the only people who can apply this are climbers who are going to high altitude because generally a drop in your oxygen saturation precedes any sort of high altitude sickness. So they’ll watch for that and if they see the saturation go down, like, ‘yeah, I gotta get off his mountain, I gotta get off now, while I still can’ and the issue with the accuracy of these devices is they tended to overestimate your saturation. So they said it’s actually can be dangerous for people, because you might be heading into pulmonary edema, but your blood oxygen saturation is looking fine, when it actually isn’t.
Rob Pickels 30:57
Right and you might not be feeling well and I know that people do this, but the device is saying I’m okay. There are days where it’s like, I’m out riding and I probably didn’t zero offset my power meter, like Dr. Cheung reminded us to do and it’s like, I am crushing myself and not going anywhere and I feel like I’m the slowest worst athlete in the world because that number is a little lower than it should be. We put so much emphasis on what this external device is saying that oftentimes we can override what our internal senses are telling us.
Trevor Connor 31:31
Let’s hear from Lindsey Gloich, who used oxygen saturation with her athletes in the preparation for the 2016 Olympic Games. She has a lot of thoughts about it, from experience.
The Value of Oxygen Saturation Data
Lindsey Golich 31:41
I think it’s not entirely new to the world in sport science. We’ve had certain devices out there probably for the last 8, 10 years and to be honest, I actually was using a device with our track cycling group on 2015 and 2016, leading up to the Rio Olympics. We were using it is that we just let it run in the background. So I actually collected data for about eight months on three training sessions a week from the athletes, all various, some on the velodrome, some on trainers, so I’m just going out for aerobic rides and I wanted to see really how that correlated with our power outputs or if it showed us or told us something that we didn’t know, or that we were overlooking and with the group of athletes that I was working with, is that it didn’t enlighten me to anything that we couldn’t already identify and I think for that group, it’s a little bit different because there are athletes that I was seeing every day. So not only are we seeing their power outputs and different things from training and in the gym, but you also have that face to face time contact. So I think from a coaching perspective, there could be some value if you’re not with your athletes all the time, but at this point, I would still look at the bread and butter of power and then have those metrics run not necessarily in the background, but in conjunction with it and see what you can learn from it and find some additional insights from a training perspective. You might learn something more quickly when you go to altitude by your power output and your oxygen saturation that might be something that can help kind of a faster learning curve for an athlete rather than just going off perceived effort.
Trevor Connor 33:22
I think that’s one of the really important things that we need to address in this episode or I hope people have as a takeaway, which is sometimes no information is better than bad information, actually, most of the time.
We become Slaves to the Data
Dr. Stephen Cheung 33:36
Yeah, and a lot of times the real challenge with getting all of this data is we become slaves to the data. We say whether it’s a recovery score that says we are really not recovered, but in all honesty, I walk around and I feel absolutely awesome. I get out on a bike, my legs feel loosened and really ready for action. What am I going to base today’s ride on? Am I going to base today’s ride on my sensations or because my recovery score says I’m really low? Well, the risk is if you are slavishly kind of fixated on these metrics, then you will only kind of become a slave to them and not actually trust your own sensations and we have sensations for a reason we have very, very finely tuned mental capacity to see how comfortable or uncomfortable, how good we’re feeling. Rating is perceived exertion is pretty well fine tuned, no matter what conditions we’re in and sometimes we should be trusting that rather than just blindly following what a number says, especially again, if we don’t know how that number is derived, if we don’t know, kind of the quality of that that number.
Calculated Training Status and Load
Trevor Connor 34:54
I’ll just say go I hate to keep going back to the well, but that that review that gave star rating to all these various things after reviewing them. Certainly what you saw as you got into more metrics that are calculated algorithms, star ratings went down and certainly the only thing that gave a one star two was training load and in their review of it basically said there are so many different ways of doing this, there are so many factors, there’s so much individualization, this is just not something you can trust. As a matter of fact, this is something that can lead you in a bad direction because as you said, you might be completely recovered and it’s telling you otherwise. Worse, you could be very fatigued, needing recovering, it’s telling you you’re ready to train today and I can actually give you some some really interesting examples of that if you want. The other one they really tore tore apart was these estimates of VO2 Max that a lot of these devices will give you and basically said there’s so many factors here, it’s really hard to trust that number and when they compared it to actual VO2 Max measurements, it could be off by a pretty significant amount, we had a chance to talk with Dr. Seiler about all these new devices and he raised the issue of the algorithms that are being built on top of the metrics. Let’s hear his thoughts.
Rob Pickels 34:54
Yeah, I’d love to dive in a little bit deeper to this part that I’m kind of calling calculated predictions, and that is your wearable trying to estimate what your training status or your training load is based off the activity. Heck, even the activity tracking that a lot of these devices have, my devices estimate my VO2 Max or my threshold, they’re trying to replace lab based standards and then Dr. Chung as you had brought up recovery status is a big one. I oftentimes try to reframe that as ‘readiness to adapt’ that might maybe be a more accurate way of looking at it as opposed to recovery, which I think is people think ‘readiness to perform’. I think that all of these are trying to replace these innate senses within our body and I’d love to get both of your opinion, have either or any of these tracked with how you feel from just a purely experience standpoint because I don’t know that there’s research on this, I don’t think that anybody has ever say, ‘use Garmin or Suunto or polars’, trading status and training load and done research based off other gold standards. I don’t even know what the gold standard is for that because I don’t think it’s TSS for what it’s worth, but I think that we have to rely on our personal experiences and I’d love to hear from both of you.
Algorithms Being Built on Top of Metrics
Dr. Stephen Seiler 37:26
It’s a good thing and it’s a bad thing and it’s a terrible thing. It’s all of those and the reason it’s all of those is because it’s no better than the basic data you’re collecting, you know trashy and trash out and unfortunately, what ends up happening with a lot of these companies is that they, you’ve got hardware and software and you work hard to get your technology so that you can actually capture some signal, like temperature, or lactate or whatever breathing frequency and often it’s engineer driven, but then they try to make money and that’s understandable, but how do they do that? They say, well, ‘we don’t want to have to keep reinventing the core technology and making new hardware, we’re just going to make new software’. We’re going to make new algorithms and there trying to squeeze the lemon and tell the consumer, ‘Well, the only thing I actually measure is heart rate, but I’m going to use some equations to try to tell you how many calories you’re burning, what your sleep time is’ and the farther you go away from the primary signal. The one thing that the technology actually measures, the more noisy and useless the metric is, but unfortunately, that’s the business model that will often emerge is let’s try to not waste money on core technology. Let’s sell algorithms.
Rob Pickels 38:56
Tis the season for spring knee as March sunshine and early spring weather inspires us to ramp up our riding mileage, our knees don’t always keep up. If you’ve got knee pain, we have the solution for you. Fast Talk Lab members can follow our new knee health pathway featuring our new Director of Sports Medicine, Dr. Andy Pruitt. See the introduction to the knee health pathway at fast talk labs.com.
Rob Pickels 39:22
Can I ask on that one specifically? You know Dr. Cheung, maybe you have some insight into this. Is this a accuracy reliability standpoint where hey, that VO2 Max number is wrong, but as long as you’re seeing it go up, you’re probably doing something right or are we like just this cloud of throwing darts and missing the target completely?
How do you Determine VO2 Max?
Dr. Stephen Cheung 39:45
I would probably summarize by saying it is one of those things you want to keep track of long term it is in some senses like stepping on a scale to check your weight. Day-to-day fluctuation is going to be there and it’s not something you should really kind of put too much emphasis on. It’s really probably the long term tracking. So, in general, what some of those VO2 and again, they’re proprietary, so I don’t know if this is actually correct, but one way of determining VO2 is really looking at how quickly you recover from your heart rate. So if you were doing some mild exercise and then you were resting for a minute or two and then your heart rate drops, that is a good signal of how fit you are and so sometimes that is what’s being used to go into your kind of VO2 Max kind of measurement and also, how your heart rate responds to mild exercise. So it’s kind of a combination of those factors and again, they’re based on you can get the really good quality data, but then you are being compared to a very broad cross spectrum of people of all ages, fitness sexes, etc. and then it’s saying your VO2 Max is predicted to be X and so, I wouldn’t put any faith in the accuracy of it, it would be kind of more, I would place some faith maybe on the reliability of it and if it is something you track periodically over weeks and months and you see it kind of steadily continue to progress upwards and if it matches your actual kind of fitness on the bike, if it matches your heart rate response, when you’re riding at a particular power speed, then I would have more faith in it, I would not just trust it in and of itself by itself as ‘Yeah, I’m fitter’. Obviously, the main thing we’re interested as endurance athletes is how we actually perform. So it has to be calibrated to that also.
Efficiency of VO2 Max Calculations
Trevor Connor 41:54
I think these these VO2 Max calculations are really a good example of what you’re talking about because I did read one study where they talked about all the different algorithms that these different companies have and how hard it is to get at the algorithm, but pointed out a lot of the calculation depends on what measures they have. So some of the cheaper watches that don’t do heart rate, don’t really have any of these optical centers, they’re literally predicting your VO2 Max based off an accelerometer, where the ones that are more sophisticated are bringing in even oxygen saturation to try to figure out your VO2 Max, but as we just pointed out, the oxygen saturation measurement isn’t very good and another issue that Dr. Cheung you just brought up is there’s also the question of efficiency or economy. So an elite athlete will tend to be a little more economical. So if you got an elite and a amateur runner going side by side at the same speed, the elite runner is actually going to need less oxygen to keep that pace up, but the watch doesn’t know that so the watch is going to overestimate the the VO2 and ultimately the VO2 Max of the elite athlete
Rob Pickels 43:04
I would love to at this point talk about what I think is the elephant in the room in terms of these algorithm calculation based predictions.
Trevor Connor 43:14
VO2 is really hard on an elephant the skins really thick, I’m sorry.
Rob Pickels 43:20
And their kgs are so big that the poor elephant, man they just get low scores all the time.
Dr. Stephen Cheung 43:27
I did read a study way back when were they actually measured VO2 on an elephant and they actually had a train to walk on a treadmill. I was just thinking, oh that poor treadmill, the force on that.
Validity of Recovery Status from a Device
Rob Pickels 43:41
No, guys, the actual elephant in the room is recovery status and I know I have gone to bed wearing more than one electronic device at the exact same time and gotten drastically different scores from each of them as to whether or not I was recovered. Trevor, you brought this up earlier. Dr. Cheung, you did too, but I’d love a little bit of a deeper take on the recovery side of things and I do want to point out before we get there that oftentimes the recovery score is based primarily on a heart rate variability score and as we mentioned earlier, optical heart rate itself, it might be accurate in a resting environment with a skin color that jives with the wavelight of the sensor, you don’t have tattoos, but that heart rate variability might be a little bit harder to extract from that and so I’m gonna throw this out there just questioning even though I want so hard to believe questioning is there any gosh darn validity to using a recovery status from a device?
Trevor Connor 44:46
So I don’t have that study in front of me but yeah, the recovery status in most of these devices and again, everybody has a different algorithm, tends to be heavily reliant on sleep. How much sleep did you get, how low was your heart rate? What was your resting heart rate? What was your heart rate variability. Certainly those are the three metrics that the whoop looks at to give you a recovery score. Other devices are going to be a little bit different, particularly if they don’t have the heart rate variability, they’re going to look at other factors and some of them as simple as they’re essentially just doing the calculations of, well, ‘you did this much work’, like you did a two hour run yesterday, you appear to get this much sleep. So it’s kind of looking at how much rest time did you do versus how much exercise time did you do and just basically doing a ratio of the two. It can be that simple and you just don’t know because they keep all these algorithms, private, but going back to let’s talk about these more sophisticated devices that look at sleep. Again, you’re gonna start having issues of not all the data is great, heart rate variability isn’t perfect. I read a study on sleep monitors and there was only one and it was the Whoop, that they said was in any way comparable to actual sleep studies in the lab. Basically said the rest of these devices, you just can’t trust them that much for their sleep measurements. So you’re doing a calculation based on not great data, then the question is, how good are the calculations and we don’t know. So it’s kind of putting bad on top of bad and maybe some of them are okay, but a lot of them are probably misdirecting you.
The Difference between Polysomnography and Whoop
Dr. Stephen Cheung 46:27
So you brought up a great point Trevor, about that sleep study and the testing of different sleep trackers and there’s an important distinction. I think, to clarify that the Whoop, in that case, they found to be pretty accurate and pretty well correlated with sleep duration, with the gold standard, which is polysomnography, which there are sensors all over your body, detecting everything from motion to brainwaves, etc. So that’s really the gold standard for measuring the different sleep stages. So if I recall that study it, it said, you know, some of the monitors specifically, the whoop was pretty good and well correlated and accurate with the sleep duration, but not necessarily with the sleep stages. So I wouldn’t put trust into kind of when Whoop says in the morning, you got more deep sleep or less deep sleep or more REM sleep and, and stuff and so I’d be hesitant about the sleep stages. I would really be hesitant about placing anything that you can do in terms, going back to what Rob was saying about what is actionable. Okay, if I got more deep sleep last night than last week, if I got less light sleep, what do I do about it, besides actually maybe going to bed earlier or getting trying to get more sleep, some of the basic stuff with sleep hygiene. Maybe not reading with blue light right before bed, those kinds of things, but, those are kind of big picture actionable things and they’re going to help you in general, but it goes back to my overall theme of don’t get fixated on a single point of data or a single thing, certainly not with did I get more REM sleep last night than the night before? I really don’t place any faith in the accuracy of those and again, even if they are accurate, what am I going to be able to do about it?
Trevor Connor 48:33
Coach Frank Overton has experienced using sleep trackers with his athletes. Let’s hear his thoughts on the devices.
Raw Data on Sleep Tracking Devices
Frank Overton 48:39
We looked into the sleep stages data and we talked to researchers and the general consensus was sleep stages, data is not that accurate and what we like to think of when we look at sleep data is just ‘total time slot’ and so we call that the raw data and we like to separate the signal from the noise and a lot of these new features that are coming out with wearables is noise, some marketing feature and then we put sleep stages data with that. Has enormous potential, but the devices need to get a little bit better at measuring the sleep stages and so it’s not to say the science isn’t bad. It’s just the the devices and the sensors need to be improved, but it’s amazing like this Apple Watch. I mean, it’s incredible what this thing can do I mean, ECG, AFIB, they get into sleep stages, but it’s not very good, but really I mean sleep and HRV we look at the raw raw data. I think I wore a Whoop and an Oura side by side all summer, looked like a dork, but generally the raw data is good sleep and HRV, but when you try to go beyond that, you’re getting into some noisy areas.
Good Sleep Hygiene is Important
Rob Pickels 49:53
Yeah Dr. Cheung, you bring up just good solid sleep hygiene is worthwhile for everyone no matter what and you don’t need to rely on a device to tell you that, just just go ahead and do it. Don’t worry about what your score is, but I do want to bring up right now a product that probably came before its time. Back in the day, gosh, at least 10 years ago, I had a ZEO sleep monitor, which was a device, it was a wearable that you actually wore on your head. So I had to go to bed every night with this head strap on my forehead and it measured through my skin and my cranium, my brain activity and a little bit more of a direct way right at what’s happening inside the old noggin there. Then it beamed all of that to a, it was an alarm clock size base unit and I wore that thing for a solid week before I just gave up on it because the headache that I got and again, it was like, what do I even do with this information, but I will say that that was 10 years ago and that was before a lot of people were interested in this topic that became a lot more, I don’t want to say important, but a bigger part of our conversation when it got integrated into the watch you wear on your wrist.
Dr. Stephen Cheung 51:09
With all the stuff that you wear to bed Rob, I’m surprised you’re actually married.
Rob Pickels 51:13
How do you know what I wear to bed?
Dr. Stephen Cheung 51:14
You’ve just been telling us in great detail.
Trevor Connor 51:18
Core, ring, whoop, some strange thing on your forehead.
Rob Pickels 51:21
Yeah, you gotta do your activity tracking.
The Future of Biosensors
Trevor Connor 51:24
But that is, I think this is where we’re going to finish this episode. So we won’t dive into it now, but I want to talk about the future of these biosensors because that’s kind of both exciting and a little scary, but let’s save that until we finish this conversation about the the algorithms,
Rob Pickels 51:43
Yeah, screw algorithms, man, let’s talk about the individual devices, Trevor, I think that those are for me some of the more exciting ones to tell you the truth, than this integrated wearable on your wrist and what I’m thinking about here are really devices that are specialized. The Moxy monitor is looking at muscle oxygen saturation very specifically, it’s the only thing that device does. We also have these brand new hydration sensors that are coming out of the marketplace that are taking small samples of sweat or looking at maybe skin conductance. I think some of them continuous glucose monitors have been all the rage and that is an invasive device because there’s a small needle that gets inserted subcutaneously to take samples and then there’s what, there’s also oh, the core body temperature sensor that’s looking at skin and environmental temperature to try to accurately assess your core temperature and Trevor, I’m sure you can add even more biosensors to that list.
Trevor Connor 52:43
I will tell you this was the most interesting engineering study I ever read was 34 pages long for this episode, I went, ‘I’ll force myself to least get through a couple pages of this’ and ended up reading the entire thing and it’s just called ‘recent progress in wearable biosensors, from healthcare monitoring to sports analytics’ and it’s talking about all these little devices that the commonality is that they measure fluids and it actually gets a little scary when they have the page about the different ones that they have. So talk about continuous glucose monitors, that’s a device that’s measuring interstitial fluid. So it’s the fluids between yourselves. It’s not blood, but a different type of fluid and so this is something you put on your skin and it just samples a little bit under the tissue and pulls out this interstitial fluid. Another one that’s become very popular or they’re doing a ton of research on is sweat analysis, the devices that you put on your body and they collect your sweat and do analysis of the electrolytes, but they took it further, there’s research going into ones that you put in your mouth. So imagine somewhere in this thing in your mouth for two weeks, that measures your saliva.
Rob Pickels 53:53
The next time you need a crown, you can just get a smart right?
Trevor Connor 53:57
Here’s one that’s really scary is one they put in your eye that measures your tear fluid.
Rob Pickels 54:02
Nice!
Trevor Connor 54:03
They’re actually researching this and then they have ones that will do continuous urine analysis and then interestingly, the last one, which is actually the most difficult is blood because if you put it on the outside of your body, there’s not many places where blood comes pretty close to the surface. So it has to sample pretty deep. Otherwise, it’s something you would have to put inside your body so that it has exposure to the blood, but this stuff has all been researched and just as I’m reading all this and reading and how advanced this research is, just having this picture of almost us becoming the cyborgs with all these different devices connected to different parts of our bodies, sampling different fluids to give us constant information about what’s going on with us. What’s interesting was the accuracy so we were just kind of beat up on devices that sit on your wrist and use photosensors to try to get you a bunch of information. Pretty consistently these devices which sample the full, they’re gonna give you very accurate information, whether it’s usable information or not, is probably another conversation.
The Value of Sweat Analysis
Dr. Stephen Cheung 55:06
Yeah, I would place great faith in the potential for sweat analysis because certainly as endurance athletes, there’s a lot of sweat available for analysis, it doesn’t have to be restricted to a particular part of your body, like the continuous glucose monitors tend to and the ability to look at the makeup of the sweat, which is a great analog or gives you a great indication about everything from your hydration status, to your electrolyte loss, to your acclimation status. Those can have great potential, not necessarily maybe for a single day of training, but whether it’s an ultra endurance event, a really long day out, or again, one of those things that you track over a long period of time, I think those have the great potential to be highly accurate and those have real long term actionable things that you can do with it of understanding of tracking, am I properly hydrated long term, am I heat adapted long term, for example. So I really think those have great potential and because they’re not trying to kind of in a way overprocess the data and we talked about kind of the challenges when you take a raw signal and you overprocess it, it tends to be like one thing that it’s doing, it’s highly accurate in it and it is telling you something pretty related to that original thing that they’re measuring. So I think those are really good in terms of the potential for those being really actionable things that you can do also.
Rob Pickels 56:46
Yeah, so I had hoped to be extremely knowledgeable on this one particular topic because I knew that we were doing this episode and I know that I’m a gadget guy. Not only am I a gadget guy, I am a sweaty, salt laden, gadget guy and so I bought a wearable hydration monitor, with the promise that an Android app would be coming and that Android app has not arrived yet. So my hydration monitor has been sitting on the shelf in my house not being used, but it is something I’m really excited about. I just can’t use it, unfortunately, but I do agree with you, I was willing to spend my own money on this product because I do see the potential benefit and the use case, depending on what you’re doing. I don’t think that a hydration monitor is worthwhile say for somebody doing a 5k. I don’t think that hydration is a limiting factor in a race like that. Somebody’s doing a marathon, somebody’s doing long training, somebody working in extreme environments, that becomes increasingly important because hydration itself becomes a limiting factor and I only really think that we should be at least if we’re looking to improve our performance that we should be focusing on these limiting factors for the information that we’re trying to gather.
The Accuracy of Continuous Glucose Monitors
Trevor Connor 58:02
Yeah, I think what’s interesting about these new types of devices, these new wearables is that they quite literally measure usually a fluid, but measure actually take a direct measure of something on your body, there is a high degree of accuracy with a lot of these. I think the issue is, as you said, interpretation and how do you use this sort of data? So one of the examples is the continuous glucose monitor. So I read a couple studies on that and they all said, yeah, these things are pretty accurate. There’s one issue, they measure interstitial fluid, they don’t measure blood and what will happen is if you see a spike in blood sugar, that will eventually get to the interstitial fluid, but there’s a delay. It’s not as immediate as you would see if you took an actual blood measurement. So you have to be careful with that, but the point that all these studies raised was all the research on continuous glucose monitors have been done from a healthcare standpoint, so they were on diabetics, for the most part and we really know how to use that information for diabetics, but that doesn’t mean we know how to use the information for athletes who are not diabetic and you have a lot of promotion of, this is gonna tell you exactly what you need to be consuming and drinking and erase and this is your real time nutrition guide and all that sort of stuff, but that hasn’t been validated. So the ability to take good measurements has been validated, but then the algorithms to say here’s how to use this information in an effective way. That’s a little more marketing right now than it is actual science. Do you agree Dr. Cheung?
Dr. Stephen Cheung 59:36
Yeah, absolutely and that’s what I highlighted earlier when I said that a lot of these devices have been originally developed and that review with all of those things, all of them are the 34 page review, Trevor, those are all developed for health care and that’s absolutely true with continuous glucose monitors and now they’re being kind of translated and transposed to a atheltic market and so yeah, I don’t think we’re there yet in terms of knowing exactly how you can on a ride by ride basis alter your your fueling strategy based on those and how well you can just directly take the diabetic population that was originally developed on and then move it into a high energy high performance sport. So yeah, I think again, the potential is there, but I think we’re in very early stages of understanding really how we can use those continuous glucose monitors the best for actual athletic performance.
Continuous Glucose Monitors for Postprandial Hypoglycemia
Rob Pickels 1:00:45
Yeah, I think I agree with you completely. Continuous glucose monitors are something that’s personally interesting to me again because I am prone to a postprandial hypoglycemia, which means that after I eat something, my blood sugar oftentimes ends up too low. More than one occasion, I’ve gone to get labs drawn after eating lunch and my glucose came back lower than it should have and so I can certainly see like, Hey, how is this affecting me while I’m riding? What do I eat prior to riding? Am I beginning hypoglycemic? Am I not? Am I am I experiencing hypoglycemia related to that during exercise? Should I be changing my carbohydrates? There are all of these questions that can potentially be answered, but if I slap that glucose monitor on my arm tomorrow, I don’t necessarily think that I get answers to these questions because as you were pointing out, we just don’t know enough and I think that we need a lot more pointed research in an athletic, a healthy athletic population. We need a lot more people to be guinea pigs to begin understanding this and then really, I think that people who want to be at the tip of the spear who are doing this just maybe to begin gathering data and trying to understand, but not action coming out of it. Maybe continuous glucose monitors work for those individuals right now, but I think that what we’re seeing is an outsized push from marketing departments to say that athletes need these things we are seeing them on so many professional athletes at this point in time, when I don’t think that they’re wearing them because they’re choosing to wear them, they’re wearing them because they’re paid to wear them. So in the hopes that everyone else sees it and begins uptaking that product and I don’t necessarily want to call out continuous glucose monitors as good or bad, but I do think that we need to understand the entire situation that’s happening.
The Problem with Power Metric Data
Trevor Connor 1:02:32
I think the issue here and I think this is kind of what we’ve been communicating the whole way through. We said, heart rate, using a heart rate strap and power, you can really trust those, but you look at the history of those, they’ve been around for decades. We had years to perfect that technology and make sure it works and then we had another 10, 15 years of figuring out how to use that information, how to create effective training zones, all this sort of information. There was a long time of figuring out how to really perfect and use these tools and I think what you’re getting at right now, Rob, which is my concern is it seems like we have a new metric coming out every year, with new algorithms based on it and I think it’s being pushed a lot by marketing for everybody to have the newest coolest tool and I can’t help but point out, I’ve got a Garmin Fenix when I bought it a year and a half ago, it didn’t do heart rate variability, there was an update on my watch and all of a sudden, it does do heart rate variability and the reason it said it didn’t do it at the start was because the optic sensors can’t do heart rate variability. So I kind of scratch my head and go, Optic Sensors haven’t changed. So how come you can suddenly do this now and so my concern is, we’re having this explosion of more and more metrics that are, literally some of them are just been in the last year or two and they’re going through a much more rapid cycle of, ‘okay, I think we got sort of something that measures this, okay’ and then very quickly, saying, let’s figure out how to use this so that athletes and other people can take advantage of this and coming up with algorithms that might be a little bit questionable and we’re not putting the scientific rigor that we put behind the original heart rate behind original power meters. It’s being pushed probably a lot by marketing right now and it’s just to me questionable. How good what we have is, I think if you gave it the years and gave it the scientific study, it probably all could be quite good. Just not sure we’re there. Let’s hear from Coach Neal Henderson, who agrees that the issue may not be the metric, but the quality of the data we’re getting right now.
Why Sleep Metrics are more Accurate to the Athlete than Power Metric Data
Neal Henderson 1:04:42
At this point, we have a lot of different technology available that’s measuring a lot of things. There are some issues number one and just the quality of that information that we’re gathering right now. If you rewind, power meters 25 years ago, yeah, we had some that were very, very good, but there was some that came out that were not very good, like indirect things, I mean, I’m not going to name names, but I think there was a chain tension and vibration.
Trevor Connor 1:05:06
I had it, I was waiting for you to mention that.
Neal Henderson 1:05:07
There was a bottom, an optical bottom bracket one and those compared to the direct force powervators, were not on the same level. The quality of information then was not reliable, repeatable and we’re seeing some of that for sure, in the wearable space in these different things. There’s also then the aspect of what is being done with the data that does come in and I would say, at this point, it’s kind of the Wild West in a lot of areas and there’s what I call algorithmic bravado, a lot of processing being done and recommendations or insights supposedly been provided by these algorithms that are not very well established and are not clearly applicable for all the users utilizing those. So I think, there’s great opportunity as we continue to move forward and there’s improved quality of the data that is being collected and then secondary to that the algorithmic aspect of how then that is being processed or is it being integrated with the other pieces of information. For sleep, as one of those things, I tend to ask, ‘what was the quality of sleep? How well did you sleep?’ That’s an important aspect. You can show me any kind of data that says how many hours and stages and this and that, but just that total volume of sleep and the subjective quality is a much better predictor than a lot of those other additional variables that may or may not be actually true at this point.
Why Core Body Temperature Sensors can be Misconceiving
Rob Pickels 1:06:37
It’s hard, right, Trevor because this is the nature of building product. Everybody has to find a unique product to put in the marketplace so that hopefully somebody purchases that product and I want to bring up the core body temperature sensor, which I have and I own. I’ve had that thing for gosh, for a couple of years now and I don’t really use it anymore because, well, first off, I don’t know if it’s accurate or not, we can discuss that, but at the same time when I was wearing it and my heart and my core body temperature was getting a bit high. What was I supposed to do? I was trying to compete that route, I was trying to compete in that race. I wasn’t going to slow down because of my core body temperature getting up to that upper limit that it was saying it was at. So I wasn’t really taking the information into account and after a couple of years of that product being on the marketplace, I don’t think that we’ve landed in a place where it’s a necessarily usable, actionable product and it’s a great number to look at and and beams right onto the head unit of my of my bike, but I don’t know. I don’t see validity there.
Trevor Connor 1:07:41
Dr. Cheung, what do you think?
What can Athletes do with the Data?
Dr. Stephen Cheung 1:07:42
This is completely in my field, Right? I mean, I study extreme heat extreme cold. So the Holy Grail, certainly from an athlete perspective, in terms of as a scientist, to be able to understand how hot an individual is getting in the workplace or as an athlete during exercise. As a scientist, it’s great. It would be amazing to have that as a tool, but again, I really side with you, Rob, is what can I as an individual athlete be doing with that data and I think there’s a big disconnect there. I mean, as a scientist, I can get lots of great information about how human physiology works, but again, as an athlete, can I actually use it to adjust my training, my pacing, my race strategy? I’m not sure that you can do anything with it, like during an actual ride or event itself and so yeah, I’m not really sold on the utility of having core temperature in real time and I’m speaking as a thermal physiologist, as someone who lives this field day to day. So accuracy and reliability issues aside, I think this is another case of is it actionable? Is it like you say, Rob, am I going to actually adjust my pacing for this marathon that I’m running because my core temperature is saying 38, five degrees Celsius? I don’t know, would that be a consideration? It might be a minor consideration, but I always say, if I’m racing, I am trying to get to the finish line as fast as I can or I’m going to try not to get dropped by the pack that I’m with. That’s gonna be a big consideration, not necessarily what my core temperature is telling me I’m at. So I’m not sure that it’s truly actionable during an actual ride, it might be one of those things that you can use long term to get an idea of how do you respond to different environments and different workouts in different environments and that might give you kind of some longer term actionable things. I know, if I am not eating enough after three hours in the heat, then I really tend to overheat or if I’m not drinking enough, as an example. Then that might be actionable, but during an actual ride itself, I’m not sure it’s something that we necessarily need to fixate on in terms of taking up a lot of real estate on our computer screens at the time during our bike ride. It would be more something I would track long term and kind of correlate it to a lot of other information, not just take it by itself.
Data can Fatigue Athletes
Trevor Connor 1:08:29
Taking it a step further, we talked about this study in a past episode, I can’t remember which one it was and I’ll try to put it in the show notes, but there was a study where they had athletes do. So cyclists do time trials and in one scenario, they could do the time trial looking at a whole wealth of data and the other scenario, all they can see was time and generally, they performed a little better when they could just see time and what they discovered was that actually looking at all that data was fatiguing the athletes and so if you have all these metrics sitting in front of your face while you’re working out and they’re not actionable, they might just be fatiguing you.
Dr. Stephen Cheung 1:11:17
Yeah and I remember I had this discussion with my PhD supervisor, like almost 30 years ago now, but we’re talking about my research, which was looking at soldiers were in chemical warfare clothing and walking into heat until they voluntarily stopped and my joke to my supervisor was, if I’m being, you know if a tank is running me down, I am going to keep going, I don’t care what my core temperature says, I’m gonna keep running and keep going until either the tank runs over me or I escaped. So it’s one of those things, is that data actionable, right then and there and I’m not sure a core temperature is. It looks great. I know when core first came out and they sponsored teams during the I think 2020 Tour de France, they were sending me files saying ‘Look at this cool data we just got from a rider’ and it was tracking their core temperatures as they were over mountain passes in a tour and as is, well that’s great, but again, like, what are you going to do about it or what are what are they going to do in terms of doing anything useful with the data? So that’s the real challenge with a lot of these measures and tools, it’s you may get super accurate data, but again, it goes back to what you said right at the outset, Rob. Is it actionable? Is it something worthwhile that we can actually do something with?
Trevor Connor 1:12:46
For both beginners and veterans polarized training is the best way to get and stay fast year after year and this is the perfect time of year to be thinking about how polarized training can help you in our new guide featuring Dr. Stephen Seiler. Explore fascinating and helpful topics like ‘how polarized training is different from sweetspot, how to bust out a performance plateaus, how to polarize all season, how to build durability and how to time your high intensity work’. With the complete guide from past talk labs, you’ll have everything you need to polarize your training like a pro and unlock your elite. Learn more at fasttalklabs.com.
Takeaways
Rob Pickels 1:13:24
Dr. Cheung. You’re bringing in this really good high level advice. If you take all of your learnings as a thermal physiologist, as a cyclist, as a guy who loves gadgets, just like me. What’s what’s your take home advice for listeners as they come out of this episode?
Dr. Stephen Cheung 1:13:40
Well again, as a scientist, I absolutely love data and I’d love to see what my body is doing. That’s what got me into this field to begin with, but I think for athletes and coaches, as they’re assessing the utility of these different tools, you first have to question the accuracy and reliability, that we talked about at the start. You have to have at least some understanding of how these devices are giving you the data, whether that is how they’re actually measuring the variable, how they are being processed and also what it means in terms of what they are telling you. So you have to have some trust in that and I think one of the challenges right now, a lot of these units, you don’t really know where they’re coming from. So how would I then deal with it? I would take each of these measures with a grain of salt. Really try to understand what they’re doing and then really track it to your own experiences. Don’t become a slave to that data. Don’t become too fixated for it. Always understand and try to get a sense of how do I feel when this unit is telling me X? How do I respond when this unit is telling me Y? So really try to calibrate your own sensation so that you have a better understanding of the tool and also how it responds to you rather than just a big group average. So I think there’s utility in them, but I think we’re really in an early stage right now and I think there’s potential for a lot of these tools coming up, but it never replaces you actually understanding how you can use it the best and relating it to your own situation and experience.
Rob Pickels 1:15:28
I’ll jump in next and Trevor, I’ll let you have the final word to close out the episode, but Dr. Cheung, I think in general, yeah I 100% agree with you. For me, this is a conversation that’s around that of like marginal gains and Trevor, as you brought up very early in this episode, we know that measures like heart rate and power they work, they’re accurate, they’re valid, we know what to do with them and that information gets you 90% of the way there. The vast majority of the information that you need is contained right there. An internal measure of your workload and an external measure of your workload. Now, there are people who like to be early adopters who like to be on the tip of the spear, who love to adopt these new technologies and to them, I’ll say, ‘Sure, go for it’, but as you just said, Dr. Cheung perfectly, don’t be a slave to that data because we don’t know exactly what the actuals are coming out of it, but if you’re interested in that device and you got the money to spend, then go ahead and get it and use it and try to understand where it works and where it doesn’t, but be very deep in your thinking and your analysis of it and don’t just take this as a superficial understanding. Don’t just take that number that it puts in front of you and say that’s good or bad and assign value to it, but very much always have an open mind. I do think the technology, the stuff that we’re talking about today that we’re like, oh my god, it’s on the forefront, we don’t quite understand it yet, give it five years, we might understand it perfectly and we might need to have a different conversation because it is the gold standard in metric at this point, but at that point, we’ll be talking about just the new thing and we’ll say we don’t understand this new thing. That’s the constant evolution. So, I did think coming out of this episode because I do love gadgets, I do love devices, that we are going to be ‘thought this is great and that is great and whatnot’ and I know that it’s a little bit negative, but I think that that conservative view is probably the worthwhile view that’s appropriate for most athletes that are out there. Stick to the things that are going to get you the biggest bang for your buck and don’t really worry about those things in the margins.
Trevor Connor 1:17:34
I think my take home is very similar, but mine is about speed and that is my concern. It could very well be that these ways of measuring heart rate variability, these ways of measuring oxygen saturation, body core temperature, all these things. 10, 15 years from now might be absolutely amazing, highly accurate, just like heart rate is now and power is now highly accurate and we know how to use them. My concern is the speed with which they’re being introduced and the speed with which they’re saying we know how to use this information and building these algorithms and the concern is right now the technology has not been perfected. So, some of the measurements at night might not be great. You might not be getting good data and the algorithms which they’re not validating, they’re not doing scientific studies on that are public. You don’t know how good those algorithms are. So right now we might be getting bad on top of bad and the ultimate information you’re getting might as I said, be worse than no information at all. So to me, that’s the thing to be careful of. All these things unfortunately, there’s just a lifecycle to them and they need to be tested. They need to go through proper third party scientific validation studies to make sure that they’re saying what they’re saying and we’re not there yet with many of them. So I would just caution people to be careful with them.
Rob Pickels 1:18:58
Awesome, guys. Well, hey, thanks for getting together and having this conversation. I think that I got a lot of value out of it and I might look at this analog watch on my wrist and be happy that it’s there.
Trevor Connor 1:19:11
Dr. Cheung real pleasure having you on the show.
Rob Pickels 1:19:14
Always great to be here. That was another episode of Fast Talk. Subscribe to Fast Talk wherever you prefer to find your favorite podcast. Be sure to leave us a rating and a review. The thoughts and opinions expressed on Fast Talk are those of the individual. As always, we love your feedback. Tweet at us at @fasttalklabs or join the conversation at forums.fasttalklabs.com. Learn from our experts at fasttalklabs.com or help keep us independent by supporting us on Patreon. For Trevor Connor, I’m Rob pickles. Thanks for listening.
References
Bowler, A.-L. M., Whitfield, J., Marshall, L., Coffey, V. G., Burke, L. M., & Cox, G. R. (2023). The Use of Continuous Glucose Monitors in Sport: Possible Applications and Considerations. International Journal of Sport Nutrition and Exercise Metabolism, 33(2), 121–132. Retrieved from https://doi.org/10.1123/ijsnem.2022-0139
Cardinale, M., & Varley, M. C. (2017). Wearable Training-Monitoring Technology: Applications, Challenges, and Opportunities. International Journal of Sports Physiology and Performance, 12(s2), S2-55-S2-62. Retrieved from https://doi.org/10.1123/ijspp.2016-0423
Fuller, D., Colwell, E., Low, J., Orychock, K., Tobin, M. A., Simango, B., … Taylor, N. G. A. (2020). Reliability and Validity of Commercially Available Wearable Devices for Measuring Steps, Energy Expenditure, and Heart Rate: Systematic Review. JMIR MHealth and UHealth, 8(9), e18694. Retrieved from https://doi.org/10.2196/18694
Ganglberger, W., Bucklin, A. A., Tesh, R. A., Cardoso, M. D. S., Sun, H., Leone, M. J., … Westover, M. B. (2022). Sleep apnea and respiratory anomaly detection from a wearable band and oxygen saturation. Sleep and Breathing, 26(3), 1033–1044. Retrieved from https://doi.org/10.1007/s11325-021-02465-2
Halson, S. L., Peake, J. M., & Sullivan, J. P. (2016). Wearable Technology for Athletes: Information Overload and Pseudoscience? International Journal of Sports Physiology and Performance, 11(6), 705–706. Retrieved from https://doi.org/10.1123/ijspp.2016-0486
Henriksen, A., Mikalsen, M. H., Woldaregay, A. Z., Muzny, M., Hartvigsen, G., Hopstock, L. A., & Grimsgaard, S. (2018). Using Fitness Trackers and Smartwatches to Measure Physical Activity in Research: Analysis of Consumer Wrist-Worn Wearables. Journal of Medical Internet Research, 20(3), e110. Retrieved from https://doi.org/10.2196/jmir.9157
Holzer, R., Bloch, W., & Brinkmann, C. (2022). Continuous Glucose Monitoring in Healthy Adults—Possible Applications in Health Care, Wellness, and Sports. Sensors (Basel, Switzerland), 22(5), 2030. Retrieved from https://doi.org/10.3390/s22052030
Pevnick, J. M., Birkeland, K., Zimmer, R., Elad, Y., & Kedan, I. (2018). Wearable technology for cardiology: An update and framework for the future. Trends in Cardiovascular Medicine, 28(2), 144–150. Retrieved from https://doi.org/10.1016/j.tcm.2017.08.003
Schiefer, L. M., Treff, G., Treff, F., Schmidt, P., Schäfer, L., Niebauer, J., … Sareban, M. (2021). Validity of Peripheral Oxygen Saturation Measurements with the Garmin Fēnix® 5X Plus Wearable Device at 4559 m. Sensors (Basel, Switzerland), 21(19), 6363. Retrieved from https://doi.org/10.3390/s21196363
Shei, R.-J., Holder, I. G., Oumsang, A. S., Paris, B. A., & Paris, H. L. (2022). Wearable activity trackers–advanced technology or advanced marketing? European Journal of Applied Physiology, 122(9), 1975–1990. Retrieved from https://doi.org/10.1007/s00421-022-04951-1
Vijayan, V., Connolly, J. P., Condell, J., McKelvey, N., & Gardiner, P. (2021). Review of Wearable Devices and Data Collection Considerations for Connected Health. Sensors (Basel, Switzerland), 21(16), 5589. Retrieved from https://doi.org/10.3390/s21165589
Xie, J., Wen, D., Liang, L., Jia, Y., Gao, L., & Lei, J. (2018). Evaluating the Validity of Current Mainstream Wearable Devices in Fitness Tracking Under Various Physical Activities: Comparative Study. JMIR MHealth and UHealth, 6(4), e94. Retrieved from https://doi.org/10.2196/mhealth.9754
Ye, S., Feng, S., Huang, L., & Bian, S. (2020). Recent Progress in Wearable Biosensors: From Healthcare Monitoring to Sports Analytics. Biosensors, 10(12), 205. Retrieved from https://doi.org/10.3390/bios10120205
