Sleep tracker validation study: 8 wearables vs. PSG

In our commentary on the criticism of sleep trackers we made a claim: the criticism of many devices is justified, but a wearable is not a wearable. This article presents the study behind that claim.

Search for a sleep tracker review and you will mostly find impressions: how a ring feels, how long a battery lasts, how the app looks. What is usually missing is the question that decides whether the data means anything: how close does the device get to what a sleep laboratory measures? That is exactly what a sleep tracker validation study at the University of Salzburg (Topalidis et al., 2025, preprint) set out to answer, with eight systems, 18 participants, five nights each and polysomnography (PSG) as the reference.

This article walks through how the study was designed, what it found, where its limits lie, and what the results mean for users and for partners such as health insurers or sensor manufacturers. sleep² was one of the eight systems tested. We publish the numbers in full, including the ones that are not flattering.

Why polysomnography is the only meaningful benchmark

Polysomnography is the gold standard of sleep medicine. It records brain activity (EEG), eye movements (EOG), muscle tone (EMG) and heart activity (ECG), and every 30-second epoch is assigned to a sleep stage according to the rules of the American Academy of Sleep Medicine. Even this standard has a ceiling: two experienced scorers agree on roughly 88 % of epochs from the same recordings. No device can meaningfully claim more.

The rule that follows is simple, and it applies to any sleep tracker validation study worth the name: an accuracy figure is only credible if it was measured against PSG, epoch by epoch, in the same night. Comparing two trackers with each other or quoting a proprietary sleep score says nothing about how well a device captures actual sleep. More on this in our article on sleep analysis with sleep trackers.

The study design: five nights at home, with disturbance conditions

The Salzburg team brought the PSG to the participants. 18 participants each slept five consecutive nights in their own bed, wired to an ambulatory polysomnography system and wearing the tested devices in parallel. This produced around 90 recorded home PSG nights and over 80,000 30-second epochs (Topalidis et al., 2025, preprint, osf.io/preprints/psyarxiv/27wun_v1; updated version on Research Square).

The five nights were not all alike. After an adaptation night and a normal night came a night with two disturbance conditions at once: participants were woken at 3 a.m. and stayed awake for 20 minutes, ten of them lying still in bed and ten on their feet; in addition, sleep that night was restricted to 20 % less than their habitual sleep time. Then came another normal night, to capture any rebound effect, and finally a night with time in bed extended by 20 %, in which participants were to remain in bed at rest even if they were already awake. This is the crucial difference from many manufacturer studies, which evaluate only quiet nights. Waking up during the night, sleeping too little, lying awake in bed: these are precisely the situations in which devices most often get it wrong, and precisely the ones people with sleep problems experience regularly.

Eight systems were compared: sleep² with the Polar Verity Sense (upper arm) and with the Polar H10 (chest strap), Oura Ring 3, Apple Watch Series 9, Fitbit Charge 6, Garmin (Vivoactive/Venu), WHOOP 4 and the Circul+ ring. All devices ran their regular consumer versions as of the time of the study. One detail matters for reading the results: the two Polar sensors deliver only the cardiac signal; the sleep² algorithm performs the staging. The other six devices stage sleep with their own software.

Result 1: total sleep time

For total sleep time, sleep² with the Verity Sense deviated from PSG by about 6 minutes on average, and by about 12 minutes with the H10. For context, the best consumer trackers typically land within 15 to 30 minutes. Other devices overestimated sleep by up to an hour on average. Oura Ring 3 and Garmin were close to PSG on average, but varied considerably from night to night.

An average alone hides the real pattern: for most devices, the size of the error depends on how the night went. On quiet nights the numbers are right; on nights with a lot of wake time, the overestimation grows. Someone who lies awake for half an hour and is still credited with uninterrupted sleep receives feedback that contradicts their own experience. Why sleep duration alone tells only half the story is the subject of our article on sleep quality versus sleep quantity.

Result 2: sleep stages in the sleep tracker validation study

The harder test is four-class accuracy: how often does a device assign an epoch to the same stage as PSG (wake, light, deep, REM)? The results at a glance:

The roughly 84 % achieved by sleep² (Cohen’s kappa 0.76, practically identical for both sensors) corresponds to roughly 95 % of what human scorers achieve among themselves. Among the ring and wrist trackers, the Oura Ring 3 performed best at around 73 %, just ahead of the Apple Watch Series 9 at 72.3 %, followed by WHOOP 4 at around 69 %, Fitbit Charge 6 at around 66 %, Garmin at around 63 % and Circul+ at around 56 %. For Oura, that is a respectable result and a good example of a finger-worn ring with an optical sensor delivering usable sleep data when a manufacturer invests in validation; Oura’s deviations from PSG did, however, vary more from night to night. Apple Watch and Oura also score clearly above what the literature reports for their predecessor models. At the lower end, the values approach what informed guessing would achieve with four classes.

Cohen’s kappa is the stricter metric, because it removes what a device gets right by chance alone. A value of 0.76 counts as substantial agreement, values between 0.41 and 0.60 as moderate, and anything below that as fair at best.

The honest reading: wake and deep sleep

Broken down by class, sleep² reached about 90 % for REM sleep, about 87 % for light sleep, about 79 % for deep sleep and about 74 % for wake. That order deserves two honest remarks.

First, wake is the weakest class for every system. Someone lying still but awake looks very much like light sleep to movement and heart-rate sensors. The wrist-worn devices label lying awake mostly as light sleep, which pushes total sleep time and sleep efficiency upwards. Even 74 % is a good, not a perfect, value here.

Second, deep sleep is the least reliable class for every device, sleep² included. That is not only the trackers’ fault. Deep sleep is also among the least reproducibly scored stages for human scorers, because the boundary between light and deep sleep is drawn by rule through a physiological continuum. And what a consumer device displays as “deep sleep” is a manufacturer’s own label that need not correspond to the clinical stage N3 at all. A deep sleep figure in minutes therefore deserves more scepticism than total sleep time, whichever device shows it. 

Why the differences are so large

The study suggests that three factors decide together: the signal quality of the sensor, its position on the body, and an algorithm trained against PSG. An optical sensor on the wrist works under harder conditions than a sensor on the upper arm or a chest strap with an ECG signal. And a model that has learned from thousands of PSG nights recognises patterns that a model calibrated on movement does not see. That the Circul+ sits on the finger just like the Oura Ring and still performs worst shows at the same time that placement alone is not enough; sensor and model have to fit together. A sleep tracker is not a measuring instrument by itself. It is a sensor plus a model, and both have to be checked against the reference. The spread from around 56 to 84 % within a single device category shows one thing above all: a wearable is not a wearable.

What this means for users and for partners

For users, the takeaway is this: total sleep time is reliable with validated systems, while individual wake periods and deep sleep remain the least certain quantities for every device. Trends over weeks are meaningful; the verdict on a single night is not. Anyone who ranks the tracker’s number above their own sense of how they slept risks what Baron et al. (2017) described as orthosomnia: preoccupation with the measurement makes sleep worse.

For health insurers and payers, the study is a touchstone. Whoever funds a digital sleep programme wants to know whether the underlying measurement is sound, and specifically on the bad nights that insomnia is about. Accuracy documented against PSG under disturbance conditions is the only solid basis for that. For sensor manufacturers, the study shows that a good sensor only becomes a sleep measurement device once it is paired with a validated algorithm. For research partners, over 80,000 annotated epochs form a dataset against which future algorithms can be benchmarked. More on the care perspective in our article on insomnia care for insurers.

Limits of the study

Three caveats belong to any fair reading of this sleep tracker validation study. First, the devices were tested in their consumer versions at the time of the study. Manufacturers update their algorithms continuously, and a firmware update can move the numbers, up or down. Second, 18 young, healthy adults over five nights each provide a solid basis at the epoch level, but not a cross-section of the population. For people with diagnosed sleep disorders, accuracy should be examined separately, because wake periods are more frequent there, and that is exactly where the errors described here would be largest. Third, the study was conducted at the University of Salzburg, from which sleep² emerged as a spin-off. That has to be disclosed. The same conditions and the same reference for every device make the results transparent, but do not replace independent replication. The study has appeared as a preprint and is under peer review; we will update the numbers here once it is published.