The Dec. 17th New England Journal of Medicine article, “Racial Bias in Pulse Oximetry Measurement,” is correct that Black patients have been a challenge for conventional pulse oximetry, resulting in the overestimation of arterial blood oxygen saturation.
When I started Masimo in 1989, we worked on this variable in our SET Pulse Oximeter by making sure to have an equal number of dark-skinned and light-skinned people represented in our calibration studies. However, I do not believe that the whole picture has been painted here, leaving room for some to create an inaccurate narrative with potentially lethal consequences.
For context, multiple studies over the years have examined this issue and reported a bias between -1.6% to +3.9% between different brands of pulse oximeters. A 2017 study of infants with hypoxemia comparing Masimo and Medtronic pulse oximeters found an overall bias with Black infants of 0.8% for Masimo devices and 3.9% for Medtronic devices. The authors concluded at the time that there was “no significant difference in systematic bias based on skin pigment for either oximeter.”
Our Internal Data
Given that the University of Michigan study, published as a letter to the editor in The New England Journal of Medicine, had many more subjects than the previous studies, I wondered if maybe we had not noticed the bias because the sample sizes in the previous studies were smaller.
Therefore, we did a further review of our internal data—which covers over 2,000 subjects with more than 1,000 dark-skinned people (more than the number of subjects in the Michigan study)—and found a 0.3% difference between the groups across an oxygen saturation range of 70% to 100%, and a 0.25% in the more limited pulse oximetry range the Michigan study focused on.
Applying the statistical test used in the Michigan letter to our internal data, we found a difference between dark and light skinned subjects of 1%, whereas that same test applied in the Michigan study found a 325% difference.
Although we recorded skin color for all of the subjects in our validation dataset (using the MASI pigmentation scale), most subjects did not report ethnicity. To expand our analysis, we also compared just those Black and white subjects who had reported ethnicity—394 subjects, 200 Black and 194 Caucasian, a similar ratio as in our overall dataset—and found a similar bias, 0.4%, between the two groups.
Given that our bias was so much less than that of the Michigan study, we questioned what could be the cause of the disparity between the results presented by the University of Michigan authors and all the data we had seen in the past two decades.
Hypotheses
We came up with several hypotheses, and here are some of them: Sickle-cell disease is one potential confounder. Sickle-cell trait affects nearly 10% of the Black population. Sickle-cell disease has been shown to cause significant errors between invasive CO-oximeter and noninvasive pulse oximeter measurements. If sickle-cell patients were not excluded in the Michigan study, that alone could account for most of the difference between what we have seen and what Michigan reported. The question is, did the Michigan study account for this?
An additional error source for pulse oximetry is the presence of high carboxyhemoglobin (COHb) and methemoglobin (MetHb) in the blood. The Michigan researchers briefly discussed COHb, but did not disclose if they excluded patients with high MetHb. There are over 40 drugs commonly given in hospitals that unfortunately can elevate MetHb to dangerous levels.
One of them, hydroxychloroquine, which has been recently used on COVID-19 patients, has been shown to dramatically elevate MetHb in Black patients. MetHb not only causes huge errors in pulse oximetry, including biasing pulse oximetry readings, but also can kill the patient if it’s not detected and treated immediately. Did the Michigan study account for this?
Another confounder could be tissue damage and poor circulation, which afflicts Black people more than any other racial or ethnic group and can also negatively affect the accuracy of the pulse oximetry readings. So again, did the Michigan study account for this?
The Brand’s Reputation
In addition, to speak of pulse oximeters as though they are all the same in accuracy and reliability is wrong. While the University of Michigan has been our customer for many years, and we must assume the pulse oximeters they used in their study are ours, we are not certain that they were.
It is important that researchers report the brand and version of the pulse oximeters and sensors they used. For example, the data as shown in the first figure in the Michigan study exhibit a very large spread, which is inconsistent with many independent peer-reviewed studies of Masimo SET pulse oximeter accuracy over the past decades. These inaccuracies are usually associated with conventional pulse oximeters, or worse, the cheap finger-clip pulse oximeters that are sold at local pharmacies.
Another confounder is the unacceptably large delay between pulse oximetry readings and invasive blood sampling, during which a patient’s oxygen saturation may be changing. In our internal studies, care is taken to record pulse oximetry measurements simultaneously with invasive blood samples.
While I do not know if these potential sources of error are the confounders that created such a large bias between Black and white patients in the Michigan study, I do believe that the Michigan study should prompt further investigations, with the goal of removing systematic sources of error from the data collection to uncover any true source of pulse oximetry bias.
We March for Black Lives
What these publications did is regretful:
• With very little explanation and underlying data, the Michigan authors sent in their findings.
• The New England Journal of Medicine published their findings seemingly without asking for the kind of data that you’d expect in a scientific journal.
• The Boston Review and the New York Times rushed to give the purported bias in a pulse oximeter a racist narrative.
My family and I, along with our friends and colleagues, marched for Black Lives Matter. We seek opportunities to stand up for justice and peace everywhere we can. We join every other citizen who understands the brilliance of action out of kindness.
When it comes to developing products and running our company, we stick with data and science done by the best people, no matter their beliefs, race, sex, or sexual orientation.
In the pursuit of science and patient safety, we will further test our hypotheses about the source of the high pulse oximetry bias on black patients in the Michigan study, and will report our findings in the near future.
We have been in touch with the Michigan researchers and hope to work with them as well any other committed researchers toward helping to ensure the health and safety of ALL humankind.
Editor’s Note: Irvine-based
