Feds Say E.R. Doctors Kill With Incompetence

The government's science is appallingly bad, but media embrace it. Sound familiar?

Ten days before Christmas, a federal government agency dropped a massive, 744-page report alleging that E.R. doctors are making errors due to “inadequate clinical knowledge, skills, or reasoning” — that we are getting the diagnosis wrong in every 18th patient — and that this leads to a quarter of a million avoidable deaths of American citizens every year.

“As many as 250,000 people die every year because they are misdiagnosed in the emergency room, with doctors failing to identify serious medical conditions like stroke, sepsis and pneumonia,” reported The New York Times.

CNN piled on as well, citing the federal report to say that 2.6 million people — a huge number, more than the entire population of Nebraska or New Mexico! — suffer a preventable harm every year at the hands of bumbling E.R. docs.

The New York Times asserted a race/gender aspect to it all: “[T]he researchers also found that women and people of color had a roughly 20 to 30 percent higher risk of being misdiagnosed.”

Actually, that’s not what the federal researchers said. They state explicitly that after reviewing multiple studies, they couldn’t draw a clear conclusion about the relevance of race or gender. They phrased this clumsily — but The New York Times could have read it more carefully. The report actually says:

Female sex and non-White race were often associated with important (20–30%) increases in misdiagnosis risk; although these disparities were inconsistently demonstrated across studies, being a woman or a racial or ethnic minority was generally not found to be “protective” against misdiagnosis (i.e., was neutral at best).

Federal government says: Some studies show women and people of color get misdiagnosed more often, but in our large-scale literature review we couldn’t confirm that.

Journalist reads this and then writes: Women and people of color get misdiagnosed more.

For its part, CNN paired the government report’s allegations with:

• an emotional op-ed by reporter Jake Tapper’s 15-year-old daughter, describing a miserable-sounding delayed appendicitis diagnosis, and

• the most tone-deaf quote they could find in an otherwise-eloquent letter by medical professional societies who responded angrily to this report. (The quote CNN selected says that we E.R. doctors are “less concerned with diagnosis,” because we’re doing other stuff).

So the E.R.s are killing people. But CNN, The New York Times and the Feds are on the case. Surely such esteemed authorities couldn’t all be wrong about such a huge medical science question?

After all, it’s not a small allegation. The death and suffering supposedly unleashed by emergency departments — millions of deaths and tens of millions of harms over the past 20 years — would rival in scale that of U.S. foreign wars.

(Calculate it out over a 20-year time frame: The emergency departments stand accused of 5 million deaths and 52 million harms. By comparison, in combat zones from Iraq to Afghanistan over a similar time frame, Brown University estimates nearly 1 million deaths by violence and “several” million more due to war-associated famine, bad sanitation, etc.; and also 38 million harms in the form of losing one’s home.)

In short, when a patient comes to the E.R. for nagging chest pressure, they’re about as safe as if they were fighting house-to-house for the streets of Fallujah.

And yet … is any of it even true?

The more one digs into this report, the weirder it all becomes.

Feds to the Rescue

The report is called “Diagnostic Errors in the Emergency Department: A Systematic Review,” and it was commissioned by the Agency for Healthcare Research and Quality (AHRQ), which describes itself as “the lead Federal agency charged with improving the safety and quality of healthcare for all Americans.”

It turns out AHRQ shared early drafts of its report with emergency medicine professional societies, who were aghast — because it truly is shockingly bad science. A behind-the-scenes lobbying effort was launched to convince the agency to send the report back for revision.

Yet the Feds plowed rapidly ahead and published. Why the rush?

Well, this was just days before the U.S. Congress finally approved the massive $1.7 trillion, 4,000-plus-page Omnibus spending bill. Among other things that bill doubled — from $10 million to $20 million — the AHRQ’s budget for research specifically into diagnostic errors.

As always in medicine, the incentives are many and overlapping. There are millions of dollars in play; but there is also the passion of people on a mission to improve medical care, people who may well believe their own calculation that a typical small E.R. will witness 50 preventable deaths-by-misdiagnosis every year.

That’s a death per week! And that’s for a small hospital. By this math, avoidable deaths at the larger E.R.s where I work would be an every-other-day occurrence. Now it’s starting to feel like a COVID-19-level emergency!

CNN says the authors of the federal report “reviewed nearly 300 studies” to come up with this indictment. That sounds scientifically powerful.

But CNN and others should have read the fine print — or at least bullet point #2 in the abstract! — which states:

“[O]verall error and harm rates are derived from three smaller studies conducted outside the United States (in Canada, Spain, and Switzerland, with combined [total number of patients studied] = 1,758).”

That’s right. The claim that America’s E.R. docs are misdiagnosing every 18th patient, and that in our larger emergency departments we are killing a patient every other day — that’s based on three small studies from other countries.

Let’s look at those three studies.

Two of them were used to calculate a U.S. misdiagnosis error rate. Each involved a single hospital, one in Spain and the other in Switzerland. Neither country at that time had emergency residency-trained physicians. (Switzerland still does not have emergency medicine as a discrete medical specialty; Spain brought emergency residencies on-line in 2011).

The Spanish study was a look at 250 bounce-backs (patients discharged but who return within 72 hours, often because something was missed) to a hospital in the Canary Islands. The authors of that small, somewhat odd study reported that the ED physicians there would see three patients per hour, which is “within the international standards” — actually, that is pretty crazily busy! Oh and by the way, the data was collected “during the [three-month] period of the year with the highest patient census.” The authors note that picking the three months when your hospital gets smashed will, of course, rig the year-long estimate against your docs.

Using the magic of comparing 250 bounce-backs to 250 controls, the authors calculate a diagnostic error rate — at a Canary Islands hospital, in 2004 — of 4.1 percent. How did they calculate it? Well, there are Kruskal-Wallis non-parametric comparisons, there is linear regression, there are Kendall tau-b linear rank correlation coefficients. Really the only thing missing is Vinz Clortho, Keymaster of Gozer, wearing an EEG made out of a spaghetti colander and telling us that “During the rectification of the Vuldronaii the Traveler came as a very large and moving Torb!”

So, yes: A 4.1 percent miss rate, calculated magically, during the worst time of the year in a single Canary Islands E.R., where clinicians without emergency medicine training cranked through three patients per hour, 19 years ago.

The Swiss study is newer, published in 2019, and has all sorts of bells and whistles: they are surveying physicians about their training (many do have formal emergency medicine training, although there is not yet a residency), they are calculating ED crowding scores, they even have decibel-measuring devices at physician workstations. They looked at 755 consecutive admitted patients, and to quote the abstract, “The discharge diagnosis differed substantially from the admittance diagnosis in 12.3 percent of cases.”

Interestingly, the Swiss authors find only one thing that predicts this looming diagnostic discrepancy. It’s not the level of training of the provider, not the overcrowding, not high noise-level days at work. It’s when the emergency physician reports upon admission that he or she is not sure what is going on with this patient.

Is that really an error? The Swiss authors, in arguing a separate point — that they do think the inpatient discharge diagnosis can be relied upon as the more correct of the two — note that internists have days with the patient, not hours like the E.R. doctor. That’s more time to figure things out, order additional tests, get consults. Again, are we describing errors in this Swiss study, or an ongoing collaborative process?

But never mind all that. There’s an Omnibus spending bill lumbering through Congress, growing fatter by the hour, and there’s $10 million extra dollars in play if the stars align. Let’s get this U.S. government report into some headlines!

So, once again through the magic of statistics, “Diagnostic Errors” takes this 12.3% discrepancy in Switzerland between the E.R. admitting diagnosis and the internal medicine discharge diagnosis; combines it with 250 bounce-backs and 250 controls from the Canary Islands somehow being read as a 4.1% misdiagnosis rate; and then calculates that the U.S. emergency physician is making a misdiagnosis 5.7% of the time.

It sure sounds like voodoo. Or as U.S. medical societies put it in their rage-letter: “We strongly believe that it is scientifically invalid.”

What about that death toll? A quarter of a million U.S. dead, every year?

That is apparently extrapolated from the third study, of 503 patients seen in the high-acuity side of two Canadian emergency departments, in which a single patient died.

That’s right: This one missed aortic dissection divided by 503 patients equals 0.199%. This is then taken south to the United States, multiplied by 130 million U.S. emergency department visits, which gives us “more than 250,000 deaths” associated with a missed diagnosis. (It’s actually 258,964. I’m surprised the study authors gave us a pass on those extra 8,964 hypothetical shameful failures).

It’s pretty incredible to make headline assertions that U.S. physicians kill more than 250,000 people every year with misdiagnosis — based on one death that happened in 2004 in a hospital in Ottawa.

“Highly misleading,” fumed our professional societies, “if not outright unconscionable.”

The Missing Context

The Canadian study enrolled its patients prospectively, and followed up on outcomes either by telephone in two weeks (if discharged), or by review of inpatient charts (if admitted).

Interestingly, the Canadians only identified 10 diagnostic errors total in 503 cases. That’s a diagnostic error rate of 2% — among both discharged and admitted patients, and arrived at by a solid, straightforward methodology (and, in the highest acuity side of the emergency department, which the authors note they intentionally focused on because it was higher risk for adverse events and errors).

A 2% miss rate among high-acuity cases sounds plausible (and worth working to improve). Yet the U.S. government’s report looked at a sliver of data out of Switzerland and another out of the Canary Islands, and magically came up with a 5.7% miss rate in the United States for all emergency department visits — including the never-ending river of low-acuity cases, the stuffy noses, ankle sprains and work note requests. Again, it seems like a massive over-estimate of the problem. But to believe that, you’d have to believe that CNN, The New York Times and the federal government could somehow all be wrong about something together.

Yet whether it’s 2% or 5.7% begs a larger question: Just because a diagnosis was missed and a person died does not mean the person could have been saved. For example, it’s by no means certain that timely identification will save the life of an aortic dissection. It certainly gives that person a better chance! But even when everything goes perfect, an aortic dissection often dies in our care.

Hypothetically, if all 250,000 of our supposed annual misdiagnosis-associated deaths had been aortic dissections — how many would we have saved if we’d gotten the diagnosis up front?

The literature suggests about ¼ of aortic dissections who undergo an operation don’t survive the hospitalization. If that’s so, then now you’re looking at only 187,500 avoidable deaths.

Does that matter? In the world of big research dollars and public health interventions, probably yes. When emergency department misdiagnoses are killing 250,000 people a year, that makes it the 4th leading cause of death in America, after heart disease, cancer and COVID-19. If it’s only 187,500 avoidable deaths, it moves into 5th place, behind trauma and accidents. It’s probably a lot lower than that, too — we don’t know — we don’t have a lot of great science.

In the real world, the opportunities to improve outcomes look a lot less impressive than “just make the right diagnosis.” Many problems can’t really be fixed, so what we do in medicine is simply comfort, assist, provide guidance about what happens next, and bear witness.

The Final Insult

In the middle of raking the emergency departments over the coals, the authors of “Diagnostic Errors” state in an aside that the error rates they believe they have found are “comparable to those seen in primary care and hospital inpatient care.”

Wow, did that message get buried in the press coverage!

So the ED is not worse than anywhere else — it’s the same? If that were so, it would be incredible, because the emergency physicians have always been the most under-resourced, over-harassed and over-taxed of all diagnosticians. We should be doing worse at this than, say, a primary care doctor or a hospital medical team — not the same! That we actually do well at it is a testament to emergency medicine training and skill and, yes, systems built over many years: from sepsis alerts to trauma activations to getting EKGs within 5 minutes of arrival for chest pain.

I would have liked to see more discussion in “Diagnostic Errors” about those kinds of systems-level fixes. Why aren’t hospitals required to staff fully to meet the needs of their communities? Why aren’t they federally incentivized to build surge capacity, for the utterly predictable “crisis” of an influenza-and-RSV season known as “winter”? How can the computer be leveraged to truly help a doctor, instead of just harassing him or her?

“Diagnostic Errors” goes in a different direction: It targets the physician.

“ED diagnostic errors were mostly cognitive errors linked to the process of bedside diagnosis,” the report claims. As evidence, it cites, as mentioned above, malpractice claims.

This is an appallingly non-scientific source. It’s basically a collection of the most emotionally-affecting, gut-wrenching accusations that can be made about some poor patient’s bad outcome. A jury needs a villain — the lawyers target the physician — the whole point of the exercise is to show that this was someone’s fault. So it’s truly bizarre to see allegations by lawyers cited as actual scientific data of some kind:

“Although errors were often multifactorial, 89 percent … of diagnostic error malpractice claims involved failures of clinical decision making or judgment … Most often these were attributed to inadequate clinical knowledge, skills, or reasoning.”

Ah, I see. It’s my fault. I’m deficient in knowledge, skills and reasoning. It’s nice to know that the government has my back.