Categories
Uncategorized

False Precision

In a May 8th Strib op-ed (“Minnesota must recover from its pandemic of fear”), Katherine Kersten asserts: “99.24% of [Minnesota’s COVID-19] deaths involve[e] nursing home residents or people with underlying medical conditions.”

Two days later, the STRIB in an unsigned editorial (“Minnesotans need to understand and act on COVID-19 risks”) used the same precise statistic to make a slightly different assertion: “Of all those who have died from the mysterious viral illness statewide, 99.24% had an underlying health condition.”

On May 12th, the STRIB ran another unsigned editorial (“Making sense of COVID-19 fatality rates”) citing yet again the same precise statistic: “So far in Minnesota, about 4 of every 5 fatalities have occurred in nursing homes or assisted-living facilities, and 99.24% of those who have died had an underlying health condition (as discussed in an editorial Sunday).”

Aside for the obvious inconsistency in their uses (unless the editorial board considers residence in a nursing home to be “an underlying health condition” or Kersten was being redundant in that regard), using a percentage carried to the hundreds place in this context seems laughably ignorant of the underlying data. It implies that the user knows the precise number of Minnesotans who meet at least two of three conditions: (1) they died from COVID-19, and (2) were a resident of nursing home or (3) had an underlying medical condition.  That follows because Minnesota has had fewer than 500 deaths (when the columns were published) and because deaths must be whole numbers (not to put too fine a point on it, but a person is either dead or not – no fractions here).  Thus, hitting a precise percentage like 99.24% must mean that there were 3 such COVID deaths out of a total of 395 (the May 2nd number reported by MDH: 392/395 = 99.24%). For any of MDH’s reported death totals after May 2nd (through May 12th), one cannot derive 99.24% using an integer for deaths.

Of course, each day the number changes as more people die. (Rounding to something reasonable and it might not, but that could soften the rhetorical point of how really, really small the number is.) The bigger issue is that the underlying data is inherently imprecise; undoubtedly individuals are dying of COVID-19 who do not appear in the MDH counts because they died at home and/or were not tested. It has been widely recognized that current COVID death data is subject to substantial undercounts. For example, see this AP report from April 30th that reports 66,000 excess deaths, many/some of must be due to unreported COVID-19.  Moreover, it is also unclear how reliable MDH’s data on residence status and underlying health conditions is – in fact, MDH admits (listing “unknown/missing” category for residence data) that it doesn’t always have data on the residence status for all decedents, the seemingly easier of the two to verify.

Bottom line: The Strib should be ashamed of publishing numbers like that in its editorials; it evidenced either carelessness or a lack of understanding of the imprecision of the data and how research statistics work.  False precision can be inadvertent, but often implies a desire to mislead. Allowing contributors like Kersten to do so also seems negligent to me.

Kersten piece is a separate case. I think a fair (probably charitable) characterization is that it is an intemperate effort to advocate for the latest right-wing hobby horse (keeping the economy semi-shut down for public health reasons is foolish) and evidences an extreme degree of confidence in its conclusion, while failing even superficially to satisfy the standards she sets out for reaching such a conclusion.

Its intemperate nature is obvious from a selection of her language characterizing Governor Walz’s actions to limit, the media coverage of, or the public’s perceptions of the risks of COVID-19 and SARS-CoV-2:

  • “coronavirus hysteria”
  • “apocalyptic scenario”
  • “irrational panic”
  • “frenzied, overblown ‘body count’ headlines”
  • “herded into a massive new regime of political control over the details of ordinary life”
  • “pummeled by apocalyptic propaganda”
  • etc.

Whew!

It fails by its own terms. Kersten correctly notes that the government officials have a “duty to responsibly balance the risks of COVID-19 with the shutdown’s * * * costs” and the appropriately way to do is with “objective, data-based cost/benefit analysis that is indispensable to responsible crisis management.” She damns the Walz administration for failing to do that (to be fair she only says there is little evidence that they did).

I will not defend the Walz administration’s efforts in that regard since I am not competent to do so.  That would require combined expertise in epidemiology and economics; I have neither.  However, I would observe that the administration does appear to be making concerted and regular efforts to measure and weigh risks and benefits. They have models and are regularly receiving advice based on analysis of evidence by experts in the relevant fields.

That is certainly more than can be said for Kersten. She obviously disagrees with the administration’s models and experts. But she provides virtually no evidence why beyond two data points – the 99.24% (most people who die live in nursing homes or have some medical condition) and only a very small percentage of younger New Yorkers have died (again carried out to the hundreds of a percentage point – but at least New York has 20,000+ COVID-19 deaths!).

With regard to doing a cost-benefit analysis (as she says is indispensable), her piece provides no evidence that she is relying on such an analysis. If one has been done that provide the basis for her confident assertions, she makes no reference to it. Rather, we are left with simply trusting on her conclusory statement – no supporting evidence beyond the fact that almost all the Minnesotans who die live in nursing homes or have some underlying health conditions and that the economic cost is obviously high. (Note, as I have pointed out, Minnesota is an outlier in that regard.) I guess that is enough for her. To me it is not even close to “an objective, data-based cost/benefit analysis” that she says is indispensable. On that I agree with her.

Where is her or the Center of the American Experiment’s (CAE, Kersten’s employer) model and projections? What R0, R, CFR, IFR, and so forth is her model using? How many more people does she think will die if the shutdown ends (as she says it “must”)? How many more will become gravely ill but recover? How many will suffer organ damage as a result? How much will medical costs increase as a result, including those paid by the public? What are her assumptions about the values of the lives that will be lost? Is she discounting them because they typically are old or have high blood pressure, are obese, etc.? What is she assuming for the values of the hours of lost work (much less pain and suffering) for the individuals who will fall sick but not die as a result of ending the shutdown? There are many more questions (especially if someone who is actually knowledgeable starts asking) – for which there is no evidence that she or anyone else in her organization have carefully tried to analyze (using evidence and credible models) and answer. I guess we need to take it on faith, Kersten’s faith, for whatever that is worth. That is why her piece fails by the standard that she sets out. It appears to me to be faith-based, not evidence-based.

I would observe that CAE appears to have a lot of resources. (Its 2018 Form 990 shows nearly $4 million in revenue.) If they want to make a useful contribution, they could use some of that money to hire reputable researchers (epidemiologists in this case), rather than just lawyers, wordsmiths, rhetoricians, and similar to advocate for positions that largely appear to be based on their priors, rather than evidence and analysis.

Leave a comment

Design a site like this with WordPress.com
Get started