Something extremely odd happened this afternoon. The press, including the New York Times, the Washington Post, CNN, the Wall Street Journal, USA Today, and Axios, reported that the CDC had announced that 73% of US COVID-19 cases were now being caused by the Omicron variant, and major public health gurus like Ashish Jha, Eric Topol, Florian Krammer, Eric Feigl-Ding, and Scott Gottlieb circulated the reports at face value.
But that’s not really what’s happening. There was some kind of bug in an automated statistical algorithm at the CDC, known as the variant proportions Nowcast. Omicron isn’t really dominant in most of the USA yet (although all evidence continues to indicate it will be soon). And the bug in Nowcast went from a quiet database update on a quiet CDC web page, to a national press sensation, in a matter of hours.
What actually happened?
No humans at the CDC stood up in front of a podium and announced the 73% number, or wrote a press release. It came from the CDC’s automated COVID data tracker website.
But it’s also not a statement of hard data, it’s the output of an algorithm called the Nowcast.
The Nowcast algorithm is based on the CDC’s variant proportions data. Every week a small percentage of US COVID-19 cases are subjected to genome sequencing, which identifies them by strain. Right now it’s about 3.6%, but it varies by state. The CDC collates this data to produce weekly reports on the prevalence of each variant in each of ten regions of the continental USA. However, the genomic data takes time to generate and process, then get deposited into genomics databanks, then get analyzed by the CDC, so the newest data is about two weeks old at each weekly refresh (today, sixteen days old; the week ending December 4).
To display an estimate of current prevalence, the CDC uses an algorithm it calls a “Nowcast,” to project forward from two weeks ago and give an estimate of current prevalence for different strains. The Nowcast algorithm has been remarkably accurate and useful in the past; I followed it with great interest as Alpha, Gamma, Delta, Mu, and other strains washed across the USA from early 2021 until Delta attained total dominance in the summer. It’s a significant achievement for the CDC, and I celebrate them for it.
But something is on the fritz now. In this week’s update, which dropped Monday afternoon, the Nowcast algorithm is presenting results that don’t make sense.
How we know the Nowcast is wrong in this case
All the press coverage has focused solely on the top level, nationwide number. But digging into the regional and time series numbers, and comparing them with the hard data from two weeks ago, the national and regional case numbers, and what we know about Omicron’s rate of growth from other contexts, shows clearly that something is fishy with this update of the Nowcast.
From the top down, we can look at the national numbers. The last two weeks of data say that Omicron made up 0.0% of cases sampled (regionally adjusted) in the week ending November 20, .1 in the week ending on November 27, and .7% in the week ending December 4. Then, the Nowcast says these numbers shot up to 12.6% in the week ending December 11, and 73.2% in the week ending December 18.
That is, the Nowcast is not saying Omicron is making up 73% of US COVID-19 cases now. It’s saying that Omicron was already making up 73.4% of cases during the average day in a week that began all the way back on December 12. And that it escalated from .7% to 12.6% in one week, and from 12.6% to 73.2% in another week. Assuming, as data has indicated, that Delta cases stayed approximately constant during this time, these numbers correspond to Omicron case numbers growing approximately twenty fold in one week, a doubling time of approximately thirty six hours, sustained over a two week period.
This is out of touch with what we know about Omicron. Responsible estimates of the growth rates of Omicron in populations with better data (like the UK and Denmark) and prior data on the USA, have suggested doubling times in the range of two to four days. This has caused us (and the real experts) to characterize its rate of growth as “explosive,” and to say in late November and early December that it might attain dominance in the USA in a matter of weeks. And that’s all true. But a doubling time of 36 hours, sustained for over two weeks, is out of step with all the other data. It’s not credible.
And a quick look at the regional numbers shows that something truly crazy is going on. Region 10 is composed of Oregon, Washington, and Idaho. The last week of data (ending December 4) shows a 0.0% prevalence for Omicron there, but the latest Nowcast says that jumped to 43.4% in the week ending December 11 and 96.3% in the week ending December 18. Region 10 has the lowest numbers in the actual data, but the highest numbers in the Nowcast. When the data say 0.0% in the latest week, what algorithm could accurately extrapolate 43.4% the following week?
And this points to another big problem. These numbers don’t comport with the national case numbers and what we know about how variants interact. While huge case surges have been seen in the states in Region 2, which was already 2.4% Omicron in the latest measurements (the week ending December 4), as well as other Omicron-dominant areas like the UK, they haven’t been seen in most states elsewhere in the country. This pattern was also seen as Delta took over from Alpha in the USA this summer, but we’re not seeing it in most of the USA now.
Until the last few days, national case numbers were flat, and have since grown by only a few percent, as a result of explosive growth in just a few states. If Omicron were already nationally dominant last week, this would imply that Delta happened to nationally completely implode (with R suddenly dropping to near zero) at the exact same moment and exact same speed that Omicron exploded. And that Omicron’s explosion suddenly slowed down at the moment Delta cases hit (near) zero, since we have not seen an explosive increase in national case numbers yet. Region 10, which supposedly saw the fastest growth and highest prevalence of Omicron, has even had decreasing numbers over this period.
But in reality, none of this is true. The CDC’s Nowcast algorithm has some kind of bug, and the prevalence of Omicron nationwide wasn’t really 73% in the week ending December 18. Take that to the bank.
What does this mean? What will happen now? What can we do about this?
A few things to take away from this:
- You will see this walked back. The CDC will explain what happened. The Twitter experts will have to walk back their initial tweets. The press will have to publish a new round of articles clarifying their prior wave of headlines. It might happen tomorrow, later this week, or some time next week, but it will happen. This result won’t stand.
- None of this should be taken to impugn the CDC. They didn’t announce this. It was an automated algorithm, and one that had a good record prior to this. But the emergence of Omicron has reset the speed of events back to “fast,” and the modern infrastructure of COVID-19 genetic surveillance, including the Nowcast algorithm, hasn’t yet been vetted in a period of extremely rapid exponential growth like December 2019-March 2020, and now. It was inevitable that bugs would come up, and this one just happens to have suddenly turned into a media sensation after hours.
- The press missed the boat on this one because they’re not keeping track of the details. The few outlets with a record of reporting on variant prevalence data in depth, like Science magazine, Endpoints, and the MIT Technology Review (along with yours truly) haven’t run this story (that I know of). Although, disappointingly, Stat News has run it, and so has the Financial Times. The outlets that are running it are probably encountering the Nowcast for the first time, don’t really understand it, and didn’t dig into the details before running the topline number. Twitter pundits, including some genuine experts who should have been more careful, then fed off of this initial round of unsophisticated media coverage. The press has a lot of failings in covering science issues generally, which go over the head of most journalists, and during the pandemic, this general issue has manifested as a serious systemic problem with press coverage. This is just another example.
- Omicron really is growing explosively, and almost certainly will dominate the entire USA pretty soon. It is probably already dominant in certain areas, like New York and New Jersey, Florida, Hawaii, and Puerto Rico, which are seeing explosive growth in overall case numbers. This algorithm bug and media frenzy have jumped the gun, but in the long run, they are just a blip. The overall story on Omicron is still that it’s growing explosively, will take over, and should be an imminent object of concern.
So, what should you do?
- Still prepare for Omicron to become dominant in your area within weeks (or already). That’s real, and it’s still on.
- Enjoy watching this story get revised.
- Continue to be careful about what you read.
You are reporting the comment """ by on