Florida changed its COVID-19 data, creating an ‘artificial decline’ in recent deaths (4 Pics)
As the delta variant spreads through Florida, data published by the Centers for Disease Control and Prevention suggest this could be the most serious and deadly surge in COVID-19 infections since the beginning of the pandemic.
As cases ballooned in August, however, the Florida Department of Health changed the way it reported death data to the CDC, giving the appearance of a pandemic in decline, an analysis of Florida data by the Miami Herald and el Nuevo Herald found.
On Monday, Florida death data would have shown an average of 262 daily deaths reported to the CDC over the previous week had the health department used its former reporting system, the Herald analysis showed. Instead, the Monday update from Florida showed just 46 “new deaths” per day over the previous seven days.
The dramatic difference is due to a small change in the fine print. Until three weeks ago, data collected by DOH and published on the CDC website counted deaths by the date they were recorded — a common method for producing daily stats used by most states. On Aug. 10, Florida switched its methodology and, along with just a handful of other states, began to tally new deaths by the date the person died.
If you chart deaths by Florida’s new method, based on date of death, it will generally appear — even during a spike like the present — that deaths are on a recent downslope. That’s because it takes time for deaths to be evaluated and death certificates processed. When those deaths finally are tallied, they are assigned to the actual date of death — creating a spike where there once existed a downslope and moving the downslope forward in time.
Shivani Patel, a social epidemiologist and assistant professor at Emory University called the move “extremely problematic,” especially since it came without warning or explanation during a rise in cases.
Patel said Florida death data now show an “artificial decline” in recent deaths and without an explanation or context, and “it would look like we are doing better than we are.”
The change came the day after the state health department’s official Twitter account posted a series of late-night tweets accusing the CDC of publishing incorrect numbers, but offering little explanation.
“As a result of data discrepancies that have occurred, this week, FDOH worked quickly and efficiently with CDC to ensure accurate display of data on their website the same day,” DOH spokesperson Weesam Khoury told the Herald in a statement at the time. “To proactively ensure accurate data is consistently displayed, the Department will begin daily submission of a complete renewed set of case data to CDC, including retrospective COVID-19 cases.”
The health department did not acknowledge the subsequent change in the data structure or its abrupt onset, leaving the public scrambling for answers as more than a year’s worth of data changed from one day to the next.
“It shouldn’t be left to the public, to scientists, national policy makers or the media to guess as to what these numbers are,” Patel said. “We know from the beginning that dates matter and that they tell us different things.”
Jason Salemi, an epidemiologist at the University of South Florida who has been tracking the state’s COVID data, said reporting by date of death is better for long-term studies of the disease.
“Deaths by date of death curve is the most accurate you can get,” Salemi said. “You know exactly when people died, you know how to construct the curve and exactly when we were experiencing surges in terms of deaths.”
But Florida’s new data structure is less useful for understanding the pandemic in real time, he warned.
“When you have big surges in deaths, the deaths by date reported will always show an increase while deaths by date occurred will go down,” Salemi said.
“Someone could have died yesterday and we may not know about it for a week, or two weeks,” Salemi said. As a result, new death trends as reported by the Florida health department are significantly lower when data are first reported and don’t immediately reflect the actual number of people who died that day.
During surges of cases and deaths, averaging seven days of deaths by report date provides an important early indicator of how many people have recently died of the disease — a number that will eventually be reflected in the data by date of death, Salemi said.
Although deaths by date the person died are not currently reflecting record numbers, trends based on newly reported deaths are currently almost 31% higher than previous peaks in summer of 2020 and over the holidays, the Herald analysis showed. Of the record deaths reported to the CDC between Saturday and Monday, the Herald found that the vast majority of the 902 victims died within the past two weeks.
The Herald also found that during the last two surges the trend lines using date of death showed peaks 25% and 8% higher respectively than the corresponding peaks by report date.
‘STATISTICAL SLEIGHT OF HAND’
The Florida health department has made several, unannounced changes to its data methodology over the span of the pandemic, abruptly switching between including and disregarding non-resident deaths in its total counts, for example. Salemi said such frequent variations make it difficult to report numbers in a consistent and transparent manner that’s easily understood by the public.
Florida hasn’t always depended on the CDC to be the exclusive publishing house for its daily numbers. Until June 4, the department published its own data, available in daily PDF reports and also provided to view and download through an online dashboard.
The downloadable data sets on cases and deaths included the report date as well as the date a person died or got sick, allowing journalists and independent researchers to select the best metric for their purposes. The daily reports showed additional cases and deaths added from one day to the next.
In June, as case numbers dropped and vaccination rates continued to rise, the health department discontinued the dashboard and changed to a weekly report. The only near-daily data was submitted by the health department to the CDC and published on the CDC Trend Tracker website.
At first, the data on the CDC website was updated in a largely predictable manner, similar to the way that the DOH had reported daily changes throughout the pandemic. Then on Aug. 10, without warning or any explanation from the health department or the CDC, the data for nearly every day of the previous year changed. Neither agency immediately explained the changes.
The CDC eventually confirmed what experts had hypothesized after comparing the new data to previous reports — that the Florida Department of Health had begun to report deaths by date of death. The change was also reflected in data about new cases, which went from being counted by date of report to “the date of specimen collection, confirmed COVID-19 laboratory test result, or clinical diagnosis,” according to the CDC website. The case data show less of a dramatic shift than death data because case data are reported more quickly than deaths.
The CDC website listed Florida as one of just 12 jurisdictions — 11 states and New York City — reporting new deaths by date of death as of the end of August. The choice of how to report is determined by each jurisdiction, according to the website.
Florida’s weekly report, published on Fridays, also shows a decline in deaths in recent weeks even as the number of total dead across the state spikes. Last Friday, the DOH report showed 389 COVID-19 deaths for the previous week. But a comparison of cumulative deaths from the report the week before showed that 1,727 additional deaths were logged by the health department over that seven-day period. The DOH chart of deaths shows a decline over the past two weeks.
The DOH weekly report notes “death counts include individuals who meet a standardized national surveillance case definition” but includes no descriptions of how the health department presents the numbers.
“Due to inherent delays in deaths being reported to the [Health] Department, the previous 14 days may be incomplete and are updated over time,” said Khoury, the health department spokesperson, in an Aug. 30 statement to the Herald. “The Department reports deaths by date of death to the CDC upon notification to the Department.”
The Florida health department does not provide that explanation on its website or anywhere in its weekly reports.
In consultation with epidemiologists the Herald continued to report additional cases and deaths added to the total each day in daily updates rather than relying on the number of “new cases” and “new deaths” DOH data attribute to the previous day. However, data still appear inconsistent, as deaths backfill in large semi-weekly batches, rather than in all weekday updates.
“The CDC has started displaying the Department’s submitted retrospective file twice a week on Mondays and Thursdays, which updates previous day deaths that were subsequently reported to the Department,” Khoury explained for the first time in Monday’s statement. The difference can be an addition of eight deaths one day, and 901 the following day, as happened in the middle of last week.
Without context about how data are reported, when recent numbers are low due to reporting lags, “people will think there’s nothing going on,” said Mary Jo Trepka, an infectious disease epidemiologist and professor at Florida International University.
Economist Tim Harford, author of “The Data Detective,” who spoke generally about how data can be manipulated, said that changes in data definitions are unlikely to have long-term negative effects on scientists, but could easily confuse or mislead the layperson.
“When numbers are presented in a flattering light, an expert will generally be able to see through the dazzle quite quickly,” Harford said. “That said, I still think the truthful-yet-deceptive framing of numbers is a serious problem.”
When data are subtly distorted or presented differently than expected — something Harford called “statistical sleight of hand” — it can prompt dangerous levels of general mistrust and cynicism, he said.
“COVID is a matter of life and death and people deserve to have information that is both accurate and understandable without having to decode it,” he said.
HOW SERIOUS IS THIS THIRD WAVE?
Current data from Florida also show that cases and hospitalizations are at record levels, although trends seem to have plateaued over the past week.
“What we’re seeing is an active rise in cases where we can’t keep up, an active rise in deaths that, because of using actual date of deaths, has been shifted back in time and we have no idea where we really are,” said Patel, the epidemiologist from Emory University.
This plateau, she said, might not reflect reality.
“It just looks like Florida is unable to count and report its cases fast enough,” she said.
Patel said data indicate the worst has yet to come for Florida, and it’s likely that deaths will surpass past records. But it’s still too soon to know with certainty, she said.
No comments