Florida is familiar with the ‘cone of uncertainty.’ This model has your COVID-19 forecast
While some projection models are being used to identify when the country, a state or county might expect to see a peak in COVID-19 cases — when healthcare systems are threatened to be overcapacity — one model is helping local jurisdictions make more informed short-term emergency planning decisions.
IEM, a consultant for the U.S. Department of Health and Human Services for 15 years, decided it could help local governments better plan their COVID-19 responses by sharing its seven-day model projections with state and local jurisdictions hit by COVID-19.
“Every emergency is a local emergency, which is really why we wanted to get down to the county level,” said Dr. Prasith “Sid” Baccam, IEM’s lead computational epidemiologist.
Baccam understood, he explained to the Bradenton Herald earlier this month, that people were much more interested in knowing what projections looked like for the counties or cities they live in so he decided to break IEM’s model down to the county level.
At the national and state levels, government officials are relying heavily on model projections being provided by IHME, the Institute for Health Metrics and Evaluation, an independent population health research center at University of Washington’s School of Medicine.
Earlier this month, county emergency management directors in Florida began receiving IEM’s model projections. IEM had not been asked to provide this data, according to Bryan Koon, IEM’s vice president for emergency management.
“We want that information to be out there for everybody to be able to utilize. It’s not the only data point that they should be using, but we do feel that when they add it into all the other data points they have, it will help with their decision making,” Koon said.
Here in Manatee County, Public Safety Director Jacob Saur has used the data in making his recommendations to the Manatee County Board of County Commissioners, including for the stay-at-home order that is currently in place.
As of Friday morning, when there were 200 confirmed cases in Manatee County, IEM was projecting Manatee would have 432 cases of COVID-19 by Wednesday — more than doubling the number of cases in a week’s time. The state, which had about 18,000 cases, is expected to have nearly 26,000 cases by then.
“One of the things you see in the models are that they are a bit broader sometimes in their time frame and lack a bit of specificity that government officials are looking for,” said Koon, who as the former director of the Florida Division of Emergency Management is familiar with the needs of state and local emergency managers.
The COVID-19 ‘cone of uncertainty’
The artificial intelligence based model used by IEM is a spaghetti model, which much like those used to forecast the future track of a hurricane, features a “cone of uncertainty.”
The model uses real data from confirmed cases and works backwards to set its parameters before trying to forecast how the disease will continue to spread.
“We run our model 11 million times to try and find the best parameter values to explain how did we get to where we are today,” Baccam explained. “Then from there, we shoot out thousands of projections into the future, and just like a hurricane model, it’s a spaghetti model. You are going to see a bunch of them going forward into the future, some of them might go a little higher, some of them might go a little lower but we need all that data to find the best projection.”
The further out the model goes, the wider the “cone of uncertainty” becomes.
This is something a Floridian can understand. It is also why the IEM model each day only makes projections seven days out.
“We only show it out to seven days because that’s when we are very confident in the accuracy,” Koon said. “That allows those folks to make real-time planning assumptions.”
Model explains spikes in COVID cases
“Data is only as good as what it is,” Baccam said. “We can’t make it any better.”
But as testing is increasing, the data starts to level out. That in turn is making IEM projections more accurate.
“So that’s why their numbers jump sometimes,” Baccam said. “That spike sometimes won’t be real but because those tests results just came in ....The data is not always so clean but our modeling tries to take that into account.”
This could explain spikes in the number of cases in Manatee County after a county health department four-day drive-thru testing event and when testing became available at other facilities, according to Baccam.
“We hope that all jurisdictions get to the point where they are happy with testing, the speed of the testing, the number of tests and so on, and when that happens, their numbers are going to start leveling out and absolutely become more regular.”
“Your jurisdiction and others are still trying to get where they want to be with testing,” Baccam said.
Social distancing effect
Public health officials and emergency managers were planning in a vacuum early on during the COVID-19 pandemic. But IEM says their models gives them the tools to make decisions, including what measures to implement in attempts to flatten the curve.
“To me it all comes down to compliance. We have these social distancing strategies and we put them out there and it really all comes down to are people listening,” Baccam said. “You’re going to see it in the data seven to 10 days later. It’s not instant gratification.”
Some social distancing strategies have worked better than others.
But some places like New York City, where the pandemic has hit the hardest, they are making decisions without that data.
Other communities that are beginning to see their own outbreaks are able to look at transmission rates that follow various different social distancing policies.
“So as this spreads across the country in different areas at different times, we’ll be able to combine the policy guidance and the processes they put in place, combined with the data from the models and help recommend which policies had better outcomes than others so they can go straight to the ones that make the biggest difference,” Baccam said.
It’s the incubation period with the coronavirus that causes a delay in a reduction of cases following the implementation of social distancing policies.
For individuals looking at projections, Koon hopes that they use it take social distancing policies or other guidelines seriously, and do their part to stop the spread.
“I hope that the efforts that the governor and local governments are doing to improve social distancing and reduce the spread are effective and our numbers are too high because we’re able to slow down the transmission rate. That’s my overall hope,” Koon said.
Operationally, Koon hopes that emergency managers are using their model when considering how much personal protection equipment they need and the availability of hospital rooms, ventilators and other medical equipment.
“Then we can look across the state, and balance that out, their needs in parts of the state may be greater than in other parts of the state so we hope by providing the statewide picture and the county-by-county picture, we can mobilize those resources and get them where they need to go so we can save as many lives as possible,” Koon said. “So that’s what I hope, that the model data translates into operational response and into the way we are managing our logistics response so that it could achieve the response everybody wants, saving lives.”
