A Note on Projections

1 April 2020 | David Merrick

It's not that simple...

When I teach EM planning, I talk about models up front. In emergency management, we depend on models because we have to make policy decisions (such as evacuations) in advance of the event itself. If we wait for concrete/certain information, it will be too late to affect the outcome.

So.. we depend on models. A model is only as good as the data that goes in, and much of those inputs are assumptions. We can't know exactly what is going to happen in the future, so we are forced to estimate. We also refine models constantly, and experience can help us create more and more accurate models.

An example of this is hurricane forecasts. The National Hurricane Center improves forecasting every year, as their models have evolved over the decades.

But even mature models are wrong. Every time. Sometimes they are wrong by a little, and sometimes they are wrong by a lot. But there is always a difference between what the model projects, and the reality that materializes.

The other issue is how the public interprets models. Short explanation - they do a poor job of it. Some models we put out to the public because we must. Hurricane forecast maps are one example. But even a simple representation such as a map can be misunderstood. Few Americans know what that cone of uncertainty actually means, and entirely too many people assume that the storm will follow the forecast track like a train following the rails.

And nowhere on that forecast map will you see - "County X will have Y fatalities from the storm".

That number is hard to predict. Even if the storm does follow the forecast exactly (including track, intensity, forward speed, etc.) we have no way to accurately model the impacts. We can model storm surge. We can model flooding. We can model the damage from winds. None of that directly translates to fatalities. How many people evacuated? How many are sheltering in vulnerable buildings? How will surge actually manifest over terrain? The list of variables is long. So even our best, most mature disaster models are flawed.

And I want to make sure my students understand that up front.

As COVID-19 began to unfold, there were models out there. Predictions that this was much more than 'just the flu'. Some, such as the Imperial College of London reports, indicated such massive levels of fatalities that they purportedly caused a major policy shift in the US and UK. But while these models where publicly available, they were not universally accepted. The numbers were simply astronomical. 2,000,000 deaths in the US was a mind-numbing number, and much more than just the flu. But we set policy based on that model, even if the policy makers didn't want to discuss specifics.

Now we have the IHME COVID-19 forecasts - and they provide easy to understand graphical depictions, based on some key indicators. Hospital bed supply/demand, ICS bed supply/demand, and ventilator supply/demand. The current forecast there is an average of 94,000 fatalities in the US, and that number ranges from 41,000 to 177,000. The numbers from this model and several others were quoted yesterday in the White House COVID-19 press briefing. They were widely reported last night.

These models assume that current protective actions (social distancing, school closures, etc.) continue moving forward. They show that these measures are working, and will continue to do so. That's the good news.

It's also important to remember that these are complicated models (and models are always wrong) dependent on a lot of variables that are difficult to measure. Here is a good writeup from fivethirtyeight.com on this issue.

So for the average public user - here is your takeaway:

What can you do?

Don't focus on the numbers of fatalities. It could be high, it will be wrong. Just stay at home.

Understand that this is not the flu - not by any measure. By staying home, keeping your distance, and washing your hands, you will have a meaningful impact on the outcome of this disaster. Need food? Send one person to the grocery store (or get it delivered). Feel sick? Stay home. Want to go get a burger? Get takeout, or better yet, make it yourself. Want to hang out with friends? Do it virtually. If it's not essential to your survival, skip it.

If you read the statistics or forecasts, make sure you understand what you are looking at. Read the backgrounds of the models, where the data comes from. Reliable sources will be up front with you about all this. If you find something you don't understand - ask an expert, don't assume. FYI, I'm probably not that expert, but I can find you one.

This will pass, we'll get to the next stage. We will soon be able to get our schools open again, and begin the return to normal life. But that will require we maintain this sacrifice up front.

David Merrick
1 April 2020