July 04, 2020

Weather Forecasting is Fifty Years Ahead of Epidemiological Prediction: That Must Change

In October 2012, Hurricane Sandy approached New York from off the Atlantic.  Seven days before, a near perfect track forecast was made by the European Center and two days later the U.S. model (the GFS), run by the National Weather Service's Environmental Modeling Center, has locked on as well.


The stunningly accurate prediction of an extremely unusual event, saved countless lives, with only about 100 deaths in a region of tens of millions of people (and most who died made a bad decision to ignore the forecast).


Although a hurricane hitting New York is exceedingly rare, the excellent forecast was based on an extraordinarily weather prediction infrastructure that had been perfected over the past half century, with sustained investment and development:
  •  A comprehensive observing system, based on weather satellites, surface observations, and more.
  • A complex quality control and data assimilation system the ensured a good idea of what happening in real time.
  • Highly complex numerical models for simulating the evolution of the atmosphere, models that had been tested and perfected over decades.  And a dedicated U.S. forecasting center responsibly for state-of-science prediction.
  • A mature statistical postprocessing system capable of improving the model forecasts based on past performance.
  • A comprehensive verification system to provide detailed evaluations of the skill of the forecast.
  • A highly developed communication system, that provided the public with clear interpretation of the forecast.
Excellence in prediction took time and investment over decades, and paid off in warning the public and guiding public officials in protecting the population.
Stunningly, U.S. epidemiological modeling has almost NONE of the above components or systems, and the performance of the U.S. Centers for Disease Control and Prevention (CDC) as poor.    Consider:
  • There is no large, permanent epidemiological prediction center in the U.S. government analogous to the NOAA/NWS.   U.S. response to COVID had to depend on a hodgepodge of university forecasting efforts, some spun up for this event.  Untested, unverified, and often very wrong.
  • Unlike weather prediction, U.S. epidemiologists do not have an accurate description of what is happening NOW.  Testing was slow to begin, in fact, the CDC made serious errors in test development.  Even today, six months into the pandemic there are not enough tests.  There is no attempt being made to randomly sample the population to understand how many folks have or were infected.  Unbelievable.
  • Unlike the comprehensive and smoothly running weather observational system, there is poor organization to the data collection by CDC. There are not even standard reporting approaches.
  • The epidemiological models are generally primitive affairs, many of which do not consider the complex, variable transmission properties of a heterogeneous population, and lack clear information of what is occurring right now (called the initialization in weather prediction).
  • Communication by CDC of both the threat and how to deal with the disease has been inconsistent and often wrong.  For example, they initially discouraged the use of masks, before reversing their guidance 180 degrees.  Similarly, CDC downplayed the threat in January and February, before reserving in March.  The NWS works very hard to start with reliable forecasts, to communicate the uncertainties, and not to go back and forth in their warnings.  They are masters at this.

So let us imagine what the Hurricane Sandy forecast would have been like if the NWS followed the CDC approach.
  • The storm would have been out there, but without comprehensive observational assets, they would not have known where it was.
  • Without a government prediction organization, the NWS would have asked for volunteer forecasts of University modeling research groups (like the University of Washington).  They would have had 5-10 forecast from various universities that would have diverged by position and intensity.  Without good initialization data, none of the forecasts were skillful.  And furthermore, without long-term verification, no one knew how good the forecasts were.
  • Based on this guidance, the NWS could not provide specific, accurate forecasts, suggesting that there was a storm out there, but it could hit anywhere from Georgia to Maine, or might go out to sea.
  • With such uncertain forecasts, political leaders pressured to evacuate the entire coast from Georgia to Maine at a huge cost.  Many would not evacuate under such vague warnings. With landfall on NY and many remaining in their homes, nearly 4,500 people died

The Political Opportunists

        There are some folks and many media pundits who are claiming this COVID disaster is all the fault of President Trump and that things would have been much better under a Democratic President.

This is either very naive or very cynical.

There is no doubt that the President and his administration has been startlingly misinformed and ineffective. His abysmal leadership has made things much worse.

But the problems noted above are not recent developments and have been allowed to fester in recent administrations, including the 8-years of President Obama.  I suspect we would not have been in much better place if Hilary Clinton would have won, because the basic institutional infrastructure was not put in place.

That is what we must do together as a nation, following the example of the weather prediction community.  And speaking as one of them, we would be glad to help.









26 comments:

  1. We have the institutions but they don't work. Center for Disease Control, budget $7+ billion in 2019, not only did not do its job but prevented other organizations from doing theirs.
    https://www.washingtonpost.com/investigations/lessons-unlearned-four-years-before-the-cdc-fumbled-coronavirus-testing-the-agency-made-some-of-the-same-mistakes-with-zika/2020/07/03/c32ca530-a8af-11ea-94d2-d7bc43b26bf9_story.html

    ReplyDelete
  2. Last few days of December - probable/possible evidence US security agencies were raising the possibility of serious epidemic in China

    Trump Jan. 22 "We have it totally under control... "

    WHO Jan 23 "it may become one (global health emergency)

    Jan. 24 Trump praises China and Xi

    Jan. 30 Trump "its all going to be great .."

    Tedros announced that the outbreak had become a "public health emergency of international concern over the global outbreak of novel coronavirus."

    All of February, more of the same

    March 11
    Trump said in an Oval Office address: "The vast majority of Americans, the risk is very, very low."

    Tedros "Pandemic"

    March 16, March 17

    Trump told reporters: "This is a pandemic. ... I felt it was a pandemic long before it was called a pandemic."

    No presidential leadership ramping up testing, PPE, as well as Federal and State health agencies.

    I cannot think of another national politician of either party who would not have been taking preliminary actions in mid January.


    ReplyDelete
    Replies
    1. Thank you for this timeline.

      Delete
    2. It is half truths because it ignores the completely false information coming out of China at the time. Go back to twitter with this nonsense. I wonder why this comment was even approved?

      Delete
  3. Despite the gross failures of the CDC the fundamental problem in the US is lack of national leadership. Many countries with much smaller scientific budgets, such as New Zealand, South Korea, Taiwan, and Singapore managed to control the virus spread and return to pre-pandemic life. As Cool Hand Luke said: What we've got here is failure to communicate.

    ReplyDelete
  4. Excellent blog post... point "right on!"

    ReplyDelete
  5. At the very least he still could have worn a mask and not talked about how the virus was going to just disappear. These facts in and of themselves have cost lives and turned the idea of wearing a mask into another politically divisive ploy. I seriously doubt any other candidate (republican or Democrat) would have made such ridiculous claims and STILL not be wearing a mask, which is proven to save lives.

    I agree with your other take on our ability to determine the course of these things.

    ReplyDelete
  6. Agree with the sentiment; we have not invested in public health very consistently.

    But there are big differences when one is not just moving air-moisture-heat between voxels. In spreading infectious disease, it isn't just neighbors that are targets. Carriers drive long distances to participate in events like choir practice, causing disease to jump locales. Duration of exposure matters, in order to accumulate enough viruses to start an infection. That's how one carrier infected 52 others at a 2.5 hour choir practice in the Skagit.

    ReplyDelete
  7. You write that pundits who claim that “that things would have been much better under a Democratic President” are “either naive or cynical.”

    In the next paragraph you write that Trump’s “abysmal leadership has made things much worse.”

    And then in the paragraph following that one you write that you “suspect we would not have been in [a] much better place if Hilary Clinton would have won, because the basic institutional infrastructure was not put in place.”

    So Trump’s bad leadership matters — it’s made things “much worse” — but Clinton’s presumably better leadership wouldn’t. The reader struggles to understand how both assertions can be true.

    ReplyDelete
    Replies
    1. It's really not that difficult to understand. Leadership of the US president is just one factor of many (and a pretty small one), but it cannot replace the lacking technology, data collection and infrastructure described in this blog.

      Think of it another way - can Hillary Clinton today predict the upcoming hurricane season on her own by just using her presumably superior "leadership" skills, without any other tools? It's naive to think even the best leader can fully grasp all the variables related to the weather, or a global pandemic for that matter, without the right tools/technology.

      Delete
    2. I was mostly pointing to a contradiction in what Cliff wrote. It doesn’t make sense to say that Trump’s leadership “has made things much worse,” but that better leadership wouldn’t matter.

      The idea that leadership is a small factor compared to scientific and technological shortcomings in epidemiology is belied by the vastly different experiences different nations have had with this pandemic. Or to put it more directly: every other advanced nation in the world has done a much, much better job controlling the virus. (Here are some graphs that indicate the scale: https://coronavirus.jhu.edu/data/new-cases. Compare the US with the UK.) If the most important factor was the state of epidemiology, which seems to be what Cliff is claiming, we would not see such dramatic variation among advanced nations that have access to the same technology and same level of scientific expertise. This is not to say that Cliff’s description of problems with epidemiology is mistaken; I’m in no position to judge. But that is not the fundamental problem here.

      Delete
    3. I can't imagine a scenario where bad leadership would help any situation, but trying to narrow down something as complex as a pandemic to a single variable is loserthink. It's especially true when that one variable is difficult to even measure objectively.

      There are thousands of variables to all of this. I could even name a few very quickly, for example comparing the UK to the US - we can discuss population size, business/personal travel with China, economic ties with the source of the outbreak (again, China), the aversion to wearing masks in public, the number of sports stadiums, the number of dense population centers, the number of densely populated tropical beaches, access to free health care - and on.

      I would invite you to consider all these - in fact you must since you are the one with the bold claim that "we would not see such dramatic variation among advanced nations" if we only tweaked single small variable called "leadership".

      In any case, I think we've found some common ground by asking the simple question - why is leadership even a factor at all? I hope we agree that this one must be removed as much as possible. After all, no-one is waiting on the president to approve the weather report.

      Delete
  8. "I suspect we would not have been in much better place if Hilary Clinton would have won, because the basic institutional infrastructure was not put in place."
    According to one NPR report, she did not know how to operate a computer!

    ReplyDelete
  9. The two aren't mutually exclusive. Cliff isn't wrong when he says Trump has made it worse. Trump has lied to the world about this situation multiple times.

    Would Clinton have lied about it? Who knows?

    ReplyDelete
  10. When will we be privy to the results of Cliff's Great Fireworks Experiment of 2020??

    ReplyDelete
  11. Imagine if a hurricane's track and intensity depended on 1) whether evacuation orders are issued and 2) the fraction of people heeding the evacuation order. Then imagine if evacuation orders became politicized! That 5 day cone of uncertainty would be much larger.

    ReplyDelete
    Replies
    1. "Hurricanes are a hoax" adherents (in Cliff's hypothetical): "Hurricanes aren't real -- they're media fabrications!"

      "COVID-19 is a hoax" adherents in the real world: "COVID-19 isn't real -- it's a media fabrication!"

      The good news: These are usually self-correcting problems.

      Delete
    2. the cone wouldn't change but the impact would, just as it did with the politicization of the covid data....and of course the cone would have to be updated via sharpie

      Delete
  12. Cliff,

    I admire your analysis and offer to help with modeling especially give all of your experience with weather modeling. I hope leaders will accept your and your colleagues help. Our lack of preparedness and not quite getting what to do next is leading to unnecessarily lost lives. No one wants this result.

    ReplyDelete
  13. When you have a whole administration whose purpose is to deny science, enrich themselves and make the President look good, you will always have a disaster. Nearly anyone would have done better if they would have put our best minds tackling the problem, not STILL denying it exists.

    ReplyDelete
  14. "There is no attempt being made to randomly sample the population to understand how many folks have or were infected. Unbelievable."

    https://www.cdc.gov/coronavirus/2019-ncov/cases-updates/geographic-seroprevalence-surveys.html

    ReplyDelete
  15. The average person can't comprehend this virus threat in the same way as they would the threat when seeing and feeling a hurricane coming full-bore at them. After all, standing out in a 100 MPH wind watching your house get blown apart definitely leaves an impression. So, this kind of thing is not given a lot of immediate attention. It's completely expected, absolutely normal human behavior.

    ReplyDelete
  16. "Even today, six months into the pandemic there are not enough tests. There is no attempt being made to randomly sample the population to understand how many folks have or were infected. Unbelievable." Thanks Cliff. I too find this all unbelievable. Why isn't the state of Washington taking measures to randomly sample for infected persons? Or to look for an accurate test for immunity? Why do media outlets fail to note that the increase in infected persons has also corresponded to an increase in testing? Why does the governor seem to focus on only wearing masks rather than develop a real strategic state testing plan? Washington is abound with intellectual resources to assist the state. I am baffled as to why the best we can do is wear a mask... that should be a piece of a more comprehensive plan.

    ReplyDelete
  17. I have enjoyed your blog, but I cannot forgive the stupidity you show in your comments about our President. He acted swiftly and decisively, while the Democrats looked the other way because they were so focused on impeaching him. He was lambasted by the left for prohibiting travel as early as he did, and for implementing stay-at-home. Now he didn't do enough? He did remarkable in getting resources where they needed to be. You can agree or disagree, but I am done with anyone who bad mouths our President. You should stick to the weather. I am gone, and will never read your page again.

    ReplyDelete

U.S. Numerical Weather Prediction is Crippled by the Division between NOAA and the Academic Community. But a Rare Opportunity Beckons.

U.S. global weather prediction run by NOAA/National Weather Service is now in fourth place among national centers and FAR behind what one wo...