The U.S. is Falling Further Behind in Numerical Weather Prediction: Does the Obama Administration Care?

The computational resources available to the U.S. National Weather Service (NWS) for numerical weather prediction is rapidly falling behind leading weather prediction centers around the world.

Unfortunately, the Obama administration does not seem to care and the U.S. is retreating into second tier status.  Such a degradation is not only completely unnecessary, but needlessly weakens the economic competitiveness of the U.S. and puts our citizens at risk.   Amazingly, Congress appropriated the money to address this problem a year and a half ago, but the administration has not made use of the funds.  There are words to describe such inaction, but this is a family oriented blog.

Numerical weather prediction (NWP) is the central technology of weather forecasting.  State-of-the-art weather prediction demands huge computer resources and thus the ability to forecast well depends on access to the top supercomputers in the world.   Some numerical weather prediction models are run globally at moderate resolution, while others are run at ultra high resolution over smaller domains to  predict small-scale features such as severe thunderstorms.   Thus, a large nation, like the U.S., requires far more computer power than, say, South Korea or the United Kingdom.

During the past several years, I have blogged repeatedly about lack of computer power available to U.S. operational weather prediction, and particularly the  forecasts made at the NOAA/NWS Environmental Modeling Center (EMC).  Many others in the meteorological community have done the same.  One and a half years ago, the U.S. Congress, recognizing the problem, provided NOAA with 25 million dollars to buy a more powerful supercomputer.  Amazingly, the U.S. administration has still not ordered the machine.  

The reason is that NOAA had signed a long-term contract with IBM (a bad move, by the way) and IBM sold their supercomputer hardware business to Lenovo, a Chinese firm.   The administration did not want to purchase such a computer from a Chinese firm.   And so nothing has happened.

 

There were many options that could have fixed the problem.  IBM could have purchased a supercomputer from CRAY, a U.S. firm.  NOAA could have broken the contract with IBM.  Or the administration could have gone ahead with the Lenovo machine (which was the same computer they would have bought anyway).    But the Obama administration clearly is not very interested in weather prediction, and the problem has festered.

But it is worse than that.  Other nations and groups are pushing ahead rapidly in weather computer acquisition, leaving the National Weather Service in the dust.

Yesterday, CRAY Computer announced the UK Met Office has ordered an extraordinary 125 million dollar system (CRAY's newest XC-40 hardware)  that will delivery a throughput of roughly  15 petaflops (a petaflop is one quadrillion operations per second).  The current NWS computer is capable of .21 petaflops and they are upgrading this fall to a machine of .8 petaflops.  So the UKMET office will have TWENTY TIMES the computer power of the U.S.  The area of the  US lower 48 states is 33 times larger than that of the UK.

In June, the Korean Meteorological Administration (KMA) purchased TWO CRAY XC-30 computers, each capable of 3.1 petaflops.  Yes, their new machines will be nearly FOUR TIMES faster than the UPGRADED U.S. weather computers.  Let's see,  Korea is 1/81 the size of the lower 48 states.

The European Center For Medium Range Weather Forecasting (ECMWF) just completed their first of several upgrades, buying two XC-30 computers from CRAY, each with 1.8 petaflops capacity--more than twice as fast as the U.S. upgrades.  And importantly, ECMWF only does global prediction and thus does not have the responsibilities for high-resolution local forecasting like the National Weather Service.  They need far less computer power, yet possess far more than the U.S. operational center.

Heard enough?   I have more examples, but the message is clear:   the U.S. is rapidly falling behind in  thecomputational resources necessary for high quality numerical weather prediction.  Sadly, this administration has the funds for a major upgrade, one that would at least secure a petaflop machine capable of revolutionizing U.S. weather prediction, but they can't seem to figure out how to buy it.

I know a lot of people inside NOAA and National Weather Service, including scientists working on the next generation of weather prediction models.  Many are frustrated by the lack of computer power--one of them recently complained to me there is not enough computer resource to test promising advances.

My back-of-the envelope-calculation is that the National Weather Service needs a minimum of 20-30 petaflops of computer power to provide the American people with state-of-the science weather prediction that would improve the life of everyone in important ways.

For example, there are several reports by the U.S.  National Academy of Sciences and  other advisory groups suggesting that the U.S. needs ensembles (many forecasts run simultaneously) run at high resolution (2-3 km grid spacing) to provide better forecasts of thunderstorms, and particularly severe ones.  Such ensembles would greatly improve the detailed weather forecasts for smaller-scale features in the rest of the country (Northwest folks, think Puget Sound convergence zone or mountain precipitation).  But the NWS simply does not have the computer power to do it.  New multi-petaflop machines would make it possible.

U.S. companies fork over millions of dollars a year to the European Center for the best forecasts...that would end with the new computers.  And there are so many other critical forecast problems that would be lessened with more computer power, like better hurricane predictions days to a week out.

The U.S. atmospheric sciences community is the intellectual leader in meteorology and weather prediction and many of our research advances are applied overseas, such as at ECMWF and the UKMET office.   The American people deserve to take advantage of the research they are paying for, but that can't happen with inferior computers and inferior forecasts. And yes, our forecasts are still inferior, with the NWS unable to match the resolution and data assimilation approaches of its rivals overseas.

Want proof?  Here are the latest statistics for global  5-day forecasts at 500 hPa (about 18,000 ft above sea level) for several major international forecasting centers during the past month.  Higher (closer to 1) is better.  The top group is the European Center (ECM,the red triangle), with an average score of .911.  The U.S model (GFS) is nearly always below them and had frequent and disturbing "drop outs" where forecast skill plummeted for a day or so (U.S. average is .876).  Second place is the UKMET office (orange circles, .897) and expect them to soar with their new hardware. U.S. forecasters in their weather discussions frequently talk about their dependence on the European Model.  Unfortunate.

And we can't simply use the European Center for our weather predictions, since they will never do the high-resolution prediction over the U.S. than we need, among other things.  That is the job of the National Weather Service.

So folks, how do we fix this?  

First, the Obama administration needs to start taking weather prediction seriously, which they obviously don't.   The President's Science Adviser John Holdren seems to be fixated on climate issues and does not appear to appreciate that good weather prediction is a primary means of protecting the American people from current and future extreme weather events.  The administration needs to figure out a way to order a large multi-petaflop machine for the National Weather Service, getting past the objections of some bureaucrats about Lenovo computers.  Or simply order a CRAY (I had lunch with a CRAY representative and they are enthusiastic about helping).

The President and Science Advisor John Holdren need to give more priority to weather prediction

Second, the American people and the weather community need to complain loudly about the current situation. The media can help us get the message out, something they did to great effect to secure the funding in the Sandy supplement in the first place.

Third, our congressional representatives need to make this a major issue and push the administration to act.

As I have noted in my earlier blog, securing adequate computer resources is only the first step in producing a renaissance in U.S. weather prediction capabilities.  But it is a critical and important first step, and it is time to finally deal with this self-inflicted problem.  Weather prediction is essential national infrastructure, like highways and education.  With second rate infrastructure, a nation declines.

If nothing is done by September 2015, the money for the new weather supercomputer will be lost.  It would be a tragedy for U.S. weather prediction and the American people.  Let's make sure this does not happen.

Announcement

Want to sign an online petition supporting improved computers for the National Weather Service?   Go to this link!

Time to teach ECMWF some humility