Where is the National Weather Service's New Supercomputer?

It is nearly a year since the U.S. Congress supplied the money for a  new cutting-edge National Weather Service weather supercomputer, using Superstorm Sandy supplemental funds.

The computer promised to greatly improve weather prediction in the U.S. and was cited as a "game changer" by the head of the National Weather Service.

It offered the U.S. a chance to finally catch up with or exceed  the state-of-the-art predictions of the European Center, resulting in saved lives, improved warnings, and  large economic benefits for the United States.

Now a year later, the computer has not even been ordered, while the the European Center has just secured a brand-new American computer to push the envelope of weather prediction far beyond that practiced in the U.S.

This blog describes this situation and calls on Congress and others to demand immediate remedy of this sad state of affairs.

U.S Inferiority in Numerical Weather Prediction Continues

By objective verification statistics (see below for an example) and the admission of National Weather Service leadership, the U.S. trails behind many other nations in global weather prediction,with groups like the

A verification of the five days forecasts for sea level pressure over the northern hemisphere reveals that the National Weather Service global model (the GFS) is in fourth place, behind the European Center (ECM), the UK Met Office (UKM), and the Canadian Met. Center (CMC).  The higher the numbers the better (1 is a perfect forecast).  Note the consistent high numbers for the ECM, (red dashed) with some major dropouts (e.g., Feb 22) for the US GFS model (black line).

European Center for Medium Range Weather Forecasting and the UKMET office consistently outperforming the U.S. operational numerical weather prediction center (the National Weather Service's Environmental Modeling Center, EMC).  Lately, the Canadian Meteorological Center has been superior as well.  As shown inthe above figure, the U.S. would not even get the bronze medal in weather prediction.

This inferiority was made clear to the general population during the period before Superstorm Sandy's landfall, when the U.S. models took several days to come into line with the highly accurate 6-8 day forecasts of the European Center model (discussed in my previous blogs here and here). More seriously, U.S. weakness in numerical prediction is not limited to global modeling:  for example, we are failing to run high-resolution ensemble forecasts that would allow far better prediction of severe thunderstorms and other local weather features.

Lack of Computer Power is Crippling U.S. Weather Prediction

As noted by many, including NWS Environmental Modeling Center staff, a major reason why the U.S. lags in numerical weather prediction is the lack of computer resources available to the National Weather Service. Today, the National Weather Service has two computers with peak performance of approximately  210 teraflops and 10,240 processors (Note:  1 petaflop equals 1000 teraflop, a teraflop is a trillion operations per second).   This US computer recently was upgraded from 80 to 210 teraflops, with a change in computer architecture from IBM proprietary to commodity chips.   In contrast, the European Center has just acquired TWO U.S.-made CRAY XC-30 computers, each with 80,000 processors.  My sources indicate that each EC computer has a peak throughput of 2-5 petaflops and a sustained 200 teraflop capability, dwarfing the computers the NWS has today. Translation:  the new European Center computer is approximately 15 times faster than the current National Weather Service computer.  And our computer has to do FAR more tasks.

Current Computer Resources (peak computational capacity in petaflops) of the European Center Versus the National Weather Service Environmental Modeling Center.  The NWS has far more modeling responsibilities than the EC

The funds from the Sandy supplemental funding would be enough to secure a computer from IBM with a peak performance of 2 petaflops, substantially less than the European Center computer.  And keep in mind that the European Center only has to do global weather prediction while the National Weather Service has vastly more modeling responsibilities, including high-resolution forecasting over a vast nation.   The U.S. really needs a 20-30 petaflop computer resource to provide U.S. citizens with state-of-the-art weather forecasting.  Such machines exist and are being used for climate prediction and simulations of nuclear explosions....do we have our priorities wrong?

This lack of computer power guarantees U.S. inferiority in weather prediction.  Some examples:

1.  The U.S. global model, the GFS runs with 26 km grid spacing, while the EC is at 16 km, and they plan to go to 10 km very soon.
2.  The U.S. does not have the computer power to run high-resolution ensembles (many forecasts) over the U.S., a necessity for prediction of thunderstorms and other local weather features.
3.  The U.S. has lagged substantially in the area of data assimilation, the technology of effectively using observations to describe the structure of the atmosphere...the starting point of all numerical weather predictions.
4.  The U.S.lacks the computer power for high-resolution global ensembles, which are critical for prediction of uncertainty and probabilities.  Such ensembles are also critically needed for weather data assimilation.

The effective use of a large computer system, like the European Center has, would greatly improve U.S. weather prediction, clearly saving lives and promoting the economic needs of the U.S. (e.g., agriculture, water resources, renewable energy).    U.S. companies now spend many millions of dollars to get the European Center model forecasts; they would not need to do so if the U.S. predictions were the best.

Why the Delay?

With such an acute need for enhanced computer power, you would think the U.S. government would have initiated the acquisition of new weather supercomputers immediately and just as quickly begin testing the improved weather forecasting models that would move to the new system.

If you thought that, you would be wrong.  Today, a year after the funds became available a new NWS supercomputer has not been ordered.  The reason given?  IBM's sale of its server division to Lenovo.

It turns out that NOAA made a very bad decision in their computer acquisitions.  They committed themselves for an extended period to buy only from IBM, thus losing the ability to shop around for the best deal during each replacement cycle.  And now IBM is planning to sell its server business to Lenovo, a Chinese company, and this is setting off all sorts of warning bells in the U.S. government, and specifically for the new weather computer acquisition.   If NOAA was not stuck with IBM they could go with other vendors, like CRAY, which sells U.S. machines that have no security issues (in fact, the European Center liked them so much they bought one!).

The worry about the Chinese server business is particularly ridiculous for a weather forecasting computer.  Does the U.S. government think the Chinese will turn off U.S. weather prediction in case of a conflict? Even if they could do that, it would make little sense.  Many Chinese groups download and use the use U.S. GFS weather model output.  If they somehow turned off our weather forecasts, we would always have the European Center, the Canadian model , the UKMET office model , and the forecasts from the U.S. Navy!  So it would not get them anything.

And there is something else you might not know of.  A huge proportion of the U.S. modeling community hails from China.  I do a lot of numerical modeling, and when I go to conferences and attend sessions on topics like weather data assimilation, it is evident that if the presenters changed language to Mandarin, the majority of the audience would be quite content.  China and the U.S. have a close partnership in weather prediction with profound interdependence.  They are not going to destroy our weather modeling capability.  And this far fetched threat should be balanced against a real threat.  In fact, not a threat, but a reality:  the inferiority of U.S. weather prediction, the cost of which is very real to each U.S. citizen.   There is all kinds of talk about extreme weather and climate change, yet we don't do the one thing that is guaranteed to afford substantial protection for our population from extreme weather:  better weather prediction.

A year ago, high NOAA and NWS administrators were flashing powerpoints with time lines showing that the new NWS supercomputer coming online during the second quarter of 2014.   Their latest presentations say nothing about upgrades during the next few years and talk about what they will be doing in 2018!  And there is NO sign of an active process of designing and testing the next generation models that could immediately be used on a far more capable supercomputer.    Just a continuation of the go slow approach that has kept U.S. weather prediction mired in the mud for so many years.

Recently, there have some changes in NWS leadership, with a particularly capable individual becoming head of NCEP (National Centers for Environmental Prediction).  There are some very capable modelers/scientists in NOAA/NWS.   But they need sufficient computer resources and they don't have them now.  And there is an acute need for a detailed strategic plan on how the U.S. will regain leadership in weather prediction...which they don't have.  It is time for Congress or the administration to end the foolishness with the computer contract, or to void the IBM contract and find another vendor.

An interesting possibility is for IBM to purchase a CRAY XC-30 and use that for the NOAA/NWS contract.

The European Center folks are just loving the stalled U.S. weather prediction effort.   As noted above, they are making out like bandits selling their model forecasts to U.S. and other countries...we are talking about many millions of dollars.   And I understand that European Center staff bonuses are partly based on how much they are ahead of the U.S.  So the current situation is like money in their pockets.  A year ago, I was at a meeting where I had a chat with the head of the European Center.   He had seen my blogs and told me that the U.S. should not worry about leading weather prediction since we were all in this together...or something like that.  Right.  Keep in mind that the European Center is never going to take care of most of the U.S. numerical weather prediction needs, like high-resolution national ensembles for thunderstorm prediction. And the money is rolling in for them as long as the U.S. is behind.

Why Not Strive for the Gold?

     In Olympic sports, the U.S. strives for and often achieves gold.  We do the same in so many areas of technology and science.  So why is the U.S., and particularly NOAA/Dept of Commerce management,

Why not the U.S.?

satisfied with bronze status, particularly when it costs the American people dearly in lost lives, injuries, and a multitude of economic costs?   How long will the political leadership of our country accept this situation?

Perhaps we need a bit of General Patton's attitude towards winning and leadership (click on picture)