Monday, January 5, 2015

A Major Advance for Numerical Weather Prediction in the U.S.

This is important.

Today, NOAA's administrator Kathryn Sullivan announced that the National Weather Service will acquire two very powerful CRAY supercomputers to support U.S. numerical weather prediction.   A machine that will FINALLY allow the U.S. to do world-class forecasting.  (The press release is here).

In a number of my previous blogs, I complained about the inferior computing resources available for numerical weather prediction in the National Weather Service (NWS).  I noted that groups such as the European Center for Medium Range Weather Forecasting (ECMWF) and the UKMET office have had substantially more skillful global predictions and that their superior computer resources gave them a substantial edge.  I have also described how the U.S. has lacked high-resolution forecast ensembles (where the model is run many times), which undermined the NWS ability to predict severe thunderstorms and other small-scale features.

Everything can change now, IF the National Weather Service uses these powerful new computers wisely.

So here is why you should be excited.

Numerical weather prediction is the underlying technology of weather prediction and it dependent on computer power.

More computer power allows more resolution:  the ability to simulate smaller-scale weather features (like thunderstorms or mountain precipitation)

More computer power allows better physics, which includes the description of how clouds form or the turbulent processes in the lower atmosphere (among others).

More computer power allows better data assimilation, the use of observations to create a physically consistent three-dimensional description of the atmosphere.

More computer power facilitates ensemble forecasting, in which running the model many times allows forecasting the probabilities of weather events.

The Details

Currently, the National Weather Service has two computers (one is for backup and research), each with a throughput of roughly .21 petaflops (quadrillion operations per second).   The European Center for Medium Range Weather Forecasting has two computer that run at roughly 2.5 petaflops.   In other words, they have ten times more computer power to do FAR LESS than the U.S. NWS (the European Center does not do high resolution local prediction, the NWS does).   And it shows.

This month the National Weather Service will upgrade to two machines of .75 petaflops.

The new machines announced today will be 2.5 petaflops each, roughly TEN TIMES what we have today.

A major part of the announcement is that the new computers will be provided by CRAY, the leading supercomputer maker for numerical weather prediction (they are supplying the machines to the European Center and the UKMET office, among others).   IBM, who has held the computing contract with NOAA, could not supply the machines, because they sold their server division to a Chinese firm, Lenovo.  Thus, they subcontracted the acquisition to Seattle-based CRAY.

I have learned that the new machines will be CRAY XC-40 systems.    I suspect they will each have 50,000-100,000 processors.

The Potential

Properly used, this new computer power can revolutionize and greatly improve the skill of U.S. numerical weather prediction, with huge positive impacts for the country.  Problems in U.S. numerical weather prediction have not been limited to lack of computer power and these problems need to be addressed, such as an inability to entrain the huge knowledge based in the huge U.S. academic research community or inefficiencies/duplication of effort in U.S. government research and development.  These are issues that must be taken on now.   But the excuse of lack of computer power is gone and a renaissance in U.S. NWP is possible.

Finally, it is important to acknowledge the leadership of NOAA's Kathryn Sullivan and NWS Director Louis Uccellini in making this happen.  The environment in NOAA seems to changing in a positive way and they deserve some credit for it.


Sysiphus said...

This is great news. However, I think it would have been better to "leapfrog" the EU and go with two 5 petaflop machines or even more. This puts us on par with the EU, but with the way IT processing power grows exponentially we should have set the bar a lot higher. Very pleased to see a local company is doing it, BTW.

Dan Satterfield said...

Cliff, I'd like to put this as a guest post on my blog on the AGU Blogoshere. A link back to your blog as well. Just give me the Okay.

Dan Satterfield

Cliff Mass said...

That is fine Dan..cliff

Rod said...

Great news on the Cray computers, Dr. Mass.

And thanks once again for the great forecast. My home in West Seattle only had a quarter of an inch of rain during the very recent deluge.

Yes. The rain shadow. That you forecast.

Thank you, Cliff.

-Rod in West Seattle

Keith Patterson said...

When will the new systems begin operation, and when will they begin to provide improved forecasting data?

JonG said...

Great news. This raises a question: Is forecast quality more limited by available computational power, density and quality of input data, or the quality and completeness of the physical models?

I'm sure the answer is 'all of the above', but the perspective of this post suggests that the main limitation is computational power.


Patrick said...

This is great, Cliff!

Wow, you say get a coastal radar, and they do. You say get Cray supercomputers, and they do. You're batting 1000.

Steve said...

FWIW, this an earlier blog posts have incorrectly stated that IBM sold their "server division" to Lenovo. They sold their x86 (Intel architecture) server business to Lenovo, but retained RISC, Mainframe, and custom supercomputer assets (like Blue Gene).

NOAA must have really wanted Intel architecture, though. The Cray uses Xeons.

So what IBM sold is moot to this discussion, but I feel like I should point out that IBM still offers various other faster supercomputers for sale.

Gpacharlie said...

Professor Mass,
Your dogged determination, well reasoned argumentation, utilization of social networking, and tenacity in pressing the politicians to provide for common sense improvements are laudable and should be publicly applauded. When will you be running for Mayor of Seattle?

Chuck Walsh

JewelyaZ said...

Thanks again for your outstanding leadership, Cliff. When are we going to help you apply pressure on the folks elected to represent us to address the other radar gaps in the state and on the coast of Oregon? Your track record is stellar, and if you ever run for office, you have my vote and my volunteer hours. :-)