U.S. Numerical Weather Is Still Behind and Not Catching Up: What is Wrong and How Can It Be Fixed?

The skill of U.S. global weather prediction still trails behind major international centers, such as the European Center and the UKMET office.  And we are not catching up.

The U.S. National Weather Service is failing to run state-of-the-art high resolution ensemble forecasting systems over the U.S. and there is no hint when we will do better in the near future.

Why is U.S. operational numerical weather prediction, the responsibility of NOAA and the National Weather Service, lagging behind? 

It is not because NOAA doesn't have good scientists.
If is not because NOAA administrators don't care.
If is not because NOAA unions or employees are dragging their feet.
It is not because NOAA lacks financial resources or the support of Congress.
And it is not because the U.S. lacks the scientific infrastructure and human resources.

The reason for U.S. lagging performance?

A dysfunctional, disorganized, and fragmented organizational structure for U.S. operational numerical weather prediction and associated research that makes it impossible for NOAA's weather prediction to be world class. 

Things won't get better until that structure is replaced with an intelligently designed, rational organizational structure,that effectively uses both governmental and non-governmental resources to give Americans state-of-science weather forecasts.

The Current Situation

Ever since Hurricane Sandy in 2012, where the European Center model did far better in predicting landfall than the U.S. GFS model, there has been a national recognition that U.S. numerical weather prediction, the foundation of all U.S. weather forecasting, had fallen behind.  Story after story have appeared in the national media.  Congressional committees held hearings. And Congress, wishing to address resource issues, provided substantial funding in what is known as the "Sandy Supplement."  Six years before, after the devastating landfall of Hurricane Katrina, Congress had provide similarly large amounts to improve hurricane forecasting and warnings, creating the HFIP program (Hurricane Forecasting Improvement Project).

The HFIP program funding led to the development of a new hurricane modeling system (HWRF, Hurricane Weather Research and Forecasting model), and the Sandy money went for a new computer, support of extramural (outside of NOAA) research, and the the start on a new global modeling system (NGGPS, Next Generational Global Prediction System).

So with huge public investments in 2006 and 2012, where are we today?

The unfortunate answer is: still behind, with little evidence we are catching up.

Let me demonstrate this to you, with hard numbers, many from NOAA's own web sites.  First, here is a measure of the skill of major global models verified over the entire planet for five-day forecasts at one level, 500 hPa, for the past month  (anomaly correlation is shown, with 1 being perfect).

For virtually, every forecast the European Center (red dashed) is the best, with U.S. GFS model (the black line) indicating lower skill.   Second best, is generally the UKMET office model (yellow color) run by the British, and for this month the U.S. is even behind the Canadians (green line, CMC).  The overall summary of skill is found in the lower left corner. The U.S. Navy also does global prediction (FNO), and is behind the others.  I could show you other months, but the results don't really change.

But this is just a snapshot.  What about a longer-term view?  The top figure below is the skill of the U.S. (GFS, red) and European Center (ECM, black) global models for the past 20 years for the five-day forecasts over the northern hemisphere at 500 hPa.   The bottom shows the difference between the modeling systems, with negative indicating that the U.S. model is behind.

The bottom line:  (1)  the U.S. model skill has been lower than the European model, (2) the U.S. has made little progress in catching up during the last ten years, when a huge investment has been made.  To be fair, one should note that both the U.S. and European models have gotten slightly better during the period.  U.S. skill is not declining.  But we both are improving at about the same rate.

Hurricane forecasting is a critical prediction responsibility of NOAA, and certainly in the news of late, with the landfalls of Harvey, Irma, Maria, and Nate. The evaluation of hurricane forecast of numerical prediction models can be separated into track and intensity error.   Track error is clearly the key parameter, since a skillful intensity error is of little value if the storm is in the wrong place. And bad tracks inevitably degrade intensity prediction.  There have been huge gains in reducing track error, but only modest improvement in intensity forecasts.

Below are the 48-h track errors for Atlantic Basin tropical cyclones/hurricanes for the past 22 years for a number of models and prediction systems (this evaluation was done by the NOAA/NWS National Hurricane Center).  The official forecast (humans using all the model guidance) is shown in black.
There has been notable improvement in 48-h track errors from roughly 150 nautical miles to about 70 for the better systems.  Impressive.  But all models are not equally skillful.  The best the last few yeas has been the European Center's global model (light blue color, EMXI).  The NOAA/NWS GFS global model is not as good and NOAA hurricane model (HWRF) is inferior as well.

What about recent hurricanes in 2017?  Hurricane Irma, which hit the Caribbean and Florida very hard, was forecast far better by the European Center than by the U.S. global or hurricane models.  To prove this, here are the mean absolute errors of the track forecasts (in km) for Irma from Professor Brian Tang's website at the University of Albany.   The European Center is far superior for all shown projections (12-120 hr) in both the US global model (indicated by AVNO) or the U.S. hurricane models (HWRF and the new HHOM).

What about Hurricane Harvey?   Same thing...the EC is the best. And the new NOAA hurricane model was very poor.

Now to be fair, although the European Center is generally superior to the American models in terms of track, there are some exceptions.  For example, Hurricane Maria, which devastated Puerto Rico.    In this case, the European Center was better at 12 and 24h, about the same at 48h, and modestly worse at 72-120 hours.

But Maria is the exception; generally, the European Center has better track predictions.  To prove this, here is graphic from Albany's Brian Tang for many of the significant storms of the past few years, showing the 5-day forecast track error.  For most storms, the European Center is best. On average, the European Center 5-day track error is around 280 km, while the U.S./NOAA GFS error is about 420 km.  A significant difference.

There is a new player in global modeling system , one run by a private sector firm (Panasonic).  They started with the U.S. global model (GFS) and assigned a team of about ten people to improve it.   The results have been impressive-- they appear to do consistently better than the NOAA GFS model in general and for major storms.

Here is the proof for Hurricane Irma.  The PWS forecasts were startlingly good and MUCH better than the NOAA GFS for all projection beyond two days.  Even better than the European Center beyond 85 hr!

This has major implications.   A modest size effort by a private sector firm was able to substantially improve NOAA's own model.  Why were such improvements not made by NOAA itself?

As noted above, NOAA is now developing a new global modeling system, based on the NOAA GFDL FV-3 model, which will go operational in two years.  The latest version of this model, with improved physics, was tried on a large number of hurricanes/tropical storms for 2015-2016.  The results below, for storm track error, are sobering.  The new model's track forecast are only slightly better than the current NOAA global model (GFS) and far worse than ECMWF.  Just as disturbing, the NOAA high-resolution hurricane model (HWRF) has worse track forecasts than the others.

The bottom line in all of this, is that after years of investment,  U.S. global prediction and hurricane track forecasts are lagging the European Center and are not catching up.  For those knowledgeable about the technical details of weather prediction, this is not surprising.   U.S. global data assimilation is not as good as the European Center's and U.S. model physics (e.g., the description of clouds, precipitation, convection, radiation, etc) are generally inferior.

But the lack of NOAA progress is worse than that.   Report after report, workshop after workshop, advisory group and after advisory group, has recommended that NOAA/NWS get serious about ensemble-based (many forecasts, each slightly different) probabilistic prediction, and particularly to field a large convection-allowing ensemble with grid spacing of 4-km or less.  Such an ensemble system is critical for prediction of severe thunderstorms, heavy mountain precipitation, and more.  But little happens at the NWS.  In desperation, the university governed, National Center for Atmospheric Research established their own high-resolution ensemble system as a demonstration of what the NWS should be doing.  It is very popular among NWS forecasters, but can not be maintained indefinitely.  The NOAA/NWS Storm Prediction Center kludged together a Storm-Scale Ensemble of Opportunity with 6-7 members--too small and ad-hoc to address the needs.

And NOAA has lagged in the area of statistical post-processing, improving model predictions by combining several models and observations using advanced statistical techniques.   The private sector, using university/NCAR research, has surged ahead in this, leaving National Weather Service forecasts in the dust.

Want some proof?  Go to forecastadvisor.com and pick your favorite city.  Here are the results from Chicago and New York.  The NWS prediction are far behind companies that use multi-model approaches using more advances statistical techniques.

Why is NOAA/NWS lagging?

The evidence for the lagging performance of NOAA/NWS weather prediction is overwhelming, and even my lengthy description above hardly scratched the surface of the problems (e.g.,  major deficiencies of their seasonal prediction system (CFS), duplicative models, very poor performance of their new hurricane model (HMON), and much more).

How can this all be happening with many good scientists, concerned and interested administrators, and NWS personnel who want to do an excellent job?

I believe the fundamental problem is a deficient organizational structure that has grown increasingly incapable of dealing with the support of a complex, modeling/prediction effort.  And until the structural deficiencies are dealt with, U.S. weather and environmental prediction will be second (or third) rate.

The essential problem:  responsibilities for numerical weather prediction are scattered around NOAA.  

Operational numerical weather prediction is the responsibility of the Environmental Modeling Center (EMC) of the National Centers for Environmental Prediction (NCEP), which is part of the NWS, which is part of NOAA (see NWS org chart below).    But the heads of EMC or NCEP, responsible for running the weather models, do not control the folks developing the models, which are in NOAA (in the NOAA ESRL and GFDL labs).

Imagine being responsible for winning a race, but you had to accept the car given to you by others.

And responsibilities for model development/application inside of the NWS are shared with folks outside of NCEP, including the Office of Science and Technology Integration (OSTI).   Responsibility for developing post-processing of model output is outside of EMC/NCEP, but the MDL lab in OSTI.  And responsibility for hydrological forecasting, which requires high-resolution model simulations is in ANOTHER office (the Office of Weather Prediction).

But it is worse than that.  The heads of EMC/NCEP or even the NWS don't control the folks working on developing new models or the science/technology required.  These folks are in NOAA at the Office of Atmospheric Research ESRL and GFDL labs.   Historically, this has been a major problem, with OAR folks developing models that have never been used by the NWS ;  sometimes NOAA ESRL even planned to compete in operational NWP.

There is no central point of responsibility for U.S. numerical weather/environmental  prediction, no one individual or group for whom the "buck stops here."  No one person or group that has control of the resources needed to be the best in the world.  And there has been a lack of integration of modeling systems, combining atmospheric, ocean, hydrological, air quality and land surface modeling. The future is in integrated environmental prediction, not weather prediction.

The lack of central responsibility for U.S. numerical weather prediction has led to duplication of effort, competition rather than cooperation at some times, development that has never gone into operations, lack of coordination, and waste of public resources.

Solving the problem

The key to fixing NOAA's problems is to prune and reorganize, to create one entity responsible for U.S. environmental and weather prediction.

A NOAA environmental prediction and research center should be established, located within NOAA, not the NWS.  There should be one director, with responsibility for ocean/atmosphere/hydrological and other environmental prediction issues.  That person would control both operations and research, with resources for both. The fragmented, ineffective current system, which has grown haphazardly over the past half-century much be replaced.

A natural location for the center would be in Boulder, Colorado, the location of NOAA ESRL, the National Center for Atmospheric Research, and University of Colorado.  It is an attractive, centrally positioned location, which is important, since the new center must attract the nation's best scientists/modelers either for visits or permanent status.

The new center would be responsible not only for creating and running an integrated modeling system, but for model post-processing as well.  It would sponsor regular workshops, conferences, and tutorials.  The center would be a central point of engagement with the academic and private sector communities.  It would take on critical tasks that have been neglected by NOAA, such as extensive model verification and the creation of actionable strategic and implementation plans.

Why today is different

The nation now understands that the U.S. has fallen behind in numerical weather prediction.  New NOAA administrators, previously from the private sector, may be willing to take a fresh look at the problems.  Both sides of the political spectrum want U.S. weather prediction to be the best in the world.  Perhaps, at a divisive time, there is an opportunity for us to come together for an effort that will benefit all Americans.

There is no reason that U.S. numerical weather prediction can not be far better than the European Center's, particularly since the U.S. research establishment is far larger than Europe.  The benefit of even incrementally improved weather prediction is immense; we just lack the organization and will to make it happen.  That can change.