With only a vague sense of direction, you will waste a lot of time and effort, taking backroads, instead of superhighways. Going the long way, rather than using short cuts.
You won't get anywhere fast.
And so it is with the National Weather Service and the US operational numerical weather prediction.
This blog will show that a key reason why the National Weather Service has fallen behind in numerical weather prediction is because it has no long-term plan for where it is going and no strategy for excellence. And until it starts developing clear cut, long-term goals and implementation plans in concert with the US weather community, it will remain behind. A national tragedy, and an unnecessary one.
A Long Journey
Numerical weather prediction is one of the most difficult tasks undertaken by our species. It requires observations from space, in the atmosphere, at the surface, and below ground. The use of complex numerical models and the most powerful supercomputers. The knowledge of phenomena that range from the atomic to the planetary scale. It requires coordination of hundreds or thousands of scientists, computer specialists, and members of the public and private sectors, as well as effective interactions between the operational and academic communities. Only with clear goals and coherent planning can a national effort in numerical weather prediction be state-of-the-art.
The Situation at the National Weather Service: No Long-Term Strategic Plan for Numerical Weather Prediction
Numerical weather prediction (NWP), the bedrock of all weather forecasting in the U.S., is done at the Environmental Modeling Center (EMC) of the National Centers for Environmental Prediction (NCEP), which in turn is part of the National Weather Service and NOAA.
And now the shocker. EMC, which is responsible for US global, national, and regional numerical weather prediction, has NO strategic plan. None. No long-term direction of where it is going. No detailed implementation plans. Nada.
Don't believe me? Check out their website--you won't find one. I have talked to EMC leadership and they confirmed that there is no long term plan. But they are thinking about putting one together. Sometime, perhaps.
We can go one level up in the National Weather Service to NCEP. They do have a strategic plan. But that plan is general and wide-ranging with little detail on where US numerical weather prediction should be going. Just a few sentences. Some examples:
- Eliminate 3 significant bottle necks in the model implementation process.
- Implement a robust, multi-day convection-allowing ensemble system for US high resolution impact based decision support.
Good ideas, but not much meat on that bird. No timelines, no definition of who does what, and exactly what is hoped to be achieved.
But surely the National Weather Service as a whole has a strategic plan. They do. But this plan is full of gauzy generalities with few specifics. It calls for a Weather Ready Nation. No much that is actionable. Little on numerical weather prediction.
The Competition and World Leaders
In contrast to the plan-shy National Weather Service, their competitors DO have specific and comprehensive strategic plans. Take the European Center for Medium Range Weather Forecasting (ECMWF). As noted in my previous blogs and by many others, ECMWF is not only the world leader in global weather prediction, they are accelerating away from the National Weather Service in terms of forecast quality and resolution.
ECMWF has developed and followed a series of strategic plans, such as the one they just completed for 2016-2025. Among other things, their plan calls for a 5-km (grid spacing) resolution global ensemble by 2025. Sufficient resolution to start to get convection right without parameterization. Just amazing. And with hard work and planning they will do it. (For comparison, the current, smaller, NWS ensemble has a grid spacing of roughly 35 km)
But ECMWF planning does not stop there. They have detailed four-year implementation plans as well. Plans with a lot of detail. Thus, it is no accident that ECMWF produces the world's best numerical weather forecasts. They develop coherent, scientifically viable long-term strategies and work efficiently to make them happen. They integrate the best science into their efforts. Such long-term vision and planning is completely missing at the National Weather Service.
The Next Generation Global Model and the Lack of Strategic Planning
A great example of the impact of the absence of NWS strategic planning in numerical weather prediction is currently playing out. After Hurricane Sandy made evident to the nation that US numerical weather prediction had fallen behind, Congress and the administration pumped considerable funds into NOAA and the National Weather Service. The National Weather Service was able to upgrade its computers from 1/10th of the resources of the European Center to parity (which is very good). They also began work on a new global modeling system to replace the aged GFS (Global Forecast System). This new effort is called NGGPS (The Next Generational Global Prediction System) and had the goal of developing a state-of-the-art global prediction system that would bring the US at least to parity with those pesky Europeans.
Unfortunately, the NWS had no long-term strategic plan for a modeling system that would be in place for next 10-15 years. While the Europeans have a clear vision of moving to convection-allowing global resolution within a decade, the National Weather Service had no such long-range view. As a result, the testing and planning for the new system was NOT directed towards securing a system that would best serve the nation in ten years or more. Rather, the competition between potential new global models was decided based on using the resolution of today (13-km) and the physics (e.g., moist physics) of 20 years ago. It appears that this myopic, short-viewed approach will select a modeling system (GFDL's FV-3) that is inferior to the other choice (NCAR's MPAS) at the critical high resolution of the future. The European's will be chuckling: they are now working on a new model that from what I can tell is very similar to MPAS.
But the core dynamical model is only one aspect of a mature numerical weather prediction system, with physics and data assimilation being equally as important, if not more so. But there is no long-term plan on how the National Weather Service will integrate state-of-science physics and assimilation to their new model.
But that is not all
The lack of strategic planning has left National Weather Service modeling ineffective and inefficient. They are too many modeling systems. Too many legacy runs. Too many legacy products.
For example, there are two larger-scale ensemble systems (GEFS and SREF) over the US. Two major modeling high resolution systems run over the US (WRF ARW and NMMB). While other major weather prediction efforts have moved to a unified model approach (same model for all scales), the National Weather Service has no real plans to move in that direction. And while NWS modeling efforts are dissipated on a needlessly complex array of models and runs, they are not attending to critical US needs, like running a large, convection-allowing (4 km or less) ensemble over the US. And perhaps most embarrassing of all, the new operational computer, capable of propelling the US forward rapidly in numerical weather prediction, is only being lightly used.
Strategic Planning Requires Community Input and Consensus
The experiences of the past several decades provide compelling evidence that without long-term strategic planning the National Weather Service's numerical weather prediction will lag the rest of the world. But for strategic planning to be effective, the vision and plan must be a good one, and there is only one way to ensure the quality of the plan: to involve the larger research and operational community in producing a consensus plan. Strategic and implementation plans can not be the product of an individual within the NOAA bureaucracy or even a collection of NOAA folks. It must include the academic community, the private sector, NOAA/NASA and other government folks, and individuals from NCAR (National Center for Atmospheric Research).
An excellent organizer for the creation of such a consensus planning effort might be UCAR's (University Corporation for Atmosphere Research) Developmental Testbed Center (DTC), which already has a close working relationship with the National Weather Service. With sufficient support, they could also provide the central organization of a coordinated US numerical weather prediction effort.
The National Weather Service (NWS) Environmental Modeling Center and NOAA in general have many excellent scientists and software engineers. I know a large number of them personally. Many are frustrated that they are spending time on models that have no real future. That they are often isolated from their colleagues in research. The key problem today for NWS numerical weather prediction is one of leadership and organization, and first and foremost, the US operational numerical weather prediction effort requires a comprehensive strategic plan and implementation strategy. Not for 1 or 2 years, but for the next decade and beyond. I know of several examples of extraordinarily talented young scientists, folks that could help move US operational NWP into leadership, that are turning away from careers in the NOAA or the NWS because they don't see a future there. That must change. And a community-consensus strategic plan for the next decade would be a good start.
Keep talking like this and you're likely to get drafted into the NWS...
ReplyDeleteI agree with Robert, above. Please keep talking like this!
ReplyDeleteThis "rise to the bottom" seems to be a disturbing, yet distinctly American passion toward the destruction of scientifically-backed processes and improvements.
ReplyDeleteThe general population gets basic science facts wrong: http://www.pewresearch.org/fact-tank/2015/09/14/does-waters-boiling-point-change-with-altitude-americans-arent-sure/
There are continued positive attitudes and what appears likely to be hopes that technology and science can "improve individual's lives": https://www.nsf.gov/statistics/seind14/index.cfm/chapter-7/c7h.htm
On the other hand, knowledge of the scientific method is almost entirely missing (and the 2016 version of their site seems to lack basic functionality, so the best I can do here is http://www.nsf.gov/statistics/2016/nsb20161/uploads/1/nsb20161.pdf "26% understanding of scientific study (2014)", "understanding of experiment (34% 2012, 53% 2014)", "concern for the environment 34%", )
With such basic misunderstanding of what constitutes a "scientific theory", the methods of science, certainty, and correlation, is it any wonder that an organization, which will necessarily be comprised of a number of non-scientists, is unable to fully grasp the benefits and requirements of a scientific strategic plan? Large, complex systems, or those that are highly-self-interactive, seem least likely to be understood in any field (and I'd be happy if anyone can provide a reference on that one, or do I need to run that research myself).
@ first need to know exactly where you are going.
ReplyDeleteIf that is in regards to what sort of forecast accuracy is acceptable, I agree.
Given the complexities touched on, that may Not be completely feasible. Pareto, or some similar rule-of-thumb may be a more useful touchstone. What ought to be known before reaching for 80% , is whether the result will be close enough to get the rest of the way (whether that be to 100%, 90%, or whatever). Also, if and when 80% can be reached, it may be recognized that further improvement is becoming exponentially more difficult...
A recent book, "UN ----- CERTAINTY, The Soul of Modeling, Probability & Statistics": http://wmbriggs.com/post/19253/
ReplyDeleteWhat entity would conceive the need for and direct the development of such a plan? A previous commenter noted the scientific illiteracy of the American body politic. I think this has been documented elsewhere, but also think we can see a similar phenomenon among or elected representatives, at least I seem to frequently hear of Congressmen denying fundamental tenets of science, and it is well known that science oriented advisory boards to Congress have been eliminated. Are we seeing the fallout from this withdrawal into a fantasy world on the part of much of America?
ReplyDeletepunchrun, that is part of the problem. until we get not only more scientific literacy among leaders, but also get them thinking in 10-15 year periods and actually be willing to stick to them, we are going to potentially drop a tier in the F1-WEC-Nascar race that is weather modeling.
ReplyDeleteMaybe if we can describe these priorities in terms of say motorsports to these congresspeople, they might be more willing to listen. after all, we have a lot of damn good forecasters on tap here in the US (thought of like drivers). But without agreement and planning from the engineers (researchers) & mechanics (modelers) from a more centralized team manager and owner (weather enterprise leadership), the best drivers will only pull out so much performance no matter what the skill. And even if the drivers can do well, we need to be sure they look towards a team title, not take each other out using real stupid moves (insane and unrealistic products), pit screw-ups (bad analyses, and mis-use of data), and driver infighting (infighting among different companies/entities within the enterprise above and beyond normal competition).
sound too complicated, or too plain-spoken?