You have your choice of sources of weather forecasts on various smartphone apps and web pages.
Where can you get the best forecast? (other than this blog, of course 😊)
One service, Forecastadviser.com, evaluates predictions from many weather outlets, including the National Weather Service to the Weather Channel (Weather.com).
Consider the scores for the last year (2022) for Seattle considering 5-day forecasts. The Weather Channel is number one, followed by Accuweatther. The National Weather Service is way down the list!.
In contrast, the National Weather Service is lagging pretty much everywhere.
So why are many commercial services, whose forecasts are entirely automated, doing so much better than the Weather Service, which includes human forecasts in the loop?
These superior systems start with global observations and forecasts from many different forecast models, such as the European Center and U.S. GFS global models. Using a history of observations and model forecasts, the advanced systems then create statistical models that combine the various inputs to produce forecasts with lower typical errors.
Many of the newest systems, such as the Microsoft Start system, make use of sophisticated machine learning approaches to optimally combine the available observations and models. This process needs to be done at every forecast location, so the amount of computation is not trivial.
With better and better models and more and more observations, these objective forecasts have gotten very good.
So why is the National Weather Service lagging?
National Weather Service forecasters start with the Weather Service's own system for combining forecasts: the National Blend of Models (NBM), which is not as sophisticated or capable as the one used by corporate systems (such as IBM's weather.com). For five-day forecasts, humans add little value...sorry, but that is the truth.
Other verification comparisons show little improvement of NWS forecasters on shorter periods forecasts as well.
Do I have a weather app do I have on my smartphone? I have to admit: I do.
The WeatherChannel.
Can I beat the forecast of the Weather Channel? Yes, perhaps half the time, using the higher resolution UW modeling system and an intimate knowledge of the local weather. But not always, particularly for run-of-the-mill weather situations.
The days of human forecasters will soon be over. However, human forecasters still have a vital role to play in communicating the forecasts and helping local governments and others to make optimal use of computer-based predictions.
When I was with the NWS (ret 2011) we were evaluated not by absolute accuracy but against the models -- were we 'adding value' to the numerical forecasts? At my station KEYW we did best in the short term, next 3 hours. (Our weather often takes place on very small scales of time and distance in the Florida Keys.) But our #1 job was saving lives and protecting property by issuing accurate and dependable warnings. I sincerely hope that the NWS will never allow a machine to issue warnings but that they are always signed by a person.
ReplyDeleteCliff, you must have seen Google's GraphCast come out recently, https://www.science.org/doi/10.1126/science.adi2336? Claims to be the best out there (at least at a global scale). Could this be the new paradigm for modeling? You've argued for more compute for US weather models, but is that still the right direction? More TPUs for weather models?
ReplyDeleteLiving near Ellensburg, the forecast of the NWS comes from Pendleton, OR., 130 miles to the southeast. The local office at KELN was closed in 1955, I think. They, nor the Spokane or Seattle offices know we are here.
ReplyDeleteLike John K at 5:43, I dislike the commercial sources.
I'll look at a few of the others.
Thanks for the discussion.
Agreed. I tried the Weather Channel app and it was painful--way too many adds, etc. to even find the weather. It's $2/mo to get rid of the noise. For mobile, I use National Wx for iPhone which uses NWS data. It includes the forecast discussion as well as radar and satellite and forecast models if you pay a few bucks a year. It's a very useful app. On my PC, I use Wunderground (with an ad blocker) for their 10 day outlook. The Microsoft Start weather app Cliff mentioned looks interesting.
ReplyDeleteForeca looks pretty interesting. It's amazing that automated forecasts have gotten as good as they are.
ReplyDeleteOn the contrary, in the Financial Times today: "DeepMind published a paper in Science which suggests that the pre-eminence
ReplyDeleteof this system might soon pass. Artificial intelligence may have brought us to
the cusp of a new paradigm in weather forecasting. Rather than the laws of
physics, Google’s GraphCast system is based on an analysis of 39 years of past
weather. Rather than running on a supercomputer the size of a volleyball court,
it runs on a single laptop. Rather than taking two hours, it takes less than a
minute. And rather than “numerical weather prediction” — the technique
behind the modern discipline — it is “machine learning-based weather
prediction,” a fundamentally new approach with staggeringly precocious
results.
They also talked about it on the All-In Podcast. Be interesting to hear Cliff's take on it.
DeleteWhat variables are being forecast? Tmin, Tmax, and 24-hour precipittion at 5-days?
ReplyDeleteWeather Underground seems to use the same data as Weather Channel, in a much more interesting format. It's rarely wrong.
ReplyDeleteI agree with the posters about the value of NWS forecast discussions. It's the only way I know about to get a good sense of underlying confdence in forecasts, and they're sometimes a great source of info regarding what to expect with convergence zones across SnoHo and King Counties.
ReplyDeleteHaving said that, I'm not sure that NWS even recognizes the opportunity or its value. This is especially true when it comes to focusing attention and forecaster bandwidth on the Seattle-Tacoma-Bellevue metro, which is home to 4M out of the 6M total residents in western WA.
Some discussions, for instance, talk at length about how far south a convergence zone will drift and how strong it will be. Many others, however, don't address this issue at all, or they address only in terms of impacts on SEA or BFI. There are similar inconsistencies in addressing probability or degree of rain shadowing. And I have never seen a discussion even try to address often significant variations between post-frontal weather just downwind from the Seattle skyline vs weather just about anywhere else in the metro. (This is absolutely a real thing; I have learned to leave the house and drive north when I want a chance at interesting photos, because again and again, Capitol Hill is where convergence zones go to die).
Long story short: NWS has the potential to add value for us here in W WA, but it doesn't seem to recognize that opportunity or want to articulate it to its forecasters. It may wish it had paid more attention to such things when TPTB start asking why we need human forecasters at all, especially in a place where short-fuse severe weather products are a rarity.