June 27, 2016

Who provides the best weather forecasts?

One question I frequently get is: where can I get the best weather forecasts?   Quality varies substantially and the answer is often different between garden variety forecasts and more difficult rare/severe weather events.

So what about garden-variety weather predictions?  What source is best?  The National Weather Service, Weather.com or Accuweather?  Or what?

There is a web site, forecastadvisor.com, that provides average verification statistics for the last month or year for a large number of locations.  Our independent verification for some Northwest locations suggests that this site is relatively reliable.  

Here is the table for Seattle for the last year.  For temperature, they consider a forecast accurate if it is within 3F.    Precipitation is not so clear, since they consider the forecast as rain if there is any chance of precipitation in the forecast.   So you might dismiss the precipitation evaluation...although all forecasts are verified in the same way.

Considering temperature, the three best are the Weather Channel, the Weather Underground, and Accuweather--I doubt whether the differences are statistically significant.   The Weather Service is a step down.


What about Spokane?  The same three are on top, but this time Accuweather is the leader, particularly for low temperatures.  They are good at low temperatures.  Again, the NWS lags a bit.


Finally, lets look at Yakima, the primary agricultural center of our state.  Same three in the lead, and Accuweather is again better at the low temperatures.

So why do the Weather Channel, Weatther Underground and Accuweather produce better forecasts than the National Weather Service on average?    The key reason is that they generally make use of multiple  weather forecast models (e.g., US GFS, European Center and UK Met Office models), and  then do sophisticated sophisticated statistical postprocessing to combine the forecasts in an optimal way.   The National Weather Service has lagged in statistical postprocessing, depending heavily on a forty-year old system called MOS, Model Output Statistics.    Thus, it the guidance provided to National Weather Service forecasts has been less than state of the art.

Fortunately, this is changing.   During the past year, the NWS has developed the National Blend of Global Models, which statistically combines an international collection of global models and traditional MOS to produce better forecasts.   The results so far are encouraging, with reduced error and lower biases.

With these advances, I would expect the National Weather Service average statistics will move closer to the Weather.coms and Accuweathers during the next year.

The National Weather Service human forecasters do play an important forecasting role, particularly when unusual and extreme weather occurs.  Events in which the models and statistics can fail, but where long experience and physical insights can be critical.  Some private firms, like Accuweather, have also invested heavily in a substantial human forecaster contingent.

The National Weather Service forecasters are positioned in a two-tiered system:  the roughly 120 forecasts offices spread around the nation, and national speciality centers, such as the the Storm Prediction Center in Norman Oklahoma or the Weather Prediction Center in Washington DC.  The first group is responsible for local prediction, while the latter specializes in specific types of extreme events, like severe convection (Storm Prediction Center) and heavy precipitation/flooding (Weather Prediction Center).

The Weather Prediction Center has found that humans can provide positive impacts for very heavy precipitation events over numerical model guidance (of course the humans START with the numerical guidance).  The Storm Prediction Center is absolutely world class, providing the best severe thunderstorm warnings in the world.

The future will be one where garden variety weather is mainly taken care of by statistically corrected ensembles of models, with humans spending most of their time dealing with extreme weather, resolving with model problems, and communicating forecasts to the public and critical user communities.  

Even  today if you are looking for a weather forecast on average day, the forecast from your typical weather app (weather.com, accuweather, etc.) are fine.  But when severe weather is possible, turning to an outlet in which humans are carefully following the situation is advised.

18 comments:

  1. Invaluable data, for sure. How do the local TV stations stack up? I believe they are just repeating "professional" forecasts from the NWS, or others, and don't really do their own forecasting, although there is considerable variation in temps (I saw the four local stations on June 18, and one station predicted 6/20 would be in the low 80's, while another predicted 72, which, it turns out, was more accurate.

    Thank you for your continuing very enlightening ways, Cliff! Good work.

    ReplyDelete
  2. No rating/mention/inclusion for the UW's own redoubtable probcast?

    ReplyDelete
  3. Great article, and I mostly agree. However, Accuweather is not accurate, or at least seems to be highly inaccurate after 5 days; for example look at their crazy 30-45 day forecasts, or even 7 days. They seem to either be way too warm, or way too cold, and always predict way more snow than any of the others. If WU or NOAA says 1-3 inches expected overnight for a storm, they are always saying like 3-5 inches. Accuweather and to a lesser extent, TWC seem to thrive on sensationalism in the forecast, kind of like disreputable news sources that shall not be named. I think it is enough to just report weather, not under or over do it.

    ReplyDelete
  4. Thanks, Chris. Any suggestion about which site is best for our mountain passes? I'm especially interested in Snoqualmie, since it's often just a degree or two that makes the difference between snow and rain.

    ReplyDelete
  5. Ditto MRT's comment about AccuWeather, at least for one location in NCentral OR. I'm curious as to why Intellicast isN't include (haveN't tried to discern their accuracy).

    ReplyDelete
  6. Yup...the Weather Channel has always been my favorite.

    ReplyDelete
  7. Great question, I'd like to know the answer to this too!

    ReplyDelete
  8. I checked out forecastadvisor.com for Bend, Oregon, and it's giving me information about Eugene. Doesn't instill a lot of trust!

    ReplyDelete
  9. Thanks for the information. Our informal look at the forecasts in Port Townsend and the Olympic Peninsula slightly favor Weather Underground. That may be because they include local private stations.

    ReplyDelete
    Replies
    1. Yep 100% agreement. Weather Underground, and the network of PWS's is what I use also.

      Delete
  10. I like Tom ( above comment)would like to know how local TV stations fare. Since Jeff Renner retired I don't feel there is a knowledgeable person on TV. One exception would be Rich Marriott, but he is on way too early for me.
    Thanks for a great blog about weather.

    ReplyDelete
  11. Didn't see a place to email you so I'll just add this here. Can you comment on this story that I see flying all over social media right now? A lot of the "logic" seems contrived. Thanks.

    https://paulbeckwith.net/2016/06/29/jet-jet-stream-crosses-equator-is-this-unprecedented/

    ReplyDelete
  12. For the records the NWS Storm Prediction Center does not provide warnings, it provides watches and forecast guidance. It is my opinion that their accuracy has slipped considerable this year.

    Here is just one example: http://www.spc.noaa.gov/products/outlook/archive/2016/day1probotlk_v_20160509_1300_torn_prt.gif An EF-4 occurred in the 5% area and an EF-5 ("significant") tornado occurred outside of the 2% area. There are other examples that could be cited.

    ReplyDelete
  13. You said "The Storm Prediction Center is absolutely world class, providing the best severe thunderstorm warnings in the world." The SPC does not issue severe weather warnings, only watches and outlooks. The local WFO (Weather Forecast Office) issues severe weather warnings.

    ReplyDelete
  14. Hi Cliff.

    I've seen this statement on 2 different "news" sites (I really take the term "news" with a grain of salt from these sites), but I was wondering what you thought of this. Frankly I was surprised you didn't have anything written up about it yet.

    http://www.rawstory.com/2016/06/unprecedented-scientists-declare-global-climate-emergency-after-jet-stream-crosses-equator/

    ReplyDelete
  15. Pasha Stewart...... You may have seen by now a Washington Post article. Cliff Mass is quoted, along with other expert opinion indicating that the Paul Beckwith blog post you enquire about is sensationalist bunk. The most significant red flag for us (the inexpert) is Paul Beckwith himself, who is not a climatologist. If anyone had first paid due attention to this simple fact, his blog, one might hope, would have gone nowhere.

    The really sad thing is that, as usual, many tribal loyalists in the environmental political movement continue to fly his flag, despite being made aware of his profound lack of credibility and the repudiation of his claims. These are people who claim consistently to follow the science, such as at least one of our Federal Green Party candidates up here in BC. Delusional pseudo science and rigtheous certainty permeates the entire spectrum of climate politics, not just the right wingers.

    ReplyDelete
  16. I was a student of Cliff's in the late 80s and have worked as a meteorologist for coming up on 29 years now. If I want to know what the weather might do on my days off, I poke around at the UW WRFGFS output, I look at the very nice dprog/dt feature, and I bring up this nifty graphic that gives gfs ensemble temps aloft and precip for Seattle:
    http://www.wetterzentrale.de/pics/MT8_Seattle_USA_ens.png

    ReplyDelete
  17. I enjoy reading the WaPo's Capital Weather gang but... wow... little close to this article don't you think (https://www.washingtonpost.com/news/capital-weather-gang/wp/2016/07/06/when-you-really-need-the-best-forecast-which-app-should-you-trust/)? Hopefully you will receive the proper referencing or acknowledgment your post deserves.

    ReplyDelete

Please make sure your comments are civil. Name calling and personal attacks are not appropriate.

Should I Let AI Write My Blogs?

I spend a lot of time thinking about and writing my blogs.   Could AI apps do the work for me?   Could any of you tell the difference? Well,...