This summer, I have gotten a number of complaints about the poor quality of our local forecasts.
Several of you have suggested that the weather is often much better (generally warmer) than predicted by the National Weather Service and several media outlets.
I have to admit, I had noticed the same thing. My calibration as a long-term forecaster seemed to be off, with temperatures surging far higher than than in the past for the same large scale conditions. In the gym, I mused about this with Nick Bond, State Climatologist. He agreed, and we talked about some reasons this could be true.
But is it?
Have forecasts been unusually bad this summer?
To answer the question, I made use of a wonderful web page created by Luke Madaus, a Ph.D. student in my department. Let re show the the results.
Here are the high (max) temperature errors for the past 200 days for Seattle from the National Weather Service. Forecast minus observed is shown, so negative temperature errors indicate forecasts being too low. From Feb to May they were up and down, but during June trough early August the Weather Service was too low almost every day (by roughly 3-4F).
Now the central tools of National Weather Service forecasters (and others) is something called MOS or Model Output Statistics. MOS takes numerical model output and corrects it based on past statistics, generally using at least 2 years of history. This is called statistical postprocessing. MOS generally improves on the numerical forecasts by lessening systematic biases.
Here are the errors from MOS based on the NWS NAM model. Same problem: too cold during the summer....even worse than the NWS forecasters (who at least improved things a bit).
Another way to check out this issue is to go to the weather forecast verification site forecastadviser.com. Below is their accuracy statistics (% of time the forecasts were within 3F of the truth)...in this case for Seattle. For the last month, the high temperatures forecast by the National Weather Service was accurate about 36% of the time--a bit worse than persistence, which is a very low bar. Persistence is simply forecasting for tomorrow what happened today. Not good. The Weather Channel did far better.
I suspect there are several reasons, both connected with the warm temperatures. First, soil moisture is less than normal because of the dry/warm conditions. The figure below shows the percent of normal of soil moisture (from the UW Hydrological Monitoring Site). Parts of western Washington are at record lows, many with less than 10% of normal soil moisture. Dry soil tends to warm up more, everything else being equal (less water to evaporate and evaporation causes cooling).
Second, we have extremely warm water offshore (the Blob), which causes the air moving into our region to be warmer at low levels.
The models know about all this, but the Model Output Statistics, trained on years of normal conditions, make the wrong adjustments because they use long-statistics. This is a hypothesis, but a reasonable one.
Why is the Weather Channel doing better? Because they don't use MOS. They have their own very sophisticated model statistic postprocessing system. One that is more adaptive and able to compensate for short-term unusual conditions.
Tomorrow will be very warm by the way...enjoy.
"Several of you have suggested that the weather is often much better (generally warmer)"
ReplyDeleteThere are many of us that would argue the weather has often been much WORSE than predicted because of the warmth. When do we get our happy weather back? :-(
So our NWS friends reading this blog should be able to make some late summer adjustments, and improve their accuracy :)
ReplyDeleteAnd it is not only high temperature forecast. The rainfall predicted in Cliff's post of Thursday the 6th looks like it miss the mark by a good margin.
ReplyDeleteCliff, at what point in time is each day's forecast judged against what comes in as the actual temp? (ie 8/12 forecast used from 3 days ago, 10 days ago, first attempt, etc?)
ReplyDeleteThanks for owning up to it. Now I wish our talking heads on our local news channels would do the same!
ReplyDeleteI'd swear it's been more than 3-4 degrees off in our neighborhood (West Seattle). Might just be a microclimate effect. But there have been lots of days where the forecast says 75 and we end up at 82, the forecast says 82 and we end up at 90, or--that one terrible Sunday--the forecast says 90 and we end up at 98. Plenty of times I've checked one of the weather apps in late afternoon, and it says the high today was to be X, while prominently displaying the current temperature of X + 6 (or 7 or 8...).
ReplyDeleteI'll give Cliff credit: we did wake up to rain yesterday, and more looks to be on the way, albeit in spotty showers. But when it's hitting the high 80s with blazing sun in between those rare showers, the garden still dries right out. My withering drought-tolerant plants and I agree with Cascadian Engineer: this is not what we'd call "nice" weather. But I suppose it's nice enough if you can find a spot in deep shade with a breeze.
I'm curious how things like the thermal mass of buildings and concrete and asphalt paving of roads, parking lots etc. are accounted for in weather temperature forecasting. It seems like the thermal masses could add to warming effects.
ReplyDeleteI am more interested in the missed forecasts related to rainy/cloudy weather. Last weekend was a good example, Sunday was suppose to be the rainiest of days, but ended up one of the hottest with no rain in sight. Granted, the weather forecasts were all changed seemingly 12 hours before Sunday came.
ReplyDeleteI notice a lot of flip-flopping in the forecasts, especially when it comes to rainy weather.
Weather models are based on the old climate; perhaps this is already the newer warmer climate characterized by novel systemic changes resulting in regionally unique new weather patterns.
ReplyDeleteEnjoying it so far.
ReplyDeleteIs that online tool created by Luke available to the public? I'd be interested in looking at it.
ReplyDeleteOver the last month of so NWS Seattle has been struggling with the its computer system, AWIPS II. Implementation of this system around the weather service has been disastrous and Seattle is no exception. Among other problems, the MM5 models have not been available to us when we are editing grids, and many of the other temperature guidances we have used are either not available or are somehow damaged. Recovering from this "upgrade" will likely take several more months. As usual in the NWS, each office has to figure every computer problem out for itself, with no help from our regional or national offices. I am personally shocked that our Max T forecasts seem so bad. To lose to persistence is indeed poor--note that in the spring forecast contest our forecasts crushed persistence, like everyone else's.
ReplyDeleteAfter a couple of years of trying via these comments to sway Cliff's ironclad belief that warmer always means better, I've given up. You'll never get Cliff to stop calling 88 degree weather, in a town with very few home air conditioners, "nice". Or to refrain from saying that it's going to be very warm, so "Enjoy", as opposed to Hang In There or Buck Up or "try not to die in your apartment".
ReplyDeleteActually, the parenthetical "weather is much better (generally warmer)" is a small victory, or I'm claiming it as a victory anyway. Finally a tiny bit of acknowledgement that warmer does not necessarily equal better for a large portion of his audience. I'll take it!
My Bodacious corn matured so fast it must have thought it was growing in the Yakima Valley. Huge ripe Brandywine tomatoes in early August in West Seattle? Incredible.
ReplyDeleteWhy not just admit that we do not know as much about weather and climate as we pretend to know. It reminds me of doctors I know with the God complex. Our knowledge of our world around us is greater than it has ever been, but still largely unknown in many disciplines.
ReplyDeleteI think we all admit that, Jeff...but, I will tell you, weather forecasting has very much improved in my 64 years of life...
ReplyDeleteHave to chuckle at Joe's comment; no doubt he voices the opinion of many locals. Cliff is bound & determined to the "warmer is better" notion and while I'm one native who does enjoy warm temps & sun, at this point those who don't yearn for decreased temps and moisture are few & far between!
ReplyDeleteIf I wanted warm I'd move to California. I miss the weather that makes this special beautiful place what it is. Bring back the norm!!
ReplyDeleteRight on Joe, right on! And one can always read Sistek's blog; he comes from the opposite angle.
ReplyDeleteYou couldn't have said it better Jim! I moved AWAY from California to get to some nicer COOLER weather. You want warm and sunny.......don't live in Seattle........well that's how it used to go anyway!
ReplyDeleteI was born and raised here, but lived in So Cal for 4 years. Just moved back in March 2015. I will tell you I kissed the ground when I got back, excited to be out of the arid hot smoggy wasteland of California, back to cooler wetter weather and green landscapes, and vowing to never move away from Washington again. While I stand by that you can imagine my dismay when I discovered the horrible hot, dry and brown conditions have extended up here and further beyond. Let's just hope this blob and all the climate havoc it has caused goes away and that this change isn't permanent.
ReplyDeleteLast couple day forecasts have been wrong for Maury Island. Yesterday our high was 75F instead of the predicted 84F. Today it's supposed to be nothing but hot and sunny. Instead we have clouds 67F and at 11am, developing thundershowers.
ReplyDeleteDoes the smoke from CA change the weather here?
^^^ Wholeheartedly agree with my colleague Chris Burke's comments above. The "upgrade" to A2 crushed our forecasting tools in Portland as well, and it took a whole year to recover...mostly. That said, Cliff's comments are spot on as well imo. MOS relies on climatological norms, and this has certainly been an abnormal year in many respects. The statistical equations going into MOS are probably making a lot of assumptions that are usually okay, but are not trivial this year.
ReplyDeleteBy the way, gotta stand up for our neighbors up north... based on the scoreboard above, NWS Seattle actually "won" versus TWC/wunderground. RMSE 2.727 for NWS; 2.872 for TWC. If NWS Seattle eliminated their -0.6°F bias, it would have been a blowout! :) Impressive considering the technical challenges they are facing, which I know all too well.
What I've been noticing is that the forecast for the next day or two is often accurate, but there is a looming rain always being forecast about a week out.
ReplyDeleteI was involved in planning an event in mid-July. We got drenched on the same weekend last year so even with the hot summer we were keeping an eye on the forecast. The weekend before we had a moment of panic at the prediction of thunderstorms on the day of our event. And then as the week progressed the thunderstorms got moved further and further back.
Facing just the opposite her eon the Delmarva in Maryland. Our wet soil is causing the MOS to over forecast the high temp. I went for 82 today and so far on the money. NWS went for 84-86.Interestingly the raw 2 meter sfc temps. off of the NAM 4km have been very close. Just slightly under the actual high.
ReplyDeleteThis kind of thing is why I never tire of forecasting!
ReplyDeleteIt seems to me that not only are the temperature forecasts off, but the forecasts themselves vary significantly (wildly)from the previous release.
After my 20 plus years of fire fighting It seems obvious that poor forest practices, such as cutting arid areas,and expecting rotation harvests to be sustained, is the largest culprit in our fire history.
ReplyDeletethe first review of our western forests done under Rooselvelt and Pinchot .This multi year study indicated that 90% of the west was not sustainable for timber cutting. Congress didn't accept this and under the threat of being terminated our very first group of silva culturist found they were to redo the study to include all of the west for cutting. Many of these newly minted silva culturist, to there credit resigned instead of lying for the cut.
As long as we cut strictly to supply the timber lobby instead of following the forest services first and founding principle of watershed protection ,we will have lots of big fires.
Question on the blob. The water is warmer so is it possible that what storms we get this year will be stronger because they can gain energy from the blob. Or is the "warmer' water going from the average 55 degrees to 59 degrees and so not enough to provide much additional energy.
ReplyDeleteMy guess would be average precip as blob and el nino battle it out this winter, but warmer than normal again seems to be a lock. As Cliff pointed out though, our winter was so far below average for the passes last winter it would be hard to be on par with that anomoly again. Though can't imagine blob/el nino combo is any less rare.
ReplyDeleteThe more I read the comments on this blog by amateur "scientists", the more I appreciate Cliff's sober approach to following the science. Now the extremists are labeling everyone a "denier" if they don't see the sky falling on their heads, right now.
ReplyDeleteI've followed the science on AGW since the issue first arose (long time ago), and see it as a huge global threat to world population and political stability, but by the definition of some of these posters, I suppose I'm a "denier" too.
Some of these discussions are the reciprocal of the true deniers, who will (falsely) interpret a cold winter as evidence that AGW is bogus. These posters are taking a couple of years of weather and using it to fan the flames, if you forgive the pun.
The backlash from over-hyping weather as evidence of near term disaster will likely drive more and more people like me into a "moderate" camp that refuses to acknowledge the extremes from both sides, and instead follows the sober science. If that makes me a "denier", then so be it.