Saturday, December 5, 2015

The Lessons of a Forecast Failure

On Thursday we had one of the most substantial forecast failures in years;  one in which a major windstorm was predicted for the coast, with strong winds over the interior.   The most stunning thing about this forecast failure is that even model forecasts only 12 hours old were in substantial error.  Normally, forecasts are quite good that close in.

The analysis at 4 PM Thursday (of sea level pressure--solid lines, and lower atmosphere temps--colors) showed a modest trough over western WA.   Blustery, but nothing major.   Winds were stronger in NW Washington, but not exceptional

 But look at the 12-hr forecast from the UW WRF model (4-km resolution) valid at the same time.  A VERY different story.   Deep trough. Strong pressure differences (pressure gradients over us).  That is what provoked the major warnings.

 And there was some consistency in the forecasts.  The model run initialized 24h before the verification time (again 4 PM Thursday) was very threatening, but a few hours slower.   The UW model begins with the National Weather Service GFS model, which also went for a big event, as did the NWS NAM model.


 But there were reasons to be worried about this forecast, even if some of our best models were going for a major storm.

First, it was a very small scale storm that did much its development during the 12h before landfall.  Furthermore, it was moving very fast.   To illustrate, here is the initialization of the 12-h forecast valid 4 PM Thursday.   Just a modest trough off of northern CA.


Six hours later (10 AM Thursday), the forecast had it deepening into the 986 hPa low center with very strong pressure gradients. 

This is a very difficult forecast...small systems, fast movement, rapid development.  Get any of that wrong and your forecast is toast.  In contrast, some weather phenomena, such as atmospheric rivers and big storms, are very large in scale and generally far more skillfully forecast.

But then there were other hints.  A major new tool for forecasting is ensembles, running forecast models many times.  This allows exploration of the uncertainty of the forecasts.   Let's look at the National Weather Ensemble System (SREF) for surface (10-m) winds along the Washington (Hoquiam) for the ensemble forecasts initialized (starting) at  7 PM Wednesday evening.  Each line is another forecast, time is on the x axis (increasing to the right), and black line is the forecast mean (of all the forecasts).  Remember 00Z Dec 4 is 4 PM Thursday.

Can you see there is a large variation in the forecasts?  Some were strong, but many were not.  There was a great deal of uncertainty in the wind forecasts. 


To put it another way, many of the ensemble members were NOT producing a strong low center that moved across our region. 

Another hint of trouble was that the European Center model did not produce a strong low over us at 4 PM Thursday (see below).

So my colleagues at the National Weather Service have a dilemma:  some forecasts were going for a major wind event, but there was a lot of uncertainty.  What should they tell the public?  They decided to tell the public about the threat.  But they perhaps should have given more information about the uncertainty of the forecasts.

But there was a major new tool that was very helpful during this forecast bust:  the High Resolution Rapid Refresh System (HRRR).  Rather than being rerun every 6 or 12 hours, like the models shown above, HRRR is run EVERY HOUR with new observations (but only 15h ahead).   HRRR caught on to the lack of development during Thursday morning and by mid-day clearly indicated that the windstorm was not going to happen.

Using HRRR and examining observations is why I sent out a blog after noon, saying that the windstorm was not going to happen.  And it didn't.


---------
And while you are thinking about the winds, don't forget to express your feelings about the proposed termination of KPLU, in the ill-advised sale to UW.  More information here and here.  This sale is marked by secrecy and  incorrect information by PLU and UW administrators.  KPLU can be saved if listeners tell the UW Board of Regents and the PLU administration to back off.

If any of you are interested in attending a strategy meeting for saving KPLU on Sunday, Dec. 6th at 2 PM, please let me know (you can email me for more information--search on "cliff mass email" to my email address)

4 comments:

sandy knoller said...

I was quite taken in by the prospect of the modeled storm becoming a serious, dangerous event for us in Western Washington. Previous extended model runs had prepared me for interesting weather on Thursday. The Saturday 28th and Monday 30th 12 UTC runs both had lows forming late Wednesday off Northern California that seemed to rapidly intensify and come ashore Thursday. Saturday's 129th hr (1:00 pm Thursday) and Monday's 90th hr (10:00 pm Thursday) put a respectable low just offshore west of Portland. Other model runs weren't consistent with that particular feature, but most seemed to show offshore low pressure development of some sort on Thursday.

Tuesday's 12 UTC run had one form early Thursday morning just west of Washington, cross Vancouver Island and intensify slightly byThursday midday. While the low itself was not overly impressive, the 10m wind gust model had 50+kt gusts offshore and SW of San Juan Island. So I was ready when the Thursday 00 UTC run showed an intensifying low at 21 hrs (1:00 pm Thursday) right where the lows had been modeled Saturday and Monday. The prospect of more widespread 50 kt gusts certainly didn't seem like much of a stretch, either.

Just want to add – Yes to good quality locally oriented radio and down with overly restrictive programming formats. Followed you to KPLU and I'm hoping the effort to keep KPLU intact is successful. Please keep us posted.

sunsnow12 said...

"The lessons of a forecast failure."

Outstanding.

Why is it this is the only place I ever see a meteorologist making an honest assessment of a forecast gone wrong?

Eric Danielson said...

Pugetropolis may not have gotten the winds, but Saturday's windstorm on the north Olympic Peninsula was the most sustained one I can remember going back 15 years. It was at least 15 hours long from early Saturday morning to well into the night. Lots of damage to houses and signs. The strange thing was that it was localized to within just a few miles of the shoreline. Folks living 3 miles inland reported no wind at all.

Joseph Ratliff said...

I have another forecast failure...

Last night (Sunday the 6th) NWS had no mention or forecast for a full-fledged windstorm for the Olympia / Lacey area with the incoming front. It did forecast North Interior and Coastal areas ... and those did verify.

Seattle hardly noticed, but Tacoma and Puyallup also joined the fray.

Winds were mainly SSE in the Olympia / Lacey area ... and at some points sustained between 30 - 33 mph while gusting between 45 - 52 mph.

But no advisory was issued ... and no forecast addressed wind speeds even close to that range, at all. This event lasted from ~ 8pm to about 2am. Power went out in many places (is still out in many places) in the Olympia area.

So unless I missed something ... this was an terrible forecast issued by the NWS Seattle for the Olympia area on December 6th, 2015.