December 19, 2011

The Surface Data Revolution

An important change has occurred during the past decade that many are not aware of:  a huge increase in the number surface observations.  Go back 20 years ago and there were perhaps 1000 regular observations around the U.S., mainly at airports.  Roughly 30-40 regular observations over Washington State at primary and secondary airports.

 In contrast, today there are certainly at least 100,000 surface stations that report hourly and distribute their data in real time via the Internet over the U.S., and several thousand such observations over Washington alone.    In fact, there are so many observations available today, that if we plotted them all the weather maps would turn black with them over many parts of the Northwest.  Here is an example from earlier this year--there are a lot more stations available now:

So what has happened to provide such a treasure drove of information?  

First, there are many groups in the observing business right now...organizations that for whatever reason have established real-time observation networks.  Some local examples:  Seattle Public Utilities, the State AgWeatherNet, Washington State Ferries, Washington DOT, local private utilities, NW avalanche network, School Weather Networks associated with local TV stations, etc.---the list is long.  And then there are hundreds, if not thousands, of Northwest residents that has purchased capable weather stations and made their data available in real time via the WeatherUnderground or similar online services.

Which gets to the other parts of the revolution:  the availability of very good amateur equipment (like the Davis Pro weather stations) and the omnipresence of the Internet.  So for a very modest investment one can collect excellent quality information and make it instantly available to the world (something impossible two decades ago).

At the UW we collect more than 70 networks in real time, quality control and archive them, and make them available to all of you at our web site,  Here is a sample for Seattle right now (and our system does not plot every observation!):



 Interestingly, there are some places where there is several stations within a half mile, while gaps remain in others. 

This huge amount of surface data for the first time gives us insights into the local variations of surface weather features.   And enhances our ability to do very short term forecasting---called Nowcasting.  The national weather service also collects another of weather networks, through their MADIS system, and you can get at a lot of this information via their very nice web site:

http://www.wrh.noaa.gov/mesowest/gmap.php?map=sew


And this surface data revolution is just warming up.  Some new smartphones take continuous pressure measurements (as a way of getting height)--can you imagine if that data was communicated to a central site in real-time?  And it is matter of time before all cars have internet capability--imagine if each reported temperature and other weather parameters in real-time as they drive around.  And there is more. 

Although surface data is very useful, it is not enough.  Meteorologists need 3D data to forecast the weather since we need to describe the atmosphere's entire volume to run our models into the future.  And surface data often has problems--wind sensors placed behind an obstacle, temperature sensors too close to a building or not properly shaded; the potential issues are many.

Securing huge volumes of upper air data is another revolution...but one that will wait for a future blog.

9 comments:

  1. It appears there is an over saturation of surface data reporting sites in areas, yet many holes appear to exist in other areas. Are there places today where data is not being collected and reported that would be valuable to forecasters and researchers? If a "wish list" could be created I am sure there are a number of individuals and or groups willing to assist in filling these holes.

    ReplyDelete
  2. This is an amazing story,one more reason to brush up on math/statistics and modeling. I was wondering it we were starting to see open source weather forecasting models that allowed the curios to do their own forecasts?

    ReplyDelete
  3. Hey Cliff...great thoughts on this article. Most of us do not really think of all the technology we have and really what would we do without it now?

    ReplyDelete
  4. Thanks for recognizing the contributions of the folks with personal weather stations. I have been up loading data to Weather Underground and CWOP for three years from Copalis Beach (just three miles from the new radar).

    Those interested in weather monitoring in exposed coastal marine environments should recognize that the initial investment in equipment will probably be followed by some periodic expense for maintenance. The salt water environment, heavy rain and wind takes a toll on even the best equipment. In three years I have replaced three temperature/humidity sensors, two wind birds and one integrated sensor module on my Davis Vantage Pro 2 system.

    Paul Middents
    CWOP DW 1622
    Weather Underground KWACOPAL3
    Copalis Beach, WA

    ReplyDelete
  5. I think I will chime in on what both Thom and Paul said. Regarding holes.... I live at the very north end of the Kitsap peninsula on the water (South Admiralty Inlet). I am the only personal weather station on the water for several miles. Overall the NWS does an outstanding job on their Marine forecast. Sometimes though, there are unexpected events. A couple of weeks ago we had 2 separate events with sustained winds of 25 with gusts to 35 on a 5-15 forecast. I called the NWS service both times and talked with some incredibly nice folks. They were aware of my site, but were not using it regularly. They thanked me for the call and modified the forecast. Now I see that they are visiting my site on a fairly regular basis. Probably mostly for fun, but it is still flattering to know they hit it from time to time. Paul, I have been running my system for 6 years now in a salt environment and have only had to replace the reed switch in my rain collector. Davis gave me the part for free. Imagine that.... a rain collector wearing out in the Pacific NorthWet.... I also run 3 other Davis VP2 stations for our business and have not had a failure in 5 years. I have probably been lucky....

    ReplyDelete
  6. I found this video on "wave clouds" on an Italian newspaper website, but the video is from Alabama!
    http://multimedia.lastampa.it/multimedia/nel-mondo/lstp/104864/

    ReplyDelete
  7. Hi Cliff,

    That's an impressive looking storm out at about 156 W, 45 N (between Alaska and Hawaii). Any thoughts on this one? Its really one of the better organized I've seen this fall.

    Happy Solstice!
    KW

    ReplyDelete
  8. This comment has been removed by the author.

    ReplyDelete

Please make sure your comments are civil. Name calling and personal attacks are not appropriate.

Should I Let AI Write My Blogs?

I spend a lot of time thinking about and writing my blogs.   Could AI apps do the work for me?   Could any of you tell the difference? Well,...