March 14, 2021

Broken Water Pipe Takes Out U.S. Buoy Data

I have gotten a number of emails from folks asking about the lack of U.S. buoy data during the past several days.  

They are right...something has happened.  A broken pipe and flooding has closed down a key NOAA data facility in Silver Spring, Maryland, and there is no estimate of when this data source will become available.  And, in fact, other important data sources are also being affected, such as the coastal marine sites (CMAN) and even some airport locations.

The NOAA National Data Buoy Center page notes the situation:


Buoy data over the oceans is a very important weather data source, in support of both numerical weather prediction and marine activities.  There are, in fact, two main types of ocean buoys:  fixed or moored buoys, which possess a relatively wide range of weather/ocean observations, and drifting buoys that move around the ocean freely (see pictures below).

Fixed buoys:


Drifting buoy:

To give you an idea of the distribution and number of the buoy data, here is an example of the drifting buoy data locations immediately before the water leak (which occurred on March 9).  Quite a few! 


And the locations of fixed buoys (including coastal marine--i.e. lighthouses) are found here:


The loss of much of this data is a problem for numerical weather prediction, which uses oceanic data to provide information on surface conditions over the oceans.   The plot of the buoys coming into the U.S. GFS global weather forecasting model shows the seriousness of the problem, with 3/4 of the buoy data gone.


And the coastal marine data has gone to zero.


Now the lack of marine data is not only an issue for weather prediction, but it can be an even larger problem for those involved in marine operations, who use weather data for the management of ports and sea lanes.   Thus, the loss of surface marine observations put life and property at risk.


This outage is important because it shows the vulnerability of critical data infrastructure at NOAA, with dependencies on single servers and the lack of backup systems.  There are others.    

NOAA operations acutely need "hot" backups that can immediately take over when primary servers go down.  

This is not only true for the ingestion of observations, but for the critically important weather modeling as well.  Cloud-based backups are one way to do this at a reasonable cost.    And there is always the old fashion way of maintaining independent servers in different locations.

Whatever approach NOAA takes, creating a more robust data and modeling infrastructure should be a high priority.   

I am sure there are folks at Microsoft and Amazon who would be happy to help.😋

5 comments:

  1. Is this a normal occurrence that so many buoys are down?

    ReplyDelete
    Replies
    1. The buoys are not down. A broken pipe and flooding has closed down a key NOAA data facility in Silver Spring, Maryland causing 75% of the buoy data to be unavailable.

      I am sure that a broken pipe is not considered to be "a normal occurrence".

      Delete
  2. Hello Prof Mass. This query isn't related to this blog but an unusual phenomenon is occurring in and around Port Townsend today (3/14). It is now 3:30 and the temperature has dropped from a high of 46 to 39 degrees currently. Yet when I look at the Weather Underground map, it is in the 40's everywhere else except a for a small circular area around Port Townsend. What is causing this localized temperature variation? Thank you for your blog and podcast!

    ReplyDelete
  3. Once, while hiking the Olympic Coast alone in April from Ozette to Shi Shi, I came across a University of Oregon Oceanographic buoy with a "Reward if Found" plaque affixed to it with an 800 number. The tether was pig-tailed (clear evidence of entanglement in a ship's screw). After hauling it up on the beach above the tide line, I completed my trip and asked the Rangers for a phone to report my find. I told them that I wished to report a buoy in a particular section of beach. They said "no."

    Upon my return to land-line phone country (2003), I called the 800 number and found out that Park Service employees had claimed my find thirty minutes before me. They got $500 as a reward. There was no one on that beach but me in April. The Ozette River crossing was a self-belay epic adventure going North.

    Moral of this story...keep your buoy finds secret. Bury them, and tell no one if there is a reward offered. If there is no reward...just do it for science.

    ReplyDelete
  4. A single point failure for such an important part of our infrastructure? How disappointing.

    If the CIA can contract out their classified work to a private party, maybe NOAA should consider that in their next round of budget negotiations.

    ReplyDelete

Please make sure your comments are civil. Name calling and personal attacks are not appropriate.

The Deepest Low Pressure Center in Northwest History? Damaging Winds West of the Cascades

 We may break a significant record tomorrow, with the rapid development of the deepest low-pressure center (lowest sea level pressure) on re...