Can you rely on such predictions? Are they really worth paying attention to?
Quite honestly, probably not--and if you do consider them, do so with the knowledge that their skill is marginal at best.
Take this month (October) for example. The official NOAA Climate Prediction Forecast for October temperatures, made on Sept. 19th, was for warmer than normal conditions over the west and MUCH above normal over the southwest U.S.
What actually happened? Nearly the entire west was much colder than normal, with the northern parts MUCH, MUCH colder than normal. A miss. In fact, a big miss.
Or the official 3-4 week forecast, made on October 4th? Warmer than normal over the west.
Such poor forecasts even a month out are not unusual. UW graduate student Nick Weber and I evaluated the skill of the main U.S. long-term forecasting model (the CFSv2) and found that skill is typically lost after roughly 2 weeks (see below and published in the peer-reviewed literature). This figure shows the forecast error (root mean square error) at 500 hPa---about 18,000 ft, a good level to view atmospheric predictability. The situation is the same over Washington, the western U.S., the continental U.S. or global. Skill is rapidly lost the second week out.
While meteorologists struggle to produce improved forecast skill past two weeks, we have gained a great deal of skill at the shorter time ranges, particularly for days 3-8.
So why is our skill improving rapidly for the shorter periods, but not the longer ones?
Because the forecasting problem is very different at the different temporal scales.
For the short periods, forecasting is an initial value problem. We start with a good description of the 3D atmosphere and our models simulate how things evolve. Because of weather satellites and other new data sources, our initial description of the atmospheric has gotten MUCH better. And our models are much better: higher resolution, much better description of key physical processes, and more. That is why a plot of the skill of skill of the 1-10 day forecasts of the European Center has improved greatly over the past decades (see below)
But small errors in the initial description of the atmosphere and deficiencies in our models inevitably lead to growing errors, and by 2 weeks such errors swamp the forecast. The forecasts are not much better than simply using the average conditions (or climatology).
There is hope for some skill beyond two weeks, by taking advantage of the forecast skill available from aspects of the environment that are changing slowly (such as sea surface temperatures, sea ice extent, snow extend, soil moisture). These aspects influence the atmosphere and potentially can torque the atmosphere one way or the other. Essentially, the forecast problem has changed from an initial values problem to a boundary-forced problem (the boundary being the surface characteristics that can influence weather).
But the skill that might be available from the boundary conditions is different---not about the conditions at a specific time, but for the average conditions over a month or season. A good example of such skill is the relationship of the warmer (El Nino) or colder (La Nina) temperatures of the tropic Pacific sea surface and weather around the world. There is some skill there, but it is relatively modest.
Unfortunately, our models still have key deficiencies (such as poor description of thunderstorms) that make it difficult for us to derive all the potential skill that should be available from the slowly changing boundary conditions. A lot of work is needed, but I am hopeful that eventually forecast skill beyond two weeks will improve.
cliff can you agree tat just because it is cold now it wont be like this all winter i thing it will be a poor push threw the year the jan and feb again we will get alot of snow off topic but i need this answered by someone
ReplyDeleteI predict that Chaos Theory (which is actually fact) will never allow specific predictions more than a few weeks out. While it it possible to predict, with considerable accuracy, a general trend, such as the effect of more CO2 or a solar minimum, it will never be possible to predict whether it will be a sunny day at the beach on a specific day next month.
ReplyDeleteCliff,
ReplyDeleteIs there anything about the PNW that makes forecasting (short- and long-term) more difficult than other parts of North America? Are there any particular parts of North America for which forecasts tend to be more accurate?
No kidding! Here's 50 years of failed long range eco/climate predictions:
ReplyDeletehttps://cei.org/blog/wrong-again-50-years-failed-eco-pocalyptic-predictions
What a walk down Memory Lane. Thanks for sharing the link.
DeleteThis article is Cliff Mass at his best.
ReplyDeletehttps://twitter.com/wxbrad/status/1166161031288295431
ReplyDeleteInteresting. On another note -- what is the ski season going to be like? :)
ReplyDeleteSeriously though, this has been my understanding for quite some time. The short term forecasts are remarkably accurate. If you read the forecast discussion, it is even better. They will tell you when they have their doubts about a forecast -- typically when the models disagree. The result is that short term forecasts (a few days out) are remarkably accurate, and even five or six days out are pretty good sometimes (if they have a lot of confidence). The one exception is a snow forecast for Seattle (or basically any sea level location close to the water). As you've explained, there are just too many variables, and one being only a little bit off (too cold, too dry, too far north, etc.) changes everything.
Long term forecasts aren't like that. They are often off by a mile. You might as well look at the woolley worms.
Love this explanation, thank you! I do think the post would be improved (or perhaps a second post is in order), if you explained the difference between our confidence in long range climate (10-100 years) models and season long forecasts (1-4 months). I think your post leaves the reader with an open question: if we have poor confidence in wx predictions beyond 2 weeks, how can we have confidence in our model-based predictions regarding climate change? I think the answer is that these are a difference in type rather than a difference in scale, but I would love to have someone with your degree of clarity on the matter spell it out for me.
ReplyDeleteWe can't rely on forecasts 6 months out, but we're supposed to spend $100 trillion to mitigate global warming based on forecasts 50 years out? What kind of insanity is that?
ReplyDeleteThere's ample empirical evidence to substantiate global warming - receding glaciers worldwide over the past century. What's your explanation otherwise?
DeleteTouché.
DeleteNo, we are cooling. Need data? Try here: https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20190028769.pdf
DeleteCC, speaking of glaciers, did you notice that your cult had to pull the "information" from Glacier National Park that told people the glaciers would be gone by 2020 when that prediction didn't come true? And how how the long list of other predictions? If I were a True Believer like you are, I wouldn't be pointing to my cult's predictions to justify my gullibility.
Deletehttps://climatechangedispatch.com/50-years-failed-climate-predictions/
Mt Shasta Glaciers in Northern California are growing. While the mountain's glaciers are atypical in this respect, it does show Climate change is hardly beating to a homogeneous drummer.
DeleteLet's just say the climate isn't changing at all and this is all a big conspiracy by scientists, because obviously there's so much extra money in it for them compared to big oil. Wouldn't you agree that pollution is bad and that is causes things like cancer? Is there a common ground where people can agree that cancer is bad and alternative forms of energy that don't help kill you is a better solution? Or maybe just the fact that if humans want to stick around we'll need to use something other than oil in 100 years or so?
DeleteWhat would you consider 100% bulletproof evidence that humans are causing this planet to warm? I haven't ever heard anyone that believes it's all made up actually tell me how they'd truly believe it. Why believe in other things that scientists have learned? Is there a correlation between flat earthers, anti-vacination folks and climate change?
You have confused weather forecasts with climate prediction, which are different things, as has been explained and re-explained 10,000 times since the 90's. Educate yourself further on the distinction (if this is not in fact willful ignorance or intentional obfuscation). Nobody is predicting the rainfall on November 1st, 2050. They are predicting average trends.
DeleteRobbyRob, I don't see a conspiracy at work. I see careerism and academic groupthink. This is hardly the first time this has happened in (for lack of a better term) the science establishment. The jihad against dietary fat that began in the 1950s and continues to this day would be an example; so would the "consensus" among oceanographers that waves taller than 65 feet were a 1 in 10,000-year event, and that centuries of contrary reports from mariners couldn't be trusted. Both of those "consensus" beliefs have been overturned, but the anti-fat belief very much persists in the public mind.
DeleteBut my favorite example is right here in Washington State. If you wander out to the Grand Coulee Dam, make sure to go to the visitor center and request that they play the video about Harlan Bretz. If there was ever an example of why the scientific consensus isn't necessarily right, that story would be it. Here's a link that tells the story better than I can.
http://www.detectingdesign.com/harlenbretz.html
Would I agree that pollution is bad? Yep, but I wouldn't agree that carbon dioxide is a pollutant. And the various predictions based on the idea that CO2 is a pollutant that's raising global temps have simply not come true. You are in thrall to a cult, and for some reason you cannot even consider entertaining the idea that the groupthinkers have gotten it wrong -- even though it's far from the first time it's happened in science.
https://climatechangedispatch.com/50-years-failed-climate-predictions/
Meanwhile Antarctic sea ice hit record...
ReplyDeletehttps://www.nasa.gov/content/goddard/antarctic-sea-ice-reaches-new-record-maximum
Quotes from your source Jeff:
Delete"Studies show that globally, the decreases in Arctic sea ice far exceed the increases in Antarctic sea ice."
"The upward trend in the Antarctic, however, is only about a third of the magnitude of the rapid loss of sea ice in the Arctic Ocean"
Hi Cliff,
ReplyDeleteI'm a bit disappointed to read this from you, because your prediction for this summer was spot on. I was hoping to hear your prediction for the coming winter.
If money gets spent because of climate change, it will be after the damage has already been done and probably come in well short of 100 trillion.
ReplyDeleteProbably one thing about the weather models is there just are not enough data points to reference based upon the short time that records have been kept and the level of detail of the data that has been collected.
That leaves lots of assumptions based on if X happened in the past based on Y than X stands a good chance of happening again if Y also has the same value. However X might have also needed the rest of the alphabet to influence the outcome to unknown quantities as well too. The precedent might not be there to understand what the rest of the alphabet does since that might be recently discovered.
Weather has been around for billions of years. Its only been in the past several hundred years that anyone decided it wasn't just work of the divine and mere decades of having the level of detail to get the data along with the computational power to actually use it. This is still "the ground floor" of understanding.
Exactly. But as with medicine, that minimal level of understanding of a complex and chaotic system has evolved in to a god complex for the "doctors." Anyone who claims to have an accurate prediction of climate 50 years out is a quack.
DeletePlaceholder, there are still differences between "The Weather" and "Climate". Climate checks in as the broad brush, big picture and really represents the macro. There are lots of fine details that can be averaged over long term with climate or even outright ignored as it is a broad average over a long sample period. Outliers don't have much impact so if this winter is brutally cold and snowy but the 200 year trend is warming, the over all climate change is trending warmer regardless of one cold winter.
ReplyDeleteWeather IS those fine details. Meso detail might be the global weather for 2019. Micro might be weather in the USA for a season. Nano is Seattle weather for this coming Monday's commute. The finer the detail and the more narrowly refined the scope, the greater the chances of accuracy.
Danger can arise when definitive results from something nano or micro are used to make conclusions about the macro. Cherry picking the data to prove a point, so to speak, while ignoring the big picture. Trump looks at a cold morning in DC in February and decrees climate change is a hoax. That sort of thing. Yes, its cold like it tends to be on a February morning. Still that "cold" might be 2-3 degrees warmer than the 200 year average. Its still "cold" but not as cold in the past and it probably won't be cold on as many days in February in comparison either.
The long term weather forecasts are still science mixed with a bit of spitballing. They have to deal with a broad scope AND sweat those fine details. Some of which are not easily understood and/or are astronomically complex. In the past all the predictions were based on clues from nature, such as those caterpillars, and old farmer/old fisherman experience. That was as recently as when.... early 20th to mid century? Weather and climate really was not cracked open until satellites got launched.
Glad you brought up Satellites. Satellites don't show the warming of the heavily adjusted (read manipulated) and variable ground based data sets. Nothing to be alarmed about. Much more worrisome is the cooling due to a quiet sun. http://www.drroyspencer.com/2019/04/uah-rss-noaa-uw-which-satellite-dataset-should-we-believe/
DeleteI am quite familiar with the difference between weather and climate. The AGW cult used to love to deliver that pious lecture until recently, when they began to attribute every weather event that departed from the average as evidence of climate change.
DeleteAs for Trump, well, you might also mention that the AGW cult made sure to deliver some of their early testimony in Washington on an unseasonably hot and humid day. That game gets played by everyone, so the piety is lost on me.
https://climate.copernicus.eu/sea-ice-cover-june-2019 Here is another take on Antarctic sea ice ...
ReplyDeleteThis post is timely too. It's called "Escape from Model Land." It details another major problem with relying on long range forecasts from models called the Hawkmoth Effect. The idea is that you can be arbitrarily close to the correct equations in models, and still be fantastically far away from the correct forecasts in the output. Because compound error in a sufficiently complex and chaotic system like global climate means that like the lottery, it's easy to choose "a" result, but very difficult to choose "the" result. Climate models are simply not reliable.
ReplyDeletehttps://wattsupwiththat.com/2019/10/30/escape-from-model-land/
The weather beyond two weeks boggles the mind as I watch the forecast devolve to the status of a coin flip much beyond the tenth day. Yet the atmospheric system doesn't devolve at all but evolves in as yet an unpredictable stable manner. There is an answer, but is it incomplete physics, initialization problems with grid size or initial dataset, perhaps bigger/faster computers would be key, and then all those boundary issues of topography, atmospheric chemistry, physical cycles, or are it just those pesky seagulls and butterflies. Will have keep reading to findout.
ReplyDeletehttps://komonews.com/weather/scotts-weather-blog/octo-brrrr-spokane-records-coldest-october-on-record
ReplyDeleteIn NW Bellingham, the temperature has not been above 50F since the the afternoon of 10/28, when it reached 50.1F. Daily minimum temperatures have been below freezing since the morning of 10/29 and below 30F on 3 days since the 10/28. The absolute minimum temperature for the week was 26.8F on the morning of 10/30 and the average weekly temperature at my location is 37.5F - about a degree warmer than the temperature inside my refrigerator.
ReplyDeleteCliff,
ReplyDeleteYou highlight some of the challenges with long-range forecasting, but I have to respectfully disagree that such predictions are "probably not" worth paying attention to. The key is to clearly communicate probability and confidence.
Along the same lines, it's not quite fair to use a single *probabilistic* forecast to illustrate that these forecasts are "poor" in general; even a highly skillful probabilistic forecast will have the occasional "big miss". Performance should be judged within the probabilistic framework over many realizations.
A few more comments here:
https://wp.worldclimateservice.com/home/2019/11/01/making-sense-of-long-range-forecasts/
Pretty hard to "assess damage" (future damage) when actual current measurement is so dodgy. I read official statements about conditions that cite figures like "average state temperature" and all I can do is wince. What climate and weather have done for millions of years is VARY, and we do need more "data points" plus a strong commitment to honesty and realism...in my opinion.
ReplyDeleteAll discussions about weather modeling and prediction invariably drift towards discussions about climate modeling and prediction.
ReplyDeleteTrying to stop this from happening would be like trying to stop the rain from falling in the Cascades, or trying to stop the dust from blowing on the Palouse.
Well, OK, the earth certainly is warming, and part of that warming is certainly human caused. However, I remain skeptical of the IPCC's climate models as being capable of accurately predicting the long-term future of global warming with the IPCC's claimed high degree of certainty.
Back in 2015, I had been criticizing the IPCC’s climate models as being a messy hodge-podge of conflicting scientific assumptions and largely assumed physical parameterizations. Someone at work said to me, “If you don’t like the IPCC’s models, why don’t you write your own climate model?”
So I did. However, not having access to millions of dollars of government funding and a well-paid staff of climate scientists and computer programmers to write the modeling code, I decided to do the whole thing graphically.
The graphical climate model I created in 2015 is presented here in a comment I offered to a recent Judith Curry blog post, 'Escape from model land'.
Beta Blocker's Parallel Offset Universe Climate Model
Back in 2015, the illustration you see in the Climate Etc. comment took about thirty hours to produce. I post this illustration on WUWT and on Climate Etc. once every year in the fall, usually in early November. In October, 2019, I updated its labeling to directly include the 1860 pre-industrial baseline datum.
The earth has been warming for more than 150 years. The earth won’t stop warming just because some people think we are at or near the top of a long-term upward fluctuation cycle.
The thirty-year running average of GMT must decline steadily for a period of thirty years or more before we can be reasonably certain that a long-term reversal of current global warming has actually occurred.
In the meantime, as long as the rise in GMT continues at or above + 0.1 C per decade -- or thereabouts -- mainstream climate scientists will continue to say that the IPCC's models have been validated by real world temperature observations.
Ever tried writing up a an article and submitting your work to peer review? That might be a good next step for you to develop your ideas further, as blog posts aren't going to take you very far.
DeleteAli, that would be a fool's errand. Even though many of the world's great scientists were amateurs, there's no way that any amateur today could ever be taken seriously -- especially with a contrary view. Consider the case of J. Harlan Bretz, the credentialed scientist who contradicted the "uniformitarian" consensus.
DeleteThat guy had a Ph.D. in geology from the University of Chicago, which at the time was about as prestigous as it could get. It took him 50 years to be formally acknowledge as correct. When it happened, he quipped that it only happened because he had outlived his opponents.
Betah Blocher's viewpoint, no matter how well researched, doesn't stand a chance today.
Ali, my primary interest lies in following the public policy debate concerning what, if anything, to do about climate change. My own perspectives concerning the validity of today's mainstream climate science, and how it influences that public policy debate, are framed with this purpose in mind.
DeleteFormalized peer review doesn't serve my purposes. Spawning further informed debate on the Internet through commentary made on climate blogs does.
With a weather forecasting model, the outcome of the model's predictions are available within a week's time or less. The data which feeds those models can be updated on a daily basis as a weather pattern evolves in real time.
Climate models are different. We won't know with reasonably good certainty precisely how sensitive the earth's climate system is to the addition of CO2 until some number of years have passed.
The uncertainties associated with the IPCC's climate models are a continuing source of controversy between mainstream climate scientists and their knowledgeable critics. This is so because the physics of those particular climate system mechanisms thought to be most important in raising the earth's global mean temperature over decadal periods of time are not well understood.
Cloud physics mechanisms, and the physics of the postulated water vapor feedback mechanism, are among the most influential of the factors which affect the predictions of the IPCC's models and which determine the estimated sensitivity of global mean temperature to the addition of CO2.
At the current state of science, it isn't possible to directly observe those particular mechanisms operating in real time. The nature of their physical operation must be inferred or deduced from other kinds of observations and measurements.
Because these physical mechanisms aren't well understood, their influence must be estimated and parameterized within the climate models based on assumptions as to how they operate. Different models, and different runs of the same model, use different assumptions about cloud feedback physics and about water vapor feedback processes.
An often-discussed topic on the anti-AGW climate science blogs is the extent to which the IPCC's models use assumed physical parameters as a replacement for direct knowledge of the actual physical processes that affect the earth's climate system.
Those who follow the endless debates between mainstream climate scientists and their informed critics understand that Beta Blocker's Parallel Offset Universe Climate Model is actually a commentary about the back-and-forth discourse and the polemics of these endless debates.
All of that commentary has been condensed into a single graphical illustration which covers these major topical points:
-- The IPCC attributes most of the warming that occurred after 1950 to anthropogenic GHG emissions. And yet the rate of warming from 1900 through 1950 isn't that much different than the rate from 1950 through 2014.
-- Roughly two-thirds of the CO2 added to the atmosphere since the beginning of the industrial revolution was added post 1990. Shouldn't the rate of GMT increase post 1990 be substantially larger than it actually is?
-- If warming continues at the current rate over the next two decades, temperatures must greatly accelerate in the 2040’s and beyond in order to reach a 3C or a 4C rise above pre-industrial by 2100. What are the odds of this happening?
Here is the bottom line. My graphical climate model is based 100% on assumptions that are 100% transparent as to how they operate. Are the IPCC's models any more reliable than my highly simplistic graphical model given that two of the most important physical processes affecting global mean temperature -- cloud feedbacks and water vapor feedbacks -- are covered with parameterized assumptions, not with data from direct observations made in the atmosphere?
Placeholder, it's fabulous that you have taken an interest in J Harlen Bretz. As such, you know that he received his geology PhD at the University of Chicago and was a professor at the UW. He most definitely worked within the academic setting that you decry above, and it was his published, peer reviewed work and engagement with the scientific community that brought his ideas to light. And that's how science works :) It's not instant, and not everyone agrees on everything, but it is how ideas and concepts develop. Oh- and if you haven't already read this, you might enjoy the book "Bretz's Flood"- it's a great read.
DeleteAli, it was "the scientific community" that was in the thrall of groupthink for decades, and did their best to ridicule and marginalize him. What saved Bretz was high-altitude photography, not "the scientific community." Then, as now, there was a great deal of corwardice and craven careerism at work.
DeleteThe AGW cult has been telling people that "the science is settled" on their hypothesis, and that anyone who differs is a "denialist." This flies in the face not just of the evidence, but of the scientific method itself. Your endorsement of "the scientific community" rings hollow.
Oh, and peer review? Nice idea for sure, but the execution leaves a whole lot to be desired. There is a great deal of fraud in scientific research these days, tracing back to the lust for research grants. This is probably why there's a "replication crisis" in research these days.
https://retractionwatch.com/2018/06/29/35000-papers-may-need-to-be-retracted-over-image-duplication-says-new-paper/
https://www.washingtonpost.com/news/morning-mix/wp/2015/03/27/fabricated-peer-reviews-prompt-scientific-journal-to-retract-43-papers-systematic-scheme-may-affect-other-journals/
https://en.wikipedia.org/wiki/Replication_crisis
When Bretz finally got the Penrose Medal in the 1970s, he quipped that the only reason he received it was that he outlived his detractors. Truth in that bit of jest.
Ali, it was the same with rogue ocean waves. Recall that the craven oceanographic "consensus" was their bogus claim that waves taller than 65 feet were a 1 in 10,000 year phenomenon. This was believed and published in spite of centuries of reports from mariners worldwide.
DeleteFinally, in the 1990s, there emerged absolutely irrefutable reports of gigantic waves. I was living in Boston when one of those reports was made by the captain of the QE2 passenger liner that pulled into the harbor for emergency repairs after encountering a wave that was level with the ship's bridge, which sat 96 feet above the waterline.
After that, the Germans launched a satellite and tracked rogue waves, and found them frequent. That's when the oceanographers had to give in. Face it, the scientists are no less likely than anyone else to lie when it suits their financial or ego interests. The AGW hypothesis is only the latest lie retold for position and money.
Placeholder, you sound sort of angry very quickly. Hard to have much of a discussion when emotionally laden words like "lying" and "cult" and "fraud" and "lust" and "ego" are tossed around so casually, that tends to put the conversation in a framework where people get super entrenched in their "position" and it's not usually very useful in the end. However- seriously- go check out Bretz's Flood!
DeleteAli, it sounds like you are a Seattle "progressive" who cannot stand it when anyone doesn't bow to your cult.
DeleteThis comment has been removed by the author.
ReplyDeleteMy favorite weather forecast is the note on the blackboard above the concierge desk at Harrison Hot Springs in British Columbia. It's a note that simply says "weather See Outside" with an arrow pointing outside. You can view it by going to Google and looking for photos of the interior of the front desk, pan around till you find the desk and blackboard. My experience has been it's been remarkable accurate.
ReplyDeleteCliff,
ReplyDeleteI appreciate reading your more detailed rationale describing why long-term forecasts are less skilled.
Cliff, your outlook for the early- to mid-fall was pretty accurate. Another forecaster, Joe Bastardi, has accumulated a pretty good (but not flawless) track record of seasonal forecasting. He does it by comparing a particular year to past years. It doesn't always work, but I've been impressed.
ReplyDelete