October 06, 2015

Apple Computer Could Revolutionize Weather Forecasting: Will It Take the Challenge?

A number of popular smartphones have atmospheric pressure sensors; in fact, there may be a billion smartphones with highly capable atmospheric pressure sensors by the end of 2016. 

Pressure is the most valuable surface observation for weather prediction since it provides information about the the deep atmosphere above.  With pressure alone, meteorologists can determine the three-dimensional structures of the atmosphere.  And pressure does not have exposure problems like wind or temperature...in a building, in a pocket, inside or outside, in the sun or not....it doesn't make much of a difference where the smartphone is located.  It is the golden parameter.  And meteorologists can make use of both pressure and pressure change to help start (or initialize) our weather forecasting models.

Imagine a world with a billion barometers

So we should get ready for a weather prediction revolution?  Perhaps, but there is a major hurdle that must be crossed first:

The infrastructure for collecting this hugely promising weather data does not exist. 

But at least one company has the potential to radically improve the situation: Apple Computer.

The question is whether Apple will help.

The iPhone 6 series (iPhone 6, 6+, 6S) have now sold over 150 million units around the world and remains the most popular smartphone in many markets.  All have atmospheric pressure sensors.
 By next year, 250 million iPhone 6's will be in use around the world.


To be useful, the pressure from the smartphone must be accompanied by position data from the unit's GPS.  The history of movement of the phone would be help, as would its speed, which would tell us if it were in a moving vehicle.

Apple could assist in a number of ways in the collecting smartphone pressure data.  Obviously, the best approach would be to build pressure collection into the operating system (iOS, which Apple control).  Or pressure collection could be an option in the weather app that Apple preinstalls on every iPhone.

Want to see a shadow of what is possible?  The Weather Channel app allows the collection of smartphone pressure and we were able to get a sample of their data---perhaps we are getting data from 1/1000 of the iPhone6s out there.  Want to see what it gives us?

Let's start with a data rich country...the U.S.  On the left are the convention observation locations over the Oklahoma area, on the right, from smartphones.   Big enhancements You can even get pressures from smartphones on roads.


But the biggest advances might be in second and third world countries without a lot of weather data, such a India.  Below is an image with conventional data on the left and smartphones on the right. Big enhancement.  Many folks have smartphones in developing countries.


Smartphone pressures are also available from some Android phones (e.g., Samsung Galaxy series)--still waiting for a positive response from Google on my inquiries for help in pressure collection.

I have two graduate students working on the use of smartphone pressures for improvement of numerical weather prediction.  We have some initially favorable results with a low density of pressure available from two small companies (Cumulunimbus Inc. and OpenSignal), but we really need more density to prove the value of the technology.

Where might dense pressures really help?   We believe that large numbers of pressure might help forecast the initiation of thunderstorms.   Or provide better short-term forecasts for wind turbines. Situations in which defining small scale weather features are important.  And those are only a few of the potential weather features situation where smartphone pressures might help.


So Apple computer, please give me a call or email.  Or leave a comment on this blog.   You could potentially change the world of weather forecasting at little cost.

26 comments:

  1. Cliff,

    While Apple doesn't do this, there is a weather app for iPhones called "Dark Sky" which does collect pressure data automatically from users to improve their forecasting. Might be worth th taking a look at it, personally I love the app.

    ReplyDelete
  2. So, I have used the Cumulonimbus app on my Galaxy for the past couple of years. Does this data get sent to you? That was one of the main reasons I got the app.

    ReplyDelete
  3. Phil,
    Thank you for using the Cumulonimbus app. We are getting the data and it is very useful to test concepts, but only around 70,000 observations are available per hour globally....just a tiny percentage of what is possible. We need more to change the world of forecasting..cliff

    ReplyDelete
  4. Cliff,

    I have emailed Tim Cook (tcook@apple.com) and I recommend that you and anyone who wants to support this idea email him also.

    ReplyDelete
  5. Cliff - and all others who read this comment - go to apple.com/feedback and link to this post!

    The biggest roadblock I see to this: Apple is extremely cautious with user data. A program like this would absolutely have to be opt-in, and people aren't necessarily super OK with their location being tracked and uploaded to 3rd parties. Even if the data were anonymized and stripped down to speed, heading, current location, and pressure, that's more information than Apple is currently collecting on their users, and would probably have to be linked to a device ID of some kind.

    All that to say... I'd love to see this happen and I'd happily participate, but I don't think it's quite as simple as it seems.

    ReplyDelete
  6. Hope you'll give an update on whether hurricane Oho will hit the NW at the end of the week. Trying to get information via weather sources but would appreciate your perspective, especially what kind of storm surge we may get. Could it be like 1996 again?

    ReplyDelete
  7. You want a history of all the places my phone has been? And I'm supposed to trust that some third party is anonymizing it properly?

    Obvious privacy issues will keep most people from doing anything like this.

    ReplyDelete
  8. Good idea, similar to how smartphones have revolutionized traffic speed maps. For what it's worth Apple Computer's name was changed to Apple, Inc. in 2007.

    ReplyDelete
  9. Fascinating idea! Might be worth reaching out to tech journalists (Daring Fireball, Mashable, etc.) and notable weather apps like Dark Sky to get their take on it.

    ReplyDelete
  10. Maybe this would be a project for the new Google company. They could use all brands of smart phones and use the cloud to collect the data.

    ReplyDelete
  11. Cliff, since atmospheric pressure varies with elevation, what are you and your grad students using for your elevation model? What if a reporting smartphone goes up an elevator to the top of a skyscraper?

    ReplyDelete
  12. Cliff, what spatial and temporal resolution could one degrade the raw data to without losing too much of the value?

    Anonymizing this data is harder than you might think. "Anonymous" in the literal sense of not carrying a personal identifier is not nearly enough. Points that can be linked into a track are highly sensitive data in themselves, and scrubbing data to prevent linking is hard.

    ReplyDelete
  13. "It's just science, what could go wrong...?" - Julius Robert Oppenheimer

    ReplyDelete
  14. Could the data collection program average 2-5 inputs before discarding the raw data? Would this provide good data, and is it enough to anonymize?

    ReplyDelete
  15. Cliff, I'd love your take on the Hurricane Oho and it's NE track heading to the PNW.

    http://www.weather.com/storms/hurricane/news/tropical-storm-oho-hurricane-northwest-british-columbia

    Seems its the first time this has happened so late in the season since 1949. I'd curious to know why it's been so long and why it's happening now.

    Thanks!

    ReplyDelete
  16. Anonymizing the data seems to be the problem. I would certainly trust weather researchers to not misuse my data; but the odds are they wouldn't manage the data collection themselves, they'd work with a large company that has the pre-existing infrastructure to handle the task for them... and THAT company would also have access to all the location data. Plus even if that company pledged not to misuse it, if/when it gets taken over - what would the new parent company do?

    On the practical side, I don't know if iOS apps even have access to the pressure sensors. But, if they do, I wonder if you could just develop an app which would collect the data. It would need to request permission to keep running in the background (by default, they are restricted in this regard). Then users could "opt in" by downloading and running the app.

    Another similar alternative would be to develop an app/tweak for jailbroken devices. That's still something like 10-20% of iPhones. It would certainly make development and deployment somewhat easier; but you'd still have to convince people to download and install it.

    ReplyDelete
  17. A start is doing something like this idea:
    http://techcrunch.com/2015/10/07/sunshine-a-crowd-sourced-weather-app-goes-live/

    ReplyDelete
  18. @Westside guy - do you have Google Maps installed? Then you are probably already sending location and speed data whenever you step out without even knowing it. Ever wonder how the Google maps traffic feature offers such great, accurate coverage even on those secondary and tertiary roads? Deidentified GPS data from thousands (millions?) of phones is streamed to Google, who analyse the data to figure out which phones are on the road, and how fast they are travelling to derive the green/yellow/red realtime traffic overlay. I believe system operates on an essentially opt-out principle, and the vast majority of users are probably completely unaware of their contribution to the handy-dandy traffic feature.

    Improved weather forecasts would seem to be a far more worthy application of crowdsourcing!

    https://googleblog.blogspot.com/2009/08/bright-side-of-sitting-in-traffic.html
    https://en.wikipedia.org/wiki/Google_Traffic

    ReplyDelete
  19. Maybe this would be a project for the new Google company. They could use all brands of smart phones and use the cloud to collect the data.

    ReplyDelete
  20. Not to be pedantic, but the company has not been Apple Computer since January of 2007 and is simply now known as Apple, Inc.

    ReplyDelete
  21. @Daniel Fowler - not on iOS unless you give permission (which, admittedly, most people likely do). iOS app permissions are much more granular than current Android permissions, and happen at the time the app wants a specific access rather than at installation (FWIW Android is apparently moving towards this in Marshmallow).

    Me, I don't let things run in the background unless they're currently performing a task I want. So I kill Waze when I've reached my destination, for example.

    ReplyDelete
  22. A bit of a recommendation since I work in South Asia. People, especially engineering and scientific professionals, bristle at being called "third" world. A better term to use is developing nation, because it implies action.

    ReplyDelete
  23. There's an interesting app on hyper-local weather forecasting using smartphone measurements at http://www.wired.com/2015/10/clever-app-turns-everyone-roving-weather-reporter/#slide-1 . Might be worth a look.

    ReplyDelete
  24. We have checked several smart phone barometers and they are pretty accurate at normal pressures, usually within 1.5 mb. Some apps let users offset the readings, and a good way to do that is our free online service at www.starpath.com/barometers, which gives users a way to set to the proper slp for their location at a specific time using the average of up to 10 closest official stations. There are detailed instructions online. The "official" stations are typically ± 1 mb or so, but periodically one can be off much more. This method lets users spot a bad report and not use it in the average.

    ReplyDelete
  25. PressureNet does this already all they need to do is make a iOS app but the big leap would be getting NWS/NOAA etc to start using PressureNet Data. We need Madis to be more modern with an api where people can more easily submit crowdsourced data into it.

    ReplyDelete
  26. Maybe this would be a project for the new Google company. They could use all brands of smart phones and use the cloud to collect the data.

    ReplyDelete

Please make sure your comments are civil. Name calling and personal attacks are not appropriate.

Why Hurricane Track Forecasts are Much Better that Intensity Predictions

Tropical Storm Milton is now a hurricane heading for Florida's west coast. As I will describe below, there is great confidence in the hu...