Climate Researchers Enlist Big Cloud Providers for Big Data Challenges

And the shift hasn’t gone unnoticed by the Big Three cloud providers. AWS and others offer subscription-based remote data storage and online tools, and researchers say they can be an affordable alternative to setting up and maintaining their own hardware.

The cloud’s added computing power can also make it easier for researchers to run machine-learning algorithms designed to identify patterns and extract insights from vast amounts of climate data, for instance, on ocean temperatures and rainfall patterns, as well as decades’ worth of satellite imagery.

“The data sets are getting larger and larger,” said Werner Vogels, chief technology officer of Amazon.com Inc. “So machine learning starts to play a more important role to look for patterns in the data.”

Last year, researchers at Oxford began using AWS machine-learning tools to examine the effects of aerosols on clouds—certain aerosols are cooling to the Earth and might offset some of the effects of greenhouse gases.

“We don’t have a good handle on how much,” Mr. Watson-Parris said. Figuring out the extent to which clouds are altered by aerosols might help mitigate the effects of climate change, he added.

So researchers trained a machine-learning algorithm to sift through petabytes of satellite images and identify so-called “ship tracks,” which are clouds that look brighter, because they have been exposed to aerosols from the exhaust released by ships.

Previously, that observation was done by statistical analysis of only a select number of images. “We would have manually sat there labeling these tracks,” Mr. Watson-Parris said.

The next step is to use machine-learning algorithms to help researchers understand whether the relationships between aerosols and clouds is causal rather than statistical. Fifteen doctoral students across Europe began working with climate scientists and machine-learning experts in September to try to figure it out.

Climate researchers are following the larger trend of academic institutions and corporations moving to the cloud, said Simon Mingay, vice president of research at Gartner Inc.

“Sustainability, climate mitigation and climate adaptation is incredibly data-intensive. You’re talking about large amounts of data and you’re talking about large complex data sets,” he said. “Absolutely, cloud will play an increasingly important role.”

Some of the cloud providers have set up programs catering directly to environmental research.

Microsoft Corp.’s AI for Earth program, for example, offers grants and technical help from Azure, its cloud division, for just such projects.

AWS is seeing a growing crop of startups and nonprofit organizations using its machine-learning services to help on dozens of climate-related projects, ranging from soil-data analysis and ocean-conservation efforts to forest preservation, said Amazon’s Mr. Vogels.

More broadly, tools like Amazon SageMaker, launched in 2017, allow software developers in any industry to build, train and use machine-learning models more quickly and with less technical complexity than in years prior, he said.

“We’re really trying to democratize machine learning in order to make sure it’s not just the domain of data scientists, but something everyone can use,” Mr. Vogels said.

Access to sophisticated machine-learning tools is one of the main advantages of using the cloud, said Sid-Ahmed Boukabara, principal scientist at the National Oceanic and Atmospheric Administration’s Center for Satellite Applications and Research.

In October, NOAA’s Satellite and Information Service announced a partnership with Google to study how cloud-based machine-learning tools could provide more accurate weather forecasts and be used to monitor extreme weather such as hurricanes.

NOAA collects terabytes worth of environmental data, including satellite images, on a daily basis. “There is a big data challenge, so we’re very interested in the cloud and the AI techniques that come with it,” Mr. Boukabara said.

The sheer amount of data presents difficulties, and several techniques are used to address it. One involves “thinning,” or decreasing the amount of data in order to run an analysis on NOAA’s own high-performance computing hardware, Mr. Boukabara said.

But Google Cloud could help NOAA process much more data and analyze it using sophisticated machine-learning algorithms made to run on specific AI-based hardware, he said. The algorithms analyzing data in the cloud could help produce more accurate forecasts within a one-kilometer radius, compared with a 10-kilometer radius, he added.

“AI will help to increase the value of what our systems are producing now,” he said.

Write to Sara Castellanos at [email protected]