Thank your for your subscribe
Oops something went wrong. Please check your entry

Clouds in the Cloud: how solar radiation has become an online service

SoDa, for SOlar radiation DAta, is a provider of online data and services on solar radiation. It is both a platform guided by the needs of its users, a pivotal tool for several international institutional networks, and the endeavor of two generations of researchers, alert to the opportunities that arise in the interstices between major disciplines.

May 2017
lire en français
lire en français
Executive Summary

Paris Innovation Review - SoDa is a freemium services platform with more than one hundred customers and tens of thousands of users. Yet it all began at a time when the very notion of an online platform was still in limbo.

Thierry Ranchin – Yes indeed, and the web itself was still in its prehistory. But we already had satellite data. This is how it all began, when, in the late 1970s, a young PhD student in oceanography was asked to count the clouds on Corsica from the images provided by the Meteosat satellite. Pixel by pixel, a percentage of cloudiness was established and its evolution was measured image after image. The doctoral student, Lucien Wald, succeeded in showing a precise correlation between cloudiness and radiation on the ground.

There was indeed a lead, and his research center decided to follow it. A reference method was developed in 1980-82, christened Heliosat 0. And although its algorithm has been refined over the course of three decades, and even though a new generation of satellites has been launched since then, the method used today (Heliosat 2) remains fairly close to what had been sketched 35 years ago: every quarter of an hour, the cloudiness ratio is determined, and from that, a radiation ratio is deduced for the ground.

What has radically changed is the dissemination of this information, and that is what brings us to SoDa, whose name was chosen in 2000.

Basically, in the 1980s, the only technology available to disseminate scientific knowledge was printing. The availability of lighter computer devices and especially the advent of the web opened up perspectives for other ways of doing things. It is in this context that the team made a research proposal to the European Commission, to create not just solar radiation databases, but a tool for users, a one-stop-shop access for all data relating to solar radiation. This would be SoDa.

The initiative makes it possible to collect the data gathered by some fifteen suppliers and from there, to produce a hundred different services downstream – such as information about the sun's course, the monthly average sunshine received by a given territory, or how much shading (depending on relief) occurs at a given point...

This tool, in the vocabulary of the time, was a community portal. Today, it would simply be defined as a service platform, supported by a considerable set of databases. Over time, various projects have enabled the development of new services. The skills involved are diverse: there are physics, of course, but also applied mathematics, or image and signal processing. Initially, we had developed dedicated software, but since 2005 we have been using standard tools, integrated into an operational processing chain, which download data every quarter of an hour and process them in less than fifteen minutes. We are not very far from real time, and part of our effort aims to simplify and optimize the processing time.

You mentioned the European Commission, but your work has been used by other international organizations.

Yes, and more broadly, the academic community involved with solar radiation works on a continental scale and not at the scale of countries – if only because satellite data allows for it.

If SoDa was quickly integrated into various international programs, it was probably thanks to our involvement into several European projects, as we had learned to work within such frameworks, and because our tool was readily configured as a platform, with a collaborative approach.

In this spirit, and since 2008, the ideas behind SoDa went on in the form of of a collaborative information system on solar and wind resources and more broadly on renewable energies,, to offer a comprehensive catalog of services mining several databases. This system has been set up under the aegis of the International Renewable Energy Agency (IRENA) and the Global Earth Observation System of Systems (GEOSS). It exploits the latest technologies of interoperable web services, according to W3C, OGC and INSPIRE standards.

Interoperability is an essential prerequisite to enable the compatibility of a system and, where appropriate, its centrality. It also attests to the heed paid to the needs of users, in the same one-stop-access mindset that I mentioned earlier. This is an essential aspect of all these projects, which are basically aimed at providing truly accessible services that potential users can easily take over.

In 2009, SoDa was integrated into the Monitoring Atmospheric Composition and Climate (MACC) European Project, one of the elements of the Global Monitoring for Environment and Security (GMES, the European contribution to GEOSS), whose specific mission is to monitor pollution and climate change. In 2016, the GMES became the Copernicus  and MACC the Copernicus Atmosphere Monitoring Service.

The adventure continues, as we are now engaged in  new projects such as the Climate Change impacts on Energy or developing innovative, lightweight in-situ measurement tools with surveillance cameras in order to complement satellite data and those of weather stations, which are all too scarce, with hardware accessible to many actors. In turn, the whole thing will be complemented by a platform to manage and validate this data.

The data sets are public, and you are committed to their sharing and dissemination, but at the same time formatting them and making them available requires a lot of work. Do users have to pay a fee to access the SoDa service?

There are about one hundred paying subscribers who use personalized services and who derive added value from them. For example, one of the first customers was a horticulturist that sought to optimize the watering of his plants in greenhouses, in order to improve their growth. Another was a company that sends out warning messages to summer visitors in case of ultraviolet peaks on the beaches.

Some data sets are not open access. But a good many of both our data and services are free, and the tens of thousands of regular users who connect to the site (thus excluding curious visitors) show that this free layer also responds to a demand. There is also a social or developmental stake in offering free access to certain information. At the beginning of 2011, for example, we completely opened up the HelioClim-1 database, which includes 21 years, from 1985 to 2005, of daily data on Europe, Africa and the Atlantic Ocean. These data sets, with a spatial resolution of 30 km, substantially complement any measurements made on the ground by existing meteorological stations, which are often far and between. This provides statistical series that can inform many policy makers or economists. It is in this spirit that we have contributed, in 2012, to the development of the World Solar and Wind Atlas, under the aegis of IRENA.

If we have switched to a freemium offer, combining a free layer and paid services, it is not for the purpose of making a fortune, even if such a business model is that of many start-ups... Simply put, the European Space Agency, which was our partner at the time, asked us in 2006 what would happen if we restrict the access to the service. We experimented with it a bit, and we realized that our users were protesting! Hence an interrogation: did this need that suddenly manifested itself have a price? On our end, as researchers, we were thinking about refocusing on our core profession. In short, we began to consider creating a spin-off, with an economic model that would make it feasible to continue serving the greater good while making the platform financially self-sufficient.

One of the advantages of this reflection is that it has led us to better understand our users’ needs. And when business really came out of the laboratory, in 2009, the two people we recruited – there are five today – took over marketing and customer management, besides dealing with operational aspects. In addition to feeding and maintaining the data bank, they are thus seeking to develop high value-added services, be they for all or for more specific needs.

The academic community, alongside economic decision-makers, is among the key target users. Climate scientists, for example, need these data to improve their models. With the availability of perfectly informed vast statistical series, they will make significant advances in the knowledge of solar radiation and its spatial and temporal variations.

The implementation of a freemium model must therefore be understood in light of a sharing logic, essentially intended to disseminate information.

This is how we decided to give away the Heliosat codes (the method used to produce the radiation databases), which allowed our method to spread to the rest of the world. In return, we gained recognition and the legitimacy of a world-class standard. This is why we were invited to the discussions on the development of Copernicus: we have been identified as a key player with a strong involvement with users.

It is with this in mind, too, that we set up free training for professionals five years ago. Each year, around thirty people from different countries (Canada, South Korea, Qatar, Morocco), and with responsibilities in various fields (SMEs, big companies, research) come to spend three days to familiarize themselves both with the tools and the data. We present a case study, and a banker participates in the training to give an idea of ​​the financial aspects related to a good use of SoDa.

Let's talk about the outlets, precisely. You mentioned a few, but more broadly, what are the possible uses of these databases and of the services that exploit them?

The most obvious ones revolve around renewable energies, which, incidentally, are the core of our research center (the Observatory, Impacts and Energy Center aims to contribute to increasing the use of renewables by generating knowledge, data, applications, and their dissemination). Solar park managers, typically, have an immediate interest in accessing reliable and directly usable data on solar irradiance at ground level. It should be noted that, in terms of irradiance, simple weather reports provide information that is rather coarse, and at any rate insufficient. Satellite data sets on clouding and the algorithm that infers solar irradiance from it are much more accurate.

Agribusiness, with the widespread rise of connected agriculture and the explosion of digital data uses, is another major user, either of statistical series or of up-to-date services enabling the optimization of water supply – especially in case of scarcity of this resource, but also to optimize plant growth, manage consumption well, or even to convince an insurance company that the lack of sunlight has seriously hindered the activity of a farm. Ultimately, it is not only aimed at the technophile manager of a large agricultural estate, but also at farmers equipped with a simple smartphone, who are destined to use services based on these databases. Of course, the interface still needs to be perfected, needs must be pinpointed, and best practices identified. Nevertheless, for over 15 years now, the user has been the focus of our approach, and this vision, guided and streamlined by actual uses, is the very DNA of a platform like SoDa.

Finally, solar radiation also affects two key dimensions: the climate, which I cited earlier, and human health. We have worked with physicians in epidemiological studies on melanomas or other UV-related pathologies, such as multiple sclerosis.

Many of these uses correspond to innovations, be they emerging activities (solar/PV) or new ways of doing things (in epidemiology, connected agriculture). Were the founders of the service visionaries?

To put it another way, they had a good hunch. But behind this flair is a particular state of mind, which is a hallmark of the research conducted at Mines ParisTech. Often, one explores adjacent spheres, because one seeks the answer to a problem; and suddenly you stumble upon a vein. Research carried out at Mines often involves interstitial spaces, at the intersection of several disciplines or areas of work. And innovation almost always occurs in these interstices. Solar radiation, a marginal theme in the 1980s, was to eventually open onto energy and climate, which are major topics today. The data turned out to lead to the information economy, with new services. If you are faithful enough to stick to this field and if you learn how to seize successive opportunities, working on solar radiation data in the 1980s can go a long way and one day let you reach very fertile territories.

Thierry Ranchin
Professor and Managing Director of Centre Observation, Impacts, Energie, Mines ParisTech - PSL