March 21, 2017 From rOpenSci (https://deploy-preview-488--ropensci.netlify.app/blog/2017/03/21/camsrad/). Except where otherwise noted, content on this site is licensed under the CC-BY license.
camsRad is a lightweight R client for the CAMS Radiation Service, that provides satellite-based time series of solar irradiation for the actual weather conditions as well as for clear-sky conditions. Satellite-based solar irradiation data have been around roughly as long our modern era satellites. But the price tag has been very high, in the range of several thousand euros per site. This has damped research and development of downstream applications. With CAMS Radiation Service coming online in 2016, this changed as the services are provided under the (not yet fully implemented) European Union stand point that data and services produced with public funding shall be provided on free and open grounds. The service is part of Copernicus, a European Union programme aimed at developing information services based on satellite earth observation and in situ data. All Copernicus information services are free and openly accessible.
🔗 Satellite-based solar irradiation data
The main groups of users are planners and managers of solar energy systems; architects and building engineers; researchers in renewable energies and building engineering. Surface solar irradiation is relatively cumbersome and expensive to retrieve by ground observation. Therefore, a satellite-based modelling approach can in many cases be a more feasible option. These approaches build-upon the principle that pixels in satellite images of clouds (target 2 in figure below) appear brighter/whiter than pixels of ground (target 1). For more thorough description on underlying theory and technical details, head to the user guide and articles. CAMS Radiation Service is jointly developed and provided by DLR, Armines and Transvalor.
Illustration on using satellite images to determine level of cloudiness. Source: MACC-III Deliverable D57.5;Check out the Shiny app
🔗 The camsRad package
In a perfect world, this package would be unnecessary. Accessing data and web services should be easy, a procedure so standardized that anyone with a bit of programming experience should be able to achieve this with a few lines of code. I´ve worked with quite a few web API, but still haven´t encountered two of the same kind. It always requires time and effort familiarizing yourself with the concept of communication, formatting etc. That there exist numerous ways of disseminating data can be seen at the rOpenSci packages listing, where roughly half of the listed packages are categorized as data access packages.
The CAMS Radiation Service uses a so-called Web Process Service (WPS) interface. WPS provides standardizing for geospatial processing services. It requires you to make a POST requests carrying a XML formatted payload that specifies how the service is invoked. As I wanted to have a thorough and reliable R-client and though other users could have use of it, I decided to make it a public R-package and submit it to the rOpenSci for review. I think I´m a quite typical aspiring R developer, mostly self-learned programmer with a non-computer science degree. Another, a bit selfish, motive for submitting was that I thought it would be a good learning ground to adopt a more formal R-development style. And yes, I learned a lot! A big thank you to reviewer Jeffrey Hollister and editor Scott Chamberlain. The biggest change after the review, was getting rid of package dependencies to get a more lightweight package and to decrease the risk of breakage when upstream packages get updated.
The core of the camsRad package is the cams_api function which interfaces the WPS of the CAMS Radiation Service. It can be of interest for anyone wanting to call CAMS Radiation Service from other languages than R, or for those that want to invoke other WPS based web services from R. The two convenience functions cams_get_radiation and cams_get_mcclear are for those of you that just want to get the data into a R data frame with a little hassle as possible. Check the vignette and readme file for further instructions and examples.
🔗 My use cases
Shiny Weather Data is a web service making different sources of European gridded climate data available in hourly time series formats used by common building performance modeling tools. This web service has been around for a while and has a steadily growing user group of professional building modelers as well as students and researchers.
Smart Energy Modeler demonstrates how modeled climate data can be used to calibrate simple building models with utility bill data. This is a dissemination from my research as a PhD Candidate at Mälardalen University, Sweden. It´s in an early stage of development, but a nice showcase of how access to open climate data enables data-driven application that solves real world-problems.
🔗 Future development
The CAMS Radiation Service relies on images from the Metosat satellite which is located at the at the prime meridian of 0° longitude (covering Europe, Africa and Middle East). I´ve started a GitHub issue about alternative data sources. So, please, let me know if you know any sources of open and free, high resolution solar radiation data based on images from satellites above America and Asia.
