Frozen Datacubes: Snow & Ice Analytics
The Eurac research team of Alexander Jacob, Advanced Computing for Earth Observation (ACEO) at the Institute for Earth Observation in Bolzano, Italy, uses state of the art technology to manage big EO Data and support important research. The research activities aim at monitoring and understanding of processes driving and driven by climate change. The mountain cryosphere is severely impacted by rising temperatures and observing snow cover and melting glaciers is key to the understanding of future water availability. Following the human and nature driven transformation of land cover and land use is important to understand the influence of agricultural practices on ecosystems and their flora and fauna. The flexible representation of data in the rasdaman datacube engine allows for efficient use of complex data such as the Sentinel-1 interferometric products just as well as long and dense time series from missions like MODIS, Landsat or Sentinel-2.
Eurac´s Petascale alpine archive of data is stored and processed directly on their high-performance cloud infrastructure. Pre-processed data is organized into datacubes using the array database rasdaman. This guarantees fast and standardized access to the data via the OGC Web Coverage Service (WCS), following the principle “what you get is what you need”. Furthermore, a good portion of the classic processing tasks such as sub-setting, re-projection, resampling and time series aggregation is directly done in these datacubes using the OGC protocol WCPS (Web Coverage Processing Service), according to the Big Data principle “bring processing to the data”.
Compressing information from more than 500 images into one figure
For direct integration into daily workflows of researchers the ACEO team is developing API level access to the data stored in datacubes for development environments like python and R, which have become highly popular due to their open-source nature and ample availability of packages or libraries for all kinds of EO data analysis. Starting from the development of a simple Python module for WCPS access to the rasdaman server, over the CubeR R-package, most recently the ACEO team was involved in the development and implementation of the openEO API capable of transforming requests into WCPS queries and responses in data structures like xarray or stars that can be directly used in Python or R for further analysis and visualization.
For the land cover and vegetation classification two algorithms well-known in this field, Random Forests and Support Vector Machines, are applied. All Python scripts run in cloud-based Jupyter notebooks. Git is used as version control system to document and publish the code for other users. This fosters collaboration as other researchers can clearly see, comment and help developing the code. With the proposed approach the EO users can process and analyse large amounts of data in a cloud infrastructure, accessing services and data remotely using a high-level programming language. In this way, EO users can focus on algorithm development, instead of dealing with data preparation.
Processing flow integration with direct connection of R and python to the rasdaman datacubes
Contact: , Eurac Research, Institute for Earth Observation