With our strong, sustained contribution in international research projects, often in a leading role, we help advancing the state of the art in Big Datacube technology, application, and standardization.
BigDataCube: Flexible, Scalable User Services on Massive Spatio-Temporal Earth Observation Data
The BigDataCube project aims at advancing the innovative datacube paradigm – i.e., analysis-ready spatio-temporal raster data – from the level of a scientific prototype to pre-commercial Earth Observation (EO) services. To this end, the Germany-based, worldwide leading datacube technology (in database lingo: “Array Database”) rasdaman will be installed on CODE-DE as well as in a commercial cloud environment to exemplarily offer analytics services prototypically.
Complementing the Hadoop service already available on CODE-DE rasdaman will offer important additional functionality, in particular a paradigm of "any query, any time, on any size", strictly based on open geo standards and federated with other data centers. On this platform novel, specialized services can be established by third parties in a fast, flexible, and scalable manner.
To this end, several features crucial for operational services need to be tested and/or implemented, such as securing access (in particular in a federation context), optimal operation in a cloud configuration like CODE-DE, support for radar data, and further items to be determined in the initial requirements analysis phase. The result will be the prototype of a federation of rasdaman installations on CODE-DE, CloudEO, as well as the (external) EarthServer federation; further, best practices on the use of array databases in operational environments will be established. This will pave the way for individual value-adding services by third parties.
This project is supported by German BMWi (Ministry of Economy and Energy).
ORBiDANSe: Orbital Big Data Analytics Service
This project will drive the "ship code to data" paradigm to the extreme: it makes a Cubesat an online Web data service for real time EO acquisition, processing, and retrieval, based on the ISO Array SQL standard under adoption. Images get acquired by the on-board camera and geo-referenced via GPS. Access is done via a high-level array query language allowing ad-hoc processing and filtering on spatio-temporal raster data, similar to what standard SQL accomplishes on tuple sets.
On board, such queries are evaluated by the rasdaman Big Array Data Analytics engine. Among others, it supports spatio-temporal queries, hence is truly multi-dimensional. The configuration can be updated/reconfigured in-flight, although emphasis will be put on automatic optimization, including acquisition planning based on incoming queries. In a direct scenario, targeted subsetting/pocessing of imagery can be downlinked directly to the requesting client, effectively turning the satellite into an image database. In a federated scenario, a client may submit some complex decision support query to a data center; the rasdaman instance there finds out that data are missing and spawns a sub-request to the Cubesat; merges its locally computed results with the Cubesat response into the final result sent back to the user. As rasdaman is already cloud-parallelized, queries can be distributed automatically between ground and space instances.
Overall goal is to achieve a quantum leap in both EO service quality, data availability, and service integration.
This project is supported by German BMBF (Ministry of Education and Research).
BigPicture: Big Data in Agriculture
This project addresses diagnosis in the farm field through Big-Data-based determination of causes for satellite image derived and site-specific variations.
The BigPicture vision is to help farmers to optimize returns and to protect the environment through innovative Big Data technology. By interpreting satellite captured variations in farm fields through Big Data technologies targeted measures, such as fertilizer placement, application of plant protection products, or choice of species to grow, can be derived. Manifold information will be combined in such analyses, including satellite imagery, weather data, as well as the farmers’ experience. This way, the processing chain from satellite-based symptoms captured in the field over diagnosis up to therapy recommendation will be automated entirely.
Internationally recognized experts are teaming up: Spatial Business Integration GmbH is a specialist in satellite-image based information products for agriculture, rasdaman GmbH is an expert in Big Data services. By combining the complementary know-how of both partners BigPicture will establish large-scale automated analysis offering individual insights and support for agricultural production in Germany.
This project is supported by German BMEL (Ministry of Food and Agriculture).
EVO-ODAS: Evolution of EO Online Data Access Services
The technology project "Evolution of EO Online Data Access Services" analyses relevant scenarios and technologies for data publication and access, identifies potential for improvements of standards and their implementations, prototypes and evaluates selected improvements and proposes standard extensions for future releases.
The consultant rasdaman GmbH supports the consortium in the assessment of standards and the specification and demonstration of standard evolutions. rasdaman GmbH is a key contributor to a wide scope of geospatial data access standards within OGC and raster database access standards within ISO. It also provides a spatio-temporal Big Datacube solution with the scalable open source array engine rasdaman.
This project is funded by ESA.
Earthserver-2: Agile Analytics on Big Data Cubes
As a followup to the extremely successful EarthServer, the intercontinental EarthServer initiative, funded by EU H2020, makes Agile Analytics on Big Earth Data Cubes of sensor, image, simulation, and statistics data a commodity for non-experts and experts alike.
Large-scale data centers are setting up water, air, weather, and planetary services on 3D and 4D Petabyte-size data cubes with user-tailored clients for both visual and textual queries for ad-hoc mix&match. Being in its second phase, EarthServer maintains and extends the lead in Big Earth Data services established in the highly successful phase 1. Being already supported by ESA, rasdaman is being established as an enabling building block for COPERNICUS / Sentinel.
rasdaman GmbH provides its enabling technology as the EarthServer platform, together with support for the data centers. The rasdaman technology will be advanced in both scalability and functionality. Further, it contributes to standardization and leads exploitation work.
This project is supported by the European Commission / EINFRA under H2020.
VISEO-FUSION: Video / Image / Sensor / EO - FUSION
ViseoFusion has designed, built and tested a multi-stream geospatial data platform that supports decision-making for operational agencies and critical infrastructure operators. Earth Observation imagery, in-situ datasets and existing web services are integrated, analyzed and visualized in order to provide the end-user with actionable information in either review or near real-time mode
rasdaman GmbH is providing its technology as a backend and information turnpike for various types of high-volume raster data.
Publicamundi: Scalable Reusable Open Geospatial Data
EU FP7 PublicaMundi is delivering methodologies, technologies and software components to leverage geospatial data as first-class citizens in the CKAN open data catalogue by combining reusable software components and tools enabling scalable, responsive, and multimodal value added applications on open geospatial data. Software components providing reusable, scalable, multi-lingual, and cost-efficient processing and analysis services for open geospatial data are being established, leveraging both vector and high-volume raster data.
rasdaman GmbH contributes its scalable raster engine, rasdaman, for integration with the widely used CKAN catalogue tool. This way, CKAN gets enabled with large-scale spatio-temporal raster data handling.
[ project flyer }
EarthServer: open, flexible, scalable, and user-friendly access to spatio-temporal Earth science data
EarthServer has established services and technology for open, flexible, scalable, and user-friendly access to spatio-temporal Earth science data. Accessing such data ranges from simple navigation over extraction, download, filtering, and processing up to complex analytics. This is accomplished through versatile ad-hoc analytics suitable for extreme-size multi-dimensional Earth Science data, based on and extending leading-edge Array Database technology.
EarthServer combines both, thereby achieving tight data/metadata integration. Further, the rasdaman Array Database System is extended with further spatiotemporal coverage data types. On server side, highly effective optimizations -- such as parallel and distributed query processing -- ensure scalability to Exabyte volumes.
Several Lighthouse Applications, spanning all Earth sciences, have been established with up to 132 TB databases covering: atmosphere (climate modelling), hydrosphere (oceanography), lithosphere (geology), and cryosphere (snow and ice mapping); in addition, a air-borne science (drone) imagery database has been set up and a Planetary science use case with surface and subsurface data on Mars.
The main innovations of EarthServer are:
- Substantial simplification of use of any-size, any-dimension Big Earth Data, through flexible, scalable, integrated, standards-based access interfaces;
- Integrating data and metadata in Big Earth Data queries,
- Uniform access across space and time, enabling large image and weather timeseries;
- Web service support for across all OGC coverage types: multi-dimensional grids, point clouds, and meshes;
- Transparent queries across heterogeneous file archives and databases;
- Paving the way for Petabyte Copernicus and climate services through parallel and distributed processing of queries.
- Strict reliance on the open OGC standards for Big Earth Data access, Web Coverage Service (WCS) and Web Coverage Processing Service (WCPS), including advancing these standards themselves.
Nucleus is the pre-existing rasdaman system which has coined a new research field, Array Databases. During EarthServer, rasdaman has strengthened its position as the world‟s leading Array Database over recent competitors such as SciDB. This substantial technological advance has resulted in a series of awards for Jacobs Uni and rasdaman GmbH, such as winner of the Big Data challenge in the Copernicus Masters competition 2014.
A wide range of outreach activities has accompanied technology development and service de- ployment. Trainings, best practices, and auxiliary software tools have been provided at events and on the Web. A multitude of communication channels have been used to position the EarthServer technology worldwide (such as GEO Plenaries, Copernicus workshops, cli- mate data workshops, standardization meetings). A particular highlight is a 1h TV documentary titled “Big Earth Data – the digitized planet” (Fig. 3) done by Jacobs and rasdaman GmbH one in collaboration with German ZDF and ARTE, to be broadcast soon after the project. While focusing on Earth sciences, this project transcends Earth and Planetary data; rather, Array Databases comprise horizontal platform technology equally game-changing Life sciences, but also industry (such as oil and gas exploration) and eGovernment. In the latter context, during EarthServer INSPIRE has agree to base its Download Services for coverages on the OGC Web Coverage Service (WCS); work will commence in October 2014. To foster such uptake, industry and authorities have been involved in EarthServer.
In TeraPro, a flexible, powerful image processing tool, proStack, has been coupled with a scalable array database backend, rasdaman.