Delivering Geospatial Datasets on the Leading Cloud Data Platforms

The CARTO Data Observatory continues to evolve, giving Developers and Spatial Data Scientists unparalleled access to thousands of curated public and premium location based datasets. Recently we enhanced the Data Observatory to allow users to access sample data online and visualize these samples on preconfigured maps, simplifying the data evaluation process.

Today we further improve the user experience by enabling access to subscribed datasets directly on the leading cloud platforms, including Google Cloud, Amazon Web Services, and Microsoft Azure

Thanks to this development, Data Observatory users can now access any dataset from our catalog within their cloud platform of choice. This enables the seamless integration of this data within their existing workflows using Athena, Databricks, and countless other cloud services. This allows users to complement the geospatial capabilities of CARTO with those provided by other platforms, and at the same time eliminating the need for the complex ETLs required to have any subscribed datasets available where needed.

How it works

Your Subscriptions section in the CARTO dashboard brings together all of the user’s data subscriptions and provides a set of options and tools to access these datasets, such as quick actions to create maps with CARTO Builder or a template Python notebook to explore the data using CARTOframes.

From today, three new options are available for your Data Observatory subscriptions: Access in BigQuery, Access in AWS, and Access in Azure.

Access in BigQuery

Once users have connected their Google BigQuery account to CARTO using our connector, this option makes it possible to query the tables from their Data Observatory subscriptions directly from the data warehouse. For that, the following information is provided:

• Location of the data table in BigQuery;
• Location of the geography table in BigQuery;
• An example query to join the data and geography tables.

Exploring the dataset directly from BigQuery is as simple as pasting the example query provided in CARTO directly into the BigQuery console.

Access in AWS and Azure

Using these new options Data Observatory users can get their datasets delivered in either AWS S3 or Azure Blob Storage as AVRO files.

This process works on a request basis, that simply consists of clicking on the request button provided in the interface.

Once the process has been completed and the dataset has been delivered to the desired location, access details will be made available in the interface.

Video Demo

Check out this video of the key steps to access subscribed datasets on cloud platforms.

This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 960401.