Javier Pérez Trufero

Data Observatory: Thousands of Spatial Datasets On Demand

Continuing on our mission to improve the efficiency of Data Science teams who need to work with spatial data, and after several months of testing, validation, and continuous development under the private beta launched at the end of 2019, we are excited to announce a new release of CARTO’s Data Observatory.

Data is an essential ingredient for spatial analysis, and the main problem in data analytics today is the amount of time analysts and data scientists spend discovering, understanding, gathering and preparing the right data for their use cases. There is a clear need for a spatial data platform that enables users to stop wasting time on “data admin”, and CARTO has been doubling down its efforts to become that one platform.

Graphic showing time spent on data admin

Thanks to the new Data Observatory, CARTO now becomes your one-stop shop for spatial data and technology, with the ability to provide seamless access to thousands of ready-to-use spatial datasets from vetted sources. By incorporating CARTO’s Location Intelligence stack, our customers can dramatically reduce the cost of acquiring third party data. The platform provides a homogenous and flawless process to discover, license, and access data from a fast-growing catalog of premium providers and public sources, across many different data categories including, Demographics, Financial, Human Mobility, and Environmental.

In this new release, the CARTO dashboard becomes the central user interface to manage all things Data Observatory, providing a system to the data scientist-oriented methods that we already made available earlier this year via CARTOframes. From today, all our cloud enterprise users have the possibility from within their dashboard to:

  • Explore our Spatial Data Catalog to discover relevant datasets for your spatial analyses;
  • Subscribe instantly to all of our public datasets and submit subscription requests for premium datasets;
  • Manage all details related to your Data Observatory subscriptions so you can access data via the relevant user interface (Builder, CARTOframes, Engine, etc.).

Let’s take a quick look at these exciting new capabilities!

New Spatial Data Catalog

We have improved the design and functionality of our Spatial Data Catalog so users can better explore the specifications of thousands of spatial datasets available via the Data Observatory and find the best data based on their requirements. This catalog is built on top of a smart metadata system that has registered in an homogenous manner all characteristics defining a dataset (i.e. provider, coverage, schema, variable description, sample data, etc.), providing a unified experience to explore and understand the details of very different types of spatial data.

Screenshot showcasing the Spatial Data Catalog

Can't find what you are looking for in the Data Observatory?

Let us know!

Simple access to thousands of public and premium datasets

Using the same Spatial Data Catalog web interface within the dashboard users can now instantly subscribe to any of our public datasets and submit requests to start the licensing process of premium datasets from our vetted data sources.

We know that one big part of the Data Observatory’s value is the availability of high quality public datasets that can provide a fast way for our users to enrich their own data and run richer spatial analyses. That’s why our Data team is continuously working to expand our catalog of public datasets with Census data from different countries, digital boundaries, environmental, and socioeconomic datasets, among others. The self-service subscription process for public datasets provides users hassle-free access to our ever-growing public data offering in just a matter of a couple of clicks.

In parallel, CARTO leads the way when it comes to establishing partnerships with leading data providers worldwide, allowing our users to benefit from the highest-quality, most accurate and relevant spatial data globally. Through this new interface and after placing a subscription request, our team of experts will get in touch to help you understand all the details about the dataset, to ensure it is the best fit for your use case and take the hassle out of the licensing process. Once that process is complete, the premium data subscription will be automatically created and you will have access to a ready-to-query and always up-to-date version of the premium dataset of your choice.

Screenshot showcasing confirmation of dataset subscription

Subscription management and data access

Through the new section “My Subscriptions” (within the Data area) users can now check all relevant details to manage and access the different datasets subscribed from the Data Observatory via our different tools and interfaces. From this central page you can check the status of each subscription (e.g. requested, active, expired, etc.), access each dataset’s metadata, and start working with your subscribed dataset.

For those datasets supported by CARTO Builder, users can use this interface to directly access the data or create a new map through the options “View dataset” and “Create map” respectively. Users working in Python can either download a pre-built Python notebook illustrating how to access and explore the data with CARTOframes or use this web-based interface to retrieve the ID’s needed to call and access the data from their own notebooks. It is important to remember that through that interface CARTO offers Data Enrichment methods for enriching both point-based and polygon-based data with variables from the datasets licensed via the Data Observatory. The results of the enrichment processes can then be then uploaded back as CARTO tables if users want to work with that enriched data in Builder.

Screenshot showcasing subscriptions within CARTO Builder

Connectivity with other cloud data warehouses

We know that it is important for our users to also connect different datasets to their own data warehouses and to leverage other tools; for this reason, we are also working to ensure the Data Observatory connects to other state-of-the-art data platforms. Now, you can make your data subscriptions available in Google BigQuery, and in the near future, Snowflake and AWS platforms.

This video shows you these new functionalities in action:

With this new release, we continue our efforts to become the leading spatial data platform in the market by providing a streamlined process to reduce the operational inefficiencies of discovering, licensing, and accessing spatial data. This allows you to stop wasting time at different stages of the process and focus your efforts on generating value through spatial analysis and modelling (which is what organizations expect their Data Science dollars to be producing).

If you are a data provider interested in adding your datasets to Data Observatory 2.0, we’d also like to hear from you - please contact us.

How to get started

This new release of the Data Observatory is available right now to all cloud enterprise customers. You can find some additional information about this component of our stack in the CARTO help center.

Not yet a CARTO enterprise customer and would like to see this in action?

Request a live personalized demo
EU Flag This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 960401.
About the author
Javier Pérez Trufero

Javier is VP of Product and is globally responsible for CARTO’s product portfolio and innovation roadmap. He also leads the company strategy with respect to third party data offerings and data science activities. Javier's responsibilities span the complete product development cycle. He also coordinates new alliances with top-class data providers such as Vodafone and Mastercard.

More posts from Javier Pérez Trufero

Related Posts

Ready to optimize your territories with Location Intelligence?

Close circle icon

Contact us

Please fill out the below form and we'll be in touch real soon.