Javier de la Torre

Announcing CARTO Spatial Extension for BigQuery

Today we announced at Google’s Cloud Next ‘21 the General Availability of the new CARTO Spatial Extension for BigQuery. Our friends from Google Cloud are showcasing it today during their talk on Geospatial innovations for a cloud-first world, with the Google Maps team publishing on their blog the work we have done to integrate Google Maps in our product.

Spatial analysis in BigQuery provides some major advantages over traditional GIS platforms. We have been big promoters of Cloud Native architectures for some time and customers have also been guiding us. They don’t want to migrate their own data to external databases. And running on your own warehouse provides additional benefits around privacy, compliance, scalability and cost.

Benefits of spatial analysis in BigQuery

And today we are completing the picture.

Introducing CARTO Spatial Extension for BigQuery

The CARTO Spatial Extension for BigQuery is a complete redevelopment of our leading Location Intelligence platform. During the past months, we have been working closely with the Google team to develop a fully cloud native GIS experience for BigQuery users. When doing spatial analysis, BigQuery users are looking for:

An easy way to quickly explore geospatial data in their BigQuery account

A modern way to make maps or do analysis from SQL or tables and to be able to share them. Including ultra large datasets.

A complete framework of analysis capabilities to perform spatial analytics in SQL, and computed in BigQuery.

Access to public and premium location data inside BigQuery without ETLs or manual updates.

A simple way to develop spatial applications that represent data from BigQuery directly in Google Maps.

Well, the CARTO Spatial Extension provides a solution to all these requirements and more. Let’s take a look, one by one.

A brand new user interface

We are introducing a brand new experience for BigQuery users. Create organization accounts, connect your BigQuery, and start exploring your data.

One of the most important modules on the new interface is the Data Explorer. In this section users can easily explore their BigQuery account, dataset by dataset and table by table. CARTO connects using the native BigQuery API and recognizes special tables like Tilesets. You can get a quick map preview and summary statistics.

Screenshot of Data Explorer

A new Builder for vector map making

When working with location data and spatial analytics, seeing is believing. That’s why having a good visual interface to run SQL, work with cartography, apply filters, and much more is so important.

For this new version of Builder we have moved entirely to vector rendering with the power of the open source Kepler.gl and deck.gl frameworks. In this first iteration we have the foundations for a great map making tool. Look out for the many upcoming features planned for Builder, such as widgets for data exploration. We can’t wait to see the stunning maps and analysis you develop with it.

See how you can write arbitrary SQL, including scripts, and CARTO will do all the work of pushing down that query into BigQuery and then represent it seamlessly on the map.

You can also see on this map the support for really large datasets using our Tiler technology for BigQuery. The new Builder allows you to mix SQL, tables, or tileset in the same map, providing map makers the flexibility they need.

And of course these maps need to be rendered on top of a basemap, and we have integrated several basemap options, including Google Maps.

A complete Analytics Toolbox

Our Analytics Toolbox unlocks more than 60 advanced spatial functions inside BigQuery using an ever-increasing set of User Defined Functions (UDFs). These routines cover a broad range of spatial use cases and include data transformations, spatial indexing for Quadkey, S2 and H3, and advanced functions to carry out geocoding, clustering, route calculations, and more.

With the core part we aim at adding capabilities to BigQuery to match PostGIS capabilities until the BigQuery team adds them natively. On the Advanced modules we expose functions suitable for more complex Spatial Analysis that goes beyond what PostGIS can provide.

Diagram showing Analytics Toolbox components

The Analytics Toolbox also incorporates our BigQuery Tiler, a map visualization solution that handles massive spatial data volumes hosted in BigQuery.

All of these functions are fully integrated into the platform giving Data Scientists the ability to explore, visualise, and analyze their spatial data without switching contexts or the need to ETL data.

And of course you can use the Analytics Toolbox in the new Builder interface, as in this example where we are performing a K-means clustering on crime data in Chicago.

Screenshot showing K-means clustering in action

Easy access to public and premium spatial data

Core to the platform is making access to spatial datasets as frictionless as possible. The Spatial Extension for BigQuery lets you search, browse, visualize, and ingest thousands of curated spatial datasets from our Data Observatory. More importantly, this data is made available directly in BigQuery so you can easily run spatial JOINS, without the need for complex ETL processing.

You can explore the catalog directly through our new user interface. And get access to a large set of sample data from some of the best location data providers out there. Look how we are looking at integrating the Data Observatory experience on the Data Explorer page.

Screenshot showing Data Observatory experience on the Data Explorer page

APIs, SDKs, and templates to build your own spatial applications with BigQuery and Google Maps

Most developers are going to need to develop their own custom applications or embed maps in different places. CARTO provides a full set of APIs and SDKs to power your spatial app development.

Today we are also announcing the release of deck.gl 8.6, the underlying Open Source library we use for visualization. Together with Google Maps, we are releasing full support for Google Maps vector formats. Read the blog post from Google Maps to learn more about it.

If you have data on BigQuery and want to visualize it on Google Maps, CARTO is a great way of doing so without having to build your own geospatial backend. Fully integrated within the Builder experience is the ability to easily design maps and then use them in code.

Screenshot showing how to quickly visualize BigQuery data in CARTO

There has been a lot of work to make all of this possible, not only overlaying data on Google Maps, but also to make it effective with interleaving and with just one single WebGL context.

To showcase the potential of visual storytelling using spatial data, we have created a demo that centers on the opportunity for vehicle electrification in the US.

Screenshot of vehicle electrification demo

A new phase for Spatial Analysis in BigQuery

Today’s release is a major step towards enabling advanced Location Intelligence to Google’s cloud data warehouse platform. Now, BigQuery users have access to a fully integrated platform to visualize spatial data, run advanced location-based analytics and develop engaging web experiences using cloud hosted data.

The Spatial Extension for BigQuery is available as a free 14 day trial. Click here to sign up and get access to all of these great, new features.

EU Flag This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 960401.
About the author
Javier de la Torre

Javier de la Torre is founder and Chief Strategy Officer of CARTO. One of the pioneers of location intelligence, Javier founded the company with a vision to democratize data analysis and visualization. Under his leadership, CARTO has grown from a groundbreaking idea into one of the fastest growing geospatial companies in the world.
In 2007, he founded Vizzuality, a renowned geospatial company dedicated to bridging the gap between science and policy making by the better use of data.

More posts from Javier de la Torre

Related Posts

Ready to optimize your territories with Location Intelligence?

Close circle icon

Contact us

Please fill out the below form and we'll be in touch real soon.