Sergio Álvarez • Sep 1, 2016

Better Aggregation Methods for Exploring Big Datasets

For a long time, most GIS tools focused on creating maps with very small datasets. Now that we have more and more data we need better tools to visualize datasets that are way bigger, or that represent a high density amount of features for some particular places. For example, if you try to visualize a dataset containing 3 million points in San Francisco, it is very likely that you will end up having more points than pixels on the screen. Therefore, showing points without aggregating them makes absolutely no sense.

Because of this, we’ve decided to rebuild the way you style your maps by separating the style properties from the aggregation properties. In CARTO Builder you can aggregate your data by squares, hexbins, administrative regions (we will be adding more in the future), and pixels –which will end up as a heatmap– or time units with a single click. For those who have used the old Editor, this will remind you of the good ol’ density maps.

On the other hand, those aggregations are calculated on the fly, so if you filter the data in your layer, the aggregation will be recalculated automatically, which turns out to be very useful when analyzing different distributions of points depending on a value (analyzing tree coverage by species or by caretaker).

Of course, we also support more custom aggregations (using census tracts for example), which require using the intersection analysis and your own polygon data (or our boundary data coming from the Data Observatory.

Happy data analyzing!

About the author

Sergio has spent years bridging the designer-developer divide and making products to improve the way that decisions are made. He has a background in computer science and currently researches data visualization, information design, and interface design with a special focus on data interaction.
In 2009, Sergio co-founded Vizzuality, a company dedicated to visualization, analysis, and cloud-based services for large amounts of data, with offices in Madrid and New York.

More posts from Sergio Álvarez