The U.S. Census is an amazing resource of data and information. The U.S. Census performs a number of regular as well as ongoing surveys that document many facets of people and life in the U.S. These data can often be used to help learn about dimensions of a location and what it might contain.
As humans though when asked about what a neighborhood is like we don't rhyme off a series of census variables for that neighborhood. Instead whether it be the hipsters of Williamsburg or the stroller traffic jams of Noe Valley we tend to describe neighborhoods in terms of archetypes that we can more easily relate to. These kinds of neighborhood descriptions can add meaningful value and context to your location data.
We want to make this kind of contextual data available and easier to use in CartoDB.
Releasing segmentation layers
Today we are releasing inside the Data Observatory a new set of layers created from demographic segmentation. Demographic segments provide a kind of grouping of people that we then apply a data-driven naming method that makes them easily readable and recognizable when analyzing your data. By releasing them in the Data Observatory we are making them available for users to use for augmenting their own data quickly.
The segmentation is generated through a clustering procedure that we'll cover in more depth in a forthcoming blog post. The output gives us two granularities of clustering one that produces 10 unique groupings of people across the USA and a second that creates 55 unique groupings of people.
10 cluster resolution
To generate these clusters we used the algorithum proposed by Spielman and Singleton and for the x10 clusters we were able to adopt their naming structure.
The names of the 10 different neighborhood types are:##_INIT_REPLACE_ME_TABLE_##
|Hispanic and kids|
|Low Income and Diverse|
|Middle income single family homes|
|Low income minority mix|
|Old Wealthy White|
|Residential institutions young people|
|Wealthy Nuclear Families|
|Low Income African American|
|Wealthy urban and kid-free|
55 cluster resolution
The 55 cluster layer was more difficult as the names of each group had not been previously published. For these more detailed categories we generated names based on the dominant traits of the populations within that cluster (or the dominant omission in a few cases). For example if an area within a city population is found to be highly dominated by college age adults with some college education it was given the name "City center university campuses".
Take a look at all 55 proposed group names:##_INIT_REPLACE_ME_TABLE_##
|Middle Class educated suburban mixed race|
|Low income on urban periphery|
|Suburban young and low-income|
|Low-income urban young unmarried|
|Low education mainly suburban|
|Young working class and rural|
|Low income with gentrification|
|High school education long commuters Black White Hispanic mix|
|Rural bachelors or college degree rent/owned mix|
|Rural high school education owns property|
|Young city based renters in sparse neighborhoods low poverty|
|Predominantly black high high school attainment home owners|
|White and minority mix multilingual mixed income / education married|
|Hispanic/Black mix multilingual high poverty renters uses public transport|
|Predominantly black renters rent / own mix|
|Lower middle income with higher rent burden|
|Black and mixed community with rent burden|
|Lower middle income with affordable housing|
|Relatively affordable satisfied lower middle class|
|Satisfied lower middle income higher rent costs|
|Suburban/rural satisfied decently educated lower middle class|
|Struggling lower middle class with rent burden|
|Older white home owners less comfortable financially|
|Older home owners more financially comfortable some diversity|
|Younger poorer single parent family Native Americans|
|Older middle income Native Americans married and educated|
|Older mixed race professionals|
|Works from home highly educated super wealthy|
|Wealthy and rural living|
|Wealthy retired mountains/coasts|
|Wealthy diverse suburbanites on the coasts|
|Urban - inner city|
|College town with poverty|
|University campus wider area|
|City outskirt university campuses|
|City center university campuses|
|Lower educational attainment homeowner low rent|
|Younger long commuter in dense neighborhood|
|Long commuters White/Black mix|
|Low rent in built up neighborhoods|
|Renters within cities mixed income areas White/Hispanic mix unmarried|
|Older Home owners with high income|
|Older home owners and very high income|
|White/Asian Mix big city burb dwellers|
|Bachelors degree mid income with mortgages|
|Asian/Hispanic Mix mid income|
|Bachelors degree higher income home owners|
|Wealthy city commuters|
|Very wealthy multiple million dollar homes|
|High rise dense urbanites|
On the map
You can explore both on this map and the deep insights dashboard here or take a look at the simple map version here:
Accessing demographic segments
Using the awesome power of the Data Observatory to bring these segments into your data is as easy as calling a quick SQL statement.
To query these segments at a single point location simply use the function
SELECT * FROM OBS_GetUSCensusCategory( CDB_LatLng(40.704512 -73.936669) 'Spielman-Singleton Segments: 10 Clusters')
##_INIT_REPLACE_ME_PRE_##SELECT * FROM OBS_GetUSCensusCategory( CDB_LatLng(40.704512 -73.936669) 'Spielman-Singleton Segments: 55 Clusters') ##_END_REPLACE_ME_PRE_##
Augmenting your data
Another interesting use of the segmentation data is to augment your tables. You can do so by adding a new column to any table called
segment(or any other unique name).
Next augment your table with the segment description:
##_INIT_REPLACE_ME_PRE_##update your_table set segment = ( SELECT * from OBS_GetUSCensusCategory( the_geom 'Spielman-Singleton Segments: 10 Clusters' ) ) ##_END_REPLACE_ME_PRE_##
##_INIT_REPLACE_ME_PRE_##update your_table set segment = ( SELECT * from OBS_GetUSCensusCategory( the_geom 'Spielman-Singleton Segments: 55 Clusters' ) ) ##_END_REPLACE_ME_PRE_##
To create similar or even better visualizations you can watch our Data Observatory webinar as many times as you need to!
Today we wanted to announce the availability of this exciting set of layers in the Data Observatory. In future blog posts we will explore some of these groupings what they can tell us about the U.S. and how they can add context and insight into your data. We will also detail how these segments were created and how we plan to improve and expand on them in the future.
For further reading checkout the data services-api docs and the Data Observatory.
For now happy demographic segment mapping!