Data/SQL Guides

Connecting Twitter Data

This guide explains how to import geocoded Twitter data into a dataset that is located in your CARTO account. CARTO provides access to the Twitter Firehose, which enables users to visualize and analyze this data directly in CARTO.

Enabling the Twitter Connector

Twitter is available from the “Connect Dataset” options when adding a new dataset, or creating a new map.

The Twitter connector is included for Enterprise account plans. If you are interested in activating this service for your account, request a demo.

Connecting Twitter Data to a Dataset

After the Twitter connector has been enabled, select Twitter from the Connect Dataset options. Before you connect your data, define the Twitter search categories and terms to be used to query Twitter’s data feed.

Connecting Twitter Data to a Dataset

The following options enable you to define the search terms for your Twitter query.

Twitter Trend Options Description
Category 1-4 Represents search terms for different hashtags or keywords separated by commas. You can enter up to four search terms using the Category fields.
From / to Displays the from and to time range for your requested data. By default, it is set to search for the last 30 days. Depending on your account settings, you can click the calendar icon to open the calendar and select a different date range. Additionally, you can indicate the hour and minute for the selected date range,
Use Displays the amount of Twitter credits allocated to your account. You can use the slider to increase or decrease the percentage of credits to use. Contact us to update your Twitter credits at any time.

CHEATSHEET: Twitter Data Tips

Note the following tips and tricks for how to search with Twitter data and how credits are consumed.

  • Enter commas between multi-sentence words as a best practice. For example, if you enter a multi-word search term, such as "cars, bikes motorbikes, planes", the category is parsed as three search terms. Not "cars, bikes, motorbikes, planes".
  • Use the `postedtime` column from your connected Twitter dataset as the time and date field.
  • Use singular and plural search terms. For example, "car" does not match tweets with the plural of "cars". Add both the singular and plural forms of the word to match both search results.
  • The maximum number of search terms per category is 29.
  • You can search by hashtags or account names. For example, "gis, @carto, #mapping".
  • Enterprise account users share the same organization-wide pool of Twitter credits.
  • Once the search starts, you cannot stop the import. Be mindful to avoid broad search terms. For example, "love" may retrieve hundreds of thousands, or even millions, of geolocalized tweets.
  • If you run out of credits, the search stops and returns all the tweets that the system was able to retrieve up until the credits were used.

CARTO only retrieves geotagged tweets, which is approximately 5% of all tweets. However, through geo-enrichment enhancements CARTO increases the Twitter search results up to 15% to 20%.

CHEATSHEET: Twitter API Data Options

There are several Twitter APIs for which CARTO can provide access. You can choose from the following data plans, or contact us for details.

  • Search API: This is the default plan, that enables the Twitter Connector to pull geolocated Twitter data from the past 30 days.
  • Streaming: This allows you to retrieve up-to-the-minute search results and can be enabled at an additional charge.
  • Historical API: This helps return older search results of tweets, beyond 30 days and can retrieve data as far back as 2006 - also enabled for an additional charge.

We are testing a feature to support real-time tweets, which automatically updates your map when live tweets are posted. If you want to use it before it becomes public, contact us for more information on how to implement it.

Understanding Twitter Data

After connecting to a Twitter dataset, unique columns from your dataset contain important Twitter information. You may need this information in order to filter or plot Twitter data on your visualization.

For details about Twitter columns and attributes, see the Tweet Activities documentation for Twitter data formats.

  • For this guide, we searched the popular ‘Hillary Clinton’ election hashtags - ‘hillaryclinton’ and ‘imwithher’ for one-day (10/27/2016) to see trends over a single day.

    Styling Twitter data

  • The Data View of the selected map layer displays unique Twitter columns. Information ranges from the postedtime, the language to tweet text, the Twitter user, and so on.

    Dataset

The Data View and Map View appear as buttons on your map visualization when a map layer is selected. Click to switch between viewing your connected dataset as a table, or show the map view of your data.

Styling Twitter Data

You can visualize your Twitter data in many different ways using Builder. For example, to view your data over time and to visualize the tweets as they increase/decrease over the queried time-range, apply the ANIMATED aggregation style for the map layer.

By animating the tweets on Hillary Clinton over the hours in a day, you can effectively visualize the brackets where the tweets rise or diminish.