On-Premises Private

CARTO can be installed on a local server, or set of servers, either directly or using our commercial installer.

Architecture

Stack diagram

A common production CARTO Builder stack is splitted in diferent services and roles. Some of them work in synchronous mode and some work in asynchronous mode. For example, when you render your dasboard, a map or perform SQL requests you will receive immediate responses. However, if you want to import a new dataset, once it has been uploaded, the process import will continue in the background.

This is a very simple diagram of a simple architecture.

image

Let’s review the main services in the stack.

Nginx

As described in the NGINX wiki NGINX is a free, open-source, high-performance HTTP server and reverse proxy server. NGINX is known for its high performance, stability, rich feature set, simple configuration, and low resource consumption.

Nginx is used to proxy and control the behavior of the HTTP requests to the backend applications. It’s also used as HTTP termination.

Varnish

As described in the Varnish website Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy.

Sending SQL queries and rendering maps tiles may have a big impact on the resources usage of the platform. Common usage patterns of CARTO usually consist on requesting multiple times similar maps tiles or SQL queries. Under high requests load having a transparent cache in the middle of the client and the applications can improve performance in several orders of magnitud higher.

Builder

Builder is the web management component of CARTO. Within Builder, you can find all the features available in CARTO. Some of the most significant tasks that you can achieve with Builder are:

  • User management. Credentials, authorization, personal info and billing.
  • Connect datasets to your CARTO account, either by importing your own datasets or importing other publicly available data.
  • Create maps from your datasets.
  • Publishing and permissions management of datasets and maps.
  • Synchronized tables management.

Internally, Builder is the operations core of CARTO. It manages PostgreSQL metadata database, keep some metadata in sync with Redis, manages new datasets import queues with resque, etc..

It is developed in Ruby on Rails and like the other components of CARTO, is Open Source. You can find the source code at CartoDB/cartodb.

For user documentation, view https://carto.com/learn/guides.

Although you can checkout any branch of the repository, most of them are usually a work in progress and are not guaranteed to work correctly. In order to run a production-ready Builder service, you need to use the master branch.

Service Modes

The code of CARTO Builder needs to run in two different modes, HTTP server mode and background jobs mode.

  • The HTTP server processes the http requests sent to the service and returns a response synchronously. Any ruby rack server can be used to start Builder in this mode. Some examples of rack servers are mongrel, webrick, thin, or unicorn.
  • The background jobs mode is started with resque. In this mode, the service keep polling some redis keys in order to find pending background jobs. When it finds one, it processes it and changes the state of the job in redis. CARTO uses this mode for different type of jobs, such as dataset imports or synchronized tables.

SQL API

The SQL API provides a node.js based API for running SQL queries against CARTO.

Like the other components of CARTO, is Open Source and you can find the source code at CartoDB/CartoDB-SQL-API.

For user documentation, view https://carto.com/docs/carto-engine/sql-api/.

Although you can checkout any branch of the repository, most of them are usually a work in progress and are not guaranteed to work correctly. In order to run a production-ready SQL API service, you need to use master branch.

Maps API

The Maps API provides a node.js based API that allows you to generate maps based on data hosted in your CARTO account, by applying custom SQL and CartoCSS to the data.

Like the other components of CARTO, is Open Source and you can find the source code at CartoDB/Windshaft-cartodb.

For user documentation, view https://carto.com/docs/carto-engine/maps-api/.

Although you can checkout any branch of the repository, most of them are usually a work in progress and are not guaranteed to work correctly. In order to run a production-ready Maps API service, you need to use master branch.

PostgreSQL

PostgreSQL is the open source database powering CARTO.

CARTO uses PostgreSQL for two purposes:

  • Metadata Storage. This is the metadata used by CARTO Builder. Builder models such as user’s information, visualizations, or other metadata, is stored in this database. The name and connection information of this database is specified on the Rails app_config.yml configuration file.
  • User Data Storage. Each single user, or organization, in CARTO has a individual PostgreSQL database. This database is created during the user signup process. Its database name and connection info is generated on the fly by the CARTO application during this process. Both values are stored within the user info in the metadata database. Every user database name contains the user UUID.

Both the Metadata Storage and the User Data Storage databases can be hosted either in the same PostgreSQL cluster, or different ones. Having both in the same cluster is the recommended approach for small environments.

Builder only knows how to connect to a Metadata Storage database. However, within every request it checks the connection info of the User Data Storage database; which is stored within the Metadata Storage database, as described above.

PostGIS

PostGIS is the extension which adds spatial capabilities to PostgreSQL. It allows you to run geospatial functions in PostgreSQL. It is required for both the User Data Storage and the Metadata Storage databases.

CARTO PostgreSQL Extension

The CARTO PostgreSQL extension can be found at https://github.com/CartoDB/cartodb-postgresql.

This extension is required by all the components of CARTO and it must be installed in the server where user databases are stored.

It provides functions (and other helpers) needed by Builder, Maps, and SQL APIs such as:

  • CARTOfying functions, which convert raw PostgreSQL tables recognized by CARTO. It adds some additional columns and triggers.
  • Multiuser schema handling functions.
  • Quota helpers.
  • Cache helpers.
  • etc.

The CARTO extension depends on PostGIS and on the pg_schema_triggers extension.

Redis

Redis is a key-value store engine, used by most components of the CARTO application stack, to store configuration and cache.

In contrast to the PostgreSQL metadata (which is used only by CARTO Builder), the metadata stored in Redis is shared among CARTO Builder, the SQL API, and the Maps API.

Important

Even though it is used as a cache, there is also persistent data stored in Redis. In some environments, Redis is configured by default to act without persistency (see http://redis.io/topics/persistence).

You must ensure Redis is configured properly to keep its data between restarts.

  • Data is stored in separate databases inside this Redis:

    • Database 0: Table and visualization metadata, including map styles and named maps.
    • Database 3: OAuth credentials metadata.
    • Database 5: Metadata about the users, including API keys and database_hosts.