A common production CARTO Builder stack is splitted in diferent services and roles. Some of them work in synchronous mode and some work in asynchronous mode. For example, when you render your dasboard, a map or perform SQL requests you will receive immediate responses. However, if you want to import a new dataset, once it has been uploaded, the process import will continue in the background.
This is a very simple diagram of a simple architecture.
Let’s review the main services in the stack.
As described in the NGINX wiki NGINX is a free, open-source, high-performance HTTP server and reverse proxy server. NGINX is known for its high performance, stability, rich feature set, simple configuration, and low resource consumption.
Nginx is used to proxy and control the behavior of the HTTP requests to the backend applications. It’s also used as HTTP termination.
As described in the Varnish website Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy.
Sending SQL queries and rendering maps tiles may have a big impact on the resources usage of the platform. Common usage patterns of CARTO usually consist on requesting multiple times similar maps tiles or SQL queries. Under high requests load having a transparent cache in the middle of the client and the applications can improve performance in several orders of magnitud higher.
Builder is the web management component of CARTO. Within Builder, you can find all the features available in CARTO. Some of the most significant tasks that you can achieve with Builder are:
- User management. Credentials, authorization, personal info and billing.
- Connect datasets to your CARTO account, either by importing your own datasets or importing other publicly available data.
- Create maps from your datasets.
- Publishing and permissions management of datasets and maps.
- Synchronized tables management.
Internally, Builder is the operations core of CARTO. It manages PostgreSQL metadata database, keep some metadata in sync with Redis, manages new datasets import queues with resque, etc..
It is developed in Ruby on Rails and like the other components of CARTO, is Open Source. You can find the source code at CartoDB/cartodb.
For user documentation, view https://carto.com/learn/guides.
Although you can checkout any branch of the repository, most of them are usually a work in progress and are not guaranteed to work correctly. In order to run a production-ready Builder service, you need to use the master branch.
The code of CARTO Builder needs to run in two different modes, HTTP server mode and background jobs mode.
The SQL API provides a node.js based API for running SQL queries against CARTO.
Like the other components of CARTO, is Open Source and you can find the source code at CartoDB/CartoDB-SQL-API.
For user documentation, view https://carto.com/docs/carto-engine/sql-api/.
Although you can checkout any branch of the repository, most of them are usually a work in progress and are not guaranteed to work correctly. In order to run a production-ready SQL API service, you need to use master branch.
The Maps API provides a node.js based API that allows you to generate maps based on data hosted in your CARTO account, by applying custom SQL and CartoCSS to the data.
Like the other components of CARTO, is Open Source and you can find the source code at CartoDB/Windshaft-cartodb.
For user documentation, view https://carto.com/docs/carto-engine/maps-api/.
Although you can checkout any branch of the repository, most of them are usually a work in progress and are not guaranteed to work correctly. In order to run a production-ready Maps API service, you need to use master branch.
PostgreSQL is the open source database powering CARTO.
CARTO uses PostgreSQL for two purposes:
Both the Metadata Storage and the User Data Storage databases can be hosted either in the same PostgreSQL cluster, or different ones. Having both in the same cluster is the recommended approach for small environments.
Builder only knows how to connect to a Metadata Storage database. However, within every request it checks the connection info of the User Data Storage database; which is stored within the Metadata Storage database, as described above.
PostGIS is the extension which adds spatial capabilities to PostgreSQL. It allows you to run geospatial functions in PostgreSQL. It is required for both the User Data Storage and the Metadata Storage databases.
The CARTO PostgreSQL extension can be found at https://github.com/CartoDB/cartodb-postgresql.
This extension is required by all the components of CARTO and it must be installed in the server where user databases are stored.
It provides functions (and other helpers) needed by Builder, Maps, and SQL APIs such as:
- CARTOfying functions, which convert raw PostgreSQL tables recognized by CARTO. It adds some additional columns and triggers.
- Multiuser schema handling functions.
- Quota helpers.
- Cache helpers.
Redis is a key-value store engine, used by most components of the CARTO application stack, to store configuration and cache.
In contrast to the PostgreSQL metadata (which is used only by CARTO Builder), the metadata stored in Redis is shared among CARTO Builder, the SQL API, and the Maps API.
Even though it is used as a cache, there is also persistent data stored in Redis. In some environments, Redis is configured by default to act without persistency (see http://redis.io/topics/persistence).
You must ensure Redis is configured properly to keep its data between restarts.
Data is stored in separate databases inside this Redis: