Anyone who has programmed geospatial software has eventually come to a conclusion about data formats: there is only one truly de facto standard for geospatial data, the shape file, and the shape file sucks.
Almost since they invented it, Esri has been trying to come up with replacements.
The “personal geodatabase” used the Microsoft Access MDB format as a storage layer, and stuffed geospatial information into that. Sadly, the format inherited all the limitations of MDB, which were substantial: file size, bloat, occasional corruption, and of course Windows-only platform dependencies.
The “file geodatabase” (FGDB) got around the MDB limitations by using a storage engine Esri wrote themselves. Unfortunately, the format was complex enough that they could never fully specify it and release a document on the format (and never seemed to want to). As a result, official support has only been through a proprietary binary API.
The FGDB format is very close to a shape file replacement.
Since it shipped with ArcGIS 10 in 2010, the FGDB format has become popular in the Esri ecosystem, and it’s not uncommon to find FGDB files on government open data sites, or to receive them from GIS practitioners when requesting data.
CARTO has supported the shape file format since day 1, but we only recently added support for FGDB. We were able to support FGDB because the GDAL library we use in our import process has an open source read-only FGDB driver. Using the “open FGDB” driver allows us to use a stock build of GDAL without incorporating the proprietary Esri API libraries.
The file geodatabase format is a collection of files inside a directory named with a
.gdb extension. In order to transfer that structure around, FGDB files are first zipped up. So, any FGDB data you receive will be a zip file that unzips to a
FGDB data are loaded to CARTO just like any other format.
.zipfile or drag’n’drop it in.
.zipfile directly into the datasets dashboard.
After loading, you will have one new dataset in your account for each layer in the FGDB, named using a
For example, the Elections.zip file from Clark County, Nevada, includes 11 layers, as we can see by looking at the
ogrinfo output for the file.
INFO: Open of `Election.gdb' using driver `OpenFileGDB' successful. 1: senate_p (Multi Polygon) 2: school_p (Multi Polygon) 3: regent_p (Multi Polygon) 4: precinct_p (Multi Polygon) 5: ward_p (Multi Polygon) 6: congress_p (Multi Polygon) 7: pollpnts_x (Point) 8: educat_p (Multi Polygon) 9: township_p (Multi Polygon) 10: commiss_p (Multi Polygon) 11: assembly_p (Multi Polygon)
After upload, the file has been convered to 11 datasets with the standard naming pattern.
If the FGDB format is so much better than shape files, why doesn’t the story end there?
Because FGDB still has a couple major problems:
A couple years after FGDB was released, the Open Geospatial Consortium (OGC) took on the task of defining a “shape file replacement” format, that learned all the lessons of shape files, personal geodatabases, and file geodatabases.
The result is the GeoPackage (GPKG) format, which has become widely used in the open source world, and increasingly throughout the geospatial software ecosystem.
Loading GeoPackage into Carto now works exactly the same as FGDB: use the “New Dataset” page, or just drag the file into the dataset dashboard. All the layers will be imported, using the
You can also now use GeoPackage as an export format! Click the export button and select the GPKG format, and you’ll get a single-layer GeoPackage with your table inside, ready for sharing with the world.
All this works because of the wonderful multi-format tools in the GDAL library, which we use as part of our import process. You can exercise the power of GDAL yourself to directly solve your CARTO ETL problems using the
ogrinfo tools in GDAL, check it out!
One of my favorite parts about working at CARTO is getting to test out the new features that our product team is constantly shipping. A month or so ago, it was our Kepler c...Core Tech
Our primary goal at CARTO is to be the world’s leading Location Intelligence platform, empowering our clients with the best data and the best spatial analysis. We frequentl...Core Tech
Please fill out the below form and we'll be in touch real soon.