Tutorial 10: Importing custom geographic data into the CleverMaps
CleverMaps provides many ready-to-use geographic dimensions such as administrative units or H3 grids. But sometimes you may need to use your own geographic data - e.g. road segments, buildings or sales areas. This tutorial describes how to import such a custom geographic data into the CleverMaps.
CleverMaps uses vector tiles technology to render geographic data on a map. Currently we use Mapbox as a backend service for hosting and serving vector tiles but other technologies can be also used (e.g. MapTiler). Some further information about vector tiles can be found here https://docs.mapbox.com/vector-tiles/reference/.
This tutorial assumes that you have your own Mapbox account ready. Mapbox has very generous free pricing plan and it should be enough for many use cases. More information about pricing is here https://www.mapbox.com/pricing . Free account can be created here https://account.mapbox.com/auth/signup/ .
To transform your geographic data (SHP, GeoJSON, CSV, …) into the CleverMaps you typically need to:
create vector tileset at Mapbox
create datasets in CleverMaps:
vector tileset dataset- it only references vector tileset from Mapbox over URL
geometryLine dataset- it holds all columns which should be visible in CleverMaps (e.g. road class, building type, sales area name, ...) plus some system columns (primary key, bounding box coordinates)
More information about CleverMaps datasets types can be found here Datasets
1. Creating vector tileset at Mapbox
You have two options for creating a vector tiles from your geodata files (SHP, GeoJSON, CSV…) - using Mapbox Studio or CleverMaps tool MapboxUtils.
Using Mapbox Studio
graphical user interface
20 uploads per month
maximum 300 MB per upload
cannot set zoom levels and other parameters
you pay for tiles generation and storage
detailed instructions can be found here https://docs.mapbox.com/studio-manual/reference/tilesets/
Using CleverMaps tool MapboxUtils
command line interface
Docker required https://docs.docker.com/get-docker/
suitable for one time usage or for automatization (e.g. daily updates)
no upload limits
vector tiles fine tuning (e.g. zoom levels)
vector tiles are generated locally by tool so you pay only for storage
detailed instructions can be found here MapboxUtils (Docker)
Things to keep in mind:
Data has to be in WGS84 coordinate system.
Mapbox pricing depends on vector tiles resolution level - you pay more for hosting and processing of vector tiles with 1 meter resolution then for 10 meter resolution. The resolution level is derived from maximum zoom level (this can be set while creating vector tiles with MapboxUtils):
resolution 10 m = maximum zoom between 6 and 10
resolution 1 m = maximum between 11 and 13
resolution 30 cm = maximum zoom between 14 and 16
Vector tiles supports only simple geometries like Line and Polygon.
Columns names has to be in lowercase.
2. Creating datasets in CleverMaps
Now we need to create some tables and metadata JSON files in CleverMaps. As described earlier we need to create two datasets at CleverMaps:
vector tile dataset (
datawarehouse dataset (
dwh dataset) with subtype
To create these two datasets please follow the instructions here Datasets. Some more things to keep in mind:
dwh datasetmust contain primary key column with the same name as the Mapbox vector tileset (as the
nuts0_codein the screenshot below)
dwh datasetmust contain the bounding box coordinates columns
x_min, x_max, y_min, y_maxfor each geometry feature (road, building, sales area, ..) - you can calculate it using some standard GIS tool or you can use CleverMaps tool
dwh datasetcan contain any other columns from your original source geodata (except geometry column)
the name of the
vt datasetneeds to be same as the layer name in Mapbox (as the
eu_nuts0_vt-84hkkqin the screenshot below)
vt datasetshould only contain primary key field and geometry, other columns should be stored only in
the data type of primary key of
dwh datasethas to be same as in vector tileset Mapbox
create default public token at Mapbox Studio and place it into the key
urlTemplateinstead of placeholder
vt datasetJSON file