SpatialUtils (Docker)

This tool provides basic spatial functions for processing a CSV files with WKT geometries. You can calculate interesection, buffer, bounding box, area, dissolve, difference and so on. It is build on top of Spatialite and thanks to it has very low memory footprint and it is even ready for large datasets (millions of rows). This tool could be handy if you dont have PostGIS or similar tools available.

SpatialUtils is CSV based (input and output). There are some requirements for geometries:

• Geometry has to be stored as WKT string (https://en.wikipedia.org/wiki/Well-known_text_representation_of_geometry). 

• Coordinate system of geometry has to be 4326 (WGS84).

If your data is in other format than CSV with WKT (e.g. Shapefile, GeoJSON), you can convert it using for example ogr2ogr utility (https://gdal.org/drivers/vector/csv.html).

Spatial functions

You can use one of the following spatial function in each execution of SpatialUtils:

• area

  • computes area in square meters for each polygon

  • value for function parameter in configuration is area

• bounding box

  • computes x_min, x_max, y_min and y_max for each geometry

  • value for function parameter in configuration is bbox 

• buffer

  • computes buffer around geometry (point, line, polygon) with defined radius in meters

  • value for function parameter in configuration is buffer

• centroid

  • computes a centroid point for each geometry

  • value for function parameter in configuration is centroid

• union

  • computes an union of geometries based on same values of attribute key 

  • value for function parameter in configuration is union

• intersection

  • computes an intersection between geometries from two input tables 

  • value for function parameter in configuration is intersection

• isvalid

  • validates WKT geometries and writes result into a two new columns - isvalid and isvalid_reason

  • value for function parameter in configuration is isvalid

You can check for example https://postgis.net/workshops/postgis-intro/geometry_returning.html for further information about spatial functions.

Installation

SpatialUtils is distributed as a Docker image so you need to have Docker platform installed. You can get Docker here. 

SpatialUtils is publicly available on Docker hub: https://hub.docker.com/r/clevermaps/spatial-utils

Use following command to get the image:

docker pull clevermaps/spatial-utils:latest

Running SpatialUtils

There are few steps you need to do before running SpatialUtils:

• Create an empty directory on your local drive (e.g. /home/user/clevermaps/spatial-utils).

• Copy your CSV file you would like to process into the folder.

• Create a configuration file for SpatialUtils and save it as a JSON file into the folder. Please check the examples below in Configuration to get the idea how the configuration file should look.

Use following command to run SpatialUtils:

docker run -t -v /home/user/clevermaps/spatial-utils/:/workdir clevermaps/spatial-utils:latest --config /workdir/area_config.json

! You need to change /home/user/clevermaps/spatial-utils to your directory path !

Configuration

Here you can find examples of configuration files, each one is for different spatial function. Basicaly you need to define:

• spatial function to be calculated (area, buffer, bbox, centroid, dissolve, intersection, isvalid)

• parameters for input CSV file (or files in case of intersection)

• parameters for output CSV 

{
  "function": "area",
  "in_csv": {
    "filename": "/workdir/data/zones.csv",
    "delimiter": ",",
    "wkt_column_name": "geom",
    "out_columns": [
      "zone_id"
    ]
  },
  "out_csv": {
      "filename": "/workdir/area_test.csv",
      "delimiter": ","
  },
  "temp_directory": "/workdir"
}

{
    "function": "bbox",
    "in_csv": {
        "filename": "/workdir/data/zones.csv",
        "delimiter": ",",
        "wkt_column_name": "wkt"
    },
    "out_csv": {
        "filename": "/workdir/bbox_test.csv",
        "delimiter": ","
    },
    "temp_directory": "/workdir"
}

{
  "function": "buffer",
  "in_csv": {
    "filename": "/workdir/data/restaurants.csv",
    "delimiter": ",",
    "wkt_column_name": "geom",
    "out_columns": [
      "id"
    ]
  },
  "buffer": {
    "srs_wkid": 3857,
    "dist": 100
  },
  "out_csv": {
      "filename": "/workdir/buffer_test.csv",
      "delimiter": ","
  },
  "temp_directory": "/workdir"
}

{
  "function": "centroid",
  "in_csv": {
    "filename": "/workdir/data/zones.csv",
    "delimiter": ",",
    "wkt_column_name": "geom",
    "out_columns": [
      "zone_id"
    ]
  },
  "out_csv": {
      "filename": "/workdir/centroid_test.csv",
      "delimiter": ","
  },
  "temp_directory": "/workdir"
}

{
  "function": "union",
  "in_csv": {
    "filename": "/workdir/data/zones.csv",
    "delimiter": ",",
    "wkt_column_name": "geom",
    "group_column_name": "restaurant_id",
    "out_columns": [
      "restaurant_id"
    ]
  },
  "simplify": true,
  "out_csv": {
      "filename": "/workdir/union_test.csv",
      "delimiter": ","
  },
  "temp_directory": "/workdir"
}

{
  "function": "intersection",
  "in_left_csv": {
    "filename": "/workdir/orders.csv",
    "delimiter": ",",
    "wkt_column": "geom",
    "join_column": "restaurant_id",
    "out_columns": [
      "order_id",
      "restaurant_id"
    ]
  },
  "in_right_csv": {
    "filename": "/workdir/zones.csv",
    "delimiter": ",",
    "wkt_column": "geom",
    "join_column": "restaurant_id",
    "out_columns": [
      "zone_id",
      "restaurant_id"
    ]
  },
  "out_csv": {
    "filename": "/workdir/intersection_test.csv",
    "delimiter": ",",
    "intersection_geometry": false
  },
  "temp_directory": "/workdir"
}

Additional info:

• Parameter join_column is optional. If specified than left and right table is firstly joined using this column and then an intersection is computed.

{
  "function": "isvalid",
  "in_csv": {
    "filename": "/workdir/data/zones.csv",
    "delimiter": ",",
    "wkt_column_name": "geom",
    "out_columns": [
      "zone_id"
    ]
  },
  "out_csv": {
      "filename": "/workdir/isvalid_test.csv",
      "delimiter": ","
  },
  "temp_directory": "/workdir"
}