Deployment

Note

This tutorial is only tested on Debian Buster!

Firewall

Ports 80 & 443 need to be open, for installing the dependencies & running the server. An example for iptables to open the ports.:

$ iptables -A INPUT -p tcp --dport 443 -j ACCEPT
$ iptables -A OUTPUT -p tcp --dport 443 -j ACCEPT
$ iptables -A INPUT -p tcp --dport 80 -j ACCEPT
$ iptables -A OUTPUT -p tcp --dport 80 -j ACCEPT

Dependent Services

Postgres 12 with Postgis 3

To use the performance boost of the new Postgres Version, add the postgres repo from postgresql.org to your system.:

$ apt-get --no-install-recommends install \
    locales gnupg2 wget ca-certificates rpl pwgen software-properties-common gdal-bin iputils-ping
$ sh -c "echo \"deb http://apt.postgresql.org/pub/repos/apt/ buster-pgdg main\" > /etc/apt/sources.list.d/pgdg.list"
$ wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
$ apt-get update

Install Postgres 12 with Postgis 3.:

$ apt-get --no-install-recommends install postgresql-client-12 \
    postgresql-common postgresql-12 postgresql-12-postgis-3 \
    netcat postgresql-12-ogr-fdw postgresql-12-postgis-3-scripts \
    postgresql-12-cron postgresql-plpython3-12 postgresql-12-pgrouting

Postgres’s service is started & set to come up after each system reboot.:

$ systemctl status postgresql.service

While the installation, the user postgres was added to the system. With the user you can access the admin postgres user.

$ su - postgres

Change the postgres user postgres password (remember this password!):

$ psql -c "alter user postgres with password 'YourNewPassword'"

Now access the postgres prompt.:

$ psql

Enable Postgis & hstore extensions for postgres.:

$ CREATE EXTENSION postgis;
$ CREATE EXTENSION hstore;
$ CREATE EXTENSION postgis_topology;

Set the template postgres database to UTF-8 encoding.:

$ UPDATE pg_database SET datistemplate = FALSE WHERE datname = 'template1';
$ DROP DATABASE template1;
$ CREATE DATABASE template1 WITH TEMPLATE = template0 ENCODING = 'UTF8';
$ UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template1';
$ \c template1
$ VACUUM FREEZE;

Create the gis database with the user mapnik to access the gis database.:

$ CREATE DATABASE gis;
$ CREATE USER mapnik WITH ENCRYPTED PASSWORD 'MyStr0ngP@SS';
$ GRANT ALL PRIVILEGES ON DATABASE gis to mapnik;

Set the new mapnik database user as superuser.:

$ ALTER USER mapnik WITH SUPERUSER;

Logout from postgres prompt & user.:

$ \q
$ exit

Redis 5

Redis use as a caching server for the tiles & as a task broker for celery.

For installing redis server, use:

$ apt-get install --no-install-recommends redis-server

If you running redis on the same system as the web-service, then is redis ready to work :)

NGINX

Install NGINX and certbot for Let’s Encrypt.:

$ apt-get install --no-install-recommends nginx python3-acme \
    python3-certbot python3-mock python3-openssl python3-pkg-resources \
    python3-pyparsing python3-zope.interface python3-certbot-nginx

Create a NGINX config file for ohdm.:

$ nano /etc/nginx/sites-available/MapnikTileServer.conf

server {
    server_name a.ohdm.net b.ohdm.net c.ohdm.net;

    location = /favicon.ico { access_log off; log_not_found off; }
    location  /static/ {
    alias /home/mapnik/MapnikTileServer/staticfiles/;
    }

    location / {
        include proxy_params;
        proxy_read_timeout 360;
        proxy_pass http://unix:/home/mapnik/MapnikTileServer/MapnikTileServer.sock;
    }

}

server {
    server_name monitor.ohdm.net;

    location / {
        include proxy_params;
        proxy_pass http://127.0.0.1:5555;
    }
}

Note

Change the domains a.ohdm.net, b.ohdm.net, c.ohdm.net & monitor.ohdm.net in the NGINX config file to your domains!

Link the config file from /etc/nginx/sites-available/MapnikTileServer.conf to /etc/nginx/sites-enabled/MapnikTileServer.conf.:

$ ln -s /etc/nginx/sites-available/MapnikTileServer.conf /etc/nginx/sites-enabled

Test if the config was set up right & restart NGINX.:

$ nginx -t
$ systemctl restart nginx

Obtaining an SSL Certificate.:

$ certbot --nginx -d a.ohdm.net -d b.ohdm.net -d c.ohdm.net -d monitor.ohdm.net
2
2

Test if Let’s Encrypt was sucessfully set up.:

$ nginx -t
$ systemctl restart nginx

Test if certbot can auto renew the SSL certificate.:

$ certbot renew --dry-run

Install MapnikTileServer

System dependencies:

$ apt-get install --no-install-recommends wget unzip fontconfig gnupg

Node:

$ apt-get install nodejs npm
$ npm i -g npm@^6

Python:

$ apt-get install --no-install-recommends python3-pip python3-dev \
    python3-setuptools

Mapnik-utils for openstreetmap-carto:

$ apt-get install --no-install-recommends mapnik-utils

Dependencies for building Python packages:

$ apt-get install --no-install-recommends build-essential

Psycopg2 dependencies:

$ apt-get install --no-install-recommends libpq-dev

Translations dependencies:

$ apt-get install --no-install-recommends gettext

Fonts for mapnik:

$ apt-get install --no-install-recommends fonts-dejavu fonts-hanazono \
ttf-unifont \
fonts-noto fonts-noto-cjk fonts-noto-cjk-extra fonts-noto-color-emoji \
fonts-noto-hinted fonts-noto-mono \
fonts-noto-unhinted \
fonts-noto-extra fonts-noto-ui-core fonts-noto-ui-extra

Geodjango:

$ apt-get install --no-install-recommends binutils libproj-dev gdal-bin

Git:

$ apt-get install --no-install-recommends git

Mapnik:

$ apt-get install --no-install-recommends libmapnik-dev libmapnik3.0 mapnik-utils \
python3-mapnik

Supervisor:

$ apt-get install --no-install-recommends supervisor

Download & install more noto fonts for mapnik:

$ mkdir noto-fonts
$ cd noto-fonts
$ wget https://noto-website-2.storage.googleapis.com/pkgs/NotoSansBalinese-unhinted.zip
$ wget https://noto-website-2.storage.googleapis.com/pkgs/NotoSansSyriacEastern-unhinted.zip
$ wget https://noto-website-2.storage.googleapis.com/pkgs/NotoColorEmoji-unhinted.zip
$ wget https://noto-website-2.storage.googleapis.com/pkgs/NotoEmoji-unhinted.zip
$ unzip -o \*.zip
$ cp ./*.ttf /usr/share/fonts/truetype/noto/
$ fc-cache -fv
$ fc-list
$ cd ..
$ rm -r noto-fonts

Update NodeJS to the latest stable:

$ npm install -g n stable

Install CartoCSS with a version below 1.:

$ npm install -g carto@0

Set environment vars for running the MapnikTileServer.:

$ nano /etc/environment

Fill the /etc/environment file with the following values.

# Django DJANGO_READ_DOT_ENV_FILE=True DJANGO_SETTINGS_MODULE=config.settings.production

Create a Mapnik user, for running the MapnikTileServer.:

$ adduser mapnik

Log into mapnik user and go to the home folder.:

$ su - mapnik
$ cd

Download openstreetmap-carto:

$ git clone https://github.com/linuxluigi/openstreetmap-carto.git

Go to the new openstreetmap-carto folder, download the shape files & create the default mapnik style XML:

$ cd openstreetmap-carto
$ ./scripts/get-shapefiles.py
$ carto project.mml > style.xml

Next go back to the mapnik home folder.:

$ cd

Download MapnikTileServer.:

$ git clone https://github.com/OpenHistoricalDataMap/MapnikTileServer.git
$ cd MapnikTileServer

Install / update the python packages as root user.:

$ exit
$ pip3 install -r /home/mapnik/MapnikTileServer/requirements/system.txt
$ pip3 install -r /home/mapnik/MapnikTileServer/requirements/base.txt
$ pip3 install -r /home/mapnik/MapnikTileServer/requirements/production.txt

Note

When install an update of MapnikTileServer, also update the python packages!

Go back to the mapnik user & back to the MapnikTileServer folder.:

$ su mapnik
$ cd /home/mapnik/MapnikTileServer

Create a .env file for the MapnikTileServer settings. Go to Settings to see all possibles options. Below is a minimal configuration:

# General
# ------------------------------------------------------------------------------
DJANGO_SECRET_KEY=!!!ChangeMeToSomeRandomValue!!!!!
DJANGO_ALLOWED_HOSTS=a.ohdm.net,b.ohdm.net,c.ohdm.net

# Redis
# ------------------------------------------------------------------------------
REDIS_URL=redis://localhost:6379/0
CELERY_BROKER_URL=redis://localhost:6379/0

# ohdm
# ------------------------------------------------------------------------------
CARTO_STYLE_PATH=/home/mapnik/openstreetmap-carto

# Default PostgreSQL
# ------------------------------------------------------------------------------
DATABASE_URL="postgres://mapnik:MyStr0ngP@SS@localhost:5432/gis"
POSTGRES_HOST=localhost
POSTGRES_PORT=5432
POSTGRES_DB=gis
POSTGRES_USER=mapnik
POSTGRES_PASSWORD=MyStr0ngP
PGCONNECT_TIMEOUT=60

# OHDM PostgreSQL
# ------------------------------------------------------------------------------
OHDM_SCHEMA=ohdm

Tests the settings, migrate the database, set indexes & collect static files:

$ python3 manage.py migrate
$ python3 manage.py set_indexes
$ python3 manage.py collectstatic

Add a superuser for the admin panel.:

$ python3 manage.py createsuperuser

Add supervisor script to auto start django, celery & flower at system start. For creating the scripts, go back to the root user.:

$ exit

Open the text editor to create the supervisor file.:

$ nano /etc/supervisor/conf.d/mapnik_tile_server.conf

Fill the supervisor file with the values below, but don’t forget to change --basic_auth="ChangeMeFlowerUser:ChangeMeFlowerPassword" with your flower user & password.:

[supervisord]
environment=DJANGO_READ_DOT_ENV_FILE=True,DJANGO_SETTINGS_MODULE=config.settings.production,CELERY_BROKER_URL=redis://localhost:6379/0

[program:MapnikTileServer_celery_worker]
command=celery -A config.celery_app worker -l INFO
user=mapnik
directory=/home/mapnik/MapnikTileServer
autostart=true
autorestart=true
priority=10
stderr_logfile=/var/log/MapnikTileServer_celery_worker.err.log

[program:MapnikTileServer_celery_beat]
command=celery -A config.celery_app beat -l INFO
user=mapnik
directory=/home/mapnik/MapnikTileServer
autostart=true
autorestart=true
priority=10
stderr_logfile=/var/log/MapnikTileServer_celery_beat.err.log

[program:MapnikTileServer_celery_flower]
command=celery flower --app=config.celery_app --broker="redis://localhost:6379/0" --basic_auth="ChangeMeFlowerUser:ChangeMeFlowerPassword"
user=mapnik
directory=/home/mapnik/MapnikTileServer
autostart=true
autorestart=true
priority=10
stderr_logfile=/var/log/MapnikTileServer_celery_flower.err.log

[program:MapnikTileServer_django]
command=/usr/local/bin/gunicorn config.wsgi --workers 2 --bind unix:/home/mapnik/MapnikTileServer/MapnikTileServer.sock -t 360
user=mapnik
directory=/home/mapnik/MapnikTileServer
autostart=true
autorestart=true
priority=10
stderr_logfile=/var/log/MapnikTileServer_django.err.log

To enable the supervisor script.:

$ supervisorctl reread
$ supervisorctl update
$ supervisorctl start all
$ supervisorctl status

Use commands

For using django commands from Commands, log into the mapnik user & go to the /home/mapnik/MapnikTileServer.:

$ su mapnik
$ cd /home/mapnik/MapnikTileServer

The commands in Commands are written for the docker usage, to use them without docker, just use the command after the django keyword. For example, to use set_indexes, in the docs the command is written down as docker-compose -f local.yml run --rm django python manage.py set_indexes and to use it without docker, just use python3 manage.py set_indexes.

Download updates

Stop all services first.:

$ supervisorctl stop all

Log into the mapnik user and go to the openstreetmap-carto folder.:

$ su mapnik
$ cd /home/mapnik/openstreetmap-carto

Get the latest version with git pull.:

$ git pull

Download the latest shape files & create the default mapnik style XML.:

$ ./scripts/get-shapefiles.py
$ carto project.mml > style.xml

Go to the MapnikTileServer.:

$ cd /home/mapnik/MapnikTileServer

Download the latest code from github, for the MapnikTileServer.:

$ git pull

Update the database & static files.:

$ python3 manage.py migrate
$ python3 manage.py set_indexes
$ python3 manage.py collectstatic

Log out from the mapnik user & start the web services again.:

$ exit
$ supervisorctl start all

Remove all packages were automatically installed and are no longer required.:

$ apt autoremove

On monitor.ohdm.net you should now able to log into the flower celery monitor. The user information is in /etc/supervisor/conf.d/mapnik_tile_server.conf on the line --basic_auth="ChangeMeFlowerUser:ChangeMeFlowerPassword".