dougal-software/etc/db/README.md

# Database configuration

## Database template (`database-template.sql`)

Used on installation to create the Dougal database.

Created with:

```bash
pg_dump -U postgres --create --schema-only --exclude-schema='survey_*' dougal >database-template.sql
```

## Schema template (`schema-template.sql`)

Used by [`bin/create-survey.sh`](../../bin/create-survey.sh) during project creation to add and populate a new schema.

Created with:

```bash
SCHEMA_NAME=survey_X EPSG_CODE=XXXXX $DOUGAL_ROOT/sbin/dump_schema.sh
```

## To create a new Dougal database

Ensure that the following packages are installed:

* `postgresql*-postgis-utils`
* `postgresql*-postgis`
* `postgresql*-contrib` # For B-trees

```bash
psql -U postgres <./database-template.sql
psql -U postgres <./database-version.sql
```

---

# Upgrading PostgreSQL

The following is based on https://en.opensuse.org/SDB:PostgreSQL#Upgrading_major_PostgreSQL_version

```bash
# The following bash code should be checked and executed
# line for line whenever you do an upgrade. The example
# shows the upgrade process from an original installation
# of version 12 up to version 14.

# install the new server as well as the required postgresql-contrib packages:
zypper in postgresql14-server postgresql14-contrib postgresql12-contrib

# If not yet done, copy the configuration create a new PostgreSQL configuration directory...
mkdir /etc/postgresql
# and copy the original file to this global directory
cd /srv/pgsql/data
for i in  pg_hba.conf pg_ident.conf postgresql.conf postgresql.auto.conf ; do cp -a $i /etc/postgresql/$i ; done

# Now create a new data-directory and initialize it for usage with the new server
install -d -m 0700 -o postgres -g postgres /srv/pgsql/data14
cd /srv/pgsql/data14
sudo -u postgres /usr/lib/postgresql14/bin/initdb .

# replace the newly generated files by a symlink to the global files.
# After doing so, you may check the difference of the created backup files and
# the files from the former installation
for i in  pg_hba.conf pg_ident.conf postgresql.conf postgresql.auto.conf ; do old $i ; ln -s /etc/postgresql/$i .; done

# Copy over special thesaurus files if some exists.
#cp -a /usr/share/postgresql12/tsearch_data/my_thesaurus_german.ths /usr/share/postgresql14/tsearch_data/

# Now it's time to disable the service...
systemctl stop postgresql.service

# And to start the migration. Please ensure, the directories fit to your upgrade path
sudo -u postgres /usr/lib/postgresql14/bin/pg_upgrade --link \
  --old-bindir="/usr/lib/postgresql12/bin" \
  --new-bindir="/usr/lib/postgresql14/bin" \
  --old-datadir="/srv/pgsql/data/" \
  --new-datadir="/srv/pgsql/data14/"

# NOTE: If getting the following error:
# lc_collate values for database "postgres" do not match:  old "en_US.UTF-8", new "C"
# then:
# cd ..
# rm -rf /srv/pgsql/data14
# install -d -m 0700 -o postgres -g postgres /srv/pgsql/data14
# cd /srv/pgsql/data14
# sudo -u postgres /usr/lib/postgresql14/bin/initdb --locale=en_US.UTF-8 .
#
# and repeat the migration command

# After successfully migrating the data...
cd ..
# if not already symlinked move the old data to a versioned directory matching
# your old installation...
mv data data12
# and set a symlink to the new data directory
ln -sf data14/ data

# Now start the new service
systemctl start postgresql.service

# If everything has been sucessful, you should uninstall old packages...
#zypper rm -u postgresql12 postgresql13
# and remove old data directories
#rm -rf /srv/pgsql/data_OLD_POSTGRES_VERSION_NUMBER

# For good measure:
sudo -u postgres /usr/lib/postgresql14/bin/vacuumdb --all --analyze-in-stages

# If update_extensions.sql exists, apply it.
```

# Restoring from backup

## Whole database

Ensure that nothing is connected to the database.

```bash
psql -U postgres --dbname postgres <<EOF
-- Database: dougal

DROP DATABASE IF EXISTS dougal;

CREATE DATABASE dougal
    WITH
    OWNER = postgres
    ENCODING = 'UTF8'
    LC_COLLATE = 'en_GB.UTF-8'
    LC_CTYPE = 'en_GB.UTF-8'
    TABLESPACE = pg_default
    CONNECTION LIMIT = -1;

ALTER DATABASE dougal
    SET search_path TO "$user", public, topology;

EOF

# Adjust --jobs according to host machine
pg_restore -U postgres --dbname dougal --clean --if-exists --jobs 32 /path/to/backup

```