This is an automated email from the ASF dual-hosted git repository. skrawcz pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/hamilton.git
commit c32f2e604ec5d7fb0192e256f0776d2249c28076 Author: Stefan Krawczyk <[email protected]> AuthorDate: Tue Feb 24 18:15:25 2026 -0800 Fix PostgreSQL 18 Docker volume mount point PostgreSQL 18 Docker images changed their data directory structure. The mount point must be /var/lib/postgresql (not /var/lib/postgresql/data) to support the new pg_ctlcluster-compatible format. Changes: - docker-compose.yml: Mount at /var/lib/postgresql - docker-compose-prod.yml: Mount at /var/lib/postgresql - UPGRADE.md: Document the volume structure change and add troubleshooting - migrate_postgres_simple.sh: Add note about new volume structure Why this matters: - PostgreSQL 18+ stores data in version-specific subdirectories (18/data) - Old volumes mounted at /var/lib/postgresql/data are incompatible - The new structure enables easier pg_upgrade operations - Prevents startup error: "in 18+, these Docker images are configured..." Migration scripts handle this automatically by creating fresh volumes. Users upgrading must follow the migration process - simply changing the image tag will fail. Fixes the error encountered when starting PostgreSQL 18 containers with old volume mount points. --- ui/UPGRADE.md | 21 +++++++++++++++++++++ ui/docker-compose-prod.yml | 2 +- ui/docker-compose.yml | 2 +- ui/migrate_postgres_simple.sh | 6 +++++- 4 files changed, 28 insertions(+), 3 deletions(-) diff --git a/ui/UPGRADE.md b/ui/UPGRADE.md index 99c4e389..d9e4a7cc 100644 --- a/ui/UPGRADE.md +++ b/ui/UPGRADE.md @@ -29,6 +29,10 @@ This guide covers backward compatibility concerns and migration steps when upgra Hamilton UI Docker setup now uses **PostgreSQL 18** instead of PostgreSQL 12 to support Django 6.0.2+, which requires PostgreSQL 13 or later. +**Important:** PostgreSQL 18 Docker images use a new data directory structure. The volume mount point has changed from `/var/lib/postgresql/data` to `/var/lib/postgresql`. This allows PostgreSQL to store data in version-specific subdirectories (e.g., `/var/lib/postgresql/18/data`) and enables easier future upgrades using `pg_upgrade --link`. + +**Why this matters:** You cannot simply change the image tag from `postgres:12` to `postgres:18` without migrating your data. The old volume will be incompatible with the new structure. + ### Do I Need to Migrate? **You need to migrate if ALL of the following are true:** @@ -142,6 +146,23 @@ docker compose exec backend ls /code/blobs cp -r old_blobs_directory new_blobs_directory ``` +**Issue:** PostgreSQL 18 fails with "Error: in 18+, these Docker images are configured to store database data in a format which is compatible with pg_ctlcluster" + +This error means you have old PostgreSQL data (from version 12 or 16) that's incompatible with PostgreSQL 18's new directory structure. The PostgreSQL 18 Docker image expects data in `/var/lib/postgresql` (not `/var/lib/postgresql/data`). + +**Solution:** You must migrate your data using the migration scripts provided. Do not try to reuse the old volume: + +```bash +# Remove the old volume (after backing up your data!) +docker compose down +docker volume rm ui_postgres_data + +# Follow the migration guide above to export and import your data +./migrate_postgres_simple.sh +``` + +If you're upgrading from PostgreSQL 16 to 18 and encounter this error, the volume mount point has changed. You still need to follow the migration process even though both versions are modern. + #### Option 3: Use pg_upgrade (Advanced) For minimal downtime migrations, you can use PostgreSQL's `pg_upgrade` tool. This is more complex and generally not recommended for Hamilton UI due to its development-focused nature. diff --git a/ui/docker-compose-prod.yml b/ui/docker-compose-prod.yml index 57ff7101..7a04f7db 100644 --- a/ui/docker-compose-prod.yml +++ b/ui/docker-compose-prod.yml @@ -5,7 +5,7 @@ services: db: image: postgres:18 volumes: - - postgres_data:/var/lib/postgresql/data + - postgres_data:/var/lib/postgresql environment: - POSTGRES_DB=postgres - POSTGRES_USER=postgres diff --git a/ui/docker-compose.yml b/ui/docker-compose.yml index a7e15ceb..e8b5e40a 100644 --- a/ui/docker-compose.yml +++ b/ui/docker-compose.yml @@ -4,7 +4,7 @@ services: db: image: postgres:18 volumes: - - postgres_data:/var/lib/postgresql/data + - postgres_data:/var/lib/postgresql environment: - POSTGRES_DB=postgres - POSTGRES_USER=postgres diff --git a/ui/migrate_postgres_simple.sh b/ui/migrate_postgres_simple.sh index 201ddcf3..7611cd03 100755 --- a/ui/migrate_postgres_simple.sh +++ b/ui/migrate_postgres_simple.sh @@ -29,9 +29,13 @@ echo "" echo "This script will:" echo " 1. Export your Hamilton data using Django" echo " 2. Stop containers and remove PostgreSQL 12 volume" -echo " 3. Start PostgreSQL 18 containers" +echo " 3. Start PostgreSQL 18 containers (with new volume structure)" echo " 4. Restore your Hamilton data" echo "" +echo "Note: PostgreSQL 18 uses a new data directory structure." +echo "The old volume at /var/lib/postgresql/data is incompatible." +echo "This script will create a new volume at /var/lib/postgresql." +echo "" # Check if docker compose or docker-compose is available if command -v docker-compose &> /dev/null; then
