bobbai00 commented on code in PR #4274:
URL: https://github.com/apache/texera/pull/4274#discussion_r3120421924
##########
bin/single-node/docker-compose.yml:
##########
@@ -75,6 +105,153 @@ services:
timeout: 5s
retries: 10
+ # Lakekeeper migration init container
+ # This runs once to migrate the database before the lakekeeper server starts
+ lakekeeper-migrate:
+ image: vakamo/lakekeeper:v0.11.0
+ container_name: texera-lakekeeper-migrate
+ depends_on:
+ postgres:
+ condition: service_healthy
+ env_file:
+ - .env
+ restart: "no"
+ entrypoint: ["/home/nonroot/lakekeeper"]
+ command: ["migrate"]
+
+ # Lakekeeper is the Iceberg REST catalog service
+ lakekeeper:
+ image: vakamo/lakekeeper:v0.11.0
+ container_name: texera-lakekeeper
+ restart: always
+ depends_on:
+ postgres:
+ condition: service_healthy
+ minio:
+ condition: service_started
+ lakekeeper-migrate:
+ condition: service_completed_successfully
+ env_file:
+ - .env
+ entrypoint: ["/home/nonroot/lakekeeper"]
+ command: ["serve"]
+ healthcheck:
+ test: ["CMD", "/home/nonroot/lakekeeper", "healthcheck"]
+ interval: 10s
+ timeout: 5s
+ retries: 10
+ start_period: 10s
+
+ # One-shot init container that creates the Lakekeeper default project and
+ # the Iceberg warehouse pointing at the MinIO bucket prepared by minio-init.
+ lakekeeper-init:
+ image: alpine:3.19
+ container_name: texera-lakekeeper-init
+ depends_on:
+ lakekeeper:
+ condition: service_healthy
+ minio-init:
+ condition: service_completed_successfully
+ env_file:
+ - .env
+ restart: "no"
+ entrypoint: [ "/bin/sh", "-c" ]
+ command:
+ - |
+ set -e
+
+ # curl is used to call Lakekeeper's management API over the internal
Review Comment:
remove these comments
##########
bin/single-node/docker-compose.yml:
##########
@@ -75,6 +105,153 @@ services:
timeout: 5s
retries: 10
+ # Lakekeeper migration init container
+ # This runs once to migrate the database before the lakekeeper server starts
+ lakekeeper-migrate:
+ image: vakamo/lakekeeper:v0.11.0
+ container_name: texera-lakekeeper-migrate
+ depends_on:
+ postgres:
+ condition: service_healthy
+ env_file:
+ - .env
+ restart: "no"
+ entrypoint: ["/home/nonroot/lakekeeper"]
+ command: ["migrate"]
+
+ # Lakekeeper is the Iceberg REST catalog service
+ lakekeeper:
+ image: vakamo/lakekeeper:v0.11.0
+ container_name: texera-lakekeeper
+ restart: always
+ depends_on:
+ postgres:
+ condition: service_healthy
+ minio:
+ condition: service_started
+ lakekeeper-migrate:
+ condition: service_completed_successfully
+ env_file:
+ - .env
+ entrypoint: ["/home/nonroot/lakekeeper"]
+ command: ["serve"]
+ healthcheck:
+ test: ["CMD", "/home/nonroot/lakekeeper", "healthcheck"]
+ interval: 10s
+ timeout: 5s
+ retries: 10
+ start_period: 10s
+
+ # One-shot init container that creates the Lakekeeper default project and
+ # the Iceberg warehouse pointing at the MinIO bucket prepared by minio-init.
+ lakekeeper-init:
+ image: alpine:3.19
+ container_name: texera-lakekeeper-init
+ depends_on:
+ lakekeeper:
+ condition: service_healthy
+ minio-init:
+ condition: service_completed_successfully
+ env_file:
+ - .env
+ restart: "no"
+ entrypoint: [ "/bin/sh", "-c" ]
+ command:
+ - |
+ set -e
+
+ # curl is used to call Lakekeeper's management API over the internal
+ # Docker network; jq parses list responses so we can detect existing
+ # resources before deciding to create them (keeps init idempotent).
+ echo "Installing curl (to call Lakekeeper's management API) and jq (to
parse its list responses)..."
+ apk add --no-cache curl jq
+
+ check_status() {
+ if [ "$$1" -ge 200 ] && [ "$$1" -lt 300 ]; then
+ echo "Created $$2 successfully (HTTP $$1)."
+ elif [ "$$1" -eq 409 ]; then
+ echo "$$2 already exists (HTTP 409). Treating as success."
Review Comment:
We should not 409 as success anymore, right?
##########
bin/single-node/docker-compose.yml:
##########
@@ -75,6 +105,153 @@ services:
timeout: 5s
retries: 10
+ # Lakekeeper migration init container
+ # This runs once to migrate the database before the lakekeeper server starts
Review Comment:
What does `migrate` mean?
##########
bin/single-node/docker-compose.yml:
##########
@@ -26,9 +26,37 @@ services:
- "${MINIO_PORT:-9000}:9000"
env_file:
- .env
+ environment:
+ - MINIO_ROOT_USER=${STORAGE_S3_AUTH_USERNAME}
+ - MINIO_ROOT_PASSWORD=${STORAGE_S3_AUTH_PASSWORD}
volumes:
- minio_data:/data
command: server --console-address ":9001" /data
+ healthcheck:
+ test: ["CMD", "curl", "-sf", "http://localhost:9000/minio/health/live";]
+ interval: 5s
+ timeout: 3s
+ retries: 10
+
+ # One-shot init container that creates the Iceberg warehouse bucket on first
+ # startup. MinIO's server image has no built-in "create bucket on boot" flag,
+ # so this sidecar uses the official `mc` client to do it idempotently.
Review Comment:
can you remove the sentences like "so this sidecar...idempotently". Comments
should describe what this corresponding piece will do, and what consequence it
will bring.
I will write the comment as: "This job will create a bucket used for
workflow result storage, which is managed by the Lakekeeper service"
##########
bin/single-node/docker-compose.yml:
##########
@@ -75,6 +105,153 @@ services:
timeout: 5s
retries: 10
+ # Lakekeeper migration init container
+ # This runs once to migrate the database before the lakekeeper server starts
+ lakekeeper-migrate:
+ image: vakamo/lakekeeper:v0.11.0
+ container_name: texera-lakekeeper-migrate
+ depends_on:
+ postgres:
+ condition: service_healthy
+ env_file:
+ - .env
+ restart: "no"
+ entrypoint: ["/home/nonroot/lakekeeper"]
+ command: ["migrate"]
+
+ # Lakekeeper is the Iceberg REST catalog service
+ lakekeeper:
+ image: vakamo/lakekeeper:v0.11.0
+ container_name: texera-lakekeeper
+ restart: always
+ depends_on:
+ postgres:
+ condition: service_healthy
+ minio:
+ condition: service_started
+ lakekeeper-migrate:
+ condition: service_completed_successfully
+ env_file:
+ - .env
+ entrypoint: ["/home/nonroot/lakekeeper"]
+ command: ["serve"]
+ healthcheck:
+ test: ["CMD", "/home/nonroot/lakekeeper", "healthcheck"]
+ interval: 10s
+ timeout: 5s
+ retries: 10
+ start_period: 10s
+
+ # One-shot init container that creates the Lakekeeper default project and
+ # the Iceberg warehouse pointing at the MinIO bucket prepared by minio-init.
+ lakekeeper-init:
+ image: alpine:3.19
+ container_name: texera-lakekeeper-init
+ depends_on:
+ lakekeeper:
+ condition: service_healthy
+ minio-init:
+ condition: service_completed_successfully
+ env_file:
+ - .env
+ restart: "no"
+ entrypoint: [ "/bin/sh", "-c" ]
+ command:
+ - |
+ set -e
+
+ # curl is used to call Lakekeeper's management API over the internal
+ # Docker network; jq parses list responses so we can detect existing
+ # resources before deciding to create them (keeps init idempotent).
+ echo "Installing curl (to call Lakekeeper's management API) and jq (to
parse its list responses)..."
+ apk add --no-cache curl jq
+
+ check_status() {
+ if [ "$$1" -ge 200 ] && [ "$$1" -lt 300 ]; then
+ echo "Created $$2 successfully (HTTP $$1)."
+ elif [ "$$1" -eq 409 ]; then
+ echo "$$2 already exists (HTTP 409). Treating as success."
+ else
+ echo "Failed to create $$2. HTTP Code: $$1"
+ echo "ERROR RESPONSE:"
+ if [ -f /tmp/response.txt ]; then cat /tmp/response.txt; fi
+ echo ""
+ exit 1
+ fi
+ }
+
+ # Lakekeeper organizes warehouses under "projects" (its top-level
tenant
Review Comment:
This comment is very clear 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
