This is an automated email from the ASF dual-hosted git repository.
mengw15 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/texera.git
The following commit(s) were added to refs/heads/main by this push:
new 09828ea954 feat(k8s): use Lakekeeper as the Iceberg Catalog in the
Kubernetes deployment (#4275)
09828ea954 is described below
commit 09828ea9543102703104c330c0461b1eb92d8341
Author: Meng Wang <[email protected]>
AuthorDate: Wed May 13 17:07:22 2026 -0700
feat(k8s): use Lakekeeper as the Iceberg Catalog in the Kubernetes
deployment (#4275)
<!--
Thanks for sending a pull request (PR)! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
[Contributing to
Texera](https://github.com/apache/texera/blob/main/CONTRIBUTING.md)
2. Ensure you have added or run the appropriate tests for your PR
3. If the PR is work in progress, mark it a draft on GitHub.
4. Please write your PR title to summarize what this PR proposes, we
are following Conventional Commits style for PR titles as well.
5. Be sure to keep the PR description updated to reflect all changes.
-->
### What changes were proposed in this PR?
<!--
Please clarify what changes you are proposing. The purpose of this
section
is to outline the changes. Here are some tips for you:
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
3. If it is a refactoring, clarify what has been changed.
3. It would be helpful to include a before-and-after comparison using
screenshots or GIFs.
4. Please consider writing useful notes for better and faster reviews.
-->
This PR adds the **Kubernetes / Helm deployment** for the
Lakekeeper-based Result Service.
**Files changed:**
- `bin/k8s/Chart.yaml`: added Lakekeeper Helm chart dependency
(`lakekeeper` v0.9.0)
- `bin/k8s/values.yaml`: added Lakekeeper and `lakekeeperInit`
configuration (warehouse name, S3 bucket, region, default
project)
- `bin/k8s/templates/lakekeeper-init-job.yaml` (**new**): Kubernetes Job
that bootstraps Lakekeeper — creates MinIO bucket,
default project, and warehouse
- `bin/k8s/templates/external-names.yaml`: added Lakekeeper ExternalName
service for computing-unit-pool namespace access
- `bin/k8s/templates/workflow-computing-unit-manager-deployment.yaml`:
switched from Postgres catalog to REST catalog env
vars; added `wait-lakekeeper` init container
- `bin/k8s/templates/webserver-deployment.yaml`: added REST catalog env
vars
- `bin/k8s/templates/postgresql-init-script-config.yaml`: added
Lakekeeper database initialization
- `bin/k8s/files/texera_lakekeeper.sql` (**new**): symlink to
`sql/texera_lakekeeper.sql`
### Any related issues, documentation, discussions?
<!--
Please use this section to link other resources if not mentioned
already.
1. If this PR fixes an issue, please include `Fixes #1234`, `Resolves
#1234`
or `Closes #1234`. If it is only related, simply mention the issue
number.
2. If there is design documentation, please add the link.
3. If there is a discussion in the mailing list, please add the link.
-->
close #4610
### How was this PR tested?
<!--
If tests were added, say they were added here. Or simply mention that if
the PR
is tested with existing test cases. Make sure to include/update test
cases that
check the changes thoroughly including negative and positive cases if
possible.
If it was tested in a way different from regular unit tests, please
clarify how
you tested step by step, ideally copy and paste-able, so that other
reviewers can
test and check, and descendants can verify in the future. If tests were
not added,
please describe why they were not added and/or why it was difficult to
add.
-->
Manually tested
### Was this PR authored or co-authored using generative AI tooling?
<!--
If generative AI tooling has been used in the process of authoring this
PR,
please include the phrase: 'Generated-by: ' followed by the name of the
tool
and its version. If no, write 'No'.
Please refer to the [ASF Generative Tooling
Guidance](https://www.apache.org/legal/generative-tooling.html) for
details.
-->
co-authored with claude
---------
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
Co-authored-by: Jiadong Bai <[email protected]>
---
bin/k8s/Chart.yaml | 5 +
bin/k8s/files/texera_lakekeeper.sql | 1 +
bin/k8s/templates/external-names.yaml | 8 +-
bin/k8s/templates/lakekeeper-init-job.yaml | 137 +++++++++++++++++++++
.../templates/postgresql-init-script-config.yaml | 6 +
bin/k8s/templates/webserver-deployment.yaml | 11 ++
...workflow-computing-unit-manager-deployment.yaml | 38 ++++--
bin/k8s/values.yaml | 32 +++++
8 files changed, 227 insertions(+), 11 deletions(-)
diff --git a/bin/k8s/Chart.yaml b/bin/k8s/Chart.yaml
index cd94170595..548d9e531e 100644
--- a/bin/k8s/Chart.yaml
+++ b/bin/k8s/Chart.yaml
@@ -59,6 +59,11 @@ dependencies:
repository: oci://docker.io/envoyproxy
alias: envoy-gateway
+ - name: lakekeeper
+ version: 0.9.0
+ repository: https://lakekeeper.github.io/lakekeeper-charts/
+ condition: lakekeeper.enabled
+
- name: metrics-server
version: 3.12.2
repository: https://kubernetes-sigs.github.io/metrics-server/
diff --git a/bin/k8s/files/texera_lakekeeper.sql
b/bin/k8s/files/texera_lakekeeper.sql
new file mode 120000
index 0000000000..6ddbed9382
--- /dev/null
+++ b/bin/k8s/files/texera_lakekeeper.sql
@@ -0,0 +1 @@
+../../../sql/texera_lakekeeper.sql
\ No newline at end of file
diff --git a/bin/k8s/templates/external-names.yaml
b/bin/k8s/templates/external-names.yaml
index 69540067b8..c79ce40f33 100644
--- a/bin/k8s/templates/external-names.yaml
+++ b/bin/k8s/templates/external-names.yaml
@@ -81,4 +81,10 @@ to access services in the main namespace using the same
service names.
"externalName" (printf "%s-minio.%s.svc.cluster.local" .Release.Name
$namespace)
) | nindent 0 }}
-
+---
+{{/* Lakekeeper ExternalName */}}
+{{- include "external-name-service" (dict
+ "name" (printf "%s-lakekeeper" .Release.Name)
+ "namespace" $workflowComputingUnitPoolNamespace
+ "externalName" (printf "%s-lakekeeper.%s.svc.cluster.local" .Release.Name
$namespace)
+) | nindent 0 }}
diff --git a/bin/k8s/templates/lakekeeper-init-job.yaml
b/bin/k8s/templates/lakekeeper-init-job.yaml
new file mode 100644
index 0000000000..0a3540b605
--- /dev/null
+++ b/bin/k8s/templates/lakekeeper-init-job.yaml
@@ -0,0 +1,137 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+{{- if .Values.lakekeeperInit.enabled }}
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: {{ .Release.Name }}-lakekeeper-init
+ namespace: {{ .Release.Namespace }}
+spec:
+ backoffLimit: 3
+ template:
+ metadata:
+ name: {{ .Release.Name }}-lakekeeper-init
+ spec:
+ restartPolicy: Never
+ containers:
+ - name: lakekeeper-init
+ image: alpine:3.19
+ env:
+ - name: STORAGE_S3_ENDPOINT
+ value: http://{{ .Release.Name }}-minio:9000
+ - name: STORAGE_S3_AUTH_USERNAME
+ valueFrom:
+ secretKeyRef:
+ name: {{ .Release.Name }}-minio
+ key: root-user
+ - name: STORAGE_S3_AUTH_PASSWORD
+ valueFrom:
+ secretKeyRef:
+ name: {{ .Release.Name }}-minio
+ key: root-password
+ - name: STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET
+ value: {{ .Values.lakekeeperInit.warehouse.s3Bucket | quote }}
+ - name: STORAGE_ICEBERG_CATALOG_REST_REGION
+ value: {{ .Values.lakekeeperInit.warehouse.region | quote }}
+ - name: STORAGE_ICEBERG_CATALOG_REST_WAREHOUSE_NAME
+ value: {{ .Values.lakekeeperInit.warehouse.name | quote }}
+ - name: LAKEKEEPER_BASE_URI
+ value: http://{{ .Release.Name }}-lakekeeper:{{
.Values.lakekeeper.catalog.service.externalPort }}
+ - name: LAKEKEEPER_PROJECT_ID
+ value: {{ .Values.lakekeeperInit.defaultProject.id | quote }}
+ - name: LAKEKEEPER_PROJECT_NAME
+ value: {{ .Values.lakekeeperInit.defaultProject.name | quote }}
+ command:
+ - /bin/sh
+ - -c
+ - |
+ set -e
+
+ apk add --no-cache curl ca-certificates wget
+ wget -q https://dl.min.io/client/mc/release/linux-amd64/mc -O
/usr/local/bin/mc
+ chmod +x /usr/local/bin/mc
+
+ check_status() {
+ if [ "$1" -ge 200 ] && [ "$1" -lt 300 ]; then
+ echo "Created $2 successfully (HTTP $1)."
+ elif [ "$1" -eq 409 ]; then
+ echo "$2 already exists (HTTP 409). Treating as success."
+ else
+ echo "Failed to create $2. HTTP Code: $1"
+ echo "ERROR RESPONSE:"
+ if [ -f /tmp/response.txt ]; then cat /tmp/response.txt; fi
+ echo ""
+ exit 1
+ fi
+ }
+
+ echo "Waiting for Lakekeeper health endpoint..."
+ until curl -s -f "${LAKEKEEPER_BASE_URI}/health" > /dev/null
2>&1; do
+ sleep 3
+ done
+
+ echo "Step 1: Initializing MinIO bucket
'${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET}'..."
+ mc alias set minio "${STORAGE_S3_ENDPOINT}"
"${STORAGE_S3_AUTH_USERNAME}" "${STORAGE_S3_AUTH_PASSWORD}" || true
+ if mc ls minio/${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET} >
/dev/null 2>&1; then
+ echo "MinIO bucket '${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET}'
already exists."
+ else
+ mc mb minio/${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET}
+ echo "MinIO bucket '${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET}'
created successfully."
+ fi
+
+ echo "Step 2: Initializing default project..."
+
PROJECT_PAYLOAD="{\"project-id\":\"${LAKEKEEPER_PROJECT_ID}\",\"project-name\":\"${LAKEKEEPER_PROJECT_NAME}\"}"
+ PROJECT_CODE=$(curl -s -o /tmp/response.txt -w "%{http_code}" \
+ -X POST \
+ -H "Content-Type: application/json" \
+ -d "${PROJECT_PAYLOAD}" \
+ "${LAKEKEEPER_BASE_URI}/management/v1/project" || echo "000")
+ check_status "${PROJECT_CODE}" "Default Project"
+
+ echo "Step 3: Initializing warehouse
'${STORAGE_ICEBERG_CATALOG_REST_WAREHOUSE_NAME}'..."
+ CREATE_PAYLOAD=$(cat <<EOF
+ {
+ "warehouse-name":
"${STORAGE_ICEBERG_CATALOG_REST_WAREHOUSE_NAME}",
+ "project-id": "${LAKEKEEPER_PROJECT_ID}",
+ "storage-profile": {
+ "type": "s3",
+ "bucket": "${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET}",
+ "region": "${STORAGE_ICEBERG_CATALOG_REST_REGION}",
+ "endpoint": "${STORAGE_S3_ENDPOINT}",
+ "flavor": "s3-compat",
+ "path-style-access": true,
+ "sts-enabled": false
+ },
+ "storage-credential": {
+ "type": "s3",
+ "credential-type": "access-key",
+ "aws-access-key-id": "${STORAGE_S3_AUTH_USERNAME}",
+ "aws-secret-access-key": "${STORAGE_S3_AUTH_PASSWORD}"
+ }
+ }
+ EOF
+ )
+ WAREHOUSE_CODE=$(curl -s -o /tmp/response.txt -w "%{http_code}" \
+ -X POST \
+ -H "Content-Type: application/json" \
+ -d "${CREATE_PAYLOAD}" \
+ "${LAKEKEEPER_BASE_URI}/management/v1/warehouse" || echo "000")
+ check_status "${WAREHOUSE_CODE}" "Lakekeeper Warehouse"
+
+ echo "Lakekeeper initialization sequence completed successfully!"
+{{- end }}
diff --git a/bin/k8s/templates/postgresql-init-script-config.yaml
b/bin/k8s/templates/postgresql-init-script-config.yaml
index 104b9e1927..9b7e5488b5 100644
--- a/bin/k8s/templates/postgresql-init-script-config.yaml
+++ b/bin/k8s/templates/postgresql-init-script-config.yaml
@@ -41,6 +41,12 @@ data:
EOF
psql -U postgres -f /tmp/iceberg_postgres_catalog.sql
+ echo "Initializing Lakekeeper database..."
+ cat <<'EOF' > /tmp/texera_lakekeeper.sql
+{{ .Files.Get "files/texera_lakekeeper.sql" | indent 6 }}
+ EOF
+ psql -U postgres -f /tmp/texera_lakekeeper.sql
+
echo "Initializing Texera database..."
cat <<'EOF' > /tmp/texera_ddl.sql
{{ .Files.Get "files/texera_ddl.sql" | indent 6 }}
diff --git a/bin/k8s/templates/webserver-deployment.yaml
b/bin/k8s/templates/webserver-deployment.yaml
index 56642c5478..983c626994 100644
--- a/bin/k8s/templates/webserver-deployment.yaml
+++ b/bin/k8s/templates/webserver-deployment.yaml
@@ -60,6 +60,17 @@ spec:
secretKeyRef:
name: {{ .Release.Name }}-lakefs-secret
key: secret_key
+ # Workflow Result (Lakekeeper REST catalog)
+ - name: STORAGE_ICEBERG_CATALOG_TYPE
+ value: rest
+ - name: STORAGE_ICEBERG_CATALOG_REST_URI
+ value: http://{{ .Release.Name }}-lakekeeper:{{
.Values.lakekeeper.catalog.service.externalPort }}/catalog
+ - name: STORAGE_ICEBERG_CATALOG_REST_WAREHOUSE_NAME
+ value: {{ .Values.lakekeeperInit.warehouse.name | quote }}
+ - name: STORAGE_ICEBERG_CATALOG_REST_REGION
+ value: {{ .Values.lakekeeperInit.warehouse.region | quote }}
+ - name: STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET
+ value: {{ .Values.lakekeeperInit.warehouse.s3Bucket | quote }}
{{- range .Values.texeraEnvVars }}
- name: {{ .name }}
value: "{{ .value }}"
diff --git a/bin/k8s/templates/workflow-computing-unit-manager-deployment.yaml
b/bin/k8s/templates/workflow-computing-unit-manager-deployment.yaml
index 5241d9160a..7a0185cd46 100644
--- a/bin/k8s/templates/workflow-computing-unit-manager-deployment.yaml
+++ b/bin/k8s/templates/workflow-computing-unit-manager-deployment.yaml
@@ -33,6 +33,25 @@ spec:
app: {{ .Release.Name }}-{{ .Values.workflowComputingUnitManager.name
}}
spec:
serviceAccountName: {{
.Values.workflowComputingUnitManager.serviceAccountName }}
+ initContainers:
+ - name: wait-lakekeeper
+ image: curlimages/curl:latest
+ command:
+ - /bin/sh
+ - -c
+ - |
+ set -e
+ LAKEKEEPER_BASE_URI="http://{{ .Release.Name }}-lakekeeper:{{
.Values.lakekeeper.catalog.service.externalPort }}"
+ WAREHOUSE_NAME="{{ .Values.lakekeeperInit.warehouse.name }}"
+ echo "Waiting for Lakekeeper to become healthy..."
+ until curl -s -f "${LAKEKEEPER_BASE_URI}/health" > /dev/null
2>&1; do
+ sleep 1
+ done
+ echo "Waiting for warehouse '${WAREHOUSE_NAME}' to exist..."
+ until curl -s "${LAKEKEEPER_BASE_URI}/management/v1/warehouse" |
grep -q "\"name\"[[:space:]]*:[[:space:]]*\"${WAREHOUSE_NAME}\""; do
+ sleep 1
+ done
+ echo "Lakekeeper warehouse is ready."
containers:
- name: {{ .Values.workflowComputingUnitManager.name }}
image: {{ .Values.texera.imageRegistry }}/{{
.Values.workflowComputingUnitManager.imageName }}:{{ .Values.texera.imageTag }}
@@ -88,16 +107,15 @@ spec:
key: secret_key
# Workflow Result
- name: STORAGE_ICEBERG_CATALOG_TYPE
- value: postgres
- - name: STORAGE_ICEBERG_CATALOG_POSTGRES_URI_WITHOUT_SCHEME
- value: {{ .Release.Name }}-postgresql:5432/texera_iceberg_catalog
- - name: STORAGE_ICEBERG_CATALOG_POSTGRES_USERNAME
- value: postgres
- - name: STORAGE_ICEBERG_CATALOG_POSTGRES_PASSWORD
- valueFrom:
- secretKeyRef:
- name: {{ .Release.Name }}-postgresql
- key: postgres-password
+ value: rest
+ - name: STORAGE_ICEBERG_CATALOG_REST_URI
+ value: http://{{ .Release.Name }}-lakekeeper:{{
.Values.lakekeeper.catalog.service.externalPort }}/catalog
+ - name: STORAGE_ICEBERG_CATALOG_REST_WAREHOUSE_NAME
+ value: {{ .Values.lakekeeperInit.warehouse.name | quote }}
+ - name: STORAGE_ICEBERG_CATALOG_REST_REGION
+ value: {{ .Values.lakekeeperInit.warehouse.region | quote }}
+ - name: STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET
+ value: {{ .Values.lakekeeperInit.warehouse.s3Bucket | quote }}
{{- range .Values.texeraEnvVars }}
- name: {{ .name }}
value: "{{ .value }}"
diff --git a/bin/k8s/values.yaml b/bin/k8s/values.yaml
index c1ae399295..2d7c520ff7 100644
--- a/bin/k8s/values.yaml
+++ b/bin/k8s/values.yaml
@@ -113,6 +113,38 @@ lakefs:
access_key_id: texera_minio
secret_access_key: password
+lakekeeper:
+ enabled: true
+ postgresql:
+ enabled: false
+ internalOpenFGA: false
+ catalog:
+ replicas: 1
+ image:
+ repository: vakamo/lakekeeper
+ tag: v0.11.0
+ pullPolicy: IfNotPresent
+ service:
+ externalPort: 8181
+ externalDatabase:
+ type: postgres
+ host_read: texera-postgresql
+ host_write: texera-postgresql
+ port: 5432
+ database: texera_lakekeeper
+ user: postgres
+ password: root_password
+
+lakekeeperInit:
+ enabled: true
+ defaultProject:
+ id: "00000000-0000-0000-0000-000000000000"
+ name: default
+ warehouse:
+ name: texera
+ region: us-west-2
+ s3Bucket: texera-iceberg
+
# Part2: configurations of Texera-related micro services
texeraImages:
pullPolicy: Always
