dennismdejong opened a new pull request, #13673:
URL: https://github.com/apache/gravitino/pull/13673
### What changes were proposed in this pull request?
Add a schema-migration Kubernetes `Job` to the Gravitino Helm chart
(`dev/charts/gravitino`).
For external (pre-existing) MySQL/PostgreSQL databases configured via
`entity.jdbcUrl`, the chart now renders a `batch/v1` Job annotated with Helm
lifecycle hooks so the migration runs before any application pods are created
or updated:
- `"helm.sh/hook": pre-install,pre-upgrade`
- `"helm.sh/hook-weight": "1"`
- `"helm.sh/hook-delete-policy": before-hook-creation`
The Job stages the schema SQL scripts via a `sqlfile` init container
(identical to the Deployment's), then a `migrate` container (MySQL or
PostgreSQL client image) waits for the database, parses the JDBC URL, and
applies the newest applicable `upgrade-*.sql` (best-effort) followed by the
newest applicable `schema-*.sql` — the same selection and ordering semantics
as the existing init containers.
Bundled in-chart MySQL/PostgreSQL keep using the existing init containers
unchanged (a `pre-install` hook Job cannot reach a bundled subchart database,
because Helm creates subchart resources only after `pre-install` hooks
finish).
Changes:
- `dev/charts/gravitino/values.yaml`: add `schemaMigration` values block
(`enabled`, `backoffLimit`, `activeDeadlineSeconds`,
`ttlSecondsAfterFinished`).
- `dev/charts/gravitino/templates/schema-migration-job.yaml`: new migration
Job template (new file).
- `dev/charts/gravitino/tests/schema-migration-job_test.yaml`: new
helm-unittest suite (new file).
- `docs/chart.md`: document automatic schema migration for existing databases
and mark the manual DB-init step as optional.
### Why are the changes needed?
Currently there is no built-in mechanism to handle database schema migrations
safely when deploying or upgrading Gravitino via Helm. Running migrations
inside application containers (init containers) can race when multiple pods
start simultaneously during a rolling update. A hook Job guarantees the
migration runs exactly once per install/upgrade, and the rolling update of
application pods only begins after the Job completes successfully.
Fix: #11099
### Does this PR introduce _any_ user-facing change?
- New `schemaMigration.*` Helm values for the Gravitino chart.
- External MySQL/PostgreSQL databases are now migrated automatically by the
chart before install/upgrade (previously a manual step). Bundled databases
are unaffected.
- `docs/chart.md` updated accordingly.
### How was this patch tested?
- `helm lint dev/charts/gravitino` — 0 failures.
- `helm unittest --with-subchart=false dev/charts/gravitino` — 45/45 tests
pass
(new `schema-migration-job_test.yaml` suite included).
- Manual render checks: Job renders for external MySQL and PostgreSQL with
the
expected hook annotations; renders nothing for bundled MySQL/PostgreSQL,
the
default H2 configuration, and `schemaMigration.enabled=false`; bundled DB
deployments still render the existing `init-mysql`/`init-postgresql` init
containers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
