The GitHub Actions job "Required Checks" on texera.git/main has succeeded. Run started by GitHub user github-merge-queue[bot] (triggered by github-merge-queue[bot]).
Head commit for run: 34c0f47737081b3021481b471935951b40afa0e9 / Yicong Huang <[email protected]> feat(local-dev): auto-apply pending sql/updates before the sbt build (#6145) ### What changes were proposed in this PR? jOOQ codegen reads the live postgres at sbt-compile time, so pulling main after a PR adds a new `sql/updates/N.sql` breaks `bin/local-dev.sh up` with `not found: value <NEW_TABLE>` until the update is applied by hand (hit today with `27.sql` / `workflow_cover_image` from #5704). `infra_ensure_db_schema` only bootstrapped `sql/texera_ddl.sql` on a fresh DB; nothing reconciled an existing local DB, and the repo's official runner (`sql/docker-compose.yml`, liquibase) is manual with creds that don't match local-dev's. This PR teaches local-dev to reconcile the schema before any sbt build, on every path that builds (full-stack `up`, `auto`, single-service `up`): - `parse_changelog_changesets` parses `sql/changelog.xml` (BSD-awk compatible, skips the commented example changeset) into id/author/path triples. - `infra_apply_sql_updates` diffs those against liquibase's `public.databasechangelog` and psql-applies the missing updates in document order, with the same normalization the official liquibase compose uses: strip the `\c` meta-command (not every file has one — `23.sql` doesn't) and connect to `texera_db` directly; the files' own `SET search_path` targets the schema. Applications are recorded in `databasechangelog` (liquibase-created DDL, `MD5SUM` left `NULL` so a later real liquibase run adopts the rows and fills checksums instead of re-executing). - A fresh bootstrap seeds every changeSet as applied — `texera_ddl.sql` is kept in sync with `sql/updates/*`, so replaying them would be redundant. - A failing update aborts with a clear message BEFORE the sbt build instead of letting the compile die on missing generated tables. The all-current fast path is one psql query + one awk parse. ### Any related issues, documentation, discussions? Closes #6144. The `--help` header's docker-infra paragraph documents the behavior. Related context: the official liquibase runner in `sql/docker-compose.yml` remains the mechanism for non-local deployments (the `ddl-change` email workflow notifies admins); this PR only automates the local dev loop while staying compatible with its bookkeeping. ### How was this PR tested? - `bash bin/local-dev/tests/test_local_dev_sh.sh` — 50 passed, 0 failed. New cases: the reconcile function exists and is wired into `infra_ensure_db_schema` on both paths (apply on existing DB, seed on fresh bootstrap), tracks via `databasechangelog` with `ON_ERROR_STOP`, the changelog parser handles the real `sql/changelog.xml` (5 changesets, commented example skipped), and every referenced update file exists on disk. - Live test against a real local dev DB that had `27.sql` applied manually and no `databasechangelog` table: first run applied changeSets 23–27 idempotently (and surfaced that `23.sql`'s `notebook` tables had in fact never been applied to this DB — now healed) and wrote the 5 tracking rows with correct authors; second run took the fast path ("schema current"). Verified `notebook` exists in the `texera_db` schema afterwards. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Fable 5) Co-authored-by: Claude Fable 5 <[email protected]> Report URL: https://github.com/apache/texera/actions/runs/28755438006 With regards, GitHub Actions via GitBox
