Yicong-Huang opened a new pull request, #7076:
URL: https://github.com/apache/texera/pull/7076
### What changes were proposed in this PR?
`bin/local-dev.sh up` against a fresh docker volume never reached the sbt
build: it died on the last `sql/updates` changeSet.
Postgres applies `sql/texera_ddl.sql` itself — compose mounts `sql/` into
`/docker-entrypoint-initdb.d` — and that DDL is kept in sync with
`sql/updates/*`, so the changeSets local-dev replays immediately afterwards
re-create objects that are already there. 23–27 are incidentally idempotent
and pass; `28.sql`'s `dataset_owner_uid_name_key` is not, and
`texera_ddl.sql:291` (`UNIQUE (owner_uid, name)`) already created it under
exactly that auto-generated name.
`infra_ensure_db_schema` picks seed-vs-replay by probing for the `feedback`
table. On a fresh volume the entrypoint has just created it, so the replay
path is taken — and the `seed` branch written for this very case (record
every
changeSet as applied without executing it) is unreachable, because the
entrypoint always wins the race.
```
Before: fresh volume -> entrypoint applies full DDL -> replay 23-28 -> 28
fails -> no build
After: fresh volume -> entrypoint applies full DDL -> 28 recorded as
applied -> build runs
```
The fix is in the replay loop rather than the probe: a psql failure whose
every `ERROR:` line is `already exists` means the changeSet's effect is
already in the schema, so record it and carry on. That also covers the next
`sql/updates/N.sql` that isn't accidentally idempotent, instead of fixing
just `28.sql`.
`_sql_errors_all_already_exist` is deliberately narrow — a `duplicate key` is
a data conflict, not an applied schema change, and a failure with **no**
`ERROR:` line at all is never assumed harmless — so an incomplete schema
still
stops the build instead of reaching jOOQ codegen with tables that aren't
there.
While in the same lines: psql's stderr is kept instead of redirected to
`/dev/null`. It holds the one line that explains the abort, and the old code
discarded it and then told the operator to re-run the file by hand to find
out why.
### Any related issues, documentation, discussions?
Closes #7064
### How was this PR tested?
New unit coverage for the detector, both directions (13 cases), plus two
structural guards on the wiring, in the existing `infra`-job suite:
```
$ bash bin/local-dev/tests/test_local_dev_sh.sh
...
✓ already-applied detector: relation already exists (the #7064 failure)
✓ already-applied detector: several already-exists errors, nothing else
✓ already-applied detector: already-exists around harmless NOTICE/ROLLBACK
chatter
✓ already-applied detector: non-ASCII identifier already exists
✓ already-applied detector: syntax error
✓ already-applied detector: missing relation
✓ already-applied detector: one already-exists mixed with one real error
✓ already-applied detector: duplicate key is a data conflict, not an
applied change
✓ already-applied detector: empty stderr
✓ already-applied detector: no ERROR line at all
✓ already-applied detector: missing stderr file
✓ already-applied detector: no argument
✓ infra_apply_sql_updates consults the already-applied detector
✓ infra_apply_sql_updates keeps psql stderr for diagnosis
64 passed, 0 failed
```
```
$ python -m pytest bin/local-dev/tests/ -q
1 failed, 42 passed
```
That one failure is `test_is_dirty_after_seed_then_edit`, and it is
**pre-existing and unrelated** — it reproduces identically on a pristine
`upstream/main` checkout on this machine, and this PR touches neither
`tui.py` nor that test. Cause, for the record: `_newest_mtime_after` compares
source mtime to the stamp with a strict `>`, and on a filesystem whose
timestamp granularity is coarser than the two consecutive writes the test
performs, both land on the same `st_mtime` and the fast filter reports clean.
Filed as #7075 and fixed there rather than folded in here.
End-to-end on the real stack (Ubuntu 24.04, docker 29.1.3), reproducing the
bug and then confirming the fix:
```sh
bin/local-dev.sh down
docker volume rm texera-local-dev_postgres_data
bin/local-dev.sh up
```
Before:
```
→ postgres: applying sql/updates/28.sql (changeSet 28)
✗ postgres: sql/updates/28.sql failed -- inspect with: docker exec -i
texera-postgres psql -U texera -d texera_db < sql/updates/28.sql
```
After:
```
→ postgres: applying sql/updates/27.sql (changeSet 27)
→ postgres: applying sql/updates/28.sql (changeSet 28)
○ postgres: sql/updates/28.sql already in schema (recording changeSet 28)
✓ postgres: 6 sql/update(s) applied
...
✓ 14 of 14 services healthy
```
The changeSet is recorded, so it is not retried on the next run:
```
$ docker exec texera-postgres psql -U texera -d texera_db -tAc \
"SELECT id||':'||exectype FROM public.databasechangelog ORDER BY
orderexecuted"
23:EXECUTED
24:EXECUTED
25:EXECUTED
26:EXECUTED
27:EXECUTED
28:EXECUTED
```
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
