bobbai00 opened a new issue, #6194:
URL: https://github.com/apache/texera/issues/6194
### What happened?
`bin/local-dev.sh up` decides whether the Postgres schema needs (re)loading
by probing for a **single hard-coded table, `feedback`**
(`bin/local-dev/main.sh`, `infra_ensure_db_schema`, ~L1346-1357):
```bash
# `feedback` is one of the newer tables; if it exists we assume the
# whole schema is current.
has_feedback=$(docker exec "$pg" psql -U texera -d texera_db -tAc \
"SELECT 1 FROM pg_tables WHERE schemaname='texera_db' AND
tablename='feedback'" ...)
if [[ "$has_feedback" == "1" ]]; then
tui_skip "postgres: schema already current"
return 0
fi
```
The probe assumes `feedback` is the newest table. Any table added *after*
`feedback` is invisible to this check. When the Postgres data volume was
created before such a table existed (a very common case for a long-lived local
dev volume), the volume already has `feedback`, so the script reports
**"postgres: schema already current"** and skips applying `sql/texera_ddl.sql`
— even though the schema is actually stale.
The downstream effect is nasty: the `dao` module runs jOOQ codegen against
this stale live DB (`common/dao/build.sbt`, wired into `Compile /
sourceGenerators`), so the generated `Tables`/`Keys` are missing the new
relation, and `sbt dist` then fails with a **misleading compile error** that
points at application Scala code, not at the schema. In my case the missing
table was `workflow_cover_image` (added in `sql/updates/27.sql`), and the build
failed with 11 errors like:
```
not found: value WORKFLOW_COVER_IMAGE
```
Expected: the schema-currency check should not silently skip DDL when the
schema is out of date. Since `sql/texera_ddl.sql` is already idempotent
(`CREATE TABLE IF NOT EXISTS`), the safest fix is to **always apply it** (the
comment in the code even acknowledges re-applying is safe), or to probe the
newest expected table / a schema-version marker instead of `feedback`.
### How to reproduce?
1. Have a `texera-local-dev` Postgres volume created before
`workflow_cover_image` was added (i.e. an older volume that already contains
`feedback`).
2. Check out a branch whose code references the newer table (references
`WORKFLOW_COVER_IMAGE`).
3. Run `bin/local-dev.sh up`.
4. Observe `postgres: schema already current` is printed (DDL skipped), then
`sbt: dist FAILED` with `not found: value WORKFLOW_COVER_IMAGE` — an error that
gives no hint that the real cause is a stale DB schema.
Workaround: manually apply the missing migration (e.g. `docker exec -i
texera-postgres psql -U texera -d texera_db < sql/updates/27.sql`) and re-run
`up`.
### Version/Branch
1.3.0-incubating-SNAPSHOT (main)
### Commit Hash (Optional)
be99b3661 (observed; the check lives in `bin/local-dev/main.sh`)
### Relevant log output
```shell
▸ Infra
✓ infra: 5 containers up
○ postgres: schema already current
▸ Build
→ sbt dist (log: /tmp/texera-local-dev/logs/sbt-dist.log) (no-TTY)
✗ sbt: dist FAILED
# sbt-dist.log:
[error] .../WorkflowResource.scala:729:15: not found: value
WORKFLOW_COVER_IMAGE
[error] .../WorkflowResource.scala:730:13: not found: value
WORKFLOW_COVER_IMAGE
...
[error] 11 errors found
[error] (WorkflowExecutionService / Compile / compileIncremental)
Compilation failed
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
