kz930 opened a new pull request, #7935: URL: https://github.com/apache/texera/pull/7935
### What changes were proposed in this PR? `lakekeeper-init` refreshes the existing warehouse's stored S3 endpoint on every `up`, so that the endpoint follows the host LAN IP across networks. That update payload also carried the configured key prefix, which #7740 added to both the create and the update path. Lakekeeper treats the key prefix as immutable and rejects the entire request when a payload would change it, so a warehouse created before the prefix existed answers with HTTP 400 and `Field key_prefix cannot be updated to prevent loss of data.` The endpoint travels in that same payload, so it stays stale forever. The consequence is not obvious from the symptom. Once the host's LAN IP changes, every execution fails before any operator runs: `IcebergUtil.createTable` waits about thirty seconds on an S3 connect timeout to the old address and surfaces `RESTException: Unable to process:` with nothing pointing at the endpoint or the warehouse. A fresh checkout is unaffected, because the warehouse is created with the same prefix the refresh later sends. What is affected is every environment whose warehouse predates #7740. The update path now reads the prefix back from the warehouse listing it has already fetched and sends that. jq prints it as JSON, a quoted string or `null`, so it goes into the payload unquoted and leaves the field at whatever the warehouse holds. The create path still takes the prefix from the environment, which is where a new warehouse's prefix belongs. ### Any related issues, documentation, discussions? Closes #7921. Follows #7740, which added the field, and #6197, which added the refresh. The issue notes two further defects that this PR deliberately leaves alone, since each has its own fix: `bin/local-dev.sh` continues and reports the stack as up when the init exits non-zero, and a failed first execution leaves the workflow service cached in a state where every later run throws a `NullPointerException` on `executionRuntimeService`, which the UI shows as being stuck on "Submitting". ### How was this PR tested? The init is shell embedded in `docker-compose.yml`, so there is no unit test to add. It was verified by hand against a local stack whose warehouse is exactly the affected case, `key-prefix: null` with a stale endpoint, extracting the init script from both the current `main` and this branch and running each in an `alpine:3.19` container on the compose network. | Check | Result | | --- | --- | | `main`'s init script against the pre-#7740 warehouse | `Failed to refresh Lakekeeper Warehouse storage. HTTP Code: 400`, `Field key_prefix cannot be updated to prevent loss of data.` | | this branch's init script, same warehouse | `Refreshed Lakekeeper Warehouse storage endpoint successfully (HTTP 200)` | | endpoint genuinely changes, refreshing to `http://texera-minio:9000` and back to the LAN address | 200 both times, stored endpoint follows each value, `key-prefix` stays `null` | | the fresh-checkout case, a throwaway warehouse created with `key-prefix: "shared"` in its own bucket | refresh returns 200, endpoint updated, prefix preserved | | `docker compose config` | parses, and `STORED_KEY_PREFIX` expands inside the heredoc as intended | Read the stored profile back with `curl -s http://localhost:8181/management/v1/warehouse` to confirm the endpoint and prefix after each run. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
