The GitHub Actions job "Required Checks" on texera.git/main has failed. Run started by GitHub user github-merge-queue[bot] (triggered by github-merge-queue[bot]).
Head commit for run: 319859571e30d123a50081a918815d9b65b4300f / Kary Zheng <[email protected]> feat(local-dev): reuse the stored storage profile when refreshing the Lakekeeper warehouse (#7935) ### What changes were proposed in this PR? `lakekeeper-init` refreshes the existing warehouse's stored S3 endpoint on every `up`, so that the endpoint follows the host LAN IP across networks. To carry that endpoint it rebuilt the whole storage profile from the environment, and #7740 added the configured key prefix to both the create and the update path. Lakekeeper treats the key prefix as immutable and rejects the entire request when a payload would change it, so a warehouse created before the prefix existed answers with HTTP 400 and `Field key_prefix cannot be updated to prevent loss of data.` The endpoint travels in that same payload, so it stays stale forever. The region is immutable in the same way, so a changed region takes the endpoint down with it too. The consequence is not obvious from the symptom. Once the host's LAN IP changes, every execution fails before any operator runs: `IcebergUtil.createTable` waits about thirty seconds on an S3 connect timeout to the old address and surfaces `RESTException: Unable to process:` with nothing pointing at the endpoint or the warehouse. A fresh checkout is unaffected, because the warehouse is created with the same profile the refresh later sends. What is affected is every environment whose warehouse predates #7740. The update path now sends back the profile the warehouse already holds, read from the listing this init has already fetched, with only the endpoint overridden. No field can drift out of the stored record, whichever ones Lakekeeper decides to protect. The create path still builds the profile from the environment, which is where a new warehouse's values belong. ### Any related issues, documentation, discussions? Closes #7921. Follows #7740, which added the field, and #6197, which added the refresh. The issue notes two further defects that this PR deliberately leaves alone, since each has its own fix: `bin/local-dev.sh` continues and reports the stack as up when the init exits non-zero, and a failed first execution leaves the workflow service cached in a state where every later run throws a `NullPointerException` on `executionRuntimeService`, which the UI shows as being stuck on "Submitting". ### How was this PR tested? The init is shell embedded in `docker-compose.yml`, so there is no unit test to add. It was verified by hand against a local stack whose warehouse is exactly the affected case, `key-prefix: null` with a stale endpoint, extracting the init script from both the current `main` and this branch and running each in an `alpine:3.19` container on the compose network. | Check | Result | | --- | --- | | `main`'s init script against the pre-#7740 warehouse | `Field key_prefix cannot be updated to prevent loss of data.` (HTTP 400) | | this branch's init script, same warehouse | HTTP 200, endpoint updated, key prefix and region left as stored | | `main`'s script with a changed region | `Field region cannot be updated to prevent loss of data.` (HTTP 400) | | this branch's script, same changed region | HTTP 200, stored region stays `us-west-2` | | endpoint genuinely changes, refreshing to `http://texera-minio:9000` and back to the LAN address | 200 both times, stored endpoint follows each value, `key-prefix` stays `null` | | the fresh-checkout case, a throwaway warehouse created by this branch with `key-prefix: "shared"` in its own bucket | create 201, refresh 200, endpoint updated, prefix preserved | | `docker compose config` | parses, and `REFRESHED_PROFILE` expands inside the heredoc as intended | Read the stored profile back with `curl -s http://localhost:8181/management/v1/warehouse` to confirm the endpoint and prefix after each run. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5) Co-authored-by: Claude Opus 5 (1M context) <[email protected]> Report URL: https://github.com/apache/texera/actions/runs/32803125500 With regards, GitHub Actions via GitBox
