Yicong-Huang opened a new issue, #5681: URL: https://github.com/apache/texera/issues/5681
### Task Summary Today the two `state-materialization` Python e2e tests run on a separate macOS-only CI job (`pyamber-state-materialization-mac`, [`.github/workflows/build.yml:742`](https://github.com/apache/texera/blob/main/.github/workflows/build.yml#L742)) that swaps in a sqlite-backed `SqlCatalog` so it can run without postgres / minio / lakekeeper infra. That diverges from prod, where every iceberg write goes through either a postgres-backed JdbcCatalog or a REST catalog (Lakekeeper) on top of MinIO/S3. ### Why it's a problem 1. `SqlCatalog` (sqlite) doesn't exercise the real prod catalog code path — a regression in the postgres / REST plumbing is invisible here. 2. The job is a duplicated diagnostic leg; the regression it was meant to catch (cross-region state materialization on macOS) is itself a test-environment artifact, not a prod concern (prod runs on Linux). ### Proposal | Before | After | | --- | --- | | `pyamber-state-materialization-mac` macOS job | (deleted) | | `SqlCatalog` (sqlite) override in fixture | postgres-backed `JdbcCatalog` via `StorageConfig.initialize`, same pattern as `test_iceberg_document.py:45` | | `pytest -sv test_state_materialization_e2e.py ...` direct | `@pytest.mark.integration` + picked up by `amber-integration` job's `pytest -m integration` step | `amber-integration` ubuntu job already provisions postgres + iceberg catalog DB + MinIO + Lakekeeper and runs `pytest -m integration` as its last step ([`build.yml:300`](https://github.com/apache/texera/blob/main/.github/workflows/build.yml#L300)). ### Task Type - [x] DevOps / Deployment / CI - [x] Testing / QA -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
