Dev-iL opened a new issue, #69233: URL: https://github.com/apache/airflow/issues/69233
`breeze` refuses to start the OpenLineage integration unless the backend is Postgres **12, 13, or 14**: https://github.com/apache/airflow/blob/main/dev/breeze/src/airflow_breeze/utils/docker_command_utils.py#L1032-L1038 ```python if "openlineage" in shell_params.integration or "all" in shell_params.integration: if shell_params.backend != "postgres" or shell_params.postgres_version not in ["12", "13", "14"]: console_print( "\n[error]Only PostgreSQL 12, 13, and 14 are supported " "as a backend with OpenLineage integration via Breeze[/]\n" ) sys.exit(1) ``` This hardcoded list has drifted badly out of sync with the versions breeze actually supports: - `"12"` is not in `ALLOWED_POSTGRES_VERSIONS` at all (`["13", "14", "15", "16", "17", "18"]`), so that branch of the gate is unreachable. - `"13"` is allowed only for compatibility with older release branches — it is not in `CURRENT_POSTGRES_VERSIONS` (`["14", "15", "16", "17", "18"]`). - In practice the OpenLineage integration on `main` therefore only works with `--postgres-version 14`, which reaches EOL on **2026-11-13** and is next in line to be dropped from the matrix. When that happens, `breeze ... --integration openlineage` (and `--integration all`, which the gate also intercepts) becomes unusable with every supported backend version. ### Background The gate dates back to the introduction of the integration via Marquez (#33742, refactored in #36131). The OpenLineage compose file (`scripts/ci/docker-compose/integration-openlineage.yml`) does not run its own database — it piggybacks Marquez onto the **same Postgres container as the Airflow backend** (via `marquez/init-marquez-db.sh` in `docker-entrypoint-initdb.d`), so the version restriction encoded whichever Postgres versions the pinned Marquez (currently `marquezproject/marquez:0.49.0`) supported at the time. It was never revisited as the breeze Postgres matrix rolled forward from 12–14 to 14–18. ### Suggested resolution 1. Check which Postgres versions the current (or latest) Marquez release supports, bumping the pinned `marquezproject/marquez` / `marquez-web` images if needed. 2. Either widen the gate to the supported subset of `ALLOWED_POSTGRES_VERSIONS`, or — better — derive it from a single constant next to the other version lists in `global_constants.py` so it cannot silently drift again. If current Marquez supports all matrix versions, drop the version part of the gate entirely (keeping the `backend != "postgres"` check). 3. Alternatively, decouple Marquez from the backend database by giving it its own Postgres service in `integration-openlineage.yml`, which removes the coupling between the integration and the backend version matrix altogether. ### Affected surface - `breeze shell / testing ... --integration openlineage` and `--integration all` - `dev/breeze/src/airflow_breeze/utils/docker_command_utils.py` (the gate) - `scripts/ci/docker-compose/integration-openlineage.yml` (Marquez pin, shared Postgres) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
