mobuchowski opened a new pull request, #70182: URL: https://github.com/apache/airflow/pull/70182
`EmrAddStepsOperator` can now pass OpenLineage parent job metadata to classic EMR Spark steps. The new `openlineage_inject_parent_job_info` option defaults to `openlineage.spark_inject_parent_job_info`, matching the other Spark-based Amazon operators. The injection reuses OpenLineage's Spark property helper and translates its output into tokenized `--conf key=value` arguments for `command-runner.jar` steps using `spark-submit` or `run-example`. It leaves non-Spark steps, custom Jar steps, and steps with existing parent settings unchanged, and does not mutate the operator's original step configuration. This lets OpenLineage events emitted by Spark link back to the originating Airflow task run when parent information injection is enabled. Validation: - `uv run --project providers/amazon pytest providers/amazon/tests/unit/amazon/aws/operators/test_emr_add_steps.py -xvs` — 15 passed - `prek run --from-ref main --stage pre-commit` — passed - `breeze ci selective-check --commit-ref 2ea76a5ef7` — selected the expected Amazon/provider CI coverage - The manual provider mypy hook and selected parallel provider suite could not start because the local Buildah CI-image build rejected its configured cache reference; both stopped before running checks. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Codex (GPT-5) Generated-by: Codex (GPT-5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- Drafted-by: Codex (GPT-5) (no human review before posting) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
