kacpermuda commented on code in PR #57809: URL: https://github.com/apache/airflow/pull/57809#discussion_r2494273472
########## providers/openlineage/docs/guides/user.rst: ########## @@ -478,6 +478,51 @@ You can enable this automation by setting ``spark_inject_transport_info`` option AIRFLOW__OPENLINEAGE__SPARK_INJECT_TRANSPORT_INFO=true +Passing parent information to Airflow DAG +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To enable full OpenLineage lineage tracking across dependent DAGs, you can pass parent and root job information +through the DAG's ``dag_run.conf``. When a DAG run configuration includes an ``_openlineage`` section with valid metadata, Review Comment: Wanted to underline, that this is something rather private, used underneath by the OL provider. Also wanted to reduce the risk of collision, in case users are already specifying it in their conf. Both `openlineage` and `_openlineage` are unlikely to be used and I have checks to prevent overwriting, but the probability of `_openlineage` being present is lower IMHO. Any particular reason why `openlineage` might be better? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
