mobuchowski commented on code in PR #57809: URL: https://github.com/apache/airflow/pull/57809#discussion_r2494294345
########## providers/openlineage/docs/guides/user.rst: ########## @@ -478,6 +478,51 @@ You can enable this automation by setting ``spark_inject_transport_info`` option AIRFLOW__OPENLINEAGE__SPARK_INJECT_TRANSPORT_INFO=true +Passing parent information to Airflow DAG +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To enable full OpenLineage lineage tracking across dependent DAGs, you can pass parent and root job information +through the DAG's ``dag_run.conf``. When a DAG run configuration includes an ``_openlineage`` section with valid metadata, +this information is automatically parsed and converted into DAG's **ParentRunFacet**, from which the root information +is also propagated to all task runs. If no DAG run configuration is provided, the DAG run itself is considered the +lineage root for its tasks. + +The ``_openlineage`` dict in conf can contain the following keys: Review Comment: >Do you mean we should always require all three root info? Yes, rather all six. > otherwise, the root is empty Root should only be empty when the library does not understand the root info - uses ParentRunFacet 1.0.1 or earlier. > As I understand you propose using parent as root in that case, so that it's never empty? Yes, that's the expected semantics. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
