kacpermuda commented on code in PR #39520:
URL: https://github.com/apache/airflow/pull/39520#discussion_r1595241671


##########
airflow/providers/openlineage/utils/utils.py:
##########
@@ -268,6 +269,91 @@ def get_airflow_run_facet(
     }
 
 
+def get_dag_structure_job_facet(dag_run: DagRun):
+    dag = dag_run.dag
+    task_groups = {
+        tg_id: {
+            "parent_group": tg.parent_group.group_id,
+        }
+        for tg_id, tg in dag.task_group_dict.items()
+    }
+    tasks = {
+        task.task_id: {
+            "operator": task._task_module + "." + task._task_type,
+            "task_group": task.task_group.group_id if task.task_group else 
None,
+            "config_selective_enable": is_selective_lineage_enabled(task),
+            "config_disabled_for_operators": 
is_operator_disabled(task._task_module + "." + task._task_type),
+            # `produces_ol_events` will tell the consumer whether they should 
expect an event from given task,
+            # no matter the conditions of disablement - it's more futureproof 
than checking everything
+            # separately (selective_enable, disabled_for_operators and 
possibly more in the future)
+            "produces_ol_events": not is_operator_disabled(task._task_module + 
"." + task._task_type)
+            or is_selective_lineage_enabled(task),
+            # not sure if state is good to have in Job type facet, maybe we 
should create another facet like
+            # AirflowDagRunStateRunFacet that's below?
+            "state": dag_run.get_task_instance(task.task_id).current_state()

Review Comment:
   @mobuchowski @JDarDagran
    I'm not sure if we should add the task state to this Job facet - we should 
then attach it to both start and complete events, so we would duplicate the 
information. Maybe we should create another facet (f.e. 
AirflowDagRunStateRunFacet like below) and only attach it to dag complete run 
facets?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to