xBis7 commented on code in PR #68213:
URL: https://github.com/apache/airflow/pull/68213#discussion_r3372190542
##########
task-sdk/src/airflow/sdk/bases/resumablejobmixin.py:
##########
@@ -101,29 +107,56 @@ def execute_resumable(self, context: Context) -> Any:
Closing this window would require atomic "submit + persist", which is
not possible across
an external system boundary.
"""
- task_store = context.get("task_store")
+ operator_tag = {"operator": type(self).__name__}
+
+ with tracer.start_as_current_span("resumable_job.resume_decision") as
span:
+ span.set_attribute("operator", type(self).__name__)
+ span.set_attribute("resumable.external_id_key",
self.external_id_key)
+
+ task_store = context.get("task_store")
+
+ if task_store is None:
+ span.set_attribute("resumable.decision", "no_task_store")
+ else:
+ external_id = task_store.get(self.external_id_key)
+ if external_id:
+ incr("resumable_job.reconnect_attempt", tags=operator_tag)
Review Comment:
These are new metrics, right? If yes, then you will need to add them here as
well
https://github.com/astronomer/airflow/blob/aip-103-debugability-observability-traces-metrics/shared/observability/src/airflow_shared/observability/metrics/metrics_template.yaml
The template is used internally by stats and for building the docs. There
will be a bunch of errors if they don't exist there.
##########
task-sdk/src/airflow/sdk/bases/resumablejobmixin.py:
##########
@@ -101,29 +107,56 @@ def execute_resumable(self, context: Context) -> Any:
Closing this window would require atomic "submit + persist", which is
not possible across
an external system boundary.
"""
- task_store = context.get("task_store")
+ operator_tag = {"operator": type(self).__name__}
+
+ with tracer.start_as_current_span("resumable_job.resume_decision") as
span:
Review Comment:
I see in your screenshot that it appears as a child of the task span and
then again as a child of the dag run span.
Is there a way to differentiate between them? E.g. add the caller to the
name or a distinct attribute?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
