xBis7 commented on code in PR #49180:
URL: https://github.com/apache/airflow/pull/49180#discussion_r2063940305
##########
airflow-core/src/airflow/jobs/scheduler_job_runner.py:
##########
@@ -1058,27 +1060,25 @@ def _update_dag_run_state_for_paused_dags(self,
session: Session = NEW_SESSION)
@provide_session
def _end_active_spans(self, session: Session = NEW_SESSION):
# No need to do a commit for every update. The annotation will commit
all of them once at the end.
- for key, span in self.active_spans.get_all().items():
- from airflow.models.taskinstance import TaskInstanceKey
-
- if isinstance(key, TaskInstanceKey): # ti span.
- # Can't compare the key directly because the try_number or the
map_index might not be the same.
- ti: TaskInstance = session.scalars(
- select(TaskInstance).where(
- TaskInstance.dag_id == key.dag_id,
- TaskInstance.task_id == key.task_id,
- TaskInstance.run_id == key.run_id,
- )
- ).one()
- if ti.state in State.finished:
- self.set_ti_span_attrs(span=span, state=ti.state, ti=ti)
- span.end(end_time=datetime_to_nano(ti.end_date))
- ti.span_status = SpanStatus.ENDED
- else:
- span.end()
- ti.span_status = SpanStatus.NEEDS_CONTINUANCE
- else:
- dag_run: DagRun =
session.scalars(select(DagRun).where(DagRun.run_id == key)).one()
+ for prefixed_key, span in self.active_spans.get_all().items():
+ # Use partition to split on the first occurrence of ':'.
+ prefix, sep, key = prefixed_key.partition(":")
Review Comment:
Initially, I thought of using nested dictionaries
```
active_spans: {
"ti": {
str(ti.id): span,
str(ti.id): span,
str(ti.id): span,
...
},
"dr": {
dr.id: span,
dr.id: span,
...
},
}
```
I went with the string prefix but if we change the dag_run key type to an
integer then this approach might make more sense over the rest.
> Though we'd have to be more careful of the type of the id we put in -- cos
("ti", UUID(...)) wouldn't match ("ti", "the_uuid")
The `ti.id` is always a UUID str.
https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/models/taskinstance.py#L717
https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/models/taskinstance.py#L555-L557
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
