Taragolis commented on code in PR #39430:
URL: https://github.com/apache/airflow/pull/39430#discussion_r1609821854
##########
airflow/jobs/job.py:
##########
@@ -101,24 +95,18 @@ class Job(Base, LoggingMixin):
Index("idx_job_dag_id", dag_id),
)
- task_instances_enqueued = relationship(
+ task_instances_enqueued = relationship( # Task Instances which have been
enqueued by this Job.
"TaskInstance",
+ back_populates="queued_by_job",
primaryjoin="Job.id == foreign(TaskInstance.queued_by_job_id)",
- backref=backref("queued_by_job", uselist=False),
)
- dag_runs = relationship(
+ dag_runs = relationship( # Only makes sense for SchedulerJob and
BackfillJob instances.
"DagRun",
- primaryjoin=lambda: Job.id ==
foreign(_resolve_dagrun_model().creating_job_id),
- backref="creating_job",
+ back_populates="creating_job",
+ primaryjoin="Job.id == foreign(DagRun.creating_job_id)",
)
- """
- TaskInstances which have been enqueued by this Job.
-
- Only makes sense for SchedulerJob and BackfillJob instances.
- """
-
Review Comment:
### Some proofs
#### DAG
```python
from __future__ import annotations
from datetime import datetime, timezone
from airflow.decorators import task
from airflow.models.dag import DAG
START_DATE = datetime(2024, 2, 1, tzinfo=timezone.utc)
with DAG("pr_39430", schedule="@once", tags=["pr", "39430",
"bi-directional"], start_date=START_DATE):
@task
def do_nothing(): ...
do_nothing()
```
Run one time by scheduler and one manual run, by trigger DAG from the UI,
results in Tables
#### Table `job`
| column| | | | |
| :- | :- | :- | :- | :- |
| **id** | 3 | 4 | 1 | 2 |
| **dag\_id** | pr\_39430 | pr\_39430 | null | null |
| **state** | success | success | running | running |
| **job\_type** | LocalTaskJob | LocalTaskJob | TriggererJob | SchedulerJob |
| **start\_date** | 2024-05-22 11:45:23.867096 +00:00 | 2024-05-22
11:45:29.782799 +00:00 | 2024-05-22 11:44:49.709191 +00:00 | 2024-05-22
11:44:49.813874 +00:00 |
| **end\_date** | 2024-05-22 11:45:24.143576 +00:00 | 2024-05-22
11:45:30.040894 +00:00 | null | null |
| **latest\_heartbeat** | 2024-05-22 11:45:23.853896 +00:00 | 2024-05-22
11:45:29.766046 +00:00 | 2024-05-22 11:45:35.885734 +00:00 | 2024-05-22
11:45:37.722701 +00:00 |
| **executor\_class** | null | null | null | null |
| **hostname** | cd2591016c8c | cd2591016c8c | cd2591016c8c | cd2591016c8c |
| **unixname** | root | root | root | root |
#### Table `task_instance`
| column| | |
| :- | :- | :- |
| **task\_id** | do\_nothing | do\_nothing |
| **dag\_id** | pr\_39430 | pr\_39430 |
| **run\_id** | scheduled\_\_2024-02-01T00:00:00+00:00 |
manual\_\_2024-05-22T11:45:28.453199+00:00 |
| **map\_index** | -1 | -1 |
| **start\_date** | 2024-05-22 11:45:23.921193 +00:00 | 2024-05-22
11:45:29.821869 +00:00 |
| **end\_date** | 2024-05-22 11:45:24.085409 +00:00 | 2024-05-22
11:45:29.979149 +00:00 |
| **duration** | 0.164216 | 0.15728 |
| **state** | success | success |
| **try\_number** | 1 | 1 |
| **max\_tries** | 0 | 0 |
| **hostname** | cd2591016c8c | cd2591016c8c |
| **unixname** | root | root |
| **job\_id** | 3 | 4 |
| **pool** | default\_pool | default\_pool |
| **pool\_slots** | 1 | 1 |
| **queue** | default | default |
| **priority\_weight** | 1 | 1 |
| **operator** | \_PythonDecoratedOperator | \_PythonDecoratedOperator |
| **custom\_operator\_name** | @task | @task |
| **queued\_dttm** | 2024-05-22 11:45:23.394469 +00:00 | 2024-05-22
11:45:29.546020 +00:00 |
| **queued\_by\_job\_id** | 2 | 2 |
| **pid** | 2375 | 2377 |
| **executor** | null | null |
| **executor\_config** | 0x80057D942E | 0x80057D942E |
| **updated\_at** | 2024-05-22 11:45:23.930163 +00:00 | 2024-05-22
11:45:29.831787 +00:00 |
| **rendered\_map\_index** | null | null |
| **external\_executor\_id** | null | null |
| **trigger\_id** | null | null |
| **trigger\_timeout** | null | null |
| **next\_method** | null | null |
| **next\_kwargs** | null | null |
| **task\_display\_name** | do\_nothing | do\_nothing |
#### Table `dag_run`
|column | | |
| :- | :- | :- |
| **id** | 1 | 2 |
| **dag\_id** | pr\_39430 | pr\_39430 |
| **queued\_at** | 2024-05-22 11:45:23.351323 +00:00 | 2024-05-22
11:45:28.490357 +00:00 |
| **execution\_date** | 2024-02-01 00:00:00.000000 +00:00 | 2024-05-22
11:45:28.453199 +00:00 |
| **start\_date** | 2024-05-22 11:45:23.373351 +00:00 | 2024-05-22
11:45:29.516604 +00:00 |
| **end\_date** | 2024-05-22 11:45:24.476332 +00:00 | 2024-05-22
11:45:30.609079 +00:00 |
| **state** | success | success |
| **run\_id** | scheduled\_\_2024-02-01T00:00:00+00:00 |
manual\_\_2024-05-22T11:45:28.453199+00:00 |
| **creating\_job\_id** | 2 | null |
| **external\_trigger** | false | true |
| **run\_type** | scheduled | manual |
| **conf** | 0x80057D942E | 0x80057D942E |
| **data\_interval\_start** | 2024-02-01 00:00:00.000000 +00:00 | 2024-05-22
11:45:28.453199 +00:00 |
| **data\_interval\_end** | 2024-02-01 00:00:00.000000 +00:00 | 2024-05-22
11:45:28.453199 +00:00 |
| **last\_scheduling\_decision** | 2024-05-22 11:45:24.474830 +00:00 |
2024-05-22 11:45:30.606623 +00:00 |
| **dag\_hash** | cd6f55b5c736bf50d2f341663c0ad83f |
cd6f55b5c736bf50d2f341663c0ad83f |
| **log\_template\_id** | 2 | 2 |
| **updated\_at** | 2024-05-22 11:45:24.477225 +00:00 | 2024-05-22
11:45:30.609766 +00:00 |
| **clear\_number** | 0 | 0 |
Relationships
---
LocalJob (id: 3) create TI (run_id: `scheduled__2024-02-01T00:00:00+00:00`)
LocalJob (id: 4) create TI (run_id:
`manual__2024-05-22T11:45:28.453199+00:00`)
SchedulerJob (id: 2) queued TI (run_id:
`scheduled__2024-02-01T00:00:00+00:00`)
SchedulerJob (id: 2) queued TI (run_id:
`manual__2024-05-22T11:45:28.453199+00:00`) - Even if it Manual DAG Run
DagRun (run_id: `scheduled__2024-02-01T00:00:00+00:00`) created by
SchedulerJob (id: 2)
DagRun (run_id: `manual__2024-05-22T11:45:28.453199+00:00`) doesn't created
by any Job
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
