MaksYermak commented on code in PR #72809:
URL: https://github.com/apache/airflow/pull/72809#discussion_r4073027251
##########
providers/google/src/airflow/providers/google/cloud/hooks/dataflow.py:
##########
@@ -339,6 +339,15 @@ def _get_current_jobs(self) -> list[dict]:
jobs = self._fetch_jobs_by_prefix_name(self._job_name.lower())
if len(jobs) == 1:
self._job_id = jobs[0]["id"]
+ elif len(jobs) > 1 and not self._multiple_jobs:
+ active_jobs = [
+ job for job in jobs if job.get("currentState") not in
DataflowJobStatus.TERMINAL_STATES
+ ]
+ if len(active_jobs) == 1:
+ self._job_id = active_jobs[0]["id"]
+ else:
+ jobs.sort(key=lambda j: j.get("createTime", ""),
reverse=True)
+ self._job_id = jobs[0]["id"]
Review Comment:
@aaron-y-chen I have tested with the same code and I did not see any problem
which you had described.
I ran two Jobs with the same name sequentially:
<img width="1193" height="302" alt="Screenshot 2026-09-22 at 16 32 24"
src="https://github.com/user-attachments/assets/52386ca3-406c-4a2a-96f3-51468ba7b692";
/>
Here is the screenshot from the first run:
<img width="1847" height="710" alt="Screenshot 2026-09-22 at 16 33 14"
src="https://github.com/user-attachments/assets/51b45d34-34ed-496f-bedf-cca501390363";
/>
we can see that `start-python-deferrable` Job in `Pending` state and the
next line is starting the deferreble mode.
Here is the screenshot from the second run:
<img width="1890" height="726" alt="Screenshot 2026-09-22 at 16 33 33"
src="https://github.com/user-attachments/assets/e1afea8c-5989-46cc-9e5c-74c125b3fb04";
/>
on this screenshot we can see on line `188` that `start-python-deferrable`
Job in `Pending` state and on line `189` that `start-python-deferrable` Job in
`Done` state. On the line `189` information about previous Jobs runs with the
same name and it is not a problem for code at all. Because, we have
`start-python-deferrable` Job in `Pending` state the code goes to deferreble
mode on line `190`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
