ashokballolli commented on code in PR #77069:
URL: https://github.com/apache/airflow/pull/77069#discussion_r4569397764
##########
providers/google/src/airflow/providers/google/cloud/hooks/dataflow.py:
##########
@@ -460,6 +462,21 @@ def fetch_job_id(self) -> str | None:
self.log.info("Google Cloud DataFlow job not available yet..")
return self._job_id
+ def fetch_latest_job_by_name(self, job_name: str) -> dict | None:
+ """
+ Return the latest created job with the given name, in any state.
+
+ Dataflow API does not support getting a job by name, so all jobs in
the location are listed
+ and filtered here.
+
+ :param job_name: Name of the job. It is converted to lowercase because
Dataflow job names
+ are always lowercase.
+ :return: The job, or None if no job has this name.
+ """
+ job_name = job_name.lower()
+ jobs = [job for job in self._fetch_all_jobs() if job.get("name") ==
job_name]
+ return max(jobs, key=lambda job: job.get("createTime", ""),
default=None)
Review Comment:
Done. Changed it to a generator inside max() as you showed. Also createTime
is parsed with datetime.fromisoformat before comparing now, and I added a test
case with ...00Z and ...00.100Z. Thanks for catching this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
