ashb commented on a change in pull request #16182:
URL: https://github.com/apache/airflow/pull/16182#discussion_r649337490
##########
File path: airflow/models/dagbag.py
##########
@@ -166,6 +166,36 @@ def dag_ids(self) -> List[str]:
"""
return list(self.dags.keys())
+ @provide_session
+ def has_dag(self, dag_id: str, session: Session = None):
+ """
+ Checks the "local" cache, if it exists, and is not older than the
confiured cache time, return True
+ else check in the DB if the dag exists, return True, False otherwise
+ """
+ if dag_id in self.dags:
+ min_serialized_dag_fetch_secs =
timedelta(seconds=settings.MIN_SERIALIZED_DAG_FETCH_INTERVAL)
+ if dag_id in self.dags_last_fetched:
+
+ return timezone.utcnow() < self.dags_last_fetched[dag_id] +
min_serialized_dag_fetch_secs
Review comment:
Looking at how that is used, nothing in the DagBag deals with the case
where a dagmodel/serialized dag row is deleted -- and the cached version would
be returned, or maybe an error would be thrown.
EIther way, it doesn't really handle this case form what I can tell.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
