jason810496 commented on code in PR #71814:
URL: https://github.com/apache/airflow/pull/71814#discussion_r3811943190
##########
airflow-core/docs/administration-and-deployment/web-stack.rst:
##########
@@ -188,9 +188,10 @@ For example, to trigger a rolling restart of the API
server pods:
kubectl rollout restart deployment airflow-api-server
-The API server also supports bounded DAG caching via ``dag_cache_size`` and
-``dag_cache_ttl``, which limits memory consumed by cached SerializedDAG
objects.
-This reduces memory growth from DAG version accumulation regardless of server
type.
+The API server also evicts cached SerializedDAG objects via ``dag_cache_size``
and
+``dag_cache_ttl``, which reduces memory growth from Dag version accumulation
regardless of
+server type. Note that only ``dag_cache_size`` caps memory outright: each
re-check resets a
+cached entry's expiry, so ``dag_cache_ttl`` reclaims only the versions that
stop being requested.
Review Comment:
I rephrased the docs in a21020f68b. Thanks.no human review before posting)
##########
airflow-core/src/airflow/models/dagbag.py:
##########
@@ -79,26 +80,30 @@ def __init__(
Initialize DBDagBag.
:param load_op_links: Should the extra operator link be loaded when
de-serializing the DAG?
- :param cache_size: Size of LRU cache. If None or 0, uses unbounded
dict (no eviction).
- :param cache_ttl: Time-to-live for cache entries in seconds. If None
or 0, no TTL (LRU only).
+ :param cache_size: Max cached entries. 0 or None means no size limit.
+ :param cache_ttl: Seconds until a cached entry expires, applied with
or without a size limit.
+ 0 or None disables TTL. With neither a size limit nor a TTL the
cache never evicts.
"""
self.load_op_links = load_op_links
self._dags: MutableMapping[UUID | str, _CacheEntry] = {}
self._use_cache = False
self._revalidation_interval = conf.getint("core",
"min_serialized_dag_update_interval")
- # Initialize bounded cache if cache_size is provided and > 0
- if cache_size and cache_size > 0:
- if cache_ttl and cache_ttl > 0:
- self._dags = TTLCache(maxsize=cache_size, ttl=cache_ttl)
- else:
- self._dags = LRUCache(maxsize=cache_size)
+ # A TTL applies with or without a size limit: an uncapped TTLCache is
what lets
+ # ``dag_cache_size = 0`` mean "no size limit" rather than "no eviction
at all".
+ size = max(cache_size or 0, 0)
+ ttl = max(cache_ttl or 0, 0)
Review Comment:
Raised `ValueError` when the value is negative. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
