YAshhh29 commented on code in PR #75916:
URL: https://github.com/apache/airflow/pull/75916#discussion_r4540845065
##########
providers/common/ai/src/airflow/providers/common/ai/durable/storage.py:
##########
@@ -91,10 +101,32 @@ def _load_cache(self) -> dict[str, Any]:
path = self._get_path()
try:
- self._cache = json.loads(path.read_text())
- except (FileNotFoundError, OSError, json.JSONDecodeError, ValueError):
+ text = path.read_text()
+ except FileNotFoundError:
self._cache = {}
+ return self._cache
+ except UnicodeDecodeError:
+ # Bytes that are not text cannot hold a journal; treat them like
any other invalid file.
+ text = ""
+ except Exception:
Review Comment:
Took the offer in 5477ca95b9. The read side is dropped, so a 403 on a key
that doesn't exist yet starts an empty journal again, as on main. I kept the
non-object fix: valid JSON that isn't an object now takes the same path as a
file cut off mid-write. The check raises inside the same `try`, so once #76381
lands, a non-object journal will follow whatever it does with a corrupt one.
##########
providers/common/ai/docs/durable_execution.rst:
##########
@@ -124,6 +124,11 @@ Without the option, the task fails with:
ValueError: durable=True requires [common.ai] durable_cache_path to be
set. Example: durable_cache_path = file:///tmp/airflow_durable_cache
+If the store fails while the task runs, the task does not. A journal file that
exists but
Review Comment:
Removed the paragraph. The only docs change left is the line in "Side
effects and idempotency", which now holds for both backends.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
