kaxil opened a new pull request, #76790:
URL: https://github.com/apache/airflow/pull/76790
Under `run_durably`, a failure the model can correct (a SQL statement naming
a missing table, say) reaches an OpenAI Agents SDK agent as `{"error": ...}`
from an `AirflowTools` tool and is recorded in the durable journal like any
other result. On a task retry it replays without the tool running, and each
tool's retry budget starts every run at zero. A retry therefore gave the model
`max_retries` new corrections on top of the ones it had already used. Pydantic
AI runs count them, because `durable/capability.py` raises a replayed
`ModelRetry` again.
Each `AirflowTools` call now records, next to its result, the count it left
in its tool's retry budget and when it finished. A replay restores the
latest-finished count, so across tries the tool gets `max_retries` corrections
in total.
A run that uses the limit up now fails on the model's account:
`_RetryLimitReached` joins the SDK's model-at-fault errors, and the retry asks
the model again from its last request. The Pydantic AI path behaves the same
way, giving the model one more answer per retry with the rejections it already
got in its history.
## Design rationale
**The count is recorded, not read from the `{"error": ...}` text.**
`ToolFailed`, which `ObjectStorageToolset` raises for a missing file, reaches
the model as `{"error": ...}` too and never uses up a correction. A tool can
also succeed with a result that has an `error` key.
**The replay restores a count rather than re-applying each call's outcome.**
The OpenAI adapter has no model-turn id, so a success resets the count, and
parallel calls reach the replay in the order they started rather than the order
they finished. A slow success next to a fast failure would leave a different
count on replay than it did live. A call that runs live always outranks
anything an earlier try recorded.
**Only the tool the adapter called records a count.** A custom `AirflowTool`
that wraps a toolset tool would otherwise inherit the inner call's count, and
its replay would run the wrapper with empty arguments. Likewise, only an
`AirflowTools` step treats a used-up limit as the model's fault: an agent
called through `as_tool` that runs out of corrections fails that call, and the
outer model's answer still replays.
**Journals written by 1.1.0 still replay.** An `AirflowTools` result is
always text, so the new `{"output", "retry"}` form is told apart by being a
dict, and older plain-text entries replay as before, without a count.
## Known issues
- A live call that counts before a replayed call of the same turn restores
is ordered by wall-clock time. That holds while workers' clocks differ by less
than the time between tries, which the Execution API's token checks already
require.
- Strands durable runs have the same gap: they resume from a checkpoint with
a fresh budget. Fixing that needs the count saved with the checkpoint, so it is
a separate change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
