kaxil commented on code in PR #74770: URL: https://github.com/apache/airflow/pull/74770#discussion_r4296282011
########## providers/common/ai/docs/frameworks/openai_agents.rst: ########## @@ -145,11 +145,65 @@ To turn the SDK's tracing off altogether, call ``agents.set_tracing_disabled(Tru ``OPENAI_AGENTS_DISABLE_TRACING=1`` in the worker's environment. That stops ``AirflowTraceProcessor`` from receiving anything too. +.. _howto/frameworks:openai-agents-durable: + +Durable execution +----------------- + +When a task that calls ``Runner.run`` fails and Airflow retries it, the agent starts +again from the prompt: every model call is paid for again, and every tool call happens +again. On a task with ``retries``, run the agent with +:func:`~airflow.providers.common.ai.durable.openai_agents.run_durably_sync`, or +:func:`~airflow.providers.common.ai.durable.openai_agents.run_durably` from async code, +instead. They take the same arguments as ``Runner.run_sync`` and ``Runner.run``: Review Comment: Done in 7715f05b85d, with the reason next to it. ########## providers/common/ai/docs/frameworks/openai_agents.rst: ########## @@ -145,11 +145,65 @@ To turn the SDK's tracing off altogether, call ``agents.set_tracing_disabled(Tru ``OPENAI_AGENTS_DISABLE_TRACING=1`` in the worker's environment. That stops ``AirflowTraceProcessor`` from receiving anything too. +.. _howto/frameworks:openai-agents-durable: + +Durable execution +----------------- + +When a task that calls ``Runner.run`` fails and Airflow retries it, the agent starts +again from the prompt: every model call is paid for again, and every tool call happens +again. On a task with ``retries``, run the agent with +:func:`~airflow.providers.common.ai.durable.openai_agents.run_durably_sync`, or +:func:`~airflow.providers.common.ai.durable.openai_agents.run_durably` from async code, +instead. They take the same arguments as ``Runner.run_sync`` and ``Runner.run``: + +.. exampleinclude:: /../../ai/src/airflow/providers/common/ai/example_dags/example_openai_agents_durable.py + :language: python + :start-after: [START example_openai_agents_durable] + :end-before: [END example_openai_agents_durable] + +Each model call and function tool call is recorded in the task instance's durable +journal as it completes, the journal ``AgentOperator(durable=True)`` uses (see +:doc:`../durable_execution`). On a retry the agent runs again from the start, and each +call that asks what the failed try asked, in the same order, returns what it returned +then. From the first call that differs, that call and every call after it run live. Once +the run succeeds, what was recorded is deleted, so clearing the task later starts over. + +A retry runs some work again: Review Comment: Split into "What a retry replays" and "What always runs live" in 7715f05b85d. ########## providers/common/ai/docs/frameworks/openai_agents.rst: ########## @@ -145,11 +145,65 @@ To turn the SDK's tracing off altogether, call ``agents.set_tracing_disabled(Tru ``OPENAI_AGENTS_DISABLE_TRACING=1`` in the worker's environment. That stops ``AirflowTraceProcessor`` from receiving anything too. +.. _howto/frameworks:openai-agents-durable: + +Durable execution +----------------- + +When a task that calls ``Runner.run`` fails and Airflow retries it, the agent starts +again from the prompt: every model call is paid for again, and every tool call happens +again. On a task with ``retries``, run the agent with +:func:`~airflow.providers.common.ai.durable.openai_agents.run_durably_sync`, or +:func:`~airflow.providers.common.ai.durable.openai_agents.run_durably` from async code, +instead. They take the same arguments as ``Runner.run_sync`` and ``Runner.run``: + +.. exampleinclude:: /../../ai/src/airflow/providers/common/ai/example_dags/example_openai_agents_durable.py + :language: python + :start-after: [START example_openai_agents_durable] + :end-before: [END example_openai_agents_durable] + +Each model call and function tool call is recorded in the task instance's durable +journal as it completes, the journal ``AgentOperator(durable=True)`` uses (see +:doc:`../durable_execution`). On a retry the agent runs again from the start, and each +call that asks what the failed try asked, in the same order, returns what it returned +then. From the first call that differs, that call and every call after it run live. Once +the run succeeds, what was recorded is deleted, so clearing the task later starts over. + +A retry runs some work again: + +- A call that was running when the try failed runs again, so tools that change data + must still be safe to repeat. +- When the run failed because of the model's answers (``MaxTurnsExceeded``, + ``ModelBehaviorError``, ``ModelRefusalError``, or an output or tool input guardrail), + replaying those answers would fail it the same way, so the retry asks the model again + from its last request. A ``ModelBehaviorError`` that a tool raises, such as arguments Review Comment: Added in 7715f05b85d: the tool runs again and the model answer before it replays. ########## providers/common/ai/docs/frameworks/openai_agents.rst: ########## @@ -145,11 +145,65 @@ To turn the SDK's tracing off altogether, call ``agents.set_tracing_disabled(Tru ``OPENAI_AGENTS_DISABLE_TRACING=1`` in the worker's environment. That stops ``AirflowTraceProcessor`` from receiving anything too. +.. _howto/frameworks:openai-agents-durable: + +Durable execution +----------------- + +When a task that calls ``Runner.run`` fails and Airflow retries it, the agent starts +again from the prompt: every model call is paid for again, and every tool call happens +again. On a task with ``retries``, run the agent with +:func:`~airflow.providers.common.ai.durable.openai_agents.run_durably_sync`, or +:func:`~airflow.providers.common.ai.durable.openai_agents.run_durably` from async code, +instead. They take the same arguments as ``Runner.run_sync`` and ``Runner.run``: + +.. exampleinclude:: /../../ai/src/airflow/providers/common/ai/example_dags/example_openai_agents_durable.py + :language: python + :start-after: [START example_openai_agents_durable] + :end-before: [END example_openai_agents_durable] + +Each model call and function tool call is recorded in the task instance's durable +journal as it completes, the journal ``AgentOperator(durable=True)`` uses (see +:doc:`../durable_execution`). On a retry the agent runs again from the start, and each +call that asks what the failed try asked, in the same order, returns what it returned +then. From the first call that differs, that call and every call after it run live. Once +the run succeeds, what was recorded is deleted, so clearing the task later starts over. + +A retry runs some work again: + +- A call that was running when the try failed runs again, so tools that change data + must still be safe to repeat. +- When the run failed because of the model's answers (``MaxTurnsExceeded``, + ``ModelBehaviorError``, ``ModelRefusalError``, or an output or tool input guardrail), + replaying those answers would fail it the same way, so the retry asks the model again + from its last request. A ``ModelBehaviorError`` that a tool raises, such as arguments + that fail validation with ``failure_error_function=None``, counts as the tool's failure + instead: the tool call runs again, but the answer that asked for it replays. +- A retry whose prompt, instructions, model settings or tools differ from the failed + try's runs everything live from the first call that sees the difference. +- These are not recorded and run live on every try: tools other than function tools + (hosted tools such as web search, the tools of ``mcp_servers``, shell and computer + tools); a function tool whose ``needs_approval`` is a function, since the SDK checks the + arguments it approved only through its own invoker; the function tools of an agent + reached through ``handoff()``, and its model unless that is given by name or left to Review Comment: Reworded to match the docstring in 7715f05b85d. ########## providers/common/ai/docs/frameworks/openai_agents.rst: ########## @@ -145,11 +145,65 @@ To turn the SDK's tracing off altogether, call ``agents.set_tracing_disabled(Tru ``OPENAI_AGENTS_DISABLE_TRACING=1`` in the worker's environment. That stops ``AirflowTraceProcessor`` from receiving anything too. +.. _howto/frameworks:openai-agents-durable: + +Durable execution +----------------- + +When a task that calls ``Runner.run`` fails and Airflow retries it, the agent starts +again from the prompt: every model call is paid for again, and every tool call happens +again. On a task with ``retries``, run the agent with +:func:`~airflow.providers.common.ai.durable.openai_agents.run_durably_sync`, or +:func:`~airflow.providers.common.ai.durable.openai_agents.run_durably` from async code, +instead. They take the same arguments as ``Runner.run_sync`` and ``Runner.run``: + +.. exampleinclude:: /../../ai/src/airflow/providers/common/ai/example_dags/example_openai_agents_durable.py + :language: python + :start-after: [START example_openai_agents_durable] + :end-before: [END example_openai_agents_durable] + +Each model call and function tool call is recorded in the task instance's durable +journal as it completes, the journal ``AgentOperator(durable=True)`` uses (see +:doc:`../durable_execution`). On a retry the agent runs again from the start, and each +call that asks what the failed try asked, in the same order, returns what it returned +then. From the first call that differs, that call and every call after it run live. Once +the run succeeds, what was recorded is deleted, so clearing the task later starts over. + +A retry runs some work again: + +- A call that was running when the try failed runs again, so tools that change data + must still be safe to repeat. +- When the run failed because of the model's answers (``MaxTurnsExceeded``, + ``ModelBehaviorError``, ``ModelRefusalError``, or an output or tool input guardrail), + replaying those answers would fail it the same way, so the retry asks the model again + from its last request. A ``ModelBehaviorError`` that a tool raises, such as arguments + that fail validation with ``failure_error_function=None``, counts as the tool's failure + instead: the tool call runs again, but the answer that asked for it replays. +- A retry whose prompt, instructions, model settings or tools differ from the failed + try's runs everything live from the first call that sees the difference. +- These are not recorded and run live on every try: tools other than function tools + (hosted tools such as web search, the tools of ``mcp_servers``, shell and computer + tools); a function tool whose ``needs_approval`` is a function, since the SDK checks the + arguments it approved only through its own invoker; the function tools of an agent + reached through ``handoff()``, and its model unless that is given by name or left to + the default; and a streamed run. An agent used through ``as_tool`` is recorded as one + call. +- A ``session`` makes every retry run live: the SDK adds the failed try's prompt to it Review Comment: Added in 7715f05b85d, with a SQLite session shared across attempts: nothing replays on the retry. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
