kaxil opened a new pull request, #74901:
URL: https://github.com/apache/airflow/pull/74901
Three Common AI example Dags ran without error in 1.0.0rc1 testing but never
reached the behaviour they exist to show when pointed at a real model. They are
rendered into the docs through `exampleinclude`, so a user copying one sees
nothing happen. The same testing turned up a miscount in the retry-policy
prompt.
- **`example_agent_tool_approval.py`**: the order lookup said the order was
"delivered", so the model declined to refund it and the task never paused for
approval. The lookup now says the carrier marked the parcel lost, as
`example_agent_tool_judge.py` already does, so a refund is the obvious next
step.
- **`example_anthropic_tool_runner.py`**: on a refusal the runner's final
message has no text, and the task succeeded with an empty XCom. The example now
checks `stop_reason == "refusal"` and fails the task.
- **`example_llm_retry_policy.py`**: `task_ambiguous` is meant to show
escalation to the text model, but jev-1.13.0 picks `permanent` for its message
every time, at a confidence that sits right at that category's 0.9 bar. In RC
testing it scored 0.79 to 0.91, so some runs applied `permanent` and failed the
task without escalating. `escalating_policy` now raises the bar on `permanent`
to 0.95 and leaves `snowflake_policy` unchanged. With a text model to ask, a
wrong `permanent` costs more than an escalation, and the higher bar also
catches the wrong `permanent` at 0.90 from the calibration run noted in the
example. Scored 13 more times against the same categories, the message picked
`permanent` every time, at 0.79 to 0.86.
- **Retry-policy prompt**: it said `attempt {try_number} of {max_tries}`,
but `max_tries` is the retry count, so a task with `retries=1` read "attempt 1
of 1" with a retry still left. In one live run the model's reasoning quoted
that as "the retry budget may already be exhausted". The prompt now says `of
{max_tries + 1}`, the same total core uses in its failure email ("Try
{{try_number}} out of {{max_tries + 1}}"). The new test fails on main with
"attempt 1 of 1" and "attempt 2 of 1".
The Anthropic tool runner, Strands and ADK examples also told users to
install only the `anthropic` extra, but each uses `SQLToolset`, which needs the
`sql` extra. Without it a fresh venv fails with `No module named 'sqlglot'`.
Related to #74741
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
