kaxil commented on issue #70093: URL: https://github.com/apache/airflow/issues/70093#issuecomment-5037286401
Thanks for writing this up, and the motivation is real: LLM and agent tasks shouldn't hold a worker slot idle for the whole call. But running the LLM/agent work in the triggerer isn't the right mechanism, so I'd rather not take this direction. The triggerer is for tasks that have nothing to do but wait on external state. A trigger's `run()` is a poll (is the job done, does the file exist, has the time passed): idempotent, side-effect-free, and safe to re-run. That last property is the one that matters here, and the docs call it out directly: "you should assume that a trigger instance can run more than once ... so you must be mindful about side effects" (`deferring.rst`). Triggers are re-created from their serialized form and re-run on any triggerer restart, redeploy, or HA redistribution. An LLM call, and more so an agent loop, is the opposite of an idle wait. `agent.run()` is a control loop: call the model, parse tool calls, execute the tools in-process (user code), feed results back, call the model again, until it's done. That's active local orchestration that happens to make remote calls between steps, not one remote call whose completion you await. Moving it into the triggerer doesn't turn it into a wait, it relocates the worker's job, tool execution and all, into the shared triggerer event loop. And because the call is a billed, non-deterministic POST rather than an idempotent poll, a restart mid-flight re-runs and re-bills it. Step 3's "durable-storage replay on triggerer restart, no re-billed LLM calls" is exactly the property triggers don't give you. The two goals here are already served by existing primitives: - "Hundreds of concurrent LLM tasks without one worker slot each" is what Airflow 3.2's async tasks are for: the task process keeps running on the worker and multiplexes I/O on a shared event loop, so you get the concurrency without releasing and later reacquiring a slot. See https://airflow.apache.org/docs/task-sdk/stable/deferred-vs-async-operators.html. - "Long-running agent tasks that don't re-bill on retry" is what `AgentOperator(durable=True)` already does: it caches model responses and tool results per step and replays completed steps on retry instead of re-issuing the LLM calls and tool invocations. That's the durable replay Step 3 is reaching for, and it runs on the worker where the loop belongs. Given that, I'll close this as not planned. If there's a gap in what async tasks or `durable=True` cover for your use case, that's the more useful thing to dig into, happy to help scope it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
