kaxil commented on code in PR #74685: URL: https://github.com/apache/airflow/pull/74685#discussion_r4268210009
########## providers/common/ai/docs/why_common_ai.rst: ########## @@ -0,0 +1,201 @@ + .. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + .. http://www.apache.org/licenses/LICENSE-2.0 + + .. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +.. _howto/why-common-ai: + +Why Common AI +============= + +You can call a model from Airflow without this provider: put an agent framework or a +vendor SDK in a ``@task`` and it runs. The trouble starts when the agent does real work. +On retry, every model and tool call runs again and is billed again. A spend limit resets +with each attempt. Pausing for a person means wiring up a separate approval task, and +giving the agent tools means writing them yourself. This provider handles those cases +for agents, and its retry policies bring a model's judgment to any task that fails. + +For a single model call with no tools, where a retry costs little, a plain ``@task`` is +enough; see `When a plain task is enough`_. + +The same job, two ways +---------------------- + +An agent reads a reports bucket, queries the orders table and says whether the numbers +agree. Without this provider, a Pydantic AI agent in a ``@task`` looks like this, with +the model key in the worker's environment and the tools written by hand: + +.. code-block:: python + + from pydantic_ai import Agent Review Comment: I'd keep this one as a code block. It shows what not to do: its `query` tool runs whatever SQL the model writes, including `DELETE`, and the next section says so. Everything under `example_dags/` ships in the provider package, and those are the files people copy. A shipped Dag whose purpose is to show the unsafe version would end up being copied. The `AgentOperator` side is already an example Dag, `example_why_common_ai.py`, so the import tests in CI cover it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
