Lee-W commented on code in PR #72156: URL: https://github.com/apache/airflow/pull/72156#discussion_r4045849282
########## providers/common/ai/docs/provider_fallback.rst: ########## @@ -0,0 +1,208 @@ + .. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + .. http://www.apache.org/licenses/LICENSE-2.0 + + .. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +Provider fallback +================= + +A single ``llm_conn_id`` gives a task one provider. When that provider is down, the task +fails and retries into the same outage. ``fallback_conn_ids`` gives the connection an +ordered list of other connections to try, so a provider outage moves to the next vendor +inside the same task attempt. + +Configure it on the connection +------------------------------ + +Put the chain in the primary connection's extra: + +.. code-block:: json + + { + "model": "openai:gpt-5", + "fallback_conn_ids": ["anthropic_prod", "bedrock_dr"] + } + +Every entry is an Airflow connection ID, resolved through the hook registered for its own +connection type. A chain can therefore mix vendors whose credentials live in different +connection fields — ``pydanticai`` for OpenAI, ``pydanticai_bedrock`` for a Bedrock +standby — without the Dag knowing anything about either. + +That is the point of configuring it here rather than in Dag code: the Dag keeps naming one +connection, and whoever administers the connections owns the failover topology. Changing a +standby provider is a connection edit, not a Dag deployment. + +A *bare* model name (e.g. ``"gpt-5"`` rather than ``"openai:gpt-5"``) is forwarded down +the chain as a logical model name: each connection that has no ``model`` of its own +resolves that name against its own platform, so one bare name can reach a primary and +every fallback without repeating it per connection. It does not matter where the primary's +name comes from -- the ``Model`` field on its connection and a ``model_id`` on the operator +or hook are forwarded alike. A fallback with its own ``model`` in +extra always uses that instead -- this is how a fallback pins a spelling the forwarded +name would not produce, such as Bedrock's region-prefixed ``us.anthropic.`` model ids. A +name that already pins a platform (its segment before the first ``:`` is itself a +recognized provider, e.g. ``"openai:gpt-5"``) is *not* forwarded; a fallback with no +``model`` of its own still raises "no model specified" rather than trying a prefixed name +meant for a different provider. See :doc:`connections/pydantic_ai_azure`, +:doc:`connections/pydantic_ai_bedrock` and :doc:`connections/pydantic_ai_vertex` for how +each vendor connection resolves a bare name. + +Configure it on the operator +----------------------------- + +``fallback_conn_ids`` is also a parameter on +:class:`~airflow.providers.common.ai.operators.llm.LLMOperator`, +:class:`~airflow.providers.common.ai.operators.agent.AgentOperator`, their subclasses, +and the matching ``@task.llm`` / ``@task.agent`` decorators -- mirroring ``model_id``, +which is settable at the same two layers: + +.. exampleinclude:: /../../ai/src/airflow/providers/common/ai/example_dags/example_llm_fallback.py + :language: python + :dedent: 0 + :start-after: [START howto_llm_fallback_operator_argument] + :end-before: [END howto_llm_fallback_operator_argument] + +The operator argument overrides the connection's extra field, and passing ``[]`` +explicitly disables a chain configured there -- ``None`` (the default) reads whatever +the connection says. Use this when a task should own its own failover order instead of +inheriting it from however the connection is configured. + +Configure it in code +-------------------- + +:class:`~airflow.providers.common.ai.hooks.pydantic_ai.PydanticAIHook` also takes the list +directly, which is what a task that constructs the hook itself (rather than through an +operator) should use: + +.. exampleinclude:: /../../ai/src/airflow/providers/common/ai/example_dags/example_llm_fallback.py Review Comment: Switched to `llm_primary_down_no_chain` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
