kaxil opened a new pull request, #73523: URL: https://github.com/apache/airflow/pull/73523
## Summary The `common.ai` docs sidebar is one flat "Guides" list of 20 entries. Seven connection pages sit between the quick start and everything else, the operators are entry 13 of 20, and "Choosing a toolset" comes after both "Toolsets" and "Sandboxed execution". A reader cannot tell from the sidebar which features exist: durable execution, guardrails, code mode, message history, approval gates and structured output are subsections of the `AgentOperator` and `LLMOperator` pages, and `toolsets.rst` (1145 lines), `sandbox.rst` (821) and `choosing_a_toolset.rst` (611, which re-walks `toolsets.rst` toolset by toolset) carry most of the agent material. This PR regroups the sidebar into captions (Getting started, Models and providers, Operators, Toolsets, Running agents, Document and RAG pipelines, Reliability and operations, Examples, References), splits the three long pages, gives each feature above a page of its own, and adds an installation page, a core concepts page and a troubleshooting page. It is a move, not a rewrite: existing prose travels verbatim apart from the sentences that referred to "this page" or "below", and content fixes are left for a follow-up so this diff can be reviewed as a reorganization. ## Design rationale **Toolsets and agent runtime features are separate captions.** A reader wiring a Snowflake connection into an agent and a reader asking what happens on retry are different people, and Pydantic AI's own docs keep "Tools & Toolsets" apart from "Capabilities" for the same reason. MCP and the sandbox are entries under those captions rather than captions of their own: each is two or three pages, and the Airflow Sphinx theme renders every caption permanently expanded, so a two-entry caption reads as an afterthought. The MCP connection and hook pages nest under `toolsets/mcp.rst` through its own toctree, so they remain one click away without adding sidebar lines. **Durable execution sits under Reliability, next to retry policies.** A reader whose retries are expensive scans for "retry", lands on `LLMRetryPolicy` (which classifies failures and never mentioned durable execution), and leaves with the wrong feature. Both pages now open with a paragraph naming the other and the batch operator's re-attach behaviour. **`choosing_a_toolset.rst` is retired rather than kept as a page.** Its intro, decision table and tail sections (credentials, call barriers, layering) open the new `toolsets/index.rst`; each per-toolset section becomes a "When to choose it" section on that toolset's page, so the guidance sits beside the configuration reference instead of on a parallel page. **Agent security gets its own page.** `security.rst` is checked verbatim against a shared include by `check_provider_docs`, so the 250-line defense-layers, `allowed_tables` and production-checklist section from `toolsets.rst` lives in `agent_security.rst` under Reliability. **Sidebar labels are class names or short nouns.** Task-first labels such as "Airflow hooks as tools" wrap at the sidebar width; the page titles keep the longer form and the overview table does the "I have X, which toolset" mapping. ## Gotchas - `redirects.txt` covers the four removed pages (`toolsets`, `choosing_a_toolset`, `sandbox`, `hooks/index`). - Pydantic AI's durable-execution guide deep-links `operators/agent.html#durable-execution`, so `operators/agent.rst` keeps a short "Durable execution" heading pointing at the new page. Fragments do not survive a redirect, which is why a stub rather than a redirect. - The generated supported-models page from #72939 belongs under "Models and providers" once it lands; nothing here depends on it. ## Follow-ups Content fixes found while reading every page (stale model ids, review-process language in prose, version-pinned claims, a connection rename note pasted into three pages, the same MCP explanation on three pages) are intentionally not in this PR. --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
