kaxil opened a new pull request, #74757: URL: https://github.com/apache/airflow/pull/74757
Mermaid diagrams in the docs show up on airflow.apache.org as raw source text instead of drawings. sphinxcontrib-mermaid's default output draws them in the reader's browser after `import mermaid from "https://cdn.jsdelivr.net/npm/[email protected]/..."`, and the site is served with the [ASF Content-Security-Policy](https://infra.apache.org/csp.html), whose `script-src` does not allow that host. Chrome blocks the import ("Loading the script ... violates the following Content Security Policy directive") and the `<pre class="mermaid">` block is never processed. The policy took effect in March 2025, before the first diagram was added in #67435, so none of them has ever rendered on the published site. Today that is the JWT and security model pages and the fab and edge3 ERDs; the core ERD (#72006), the Common AI sandbox and HITL review pages and the task-sdk language spec are next in line. This draws the diagrams into SVG files at build time with mermaid-cli (`mmdc`), so the page needs no script at all. Apache Arrow and DataFusion Comet set `mermaid_output_format = 'svg'` for the same reason. ## Design rationale - **mermaid-cli and Chromium are installed on demand, not baked into the CI image.** Together they take about 1 GB on disk (598 MB from apt, 463 MB of node_modules) and at least 250 MB compressed, in an image that every CI job loads, for seven pages. `build_docs.py` checks whether a package it is about to build contains a `.. mermaid::` directive and only then, and only inside breeze, runs `scripts/in_container/install_mermaid_cli.sh`, the same runtime-install pattern as `run_agent_framework_tests.sh`. Locally a sqlite docs build skips the step, and a Common AI build installed both and finished in 79 s. - **Why not ask Infra to allow jsdelivr.** Hosts can be added through `CSP_PROJECT_DOMAINS`, but each needs approval from VP Data Privacy, and the [website privacy policy](https://privacy.apache.org/policies/website-policy.html) says all assets need to be hosted on ASF servers. - **Why not self-host mermaid.js** with `mermaid_use_local`. That also satisfies the CSP, but mermaid 11's ESM build imports 146 chunk files that would have to be vendored and kept in step, and every reader's browser would still do the drawing. - **Arial comes first in the diagram font list.** mermaid sizes each label's box from the font it measures in the build container, and the default `"trebuchet ms"` is not installed there. Readers on macOS do have it, it is wider, and labels came out clipped ("Airflow tasl", "Return outpu"). Liberation Sans, installed alongside Chromium, has Arial's metrics. - The global npm install uses `--ignore-scripts` and `--before` 14 days ago, the lifecycle-script and minimum-release-age rules `ts-sdk/pnpm-workspace.yaml` already applies. A missing `mmdc` or a mermaid syntax error produces a Sphinx warning, which the docs build already turns into a failure, so a broken diagram cannot ship as a hole in the page. The docs README has a troubleshooting entry for both. ## Gotchas - The core ERD is scaled to the content width, the same as the graphviz `<img>` in current releases. - A docs build outside breeze needs `mmdc` on `PATH`. `build_docs` prints the npm command when it is missing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
