andygrove opened a new pull request, #6064:
URL: https://github.com/apache/datafusion-comet/pull/6064
## Which issue does this PR close?
Closes #6062.
## Rationale for this change
All three mermaid diagrams are missing from the published site — not blank,
not broken images, absent from the HTML:
```
$ gh api
"repos/apache/datafusion-comet/contents/contributor-guide/memory_management.html?ref=asf-site"
\
--jq .content | base64 -d | grep -c 'svg\|mermaid\|<object'
0
$ gh api "repos/apache/datafusion-comet/git/trees/asf-site?recursive=1" \
--jq '[.tree[].path | select(test("mermaid"))] | length'
0
```
#6021 moved rendering from the reader's browser to build-time SVG, which
fixed #6020's CSP block. Its publish commit (`b9ea47c16`) removed the `<pre
class="mermaid">` blocks and the jsdelivr import from both pages and added
nothing in their place, and nothing has rendered since. `sphinxcontrib-mermaid`
downgrades a render failure to a Sphinx warning and drops the node, so the
deploy stays green and the page publishes with a diagram-shaped hole.
`docs/README.md` and the comment on the `Install mermaid-cli` step both
already said this was the failure mode. Nothing enforced it. And the `docs` job
is push-tier by POLICY, so no docs build ran on #6021's pull request either —
the change was verified locally on macOS, where it produced the expected 2 and
1 `<object>` embeds.
Two candidate causes, and rather than guess between them without the job
log, this closes out both:
- Chrome's setuid sandbox needs unprivileged user namespaces, which Ubuntu
restricts by AppArmor policy from 23.10 onwards, so Chrome can fail to launch
on an `ubuntu-24.04` runner.
- `puppeteer`'s `postinstall`, which fetches the browser `mmdc` drives,
catches its own download failure and exits 0. A flaky fetch leaves a green
install step with no browser at all, and `mmdc` dies with `Could not find
chrome-headless-shell`. That is reproducible locally and is [written up on the
issue](https://github.com/apache/datafusion-comet/issues/6062#issuecomment-5750544640).
## What changes are included in this PR?
- `docs/puppeteer-config.json` + `mermaid_params` in `docs/source/conf.py`:
pass `mmdc` a puppeteer config with `--no-sandbox`. The CI container is already
the isolation boundary. `DOCS_DIR` resolves from `__file__`, so the path is
right whether `conf.py` is read from `docs/source/` or from the `docs/temp/`
copy `build.sh` makes.
- Both workflows install mermaid-cli into `$RUNNER_TEMP` and then run
`puppeteer browsers install chrome-headless-shell` explicitly, where a failure
is allowed to fail the job.
- `dev/ci/check-mermaid.py`, the durable half, in two modes:
- default: render every ` ```mermaid ` fence under `docs/source/` with
mmdc. It reads `mermaid_params` out of `conf.py` rather than repeating them, so
the check cannot pass with flags the build does not use. Run from preflight, so
a diagram that only mmdc rejects fails on the pull request.
- `--built docs/build/html`: assert the built site carries a non-empty SVG
per distinct fence and that some page references each one. Run after the build
and before the publish step, so a bad render leaves the previous site in place.
- The pinned mermaid-cli version lives in that script and both workflows
install `--cli-spec`, so the pull-request check and the deploy cannot render
with different versions.
- `dev/ci/compute-changes.py` routes the new script to the `docs` job, with
the spot check in `check-ci-config.py`; `pom.xml` gets a RAT exclude for the
JSON, which can carry no license header; `docs/README.md` documents both checks
and the `Could not find chrome-headless-shell` case.
## How are these changes tested?
The new script is the test, and it was calibrated against the real failure
rather than a synthetic one:
| case | input | result |
| --- | --- | --- |
| today's published site | the deployed `memory_management.html` and
`ci.html` from `asf-site`, no `_images` | fails: `docs/source has 3 distinct
```mermaid fences but the build produced 0 SVG(s)` |
| healthy build | the same pages with 3 non-empty `mermaid-*.svg` referenced
by `<object>` | passes |
| a diagram rendered to an empty file | one SVG truncated | fails, naming
the file |
| mmdc absent | no mmdc on `PATH` | fails with the install instruction |
| mmdc present, no browser | mermaid-cli 11.17.0, empty puppeteer cache |
fails on all 3 fences with mmdc's stderr plus the `npx puppeteer browsers
install` hint |
| fence detection | `docs/source/**` | finds exactly the 3 real fences, at
`ci.md:35`, `memory_management.md:113` and `:395` |
| `conf.py` from `docs/temp/` | copy of `conf.py` one directory over |
`DOCS_DIR` still resolves to `docs/`, config file found |
| `-p` is actually read | `mmdc -p /nonexistent.json` vs the real path | the
first errors on the missing config, the second proceeds past argument parsing
to the browser launch |
`actionlint`, `python3 dev/ci/check-ci-config.py`, `./mvnw -N
apache-rat:check` and `prettier --check` all pass locally.
What I could **not** verify locally: a successful end-to-end render.
Chrome's download host is blocked in my environment and there is no Chrome on
the machine, so every render attempt stops at browser launch. The first docs
run on main after this merges is what proves the fix — and if a third cause is
in play, that run now fails loudly with mmdc's stderr instead of publishing a
hole.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
