This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-25586-d56955b3189f9b8725fcef8256166054f0ccfffe in repository https://gitbox.apache.org/repos/asf/datafusion.git
commit cb5714d54f546a958bda2e5544a9eebf9e172bb7 Author: Kumar Ujjawal <[email protected]> AuthorDate: Thu Sep 24 06:13:18 2026 +0000 dev: include the HTML documentation build in the local lint suite (#25586) ## Which issue does this PR close? - Part of #21048. ## Rationale for this change `./dev/rust_lint.sh` runs the non-functional CI checks locally, but not the documentation website build. Sphinx builds with `-W`, so one warning fails CI. A contributor sees that failure only after a push. ## What changes are included in this PR? - `ci/scripts/check_docs_html.sh` runs the existing `docs/build.sh` through uv. It works from any directory. It reports a missing `uv`, `cargo`, `cargo-depgraph`, `dot`, or `make` before the build starts, and installs nothing. It has no write mode. - `dev/rust_lint.sh` runs the new script as a read-only step. - The "Test doc build" job runs the same script. The path filters of the job also watch the script. - `docs/source/contributor-guide/testing.md` documents the command. `docs/build.sh` still owns the Sphinx and dependency-graph commands. The Python dependencies and the Sphinx settings do not change. ## What is the testing strategy for this PR? Local checks with command stubs and a real build: - The script starts the same build from the repository root and from other directories. - `-h`, `--help`, `--write`, `--allow-dirty`, and an unknown flag all fail before any tool runs. - Each missing tool gives one message and a non-zero exit. - Failures in `uv`, `cargo depgraph`, and `dot` keep their exit code. - A real build writes `docs/build/html/index.html` and changes no tracked file. - One Sphinx warning, a page outside every toctree, fails the build. `uv run ./dev/rust_lint.sh` passes with the new step. ## Are there any user-facing changes? No. This changes the contributor tooling only. --- .github/workflows/docs_pr.yaml | 7 ++- ci/scripts/check_docs_html.sh | 87 ++++++++++++++++++++++++++++++++ dev/rust_lint.sh | 1 + docs/source/contributor-guide/testing.md | 13 +++++ 4 files changed, 104 insertions(+), 4 deletions(-) diff --git a/.github/workflows/docs_pr.yaml b/.github/workflows/docs_pr.yaml index 64be66f350..771e950723 100644 --- a/.github/workflows/docs_pr.yaml +++ b/.github/workflows/docs_pr.yaml @@ -26,12 +26,14 @@ on: push: paths: - "docs/**" + - "ci/scripts/check_docs_html.sh" branches: - main - branch-* pull_request: paths: - "docs/**" + - "ci/scripts/check_docs_html.sh" # manual trigger # https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow workflow_dispatch: @@ -63,7 +65,4 @@ jobs: with: tool: [email protected] - name: Build docs html and check for warnings - run: | - set -x - cd docs - uv run --package datafusion-docs ./build.sh # fails on errors + run: ./ci/scripts/check_docs_html.sh # fails on errors diff --git a/ci/scripts/check_docs_html.sh b/ci/scripts/check_docs_html.sh new file mode 100755 index 0000000000..0d04afff13 --- /dev/null +++ b/ci/scripts/check_docs_html.sh @@ -0,0 +1,87 @@ +#!/usr/bin/env bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Builds the documentation website with `docs/build.sh`, the same way the +# "Test doc build" job does. Sphinx runs with `-W`, so a warning fails the +# build. + +set -euo pipefail + +SCRIPT_NAME="$(basename "${BASH_SOURCE[0]}")" +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ROOT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)" + +usage() { + cat >&2 <<USAGE +Usage: $0 + +Builds the HTML documentation with docs/build.sh and fails on any Sphinx +warning. There is no write mode. + +Needs uv, cargo, cargo-depgraph, Graphviz dot, and make. See docs/README.md +for the Python and Graphviz setup. + +Each build rewrites docs/build and docs/source/_static/data/deps.svg, which +Git ignores. The HTML entry point is docs/build/html/index.html. +USAGE + exit 1 +} + +while [[ $# -gt 0 ]]; do + case "$1" in + -h|--help) + usage + ;; + *) + usage + ;; + esac + shift +done + +# Report every missing prerequisite in one run. This check installs nothing. +missing=0 +require_tool() { + local cmd="$1" + local hint="$2" + if ! command -v "$cmd" > /dev/null 2>&1; then + echo "[${SCRIPT_NAME}] ${cmd} was not found on PATH. ${hint}" >&2 + missing=1 + fi +} + +require_tool "uv" "Install uv to get Python and the documentation dependencies: https://docs.astral.sh/uv/getting-started/installation/"; +require_tool "cargo" "Install the Rust toolchain with rustup: https://rustup.rs/"; +require_tool "cargo-depgraph" "Install it with: cargo install cargo-depgraph --version '^1.6' --locked" +require_tool "dot" "Install Graphviz (e.g., brew install graphviz, or apt-get install graphviz)." +require_tool "make" "Install make (e.g., xcode-select --install, or apt-get install make)." + +if [[ "${missing}" -ne 0 ]]; then + exit 1 +fi + +cd "${ROOT_DIR}" + +echo "[${SCRIPT_NAME}] Building the documentation website with docs/build.sh" + +# `docs/build.sh` owns the dependency graph and Sphinx commands and moves to +# the docs directory itself. +uv run --package datafusion-docs ./docs/build.sh + +echo "[${SCRIPT_NAME}] Wrote the HTML documentation to docs/build/html/index.html" diff --git a/dev/rust_lint.sh b/dev/rust_lint.sh index 74d5fc16d7..dbb6454302 100755 --- a/dev/rust_lint.sh +++ b/dev/rust_lint.sh @@ -142,6 +142,7 @@ declare -a READONLY_STEPS=( "ci/scripts/check_circular_dependencies.sh|false" "ci/scripts/check_unused_dependencies.sh|false" "ci/scripts/rust_docs.sh|false" + "ci/scripts/check_docs_html.sh|false" ) for entry in "${WRITE_STEPS[@]}" "${READONLY_STEPS[@]}"; do diff --git a/docs/source/contributor-guide/testing.md b/docs/source/contributor-guide/testing.md index 89dc7969b3..4ba7d9d011 100644 --- a/docs/source/contributor-guide/testing.md +++ b/docs/source/contributor-guide/testing.md @@ -320,6 +320,19 @@ the README: [`dev/update_function_docs.sh`]: https://github.com/apache/datafusion/blob/main/dev/update_function_docs.sh [`ci/scripts/check_generated_docs.sh`]: https://github.com/apache/datafusion/blob/main/ci/scripts/check_generated_docs.sh +## Documentation HTML Build + +[`ci/scripts/check_docs_html.sh`] builds the documentation website, the same +build the "Test doc build" job runs. Sphinx builds with `-W`, so one warning +fails the check. `./dev/rust_lint.sh` runs it. Run the script with `--help` +for the tools it needs and the files it writes. + +```shell +./ci/scripts/check_docs_html.sh +``` + +[`ci/scripts/check_docs_html.sh`]: https://github.com/apache/datafusion/blob/main/ci/scripts/check_docs_html.sh + ## Benchmarks ### Criterion Benchmarks --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
