nchammas opened a new pull request, #44680: URL: https://github.com/apache/spark/pull/44680
### What changes were proposed in this pull request? Upgrade to Sphinx 4.5.0, which is the [latest in the 4.x line][1] and includes the [fix for parallel builds on macOS][2]. Enable parallel Sphinx workers to build the Python API docs. I experimented with a few different values, and `auto` seems to work best. Configuring 4 workers seems to yield the same improvement as `auto`, suggesting parallelization beyond that is ineffective due to some sort of resource contention. But I left it as `auto` since that's more dynamic and may work better on CI. On my 16-core Intel workstation, the runtime of `make html` was cut by ~60%. ```sh # `make html` @ master real 43m51.167s user 41m43.526s sys 0m39.651s # `make html` with parallel workers real 17m8.424s user 174m42.051s sys 5m8.824s ``` [1]: https://www.sphinx-doc.org/en/master/changes.html#release-4-5-0-released-mar-28-2022 [2]: https://github.com/sphinx-doc/sphinx/pull/9793 ### Why are the changes needed? This saves developer time (and may also save CI time). ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? I manually built and reviewed the docs using: ```sh SKIP_SCALADOC=1 SKIP_SQLDOC=1 SKIP_RDOC=1 time bundle exec jekyll build ``` ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
