aaronmarkham opened a new pull request #11990: [MXNET-744] Docs build tools update URL: https://github.com/apache/incubator-mxnet/pull/11990 ## Description ## While trying figure out how to debug the Sphinx website theme for #11916, I found the build tools frustratingly slow and nearly impossible to use for problems related to a full site build with the versions dropdown. So, I rewrote parts of it so you can do amazing and practical things like: :star: Decide what documentation sets to build on a per version basis! :star: Speed up the full version site build by 2.5x (was 43 minutes, now 17 minutes)! :star: Make incremental front-end site updates and build 14.3x faster (was 43 minutes, now 3 minutes)! :star: Optimize Sphinx configurations and pass these updates to the older versions of the site! :star: Not want to rage quit every time the docs build fails 40 minutes in! :star: Actually run the R docs build! I'm pretty stoked. I hope you are too. ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [x] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) created (except PRs with tiny changes) - [x] Changes are complete (i.e. I finished coding on this PR) ### Preview I'm actively running tests here, but you can see the current output (probably): http://34.201.8.176/ ### Changes ### :boom: Each version build happens in its own folder. This lets the MXNet build be cached if there's no update to the library. Big win for time savings here. Tradeoff is it takes up more space. :boom: When you build Sphinx docs with `make html` you can pass a new param, `BUILD_VER={tag}`, like `BUILD_VER=1.2.0`, and it'll load the settings according to that version :boom: Settings per version? Yes, in a new file at `/docs/settings.ini` we now have configs for each version and each document set. I could have also used the Makefile, but this seemed to be better for handling all of the version support variations. You can configure the build to build what you want, and more importantly, turn it off for a version where it isn't possible... like for Clojure and any previous version tag. Only the most recent version or master or your fork might have it. :boom: What's this about your fork? Yes, you can now build docs with your own fork and output different branches of your fork to represent the different version on the website output. :sparkles: Cool, right? :sparkles: Gawd, why didn't I do this earlier!? And I tweaked the css a bit so the h1-h4 tags pop a little more. The mxnet.css file gets pulled from the artifacts folder now, but we can alter this design down the line. ## Usage ## ### Building Docs with Sphinx From the `docs` folder: You can just use `make html` and it'll load the defaults which will build everything: ``` [document_sets_default] clojure_docs = 1 doxygen_docs = 1 r_docs = 1 scala_docs = 1 ``` Or you can use a specific version and it'll load the settings for it: ``` make html USE_OPENMP=1 BUILD_VER=1.0.0 ``` From `settings.ini`: ``` [document_sets_1.0.0] clojure_docs = 0 doxygen_docs = 1 r_docs = 0 scala_docs = 0 ``` ### Building the Full Site with Your Fork The previous example is a precursor to the more complicated process of building the full site and its many versions. The scripts used by CI and that can be used yourself for building dev version of the site are in `build_version_doc` and are still `build_all_version.sh` and `update_all_version.sh`. Now they take optional params for your fork. #### Building Each Version and Optionally Using Your Fork Build the content of the 1.2.0 branch in the main project repo to the 1.2.1 folder: ``` ./build_all_version.sh "1.2.0" "1.2.1" ``` Using the main project repo, map the 1.2.0 branch to output to a 1.2.1 directory; others as is: ``` ./build_all_version.sh "1.2.0;1.1.0;master" "1.2.1;1.1.0;master" ``` Using a custom branch and fork of the repo, map the branch to master, map 1.2.0 branch to 1.2.1 and leave 1.1.0 in 1.1.0: ``` ./build_all_version.sh "sphinx_error_reduction;1.2.0;1.1.0" "master;1.2.1;1.1.0" https://github.com/aaronmarkham/incubator-mxnet.git ``` #### Updating Each Version and Optionally Using Your Fork Assuming you build 1.2.1, 1.1.0, 1.0.0, and master, you need to inject the versions dropdown to reflect these options for each page on the site: ``` ./update_all_version.sh "1.2.1;1.1.0;1.0.0;master" master http://mxnet.incubator.apache.org/ ``` It doesn't use your fork at this point, but it will pull the latest project README.md from your current branch and use that for the root of the output site. ## Comments ### Building R Docs I'm not adding the R deps and making CI changes here. That's just too much in one PR. However, in a followup PR, or if you want test the pipeline with the R docs build turned on you need the following in addition to the R deps already in the docs setup scripts in CI or the one in `build_version_doc`: ``` sudo apt-get install \ texinfo \ texlive \ texlive-fonts-extra \ ``` You would also want to change `settings.ini` and enable the build of the R docs (per version you want). ### Building Scala Docs The current Scala config is incompatible with version 1.1.0 or earlier. You'll get an error about the namespace change. I'm disabling the build for this in those earlier versions in the provided `settings.ini`. Please discuss with me if this is a bad choice and if we need to use a custom config for those earlier versions. The function in mxdoc.py for scaladocs could use the new version param, and make some conditional statements for how it builds...
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
