The delays getting the Ref Guides for 8.x releases out have caused me to think a bit about the Ref Guide publication process. It seems clear others aren't able to pick up the process when I can't and I’m sure there are a million individual reasons for that so I don't intend to shame or blame anyone, but a process that relies on a single person in a community our size isn’t a very good one. And, if I think about _why_ we have a process like we have today [1], I’m not sure it makes a ton of sense any longer.
So, I propose making some radical changes. My ideas here require a shift from thinking of the Guide as a release artifact like the binaries to thinking of it similar to how we treat javadocs. These ideas also allow us to finally get to the goal of unifying these currently separate processes. 1. Make the HTML version the “official” version. -- What to do with the PDF is TBD after that decision, see below. 2. Stop voting for the Ref Guide release as a separate VOTE thread. 3. Jenkins jobs are already created when a release branch is cut. We can change these jobs so they always automatically push the HTML version to the website, although before the version binaries are released the pages would still have a DRAFT watermark across them [2]. -- By ASF policy, release artifacts must be produced on a machine controlled by the committer. However, the point here is that the Ref Guide would no longer be a release artifact, so I think that gets us around that rule? If anyone sees this differently that would change things here a little bit. -- I know other projects have similar Jenkins->publish workflows, but I’m not sure exactly what’s involved in setting it up. Might need to discuss with the Infra team and other changes may be required depending. -- The goal, though, is to automate this as much as possible. 4. When a VOTE has passed, a simple step could be added to the release process to run a Jenkins job to regenerate the HTML pages without the current DRAFT watermark and automatically push them to the production website. -- Since we usually leave branch jobs configured-but-disabled for a little bit in case a patch release is necessary, typos or other things fixed “post-release" could be fixed and the Ref Guide Jenkins job would just push new commits to the branch to the live production site. -- These updates would be done without the DRAFT status, since the Ref Guide in that branch is now considered “live”. -- This part of the idea would allow us to more easily backport any docs changes and re-publish the Guide without having to do a new vote, which we would need today. This might be rare, but it is a question that comes up from time-to-time. I feel that if the publication process was easier, we might fix things retroactively more often. 5. Some tooling would be nice to automate parts of the copy edit process I do today, so it can be run by any committer at any point in the process. This can follow on later. I have a list. So, that's the idea in a nutshell - thoughts? [1] Current release process: https://lucene.apache.org/solr/guide/8_1/how-to-contribute.html#ref-guide-publication-process [2] Example of DRAFT watermark (it's all CSS, it could look however we want it to): https://builds.apache.org/view/L/view/Lucene/job/Solr-reference-guide-8.x/javadoc/ PS, As for the PDF, I believe there are mixed opinions about it. Some rely on it, others only use it when they need it (portability, easier to search, etc.), others don’t ever look at it. The fact is it’s over 1600 pages, and that’s just really too big. Joel is about to add a significant number of new images as part of a new "visual" guide (see SOLR-13105), which will make it even longer and bigger. Trying to split it to make it smaller would bring in a lot of complexity with how to deal with links between pages that end up in different PDF files (believe me, I've done it before). And finally, it holds us back a little - some things we could do with HTML/JS can't be done in PDF. I’d be fine continuing to produce it, just not as our main artifact. We could have Jenkins push that also to the SVN dist/dev repo where it currently lives. Cassandra
