Hi David, thanks, I see your points, I have just a few acknowledgements/comments to add
On Tue, Dec 21, 2021 at 7:42 PM David Jencks <david.a.jen...@gmail.com> wrote: > > Hi Zoran, inline after extracting a couple of concrete proposals. > > 1. I’m going to open a website issue and PR for publishing the site manifest > with the site, to enable immediate quick local “xref check” builds without > needing to build the full site. +1 > 2. I’m going to open a website issue proposing checking in the build UI > bundle. This is a different solution to the problem than PR#537[1], and is > certainly open to debate. +1 I see the difference between PR#537 and what you're suggesting; and the convenience build/setup wise your proposal brings, in the end a `raw` url from GitHub will give us the same advantage of publishing the artifact as proposed in PR#537. So let's give it a try and see if we actually do notice any downsides. > 2. I’m going to open a website issue to move the site publishing to a > separate repo +1 if we're arguing that the website source repository should be lighter to checkout; the keeping of history I'm a bit hesitant about because of the size increase of the git repository with every/most changes > 3. I’m going to start on a PR for the user manual to document how to do local > builds. +1 thanks! > My comments on why below... One more comment inline asking for other folk on the ergonomics of documenting the workflow, it'd be nice to hear from others > > On Dec 21, 2021, at 1:55 AM, Zoran Regvart <zo...@regvart.com> wrote: > > > > Hi David, > > > > On Mon, Dec 20, 2021 at 10:23 PM David Jencks <david.a.jen...@gmail.com> > > wrote: > >> > >> (Mostly general info, everyone please read) > >> > >> Hi Antonin, > >> > >> Sorry, somehow I missed this earlier! > >> > >> This work should help with that issue. I’m finding it really great for > >> stuff I do, it’s also really easy to adapt to create a website PR for > >> integrating lots of branch changes, cf. > >> https://github.com/apache/camel-website/pull/729 > >> <https://github.com/apache/camel-website/pull/729>. > >> > >> I’m going to be working on my “referential integrity” idea next (unless > >> more documentation fires break out :-). This will touch every active > >> branch of every subproject, and unless someone objects I’m going to > >> install the local-build stuff everywhere to make it easier for myself :-), > >> so it should soon be pretty easy for everyone to try it on their favorite > >> parts of camel. > > > > +1 this is great work and it'll help us in the long run. I think > > you're doing it carefully and conscientiously taking into account not > > to be disruptive. Did we already update the README's, so folk > > understand how to use `local-build.sh`? > > Not yet, but I think this is going well enough to assume we'll keep it. I > think the best will be if I have detailed instructions in the “working on the > website” page in the user manual and very short directions in each subproject. > > Would it be a good idea to make the local instructions named so they are > obviously instructions, maybe README_local_build.adoc? I never know what to > expect from a README :-) Any thoughts from others, I think it's important that folk find the instructions where they'd expect them first > > > >> It’s not a problem for me since I basically only work on the website, but > >> I would imagine two major annoyances of the current system for everyone > >> else are: > >> > >> (1) You have to build the full site at least once, which takes annoyingly > >> long (7 minutes?). > >> (2) You have to build the UI locally, which is tricky on non-linux systems. > >> > >> For (1), building the site manifest as part of the regular website build > >> so the latest can always be downloaded would make it so you can always do > >> a quick local partial build, although you won’t get a very usable local > >> website: just the local stuff will be there locally. It would detect the > >> same errors as a the current local build, but not xref problems into the > >> local changes: this (currently) requires a full build. > > > > Still, I think this is a great step forward, we can detect some issues > > early and quickly, I'm sure we can improve to detect most issues in > > subsequent refinements > > I think there are 3 cases for local builds: > > 1. You just want to check for errors as quickly as possible. I think this is > what most people want to do most of the time. We might even be able to get > maven to run this. Having the site manifest served from the live site (or > some other location) is needed for this. > 2. You are actually working on the docs and want a quick continuous build of > a subproject, and don’t mind having to do a full build first (maybe 7-8 > minutes). This is what the current `./local-build.sh` does, after a > `./local-build.sh full`. This detects xref problems out of the subproject but > not into the subproject. > 3. You want to check for all xref and other problems with your local changes. > This is what the current `.local-build.sh full` does. > > We don’t support (1) yet, likely by far the most common case. So, I propose > we start by publishing the site manifest. I can’t see any downside…. we can > always stop. > > > > >> For (2), one simple option is just checking the built UI bundle into git. > >> > >> A further annoyance that I experience is that the built website is checked > >> into a branch of camel-website. This means whenever you pull > >> camel-website, you get the latest published website which is a lot of > >> changes, along with the tiny changes that might or might not have occurred > >> since your last update. > > > > Nicola touched upon this in PR#537[1], would that be a better option? > > #537 is about publishing the UI bundle, not whether the asf-site branch to > publish the website is in the same repo as the antora and hugo build > machinery. > > I think that #537 would be materially less convenient and slower and harder > for everyone. It’s definitely a possibility. Here’s my comparison, checking > in bundle vs. publishing bundle > - The main build can continue to rely on the yarn workspace coordinates for > the bundle, as can local builds… vs…. either the main build and local builds > use different playbooks or the main build has to deal with getting the > released bundle. > - A camel-website clone or pull/update will come with the latest (or at least > recent enough) UI for local builds… vs… an additional fetch operation is > needed for all builds, which is slower for everyone. > > However, IMO some way of doing a local build without having to locally build > the UI bundle is needed. I think by far the easiest is to check in the built > bundle, but I don’t really care how we do it. > > > > >> I would like us to move the published website to a different repo. This > >> would also allow us to keep the website history, which would partly solve > >> the problem of docs for outdated component versions that we’ve dropped > >> from the active website. I also think from some comments about the old > >> svn site publishing system that infra expects website history to be > >> preserved, although I haven’t asked explicitly. I’ve set this up in a > >> couple other projects, and it’s pretty easy to do. > > > > I've chatted with INFRA a while back about the size of the git > > repository that was causing issues with the publish, and the > > workaround for that issue was to do what we do now and squash the > > commits before pushing. If we preserve history I think we'll end up > > with the same issue again. I don't understand the need of having the > > history of the website build result preserved, given that the output > > is optimized, changes can be seen much clearer and easier in the > > website sources. And I don't think there is much difference in having > > a separate repository vs a separate branch for publishing. > > > > My main motivation for proposing this now is that we’re going to have > everyone working on the docs cloning or pulling the camel-website repo, and > no one needs the publishing branch. Even if there are no changes in the > antora or hugo stuff, there’s still a several-second download of the latest > site content. This is, at least to me, annoying and confusing. I’d like it > if, when there are no changes to the stuff that builds the site, nothing is > pulled: I can be confident that I don’t have to investigate anything further. > Therefore I want to move the published website to a separate repo. > > If we do this, we have the option of keeping history or continuing to squash > it. > - Either way, we only need to get the repo at depth 1, with no history. > Therefore I think the git traffic and speed would be the same. Does Jenkins > have some caching that invalidates this argument? > - We’ve discussed somewhere the problem that we want to keep the website > build reasonable fast but by pruning old doc versions we make old releases > undocumented. If we keep the website history then one solution to this is to > check out an appropriate vintage commit and look at it with a local httpd. > There’s no practical way to build an old version of the site, and this would > give the docs as they existed. Something else we might do is to tag > published site commits just before we remove a doc version. Even if we > squashed history, this would keep those specific versions available. If we > kept history, tags would provide some way to find the revision you are > interested in :-). > > So, I want to move the publishing to a separate repo to make local builds > quicker: doing so also provides room for discussing keeping history, which > might or might not be a good idea. > > > 2c > > > > zoran > > > > [1] https://github.com/apache/camel-website/pull/537 > > -- > > Zoran Regvart > -- Zoran Regvart
