We're still polishing up the README and other changes in the latest version of that PR, through some iterations of code review, but I think we'll be ready to merge it soon. That will provide one more option for people to do jekyll development locally for the website.
Dave also had some reservations about removing the staging site (his last comment was a bit ambiguous "IIRC the local build is likely sufficient"). Before I do anything to streamline website publication by removing the mandatory staging site, I would like consensus from everybody who had reservations. So, Dave, what do you think about having the Docker build as an option for local staging? Would that be enough for you to get on board with removing the mandatory staging site step? On Thu, Apr 20, 2023 at 12:01 AM Christopher <ctubb...@apache.org> wrote: > > I reviewed your PR. It includes changes that are likely to break in my > dev environment (the gem updates, due to errors building the native > extensions for transitive dependencies in my environment) and make it > harder for me to finish the 1.10.3 release changes, instead of make it > easier... which is what my original goal was in streamlining this. I'm > okay with having a Dockerfile... but the other changes are probably > going to cause more problems for us in the short term than they solve. > > On Mon, Apr 17, 2023 at 6:11 PM Daniel Roberts <ddani...@gmail.com> wrote: > > > > Christopher, > > > > I added a containerized dev environment in #384 which uses Jekyll's polling > > option to dynamically pick up changes. > > > > Based on that change being included, I'm ok with removing the staging site > > as PRs can be reviewed locally with minimal concern for dev environment > > drift. > > > > Thanks, > > > > - Dan > > > > On Mon, Apr 17, 2023, 4:07 PM Christopher <ctubb...@apache.org> wrote: > > > > > Keith - If we made the change, the README would still exist, but it > > > would get a bit smaller, as there would be fewer mandatory things to > > > do. > > > > > > Dan - Streamlining would just remove the staging site as a mandatory > > > step. It will still remain an optional step, following steps similar > > > to what you describe for feature branches. It's already available as > > > an optional step today for feature branches, but we haven't had a need > > > to use it that way. For a long time, staging sites weren't even an > > > option, and we got along just fine without them. Like Keith, I can't > > > think of a single time a staging site played any kind of influential > > > role. But still, the feature would still be available after > > > streamlining... just optional instead of mandatory. > > > > > > Regarding link validation, I agree that *might* be a useful QA build > > > step (depending on how much effort it is, return on value it provides, > > > how it's implemented, false positives due to network conditions and > > > outages, etc.). But, I think that's a separate issue, and we can look > > > into options to run some checks in the GitHub Actions, or as a > > > periodic Jenkins job that crawls our website looking for broken links, > > > independently from the streamlining proposal. > > > > > > On Mon, Apr 17, 2023 at 1:33 PM Daniel Roberts <ddani...@gmail.com> wrote: > > > > > > > > Christopher, > > > > > > > > In your breakdown of my proposal, I think there's a miscommunication in > > > > regard to the number of build steps and branches needed to maintain. > > > > > > > > Looking back I should have used descriptors like Branch C, Branch S, > > > Branch > > > > P vs main and staging due to the collision with the current branch names > > > in > > > > use. > > > > > > > > Feature branches would feed into staging and staging would be a direct > > > > merge into the site branch for a total of three branches. One of those > > > > branches is deleted after merge, resulting in two long-lived branches. > > > One > > > > for the staging site, and one for the production site. > > > > > > > > The number of build job runs should remain exactly the same as our > > > current > > > > implementation. > > > > The only change would be an additional PR and the migration of the > > > publish > > > > operation from a local action to a github deploy action. > > > > > > > > Unless link paths need to be regenerated for the new site location, > > > > there > > > > shouldn't need to be an additional jekyll build run on the merge from > > > > the > > > > staging site to the production site branch. > > > > > > > > Since the additional PR is what would be used for formal review, the > > > amount > > > > of work generated for additional team members remains the same, with the > > > > benefit that changes are deployed directly to the production site once > > > the > > > > review is complete. > > > > > > > > I agree with the downside of merge commits in my proposal and > > > > that fast-forward merges are better in terms of git history management. > > > > > > > > While this could be solved with changing the merge strategy of the > > > > repository, I understand the desire to not involve the infra team. > > > > > > > > However, the majority of change tracking problems that come from extra > > > > merge commits being introduced don't exist in this scenario as multiple > > > > versions of the website aren't supported. > > > > > > > > My personal opinion is that fast-forward merges do not provide enough > > > > benefit in this scenario to outway a build pipeline that eliminates > > > > maintaining a local development environment for site updates. > > > > > > > > > > > > *Current thoughts* > > > > I agree with the desire to streamline the build pipeline and move the > > > > production publish stage from a manual step to an automated build > > > > action. > > > > > > > > Having a staging area is useful in the event any large scale changes > > > > need > > > > to be made to the site like major rendering version updates. But I do > > > agree > > > > that it provides little benefit when introducing minor changes outside > > > > of > > > > the possibility of discovering link/navigation issues. > > > > > > > > Regardless of what we do with the staging site, I don't think the answer > > > is > > > > do nothing. > > > > I think one of the things we've identified in this discussion is the > > > > need > > > > for link validation to run in our QA build step. > > > > > > > > That seems to be the most common bug that makes it into the production > > > site > > > > so identifying and eliminating those errors reduces the need for staging > > > to > > > > exist. > > > > > > > > - Dan R. > > > > > > > > On Mon, Apr 17, 2023 at 11:22 AM Keith Turner <ke...@deenlo.com> wrote: > > > > > > > > > On Wed, Apr 12, 2023 at 12:34 PM Christopher <ctubb...@apache.org> > > > wrote: > > > > > > > > > > > > I don't think Dan's suggested flow is much different than what we > > > have > > > > > > now. In both cases, the steps are: > > > > > > > > > > > > 1. Initial PR to a branch (currently against main; Dan's approach > > > > > > would be against a separate main-like staging branch) > > > > > > 2. Auto build on PR merge (currently, that's main -> asf-staging; > > > > > > Dan's approach would build staging -> asf-staging) > > > > > > 3. View on staging site to evaluate for needed changes (auto-publish > > > > > > of asf-staging branch) and repeat steps 1-3 > > > > > > 4. Then decision to publish (currently, that's fast-forward git > > > > > > merge > > > > > > from asf-staging to asf-site branch; in Dan's approach, it would be > > > > > > a > > > > > > merge from the staging branch to the main branch, which triggers > > > > > > another Jekyll build to the asf-site branch) > > > > > > 5. View on published site to evaluate for needed changes and repeat > > > > > steps 1-5 > > > > > > > > > > > > In Dan's approach, we would be maintaining two source branches and > > > two > > > > > > build branches, and there's an extra Jekyll build, and the > > > possibility > > > > > > of introducing extra merge commits if we fail to do a FF merge from > > > > > > the staging branch to the main branch. > > > > > > > > > > > > Overall, I don't think the approach gets us anything we don't > > > > > > already > > > > > > have, and it uses a bit more resources. > > > > > > If we really need the extra staging step, then I'd prefer to keep > > > > > > the > > > > > > current flow. > > > > > > > > > > > > For comparison, the streamlined process I'm proposing is: > > > > > > > > > > > > 1. Initial PR to main branch > > > > > > 2. Auto build and publish on PR merge > > > > > > 3. View on published site to evaluate if additional changes need to > > > be > > > > > > made and repeat steps 1-3 > > > > > > > > > > > > We could delete the asf-staging branch, operating only with the main > > > > > > and asf-site branches. In all cases, current and proposed, there are > > > > > > alternate ways to view/evaluate a change: > > > > > > > > > > > > 1. As rendered Markdown in the GitHub PR (navigation and Jekyll > > > > > > templated code won't work) > > > > > > 2. As local build (requires user to set up a local environment) > > > > > > 3. As built artifacts from GitHub Actions (slow download; best > > > > > > viewed > > > > > > in a local webserver, but can also be viewed directly in a browser, > > > > > > with some navigation limitations) > > > > > > 4. Temporarily create a new staging branch to stage to a separate > > > > > > named staging site > > > > > > 5. Just view it published at the main site and fix it in a > > > > > > subsequent > > > > > > change because it's fine if the site is broken slightly for a brief > > > > > > moment (we can always revert the change if a fix isn't quick) > > > > > > > > > > > > So, to recap the positions of this discussion's participants: > > > > > > > > > > > > - Chris and I are in favor of streamlining by getting rid of the > > > staging > > > > > step. > > > > > > - Dave and Dan were interested in leaving in a staging step. > > > > > > - Dan suggested an alternate staging flow. > > > > > > > > > > > > Unless others weigh in to lean one way or another, or Dave or Dan > > > > > > switch their opinions, it's kind of a toss-up (no consensus), and I > > > > > > guess we'll just keep things as they are now. > > > > > > > > > > Currently the website README makes it really easy to know what to do > > > > > and its really quick to follow the instructions. So I do not see > > > > > staging as a burden. > > > > > > > > > > I do not recall an instance where I discovered an error in staging > > > > > before updating. Staging does give us a buffer to prevent us from > > > > > breaking the website, but I think if we ever did break it that we > > > > > could also fix it quickly. > > > > > > > > > > Not seeing staging as a burden, but not having found it useful leaves > > > > > me slightly in favor of removing staging. If we didn't have such a > > > > > good README for the website I would be more strongly in favor of > > > > > removing staging. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Apr 9, 2023 at 9:56 AM Dave Marion <dmario...@gmail.com> > > > wrote: > > > > > > > > > > > > > > Christopher, > > > > > > > > > > > > > > IIRC the local build is likely sufficient. It may have been that > > > I > > > > > failed > > > > > > > to use it in all cases. I do remember in the past merging some > > > > > > > PR's > > > > > then > > > > > > > noticing on the staging site that the links or formatting were not > > > > > right. I > > > > > > > think Dan's suggested change to the git workflow might also be > > > useful > > > > > > > (Auto-deply to staging on merge to staging branch, auto-deploy to > > > > > apache.org > > > > > > > on merge to main). > > > > > > > > > > > > > > On Fri, Apr 7, 2023 at 10:03 PM Christopher <ctubb...@apache.org> > > > > > wrote: > > > > > > > > > > > > > > > Hi Dave, > > > > > > > > > > > > > > > > How often would you estimate the diff in the PR is insufficient, > > > and > > > > > > > > you need a full site build to get a sense of how the change is > > > > > viewed? > > > > > > > > I imagine this might come up with some longer blog posts with a > > > lot > > > > > of > > > > > > > > complex layout and references to docs, etc., but that doesn't > > > happen > > > > > > > > very frequently. I don't know if there are other times you feel > > > it > > > > > > > > would be helpful or how often those occur. > > > > > > > > > > > > > > > > Aside from local builds, are your concerns mitigated in any way > > > by > > > > > the > > > > > > > > following? > > > > > > > > > > > > > > > > 1. Code reviewers checking the proposed changes for > > > layout-related > > > > > > > > mistakes before merging, or > > > > > > > > 2. Subsequent commits to correct initial publication mistakes > > > would > > > > > > > > also be streamlined. > > > > > > > > > > > > > > > > (For reference, the local build is documented in the website > > > README > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/accumulo-website/blob/main/README.md#local-builds-for-testing > > > > > > > > ) > > > > > > > > > > > > > > > > On Fri, Apr 7, 2023 at 7:24 PM Dave Marion <dmario...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > > > > > I find the staging site useful for reviewing changes before > > > > > publishing. > > > > > > > > > There are some things that are not the easiest to review in a > > > > > markdown > > > > > > > > > diff. Especially since the markdown is transformed to what is > > > > > ultimately > > > > > > > > > displayed. I have no issue with streamlining this, I'll just > > > need > > > > > to > > > > > > > > > remember to be more careful. There is a way to deploy locally > > > > > IIRC, I'll > > > > > > > > > just have to use that. > > > > > > > > > > > > > > > > > > On Fri, Apr 7, 2023 at 6:34 PM Christopher Shannon < > > > > > > > > > christopher.l.shan...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > +1 to make the change. I agree that when PRs are merged we > > > > > generally > > > > > > > > just > > > > > > > > > > want to publish immediately. > > > > > > > > > > > > > > > > > > > > FWIW, this is how we do things for the ActiveMQ website > > > > > > > > > > <https://github.com/apache/activemq-website> and it works > > > fine, > > > > > we > > > > > > > > just > > > > > > > > > > publish and there is no staging. > > > > > > > > > > > > > > > > > > > > On Fri, Apr 7, 2023 at 3:49 PM Christopher < > > > ctubb...@apache.org> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Accumulo Devs, > > > > > > > > > > > > > > > > > > > > > > When I first set up the automation using .asf.yaml for > > > Jekyll > > > > > builds > > > > > > > > > > > to go to the asf-staging branch, I expected us to get a > > > > > > > > > > > bit > > > > > more use > > > > > > > > > > > out of the staging site at > > > https://accumulo.staged.apache.org/ > > > > > > > > > > > However, that really hasn't happened, and it seems that we > > > > > basically > > > > > > > > > > > almost always want to publish to the > > > > > https://accumulo.apache.org > > > > > > > > once > > > > > > > > > > > we've approved a PR against the main branch and have a > > > > > successful > > > > > > > > > > > Jekyll build. > > > > > > > > > > > Having the staging site is now just an unnecessary extra > > > step > > > > > to > > > > > > > > > > > publish (that we sometimes forget to do for a while, even > > > > > though it's > > > > > > > > > > > trivial to run ./_scripts/publish.sh). I'm not sure it's > > > > > adding any > > > > > > > > > > > value. > > > > > > > > > > > > > > > > > > > > > > Perhaps we should just have Jekyll build directly to the > > > > > asf-site > > > > > > > > > > > branch and get rid of the asf-staging branch? > > > > > > > > > > > > > > > > > > > > > > The one benefit to the staging site is that, in theory, > > > it's a > > > > > > > > > > > conscientious decision to publish. But, I don't think > > > > > > > > > > > we've > > > > > ever come > > > > > > > > > > > across any kind of situation where we wanted to stage the > > > > > site, but > > > > > > > > > > > then didn't immediately want to publish it. So, I'm not > > > > > convinced > > > > > > > > that > > > > > > > > > > > extra step is adding any value, especially since we're > > > almost > > > > > always > > > > > > > > > > > making the conscientious decision in GitHub when approving > > > a > > > > > PR to > > > > > > > > > > > update the site already. > > > > > > > > > > > > > > > > > > > > > > Any thoughts or opinions on this? I'm leaning slightly > > > towards > > > > > > > > > > > streamlining this. > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > Christopher > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
