On 30 April 2018 at 06:56, Hervé BOUTEMY <[email protected]> wrote: > Le lundi 30 avril 2018, 03:19:20 CEST sebb a écrit : >> On 29 April 2018 at 22:11, Hervé BOUTEMY <[email protected]> wrote: >> > better than a discussion: a demo >> > >> > here is the website: >> > https://asf-attic.github.io/ >> > >> > its source: https://github.com/asf-attic/asf-attic.github.io/tree/source >> > its output: https://github.com/asf-attic/asf-attic.github.io/tree/master >> > >> > the only thing that is not included here is the code for the CI to >> > checkout >> > the output branch and update content after rebuild >> >> All I see is two disjoint branches. > yes, that's what it's about > >> It's not obvious how a user is supposed to update the master from the >> source. > people who know GitHub pages know: that's now part of common culture for many > people
So how do people update the site? Is it enough to edit the source branch and commit the change? > and from an operational point of view, that's why in general there is a CI > server doing the official build from source branch then committing output to > output branch Is that part of Github's offering? >> >> The build.sh script can be used to create a docs/ tree, but then what? > when you build locally, it's just for yourself, because you want to check a > change that is not as trivial as usual > then the change in output branch is usually done by a CI server > > When used with Maven, there is scm-pulbish plugin for that: it's a common > convention, and that's what is used by Apache Cayenne for example. > > When used with Jekyll or anything else as rendering engine, I don't know how > people script the output commit: Apache Freemarker just tells in their > documentation "To publish the built site, commit the output into the "asf- > site" branch". And Apache Accumulo writes a serie of shell commands that they > tell are launched by a Git hook (and not a CI server). AFAICT that only commits the change to the asf-site branch; it does not push the updated branch. > And again, in general, there is a central official setup does the job in a > central and official way, to avoid subtle differences when rendering from > multiple personal configurations > >> >> > But what is done here with GitHub GitPubSub equivalent can be done exactly >> > the same way at Apache Software Foundation I'm not disputing that, but if I am to amend the Buildbot to mimic the GitHub behaviour I need to know what it does. >> > >> > Regards, >> > >> > Hervé >> > >> > Le dimanche 29 avril 2018, 19:13:43 CEST Hervé BOUTEMY a écrit : >> >> Le dimanche 29 avril 2018, 14:46:09 CEST sebb a écrit : >> >> > On 29 April 2018 at 11:33, Hervé BOUTEMY <[email protected]> wrote: >> >> > > Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit : >> >> > >> On 29 April 2018 at 09:41, Hervé BOUTEMY <[email protected]> > wrote: >> >> > >> > first, I want to reassure everybody: this is a discussion, to get >> >> > >> > common >> >> > >> > knowledge of how things work in other projects then may work in >> >> > >> > the >> >> > >> > future for Attic if we decide to do an equivalent setup >> >> > >> >> >> > >> +1 >> >> > >> >> >> > >> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit : >> >> > >> >> On 28 April 2018 at 12:48, sebb <[email protected]> wrote: >> >> > >> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY >> >> > >> >> > <[email protected]> >> >> >> >> wrote: >> >> > >> >> ... >> >> > >> >> >> >> > >> >> >> In Git, this would naturally be in a separate branch named >> >> > >> >> >> "asf-site" >> >> > >> >> >> >> > >> >> How would that work for Attic? >> >> > >> >> >> >> > >> >> Where would the source files used to generate the site be held? >> >> > >> > >> >> > >> > There are multiple ways of doing, and GitHub documented it as >> >> > >> > clearly >> >> > >> > as >> >> > >> > possible [2] (yes, what we do at ASF with GitPubSub is exactly >> >> > >> > what >> >> > >> > GitHub calls "GitHub Pages", with marketing bells turned on and >> >> > >> > technical >> >> > >> > details on the build solution turned off) >> >> > >> > >> >> > >> > The 2 common ways are: >> >> > >> > 1. publish html from separate branch (which would be by default >> >> > >> > "asf-site" >> >> > >> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a >> >> > >> > subdirectory >> >> > >> > on master branch (you see Attic current pattern?) >> >> > >> > >> >> > >> > I find the first option a lot more clear from a build+scm >> >> > >> > perspective >> >> > >> > than >> >> > >> > the second one. This will avoid the exact same discussion we have >> >> > >> > currently at Attic with svn to know who commits the generated >> >> > >> > content >> >> > >> > (& >> >> > >> > when as a consequence): - CI after source-only commit? >> >> > >> > - or user who builds on his machine then commits simultaneously >> >> > >> > source >> >> > >> > and >> >> > >> > generated content? >> >> > >> > >> >> > >> > Then looking at ASF gitwcsub configuration [2], I had a look at >> >> > >> > many >> >> > >> > ASF >> >> > >> > projects: the 2 ways are used. I picked Cayenne [3] case to show a >> >> > >> > case >> >> > >> > where: >> >> > >> > - master branch is a source branch, with markup and a build script >> >> > >> > - asf-site branch is a completely separate branch that contains >> >> > >> > generated >> >> > >> > html It uses Maven scm-publish plugin to update asf-git branch >> >> > >> > with >> >> > >> > generated html [6] >> >> > >> > >> >> > >> > But there is also Freemarker [4], that has a simple README telling >> >> > >> > "To >> >> > >> > publish the built site, commit the output into the "asf-site" >> >> > >> > branch". >> >> > >> > >> >> > >> > Or Accumulo [5] which uses Jekyll and has some instructions to >> >> > >> > publish >> >> > >> > generated output to asf-site branch with a git-hook that I don't >> >> > >> > fully >> >> > >> > understand, but that maybe Attic members will prefer since it >> >> > >> > seems >> >> > >> > it's >> >> > >> > more the common culture here >> >> > >> > >> >> > >> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish plugin >> >> > >> > used >> >> > >> > by >> >> > >> > Cayenne, and I use it in many projects, initially with svn as >> >> > >> > target >> >> > >> > source control (in 2012, for svnpubsub & Apache CMS) then with git >> >> > >> > also, >> >> > >> > when GitHub pages became popular. But I see that it's not the >> >> > >> > right >> >> > >> > choice at Attic because it's not the most common Attic culture. >> >> > >> >> >> > >> Attic does not produce source code. >> >> > >> The only output from its SCM is the website. >> >> > > >> >> > > yes, like any other website that I showed: source code here is a >> >> > > markup >> >> > > language (be it Markdown, xdoc, static content, or anything else) >> >> > > Attic is really exactly the same >> >> > > >> >> > > What makes Attic different is that Attic does not have any other repo >> >> > > for >> >> > > "programming" code: that's true, but does not change anything >> >> > > regarding >> >> > > site> >> >> > > >> >> > >> AFAICT, all the above examples have a branch which contains the >> >> > >> source >> >> > >> for building the website. >> >> > > >> >> > > yes, I explained I chose them exactly for that reason >> >> > > >> >> > > I can show you Airavata site, which is quite simple and did the other >> >> > > choice: https://github.com/apache/airavata-site >> >> > > >> >> > > source is in source, output is in content in the same branch >> >> > >> >> > This is equivalent to Attic. >> >> > >> >> > > it could have been: source in master branch, content in asf-site >> >> > > branch >> >> > > which is the most common setup in GitHub pages (= what people >> >> > > nowadays >> >> > > know a lot) >> >> > > >> >> > >> 1) The source is edited. >> >> > >> 2) Run the build script to create the output in a clean subdirectory >> >> > >> 3) Copy the subdirectory tree to the asf-site branch >> >> > >> 4) commit the asf-site branch >> >> > >> 5) The entire asf-site branch is then published via pubsub. >> >> > >> >> >> > >> What Attic does currently is: >> >> > >> 1) & 2) as above >> >> > >> 3) commit the changes >> >> > >> 5) as above >> >> > >> >> >> > >> i.e. there is no need to copy the generated output anywhere because >> >> > >> it >> >> > >> is part of the same repo. >> >> > >> >> >> > >> This works because svnpubsub is set up to get its source from the >> >> > >> docs/ subdirectory >> >> > > >> >> > > yes, the setup with source and output in the same svn repo or Git >> >> > > branch >> >> > > makes it simple to checkout, but it mixes 2 types of files (source >> >> > > and >> >> > > generated) >> >> > > >> >> > > separating source and generated in 2 separate locations (separate svn >> >> > > root >> >> > > or different branches in the same git repo) makes things more clear, >> >> > > at >> >> > > the cost of an extra step to check out the generated content then >> >> > > update >> >> > > with the updated content >> >> > >> >> > The workspace still contains both source and generated output in the >> >> > examples I have seen. >> >> > >> >> > I assume it is ignored by SVN/Git so does not get committed or show up >> >> > as a local change. >> >> >> >> I showed you Cayenne, Freemarker and Accumulo that are not like this. >> >> Here we go back to Freemarker: >> >> - source: https://github.com/apache/freemarker-site >> >> - output: https://github.com/apache/freemarker-site/tree/asf-site >> >> >> >> > > yes: choose your issue >> >> > > personally, I prefer the second setup (clear but a little harder to >> >> > > setup) >> >> > > I don't like having mixed content in one repo (source and generated >> >> > > output) >> >> > > >> >> > > If everybody understands that these 2 setups a completely equivalent >> >> > > but >> >> > > really prefer the mixed one (just to avoid a second checkout), I'll >> >> > > let >> >> > > you >> >> > > go: I don't have any problem myself, I make a strong difference >> >> > > between >> >> > > source directory and output directory >> >> > >> >> > There's still mixed content in local workspaces unless you generate >> >> > the output in a separate tree. >> >> > >> >> > > But if people start to edit output directory instead of source (like >> >> > > it >> >> > > is >> >> > > so easy to do in the mixed content setup), you're at risk >> >> > >> >> > It's also possible to checkin the generated output if it's not >> >> > properly ignored in a 2 branch version. >> >> > And then wonder why the site does not get updated. >> >> >> >> that's why in general there is a .gitignore or svn:ignore that is >> >> properly >> >> configured >> >> >> >> > >> I don't know if gitpubsub can take its input from a subdirectory of >> >> > >> a >> >> > >> branch. >> >> > > >> >> > > it can: see Airavata site >> >> > >> >> > Ah - I see now. >> >> > >> >> > The webserver defines the site to be under content/, so the branch can >> >> > contain other files in parallel directories. >> >> > >> >> > >> If not, then we will have to change strategy in order to use Git. >> >> > >> Otherwise, we have a choice. >> >> > > >> >> > > we have a choice >> >> > >> >> > Yes. >> >> > >> >> > I prefer the status quo, not least because it involves fewer changes >> >> > (I think only renaming docs/ to content/ if we move to Git). >> >> > Using multiple repos would involve updating instructions as well. >> >> >> >> no, it's not multiple repos but multiple branches of the same repo >> >> >> >> > But if the majority want to change I won't object. >> >> >> >> since you are the guy who does the buidbot script, that does the commit, >> >> you'll have to be confident that you can code the multi-branch option >> >> >> >> > > Regards, >> >> > > >> >> > > Hervé >> >> > > >> >> > >> > Regards, >> >> > >> > >> >> > >> > Hervé >> >> > >> > >> >> > >> > >> >> > >> > [1] >> >> > >> > https://help.github.com/articles/configuring-a-publishing-source-f >> >> > >> > or-> > >> > gi th >> >> > >> > ub-pages/ >> >> > >> > >> >> > >> > [2] >> >> > >> > https://github.com/apache/infrastructure-puppet/blob/deployment/mo >> >> > >> > dul >> >> > >> > es >> >> > >> > /g >> >> > >> > itwcsub/files/config/gitwcsub.cfg >> >> > >> > >> >> > >> > [3] https://github.com/apache/cayenne-website/ >> >> > >> > >> >> > >> > [4] https://github.com/apache/freemarker-site >> >> > >> > >> >> > >> > [5] https://github.com/apache/accumulo-website >> >> > >> > >> >> > >> > [6] >> >> > >> > https://maven.apache.org/plugins/maven-scm-publish-plugin/various-> >> >> > >> > >> > >> > tip >> >> > >> > s. >> >> > >> > ht >> >> > >> > ml#Git_branch > >
