Le lundi 30 avril 2018, 12:46:09 CEST sebb a écrit : > On 30 April 2018 at 06:56, Hervé BOUTEMY <[email protected]> wrote: > > Le lundi 30 avril 2018, 03:19:20 CEST sebb a écrit : > >> On 29 April 2018 at 22:11, Hervé BOUTEMY <[email protected]> wrote: > >> > better than a discussion: a demo > >> > > >> > here is the website: > >> > https://asf-attic.github.io/ > >> > > >> > its source: > >> > https://github.com/asf-attic/asf-attic.github.io/tree/source > >> > its output: > >> > https://github.com/asf-attic/asf-attic.github.io/tree/master > >> > > >> > the only thing that is not included here is the code for the CI to > >> > checkout > >> > the output branch and update content after rebuild > >> > >> All I see is two disjoint branches. > > > > yes, that's what it's about > > > >> It's not obvious how a user is supposed to update the master from the > >> source. > > > > people who know GitHub pages know: that's now part of common culture for > > many people > > So how do people update the site? > > Is it enough to edit the source branch and commit the change? yes, once a CI job is configured (like you do with Buildbot, and I do usually with Jenkins + Maven scm-publish)
> > > and from an operational point of view, that's why in general there is a CI > > server doing the official build from source branch then committing output > > to output branch > > Is that part of Github's offering? if you use the GitHub Jekyll, yes, the build is magic if you want your own build engine, it's up to you to do the CI setup > > >> The build.sh script can be used to create a docs/ tree, but then what? > > > > when you build locally, it's just for yourself, because you want to check > > a > > change that is not as trivial as usual > > then the change in output branch is usually done by a CI server > > > > When used with Maven, there is scm-pulbish plugin for that: it's a common > > convention, and that's what is used by Apache Cayenne for example. > > > > When used with Jekyll or anything else as rendering engine, I don't know > > how people script the output commit: Apache Freemarker just tells in > > their documentation "To publish the built site, commit the output into > > the "asf- site" branch". And Apache Accumulo writes a serie of shell > > commands that they tell are launched by a Git hook (and not a CI server). > > AFAICT that only commits the change to the asf-site branch; it does > not push the updated branch. perhaps instructions are not fully written: in general, a CI job does the work > > > And again, in general, there is a central official setup does the job in a > > central and official way, to avoid subtle differences when rendering from > > multiple personal configurations > > > >> > But what is done here with GitHub GitPubSub equivalent can be done > >> > exactly > >> > the same way at Apache Software Foundation > > I'm not disputing that, but if I am to amend the Buildbot to mimic the > GitHub behaviour I need to know what it does. sure :) it's about doing a git commit + push to the output branch, like it did previously the svn commit to svnpubsub location the demo on GitHub is just here to see it in a very concrete way, since GitHub provides the gitwcsub to publish from the branch to the web server > > >> > Regards, > >> > > >> > Hervé > >> > > >> > Le dimanche 29 avril 2018, 19:13:43 CEST Hervé BOUTEMY a écrit : > >> >> Le dimanche 29 avril 2018, 14:46:09 CEST sebb a écrit : > >> >> > On 29 April 2018 at 11:33, Hervé BOUTEMY <[email protected]> wrote: > >> >> > > Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit : > >> >> > >> On 29 April 2018 at 09:41, Hervé BOUTEMY <[email protected]> > > > > wrote: > >> >> > >> > first, I want to reassure everybody: this is a discussion, to > >> >> > >> > get > >> >> > >> > common > >> >> > >> > knowledge of how things work in other projects then may work in > >> >> > >> > the > >> >> > >> > future for Attic if we decide to do an equivalent setup > >> >> > >> > >> >> > >> +1 > >> >> > >> > >> >> > >> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit : > >> >> > >> >> On 28 April 2018 at 12:48, sebb <[email protected]> wrote: > >> >> > >> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY > >> >> > >> >> > <[email protected]> > >> >> > >> >> wrote: > >> >> > >> >> ... > >> >> > >> >> > >> >> > >> >> >> In Git, this would naturally be in a separate branch named > >> >> > >> >> >> "asf-site" > >> >> > >> >> > >> >> > >> >> How would that work for Attic? > >> >> > >> >> > >> >> > >> >> Where would the source files used to generate the site be > >> >> > >> >> held? > >> >> > >> > > >> >> > >> > There are multiple ways of doing, and GitHub documented it as > >> >> > >> > clearly > >> >> > >> > as > >> >> > >> > possible [2] (yes, what we do at ASF with GitPubSub is exactly > >> >> > >> > what > >> >> > >> > GitHub calls "GitHub Pages", with marketing bells turned on and > >> >> > >> > technical > >> >> > >> > details on the build solution turned off) > >> >> > >> > > >> >> > >> > The 2 common ways are: > >> >> > >> > 1. publish html from separate branch (which would be by default > >> >> > >> > "asf-site" > >> >> > >> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a > >> >> > >> > subdirectory > >> >> > >> > on master branch (you see Attic current pattern?) > >> >> > >> > > >> >> > >> > I find the first option a lot more clear from a build+scm > >> >> > >> > perspective > >> >> > >> > than > >> >> > >> > the second one. This will avoid the exact same discussion we > >> >> > >> > have > >> >> > >> > currently at Attic with svn to know who commits the generated > >> >> > >> > content > >> >> > >> > (& > >> >> > >> > when as a consequence): - CI after source-only commit? > >> >> > >> > - or user who builds on his machine then commits simultaneously > >> >> > >> > source > >> >> > >> > and > >> >> > >> > generated content? > >> >> > >> > > >> >> > >> > Then looking at ASF gitwcsub configuration [2], I had a look at > >> >> > >> > many > >> >> > >> > ASF > >> >> > >> > projects: the 2 ways are used. I picked Cayenne [3] case to > >> >> > >> > show a > >> >> > >> > case > >> >> > >> > where: > >> >> > >> > - master branch is a source branch, with markup and a build > >> >> > >> > script > >> >> > >> > - asf-site branch is a completely separate branch that contains > >> >> > >> > generated > >> >> > >> > html It uses Maven scm-publish plugin to update asf-git branch > >> >> > >> > with > >> >> > >> > generated html [6] > >> >> > >> > > >> >> > >> > But there is also Freemarker [4], that has a simple README > >> >> > >> > telling > >> >> > >> > "To > >> >> > >> > publish the built site, commit the output into the "asf-site" > >> >> > >> > branch". > >> >> > >> > > >> >> > >> > Or Accumulo [5] which uses Jekyll and has some instructions to > >> >> > >> > publish > >> >> > >> > generated output to asf-site branch with a git-hook that I > >> >> > >> > don't > >> >> > >> > fully > >> >> > >> > understand, but that maybe Attic members will prefer since it > >> >> > >> > seems > >> >> > >> > it's > >> >> > >> > more the common culture here > >> >> > >> > > >> >> > >> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish > >> >> > >> > plugin > >> >> > >> > used > >> >> > >> > by > >> >> > >> > Cayenne, and I use it in many projects, initially with svn as > >> >> > >> > target > >> >> > >> > source control (in 2012, for svnpubsub & Apache CMS) then with > >> >> > >> > git > >> >> > >> > also, > >> >> > >> > when GitHub pages became popular. But I see that it's not the > >> >> > >> > right > >> >> > >> > choice at Attic because it's not the most common Attic culture. > >> >> > >> > >> >> > >> Attic does not produce source code. > >> >> > >> The only output from its SCM is the website. > >> >> > > > >> >> > > yes, like any other website that I showed: source code here is a > >> >> > > markup > >> >> > > language (be it Markdown, xdoc, static content, or anything else) > >> >> > > Attic is really exactly the same > >> >> > > > >> >> > > What makes Attic different is that Attic does not have any other > >> >> > > repo > >> >> > > for > >> >> > > "programming" code: that's true, but does not change anything > >> >> > > regarding > >> >> > > site> > >> >> > > > >> >> > >> AFAICT, all the above examples have a branch which contains the > >> >> > >> source > >> >> > >> for building the website. > >> >> > > > >> >> > > yes, I explained I chose them exactly for that reason > >> >> > > > >> >> > > I can show you Airavata site, which is quite simple and did the > >> >> > > other > >> >> > > choice: https://github.com/apache/airavata-site > >> >> > > > >> >> > > source is in source, output is in content in the same branch > >> >> > > >> >> > This is equivalent to Attic. > >> >> > > >> >> > > it could have been: source in master branch, content in asf-site > >> >> > > branch > >> >> > > which is the most common setup in GitHub pages (= what people > >> >> > > nowadays > >> >> > > know a lot) > >> >> > > > >> >> > >> 1) The source is edited. > >> >> > >> 2) Run the build script to create the output in a clean > >> >> > >> subdirectory > >> >> > >> 3) Copy the subdirectory tree to the asf-site branch > >> >> > >> 4) commit the asf-site branch > >> >> > >> 5) The entire asf-site branch is then published via pubsub. > >> >> > >> > >> >> > >> What Attic does currently is: > >> >> > >> 1) & 2) as above > >> >> > >> 3) commit the changes > >> >> > >> 5) as above > >> >> > >> > >> >> > >> i.e. there is no need to copy the generated output anywhere > >> >> > >> because > >> >> > >> it > >> >> > >> is part of the same repo. > >> >> > >> > >> >> > >> This works because svnpubsub is set up to get its source from the > >> >> > >> docs/ subdirectory > >> >> > > > >> >> > > yes, the setup with source and output in the same svn repo or Git > >> >> > > branch > >> >> > > makes it simple to checkout, but it mixes 2 types of files (source > >> >> > > and > >> >> > > generated) > >> >> > > > >> >> > > separating source and generated in 2 separate locations (separate > >> >> > > svn > >> >> > > root > >> >> > > or different branches in the same git repo) makes things more > >> >> > > clear, > >> >> > > at > >> >> > > the cost of an extra step to check out the generated content then > >> >> > > update > >> >> > > with the updated content > >> >> > > >> >> > The workspace still contains both source and generated output in the > >> >> > examples I have seen. > >> >> > > >> >> > I assume it is ignored by SVN/Git so does not get committed or show > >> >> > up > >> >> > as a local change. > >> >> > >> >> I showed you Cayenne, Freemarker and Accumulo that are not like this. > >> >> Here we go back to Freemarker: > >> >> - source: https://github.com/apache/freemarker-site > >> >> - output: https://github.com/apache/freemarker-site/tree/asf-site > >> >> > >> >> > > yes: choose your issue > >> >> > > personally, I prefer the second setup (clear but a little harder > >> >> > > to > >> >> > > setup) > >> >> > > I don't like having mixed content in one repo (source and > >> >> > > generated > >> >> > > output) > >> >> > > > >> >> > > If everybody understands that these 2 setups a completely > >> >> > > equivalent > >> >> > > but > >> >> > > really prefer the mixed one (just to avoid a second checkout), > >> >> > > I'll > >> >> > > let > >> >> > > you > >> >> > > go: I don't have any problem myself, I make a strong difference > >> >> > > between > >> >> > > source directory and output directory > >> >> > > >> >> > There's still mixed content in local workspaces unless you generate > >> >> > the output in a separate tree. > >> >> > > >> >> > > But if people start to edit output directory instead of source > >> >> > > (like > >> >> > > it > >> >> > > is > >> >> > > so easy to do in the mixed content setup), you're at risk > >> >> > > >> >> > It's also possible to checkin the generated output if it's not > >> >> > properly ignored in a 2 branch version. > >> >> > And then wonder why the site does not get updated. > >> >> > >> >> that's why in general there is a .gitignore or svn:ignore that is > >> >> properly > >> >> configured > >> >> > >> >> > >> I don't know if gitpubsub can take its input from a subdirectory > >> >> > >> of > >> >> > >> a > >> >> > >> branch. > >> >> > > > >> >> > > it can: see Airavata site > >> >> > > >> >> > Ah - I see now. > >> >> > > >> >> > The webserver defines the site to be under content/, so the branch > >> >> > can > >> >> > contain other files in parallel directories. > >> >> > > >> >> > >> If not, then we will have to change strategy in order to use Git. > >> >> > >> Otherwise, we have a choice. > >> >> > > > >> >> > > we have a choice > >> >> > > >> >> > Yes. > >> >> > > >> >> > I prefer the status quo, not least because it involves fewer changes > >> >> > (I think only renaming docs/ to content/ if we move to Git). > >> >> > Using multiple repos would involve updating instructions as well. > >> >> > >> >> no, it's not multiple repos but multiple branches of the same repo > >> >> > >> >> > But if the majority want to change I won't object. > >> >> > >> >> since you are the guy who does the buidbot script, that does the > >> >> commit, > >> >> you'll have to be confident that you can code the multi-branch option > >> >> > >> >> > > Regards, > >> >> > > > >> >> > > Hervé > >> >> > > > >> >> > >> > Regards, > >> >> > >> > > >> >> > >> > Hervé > >> >> > >> > > >> >> > >> > > >> >> > >> > [1] > >> >> > >> > https://help.github.com/articles/configuring-a-publishing-sourc > >> >> > >> > e-f > >> >> > >> > or-> > >> > gi th > >> >> > >> > ub-pages/ > >> >> > >> > > >> >> > >> > [2] > >> >> > >> > https://github.com/apache/infrastructure-puppet/blob/deployment > >> >> > >> > /mo > >> >> > >> > dul > >> >> > >> > es > >> >> > >> > /g > >> >> > >> > itwcsub/files/config/gitwcsub.cfg > >> >> > >> > > >> >> > >> > [3] https://github.com/apache/cayenne-website/ > >> >> > >> > > >> >> > >> > [4] https://github.com/apache/freemarker-site > >> >> > >> > > >> >> > >> > [5] https://github.com/apache/accumulo-website > >> >> > >> > > >> >> > >> > [6] > >> >> > >> > https://maven.apache.org/plugins/maven-scm-publish-plugin/vario > >> >> > >> > us-> >> > >> > tip s. > >> >> > >> > ht > >> >> > >> > ml#Git_branch
