Le lundi 30 avril 2018, 20:47:47 CEST sebb a écrit : > On 30 April 2018 at 19:24, Hervé BOUTEMY <[email protected]> wrote: > > Le lundi 30 avril 2018, 12:46:09 CEST sebb a écrit : > >> On 30 April 2018 at 06:56, Hervé BOUTEMY <[email protected]> wrote: > >> > Le lundi 30 avril 2018, 03:19:20 CEST sebb a écrit : > >> >> On 29 April 2018 at 22:11, Hervé BOUTEMY <[email protected]> wrote: > >> >> > better than a discussion: a demo > >> >> > > >> >> > here is the website: > >> >> > https://asf-attic.github.io/ > >> >> > > >> >> > its source: > >> >> > https://github.com/asf-attic/asf-attic.github.io/tree/source > >> >> > its output: > >> >> > https://github.com/asf-attic/asf-attic.github.io/tree/master > >> >> > > >> >> > the only thing that is not included here is the code for the CI to > >> >> > checkout > >> >> > the output branch and update content after rebuild > >> >> > >> >> All I see is two disjoint branches. > >> > > >> > yes, that's what it's about > >> > > >> >> It's not obvious how a user is supposed to update the master from the > >> >> source. > >> > > >> > people who know GitHub pages know: that's now part of common culture > >> > for > >> > many people > >> > >> So how do people update the site? > >> > >> Is it enough to edit the source branch and commit the change? > > > > yes, once a CI job is configured (like you do with Buildbot, and I do > > usually with Jenkins + Maven scm-publish) > > > >> > and from an operational point of view, that's why in general there is a > >> > CI > >> > server doing the official build from source branch then committing > >> > output > >> > to output branch > >> > >> Is that part of Github's offering? > > > > if you use the GitHub Jekyll, yes, the build is magic > > if you want your own build engine, it's up to you to do the CI setup > > > >> >> The build.sh script can be used to create a docs/ tree, but then what? > >> > > >> > when you build locally, it's just for yourself, because you want to > >> > check > >> > a > >> > change that is not as trivial as usual > >> > then the change in output branch is usually done by a CI server > >> > > >> > When used with Maven, there is scm-pulbish plugin for that: it's a > >> > common > >> > convention, and that's what is used by Apache Cayenne for example. > >> > > >> > When used with Jekyll or anything else as rendering engine, I don't > >> > know > >> > how people script the output commit: Apache Freemarker just tells in > >> > their documentation "To publish the built site, commit the output into > >> > the "asf- site" branch". And Apache Accumulo writes a serie of shell > >> > commands that they tell are launched by a Git hook (and not a CI > >> > server). > >> > >> AFAICT that only commits the change to the asf-site branch; it does > >> not push the updated branch. > > > > perhaps instructions are not fully written: in general, a CI job does the > > work> > >> > And again, in general, there is a central official setup does the job > >> > in a > >> > central and official way, to avoid subtle differences when rendering > >> > from > >> > multiple personal configurations > >> > > >> >> > But what is done here with GitHub GitPubSub equivalent can be done > >> >> > exactly > >> >> > the same way at Apache Software Foundation > >> > >> I'm not disputing that, but if I am to amend the Buildbot to mimic the > >> GitHub behaviour I need to know what it does. > > > > sure :) > > it's about doing a git commit + push to the output branch, like it did > > previously the svn commit to svnpubsub location > > > > the demo on GitHub is just here to see it in a very concrete way, since > > GitHub provides the gitwcsub to publish from the branch to the web server > Ah, OK. > > I don't seem to be able to edit the source, so I cannot try it out. > > But if I did, I assume I would see my changes applied to the source > and then master branches, is that correct? I did not setup a CI job, then the build won't happen but if there was a CI job somewhere to build and commit the result, yes, this would be the process = what we expect to have at Attic once migrated to Git IIUC
> > >> >> > Regards, > >> >> > > >> >> > Hervé > >> >> > > >> >> > Le dimanche 29 avril 2018, 19:13:43 CEST Hervé BOUTEMY a écrit : > >> >> >> Le dimanche 29 avril 2018, 14:46:09 CEST sebb a écrit : > >> >> >> > On 29 April 2018 at 11:33, Hervé BOUTEMY <[email protected]> > > > > wrote: > >> >> >> > > Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit : > >> >> >> > >> On 29 April 2018 at 09:41, Hervé BOUTEMY > >> >> >> > >> <[email protected]> > >> > > >> > wrote: > >> >> >> > >> > first, I want to reassure everybody: this is a discussion, > >> >> >> > >> > to > >> >> >> > >> > get > >> >> >> > >> > common > >> >> >> > >> > knowledge of how things work in other projects then may work > >> >> >> > >> > in > >> >> >> > >> > the > >> >> >> > >> > future for Attic if we decide to do an equivalent setup > >> >> >> > >> > >> >> >> > >> +1 > >> >> >> > >> > >> >> >> > >> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit : > >> >> >> > >> >> On 28 April 2018 at 12:48, sebb <[email protected]> wrote: > >> >> >> > >> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY > >> >> >> > >> >> > <[email protected]> > >> >> >> > >> >> >> wrote: > >> >> >> > >> >> ... > >> >> >> > >> >> > >> >> >> > >> >> >> In Git, this would naturally be in a separate branch > >> >> >> > >> >> >> named > >> >> >> > >> >> >> "asf-site" > >> >> >> > >> >> > >> >> >> > >> >> How would that work for Attic? > >> >> >> > >> >> > >> >> >> > >> >> Where would the source files used to generate the site be > >> >> >> > >> >> held? > >> >> >> > >> > > >> >> >> > >> > There are multiple ways of doing, and GitHub documented it > >> >> >> > >> > as > >> >> >> > >> > clearly > >> >> >> > >> > as > >> >> >> > >> > possible [2] (yes, what we do at ASF with GitPubSub is > >> >> >> > >> > exactly > >> >> >> > >> > what > >> >> >> > >> > GitHub calls "GitHub Pages", with marketing bells turned on > >> >> >> > >> > and > >> >> >> > >> > technical > >> >> >> > >> > details on the build solution turned off) > >> >> >> > >> > > >> >> >> > >> > The 2 common ways are: > >> >> >> > >> > 1. publish html from separate branch (which would be by > >> >> >> > >> > default > >> >> >> > >> > "asf-site" > >> >> >> > >> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a > >> >> >> > >> > subdirectory > >> >> >> > >> > on master branch (you see Attic current pattern?) > >> >> >> > >> > > >> >> >> > >> > I find the first option a lot more clear from a build+scm > >> >> >> > >> > perspective > >> >> >> > >> > than > >> >> >> > >> > the second one. This will avoid the exact same discussion we > >> >> >> > >> > have > >> >> >> > >> > currently at Attic with svn to know who commits the > >> >> >> > >> > generated > >> >> >> > >> > content > >> >> >> > >> > (& > >> >> >> > >> > when as a consequence): - CI after source-only commit? > >> >> >> > >> > - or user who builds on his machine then commits > >> >> >> > >> > simultaneously > >> >> >> > >> > source > >> >> >> > >> > and > >> >> >> > >> > generated content? > >> >> >> > >> > > >> >> >> > >> > Then looking at ASF gitwcsub configuration [2], I had a look > >> >> >> > >> > at > >> >> >> > >> > many > >> >> >> > >> > ASF > >> >> >> > >> > projects: the 2 ways are used. I picked Cayenne [3] case to > >> >> >> > >> > show a > >> >> >> > >> > case > >> >> >> > >> > where: > >> >> >> > >> > - master branch is a source branch, with markup and a build > >> >> >> > >> > script > >> >> >> > >> > - asf-site branch is a completely separate branch that > >> >> >> > >> > contains > >> >> >> > >> > generated > >> >> >> > >> > html It uses Maven scm-publish plugin to update asf-git > >> >> >> > >> > branch > >> >> >> > >> > with > >> >> >> > >> > generated html [6] > >> >> >> > >> > > >> >> >> > >> > But there is also Freemarker [4], that has a simple README > >> >> >> > >> > telling > >> >> >> > >> > "To > >> >> >> > >> > publish the built site, commit the output into the > >> >> >> > >> > "asf-site" > >> >> >> > >> > branch". > >> >> >> > >> > > >> >> >> > >> > Or Accumulo [5] which uses Jekyll and has some instructions > >> >> >> > >> > to > >> >> >> > >> > publish > >> >> >> > >> > generated output to asf-site branch with a git-hook that I > >> >> >> > >> > don't > >> >> >> > >> > fully > >> >> >> > >> > understand, but that maybe Attic members will prefer since > >> >> >> > >> > it > >> >> >> > >> > seems > >> >> >> > >> > it's > >> >> >> > >> > more the common culture here > >> >> >> > >> > > >> >> >> > >> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish > >> >> >> > >> > plugin > >> >> >> > >> > used > >> >> >> > >> > by > >> >> >> > >> > Cayenne, and I use it in many projects, initially with svn > >> >> >> > >> > as > >> >> >> > >> > target > >> >> >> > >> > source control (in 2012, for svnpubsub & Apache CMS) then > >> >> >> > >> > with > >> >> >> > >> > git > >> >> >> > >> > also, > >> >> >> > >> > when GitHub pages became popular. But I see that it's not > >> >> >> > >> > the > >> >> >> > >> > right > >> >> >> > >> > choice at Attic because it's not the most common Attic > >> >> >> > >> > culture. > >> >> >> > >> > >> >> >> > >> Attic does not produce source code. > >> >> >> > >> The only output from its SCM is the website. > >> >> >> > > > >> >> >> > > yes, like any other website that I showed: source code here is > >> >> >> > > a > >> >> >> > > markup > >> >> >> > > language (be it Markdown, xdoc, static content, or anything > >> >> >> > > else) > >> >> >> > > Attic is really exactly the same > >> >> >> > > > >> >> >> > > What makes Attic different is that Attic does not have any > >> >> >> > > other > >> >> >> > > repo > >> >> >> > > for > >> >> >> > > "programming" code: that's true, but does not change anything > >> >> >> > > regarding > >> >> >> > > site> > >> >> >> > > > >> >> >> > >> AFAICT, all the above examples have a branch which contains > >> >> >> > >> the > >> >> >> > >> source > >> >> >> > >> for building the website. > >> >> >> > > > >> >> >> > > yes, I explained I chose them exactly for that reason > >> >> >> > > > >> >> >> > > I can show you Airavata site, which is quite simple and did the > >> >> >> > > other > >> >> >> > > choice: https://github.com/apache/airavata-site > >> >> >> > > > >> >> >> > > source is in source, output is in content in the same branch > >> >> >> > > >> >> >> > This is equivalent to Attic. > >> >> >> > > >> >> >> > > it could have been: source in master branch, content in > >> >> >> > > asf-site > >> >> >> > > branch > >> >> >> > > which is the most common setup in GitHub pages (= what people > >> >> >> > > nowadays > >> >> >> > > know a lot) > >> >> >> > > > >> >> >> > >> 1) The source is edited. > >> >> >> > >> 2) Run the build script to create the output in a clean > >> >> >> > >> subdirectory > >> >> >> > >> 3) Copy the subdirectory tree to the asf-site branch > >> >> >> > >> 4) commit the asf-site branch > >> >> >> > >> 5) The entire asf-site branch is then published via pubsub. > >> >> >> > >> > >> >> >> > >> What Attic does currently is: > >> >> >> > >> 1) & 2) as above > >> >> >> > >> 3) commit the changes > >> >> >> > >> 5) as above > >> >> >> > >> > >> >> >> > >> i.e. there is no need to copy the generated output anywhere > >> >> >> > >> because > >> >> >> > >> it > >> >> >> > >> is part of the same repo. > >> >> >> > >> > >> >> >> > >> This works because svnpubsub is set up to get its source from > >> >> >> > >> the > >> >> >> > >> docs/ subdirectory > >> >> >> > > > >> >> >> > > yes, the setup with source and output in the same svn repo or > >> >> >> > > Git > >> >> >> > > branch > >> >> >> > > makes it simple to checkout, but it mixes 2 types of files > >> >> >> > > (source > >> >> >> > > and > >> >> >> > > generated) > >> >> >> > > > >> >> >> > > separating source and generated in 2 separate locations > >> >> >> > > (separate > >> >> >> > > svn > >> >> >> > > root > >> >> >> > > or different branches in the same git repo) makes things more > >> >> >> > > clear, > >> >> >> > > at > >> >> >> > > the cost of an extra step to check out the generated content > >> >> >> > > then > >> >> >> > > update > >> >> >> > > with the updated content > >> >> >> > > >> >> >> > The workspace still contains both source and generated output in > >> >> >> > the > >> >> >> > examples I have seen. > >> >> >> > > >> >> >> > I assume it is ignored by SVN/Git so does not get committed or > >> >> >> > show > >> >> >> > up > >> >> >> > as a local change. > >> >> >> > >> >> >> I showed you Cayenne, Freemarker and Accumulo that are not like > >> >> >> this. > >> >> >> Here we go back to Freemarker: > >> >> >> - source: https://github.com/apache/freemarker-site > >> >> >> - output: https://github.com/apache/freemarker-site/tree/asf-site > >> >> >> > >> >> >> > > yes: choose your issue > >> >> >> > > personally, I prefer the second setup (clear but a little > >> >> >> > > harder > >> >> >> > > to > >> >> >> > > setup) > >> >> >> > > I don't like having mixed content in one repo (source and > >> >> >> > > generated > >> >> >> > > output) > >> >> >> > > > >> >> >> > > If everybody understands that these 2 setups a completely > >> >> >> > > equivalent > >> >> >> > > but > >> >> >> > > really prefer the mixed one (just to avoid a second checkout), > >> >> >> > > I'll > >> >> >> > > let > >> >> >> > > you > >> >> >> > > go: I don't have any problem myself, I make a strong difference > >> >> >> > > between > >> >> >> > > source directory and output directory > >> >> >> > > >> >> >> > There's still mixed content in local workspaces unless you > >> >> >> > generate > >> >> >> > the output in a separate tree. > >> >> >> > > >> >> >> > > But if people start to edit output directory instead of source > >> >> >> > > (like > >> >> >> > > it > >> >> >> > > is > >> >> >> > > so easy to do in the mixed content setup), you're at risk > >> >> >> > > >> >> >> > It's also possible to checkin the generated output if it's not > >> >> >> > properly ignored in a 2 branch version. > >> >> >> > And then wonder why the site does not get updated. > >> >> >> > >> >> >> that's why in general there is a .gitignore or svn:ignore that is > >> >> >> properly > >> >> >> configured > >> >> >> > >> >> >> > >> I don't know if gitpubsub can take its input from a > >> >> >> > >> subdirectory > >> >> >> > >> of > >> >> >> > >> a > >> >> >> > >> branch. > >> >> >> > > > >> >> >> > > it can: see Airavata site > >> >> >> > > >> >> >> > Ah - I see now. > >> >> >> > > >> >> >> > The webserver defines the site to be under content/, so the > >> >> >> > branch > >> >> >> > can > >> >> >> > contain other files in parallel directories. > >> >> >> > > >> >> >> > >> If not, then we will have to change strategy in order to use > >> >> >> > >> Git. > >> >> >> > >> Otherwise, we have a choice. > >> >> >> > > > >> >> >> > > we have a choice > >> >> >> > > >> >> >> > Yes. > >> >> >> > > >> >> >> > I prefer the status quo, not least because it involves fewer > >> >> >> > changes > >> >> >> > (I think only renaming docs/ to content/ if we move to Git). > >> >> >> > Using multiple repos would involve updating instructions as well. > >> >> >> > >> >> >> no, it's not multiple repos but multiple branches of the same repo > >> >> >> > >> >> >> > But if the majority want to change I won't object. > >> >> >> > >> >> >> since you are the guy who does the buidbot script, that does the > >> >> >> commit, > >> >> >> you'll have to be confident that you can code the multi-branch > >> >> >> option > >> >> >> > >> >> >> > > Regards, > >> >> >> > > > >> >> >> > > Hervé > >> >> >> > > > >> >> >> > >> > Regards, > >> >> >> > >> > > >> >> >> > >> > Hervé > >> >> >> > >> > > >> >> >> > >> > > >> >> >> > >> > [1] > >> >> >> > >> > https://help.github.com/articles/configuring-a-publishing-so > >> >> >> > >> > urc > >> >> >> > >> > e-f > >> >> >> > >> > or-> > >> > gi th > >> >> >> > >> > ub-pages/ > >> >> >> > >> > > >> >> >> > >> > [2] > >> >> >> > >> > https://github.com/apache/infrastructure-puppet/blob/deploym > >> >> >> > >> > ent > >> >> >> > >> > /mo > >> >> >> > >> > dul > >> >> >> > >> > es > >> >> >> > >> > /g > >> >> >> > >> > itwcsub/files/config/gitwcsub.cfg > >> >> >> > >> > > >> >> >> > >> > [3] https://github.com/apache/cayenne-website/ > >> >> >> > >> > > >> >> >> > >> > [4] https://github.com/apache/freemarker-site > >> >> >> > >> > > >> >> >> > >> > [5] https://github.com/apache/accumulo-website > >> >> >> > >> > > >> >> >> > >> > [6] > >> >> >> > >> > https://maven.apache.org/plugins/maven-scm-publish-plugin/va > >> >> >> > >> > rio > >> >> >> > >> > us-> >> > >> > tip s. > >> >> >> > >> > ht > >> >> >> > >> > ml#Git_branch
