Follow up. We tested with Pablo how these options work on a separate repository/job.
I just integrated these changes, and they effectively disable concurrent builds on the development branch. PRs are still being validated concurrently. On Fri, Oct 13, 2017 at 11:37 AM, Guillermo Polito < guillermopol...@gmail.com> wrote: > Well, apparently there is an option to disable concurrent builds, and we > should be able to make it work only for the development branch. > > https://jenkins.io/doc/book/pipeline/syntax/#options > https://stackoverflow.com/questions/41492688/how-to- > limit-jenkins-concurrent-multibranch-pipeline-builds > https://issues.jenkins-ci.org/browse/JENKINS-46593 > > I've opened the following issue: > > https://pharo.fogbugz.com/f/cases/20541/Diable-concurrent- > builds-for-the-development-branch > > and the following PR: > > https://github.com/pharo-project/pharo/pull/363 > > Could you review/check? I'm in the middle of a meeting right now :) > > > On Fri, Oct 13, 2017 at 9:40 AM, Guillermo Polito < > guillermopol...@gmail.com> wrote: > >> >> >> On Thu, Oct 12, 2017 at 9:23 PM, Torsten Bergmann <asta...@gmx.de> wrote: >> >>> Hi Guille, >>> >>> I understand that we want run the pull requests for contributions in >>> parallel. But do we want to run also the main build for >>> the daily official relase in parallel to have such side effects? I doubt >>> that. >>> >>> As I wrote it is not only an issue of the file server. The zero conf >>> does not give you the latest >>> although available (which is bad if you want to do the next >>> contribution) and we inform about >>> later build earlier than previous builds, etc. >>> >> >> Yes, I agree that this is a real issue. The reason is the following: >> every build, all artifacts are uploaded to the file server + a copy named >> latest.zip, which is used by zeroconf scripts. latest.zip is overwritten on >> every build. This means that the last build to finish is the last that >> writes to latest.zip. >> >> >>> >>> Sure it would be "nice" if our future Pharo 12 would already be >>> available before we release Pharo 7 - but it >>> still is not correct and confuses people ;) >>> >>> So is there anything that can be done to improve/solve this? >>> >> >> I'll check the options and come back to everybody here. >> >> >>> >>> Thanks >>> T. >>> >>> >>> Am 12.10.2017 um 16:12 schrieb Guillermo Polito: >>> >>>> >>>> >>>> On Thu, Oct 12, 2017 at 4:01 PM, Norbert Hartl <norb...@hartl.name >>>> <mailto:norb...@hartl.name>> wrote: >>>> >>>> >>>> > Am 12.10.2017 um 14:23 schrieb Torsten Bergmann <asta...@gmx.de >>>> <mailto:asta...@gmx.de>>: >>>> >>>> > >>>> > Hi, >>>> > >>>> > today there was an interesting effect visible for the Pharo >>>> build on CI for Pharo 7: >>>> > >>>> > Esteban merged three contributions for Pharo 7 into #development >>>> (see [1]). All three merges on git triggered the CI. >>>> > For each a separate build for Pharo 7 started on CI independently. >>>> > >>>> > While usually one would expect the builds in regular build order >>>> 183, 184, 185 the CI job for building 185 job was faster >>>> > (as you can see on the attached screenshot). >>>> > >>>> > So the 185 series of images was copied first to the file server >>>> [3] because the job was already finished. >>>> > Afterwards 183 build completed on CI and these images were >>>> copied to the file server with a newer timestamp. >>>> > >>>> > So you have the order 185, 183 with 183 having the files with >>>> the newest timestamp (but not being the most recent state of Pharo). >>>> > >>>> > This leads to several unfortunate effects: >>>> > 1. When you sort on file server [3] descending by date the >>>> build 183 images are newer than the build 185 images >>>> > 2. When you use PharoLauncher to download "latest-32" you get >>>> an 183 image instead of the "real latest" 185 image >>>> > 3. When you ask Zeroconf to download the latest using >>>> http://get.pharo.org/70+vm you also get 183 instead of the already >>>> > available 185 >>>> > >>>> > So as the CI build of an older "update" (here 183) finished >>>> later than the most recent version (here 185) >>>> > you get 183 as "latest" instead of 185. >>>> > >>>> > Any ideas how we could avoid such a side effect? Maybe a chain >>>> on the builds so 183 has to finished before 184 and 185 could be >>>> > built? >>>> > >>>> I don't understand why it is working like this. Usually only build >>>> for a job is active, others will be queued. So the problem here is >>>> that >>>> the jenkins spawns more builds in parallel for the same job. >>>> >>>> >>>> Yes, the thing is that the same job is being used to build the >>>> development branch and the pull requests. >>>> If you think about it, this is the same setup as in travis. >>>> >>>> The thing is that this does not bother me too much, considering the >>>> tradeoffs. >>>> >>>> - I want pull request to run in parallel. >>>> - I want to have less things to maintain :) >>>> >>>> Now, >>>> - we could split the job in two (integration vs pr validation). But >>>> this requires setting up a separate jenkins because otherwise they share >>>> the same slave configuration :/. And there is no way in the actual state of >>>> the plugins we use to limit the number of parallel builds... >>>> - we could try to make a build wait until a build with lower build >>>> number is building... >>>> - we could also forbid Esteban to push on the merge button while a >>>> build is running >>>> But doing any of those is a lot of work... >>>> >>>> - Moreover, I think the problem is that we are using the file server as >>>> something it is not. The file server is... a file server. >>>> Ideally, I'd like to attach meta-data to a release/build artifact, and >>>> be able to browse/search/sort using that metadata. Something alike >>>> artifactory or bintray. >>>> >>>> - Also, I don't know how the pharo launcher is getting the latest >>>> image, but if instead of using the file timestamp it uses the build number, >>>> then this may be fixed, isn't it? >>>> >>>> >>>> Norbert >>>> >>>> > Thx >>>> > T. >>>> > >>>> > >>>> > >>>> > [1] https://github.com/pharo-project/pharo/commits/development >>>> <https://github.com/pharo-project/pharo/commits/development> >>>> > [2] >>>> https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20p >>>> ull%20request%20and%20branch%20Pipeline/job/development >>>> <https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20 >>>> pull%20request%20and%20branch%20Pipeline/job/development> >>>> > [3] http://files.pharo.org/image/70/ >>>> <http://files.pharo.org/image/70/> >>>> > <buildOrder.png> >>>> >>>> >>>> >>>> >>>> -- >>>> >>>> >>>> >>>> Guille Polito >>>> >>>> Research Engineer >>>> >>>> Centre de Recherche en Informatique, Signal et Automatique de Lille >>>> >>>> CRIStAL - UMR 9189 >>>> >>>> French National Center for Scientific Research - _http://www.cnrs.fr_ >>>> >>>> >>>> *Web:* _http://guillep.github.io_ >>>> >>>> *Phone: *+33 06 52 70 66 13 >>>> >>>> >>> >> >> >> -- >> >> >> >> Guille Polito >> >> Research Engineer >> >> Centre de Recherche en Informatique, Signal et Automatique de Lille >> >> CRIStAL - UMR 9189 >> >> French National Center for Scientific Research - *http://www.cnrs.fr >> <http://www.cnrs.fr>* >> >> >> *Web:* *http://guillep.github.io* <http://guillep.github.io> >> >> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >> > > > > -- > > > > Guille Polito > > Research Engineer > > Centre de Recherche en Informatique, Signal et Automatique de Lille > > CRIStAL - UMR 9189 > > French National Center for Scientific Research - *http://www.cnrs.fr > <http://www.cnrs.fr>* > > > *Web:* *http://guillep.github.io* <http://guillep.github.io> > > *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> > -- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13
