Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
Hi Andrei, (Kind request - please don't top post, at least not in a long conversation thread which uses bottom-posting, it's quite hard to understand what you're replying to :-) ) On Tue, 2016-09-20 at 09:45 +, Andrei Dulvac wrote: > Hi Robert, > > Have a quick look at https://jenkins.io/doc/pipeline/jenkinsfile/ > It is a jenkins 1.X plugin, formerly known as "Workflow plugin": > https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Plugin > It is part of jenkins 2.0 now (https://jenkins.io/2.0/) and it will > definitely surpass usage of the Job DSL plugin, which is in the same > space. I've read about the pipeline plugin, I've seen comparisons with the Job DSL plugin, but nowhere have I found how to automate job generation with the pipeline plugin. With the Job DSL plugin I can have one 'seed' job which automatically generates and updates an arbitrary number of jobs. This is what I'm looking for since I don't want to manually maintain 50 or more per- module Jenkins jobs. If you can point me to that specific functionality in the pipeline plugin I'd be happy to build on top of it. Otherwise, I will stick with the Job DSL plugin. Robert > > HTH. > -Andrei > > On Mon, Sep 19, 2016 at 9:50 PM Robert Munteanu > wrote: > > > > > On Mon, 2016-09-19 at 18:02 +0200, Konrad Windszus wrote: > > > > > > Wouldn’t it make more sense to rely on the pipeline plugin > > > instead of > > > the Job DSL Plugin? This is by default shipped in Jenkins since > > > 2.0.0 > > > so probably there is not even the necessity to install an > > > additional > > > plugin. > > > For a comparison between those two approaches look at > > > http://stackoverflow.com/questions/37657810/job-dsl-plugin-vs- > > > pipeline-plugin. > > > > AFAIU the pipeline plugin defines a pipeline, but that's not what > > I'm > > looking for at first. I skipped the documentation and the examples, > > but > > I could not find an example about how to create jobs using the > > pipeline > > plugin. > > > > I want to use the Job DSL plugin to define a large number of jobs > > programatically, rather than maintain them by hand. > > > > In the future we can orchestrate these jobs by using the pipeline > > plugin. > > > > Of course, if anyone can point me in the direction of generating > > jobs > > using the pipeline plugin, I'd be happy to use just that. > > > > Thanks, > > > > Robert > > > > > > > > > > > I don’t have any practical experience with either of those two > > > approaches, but since the Pipeline DSL gained a lot more > > > attention > > > from the Jenkins community in the past I am wondering whether > > > using > > > that that is not enough for Sling. > > > Konrad > > > > > > > > > > > > > > > > > > Am 19.09.2016 um 16:15 schrieb Robert Munteanu > > > org> > > > > : > > > > > > > > On Mon, 2016-09-19 at 15:53 +0200, Carsten Ziegeler wrote: > > > > > > > > > > > > > > > The benefit of the current solution is clear: below or equal > > > > > to > > > > > zero. > > > > > Afaik, Robert look already into having per module jobs in > > > > > jenkins. > > > > > Not > > > > > sure how far this is. > > > > > > > > I pinged infra about this and noone opposed :-) > > > > > > > > http://mail-archives.apache.org/mod_mbox/www-builds/201609.mb > > > > ox/b > > > > rows > > > > er > > > > > > > > I would be happy to look into per-module builds based on > > > > Jenkins. > > > > I've > > > > done some work automating similar setups in the past using Job > > > > DSL > > > > plugin > > > > > > > > https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin > > > > > > > > I can file a request for infra to install this plug-in and > > > > start > > > > testing on a limited set of modules if no one objects. We can > > > > then > > > > gradually migrate modules off the 'main' job and move them to > > > > individual jobs. > > > > > > > > Thanks, > > > > > > > > Robert > > > > > > > > > > > > > > > > > > > > > > > > Carsten > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 for openly talking about this. I think the most > > > > > > important > > > > > > thing > > > > > > is > > > > > > clearly understanding the benefits and drawbacks for the > > > > > > different > > > > > > ways of > > > > > > building continuously. > > > > > > > > > > > > On Mon, Sep 19, 2016 at 3:43 PM Stefan Seifert > > > > > o-vi > > > > > > sion > > > > > > .de> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > for quiet some time now, it seems that the IT tests of > > > > > > > > the > > > > > > > > i18n > > > > > > > > module > > > > > > > fail in reactor build > > > > > > > > > > > > > > i also had a brief look at the tests of the i18n project > > > > > > > and > > > > > > > was > > > > > > > not able > > > > > > > to spot the problem easily. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I suggest we turn off Jenk
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
Hi Robert,
Have a quick look at https://jenkins.io/doc/pipeline/jenkinsfile/
It is a jenkins 1.X plugin, formerly known as "Workflow plugin":
https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Plugin
It is part of jenkins 2.0 now (https://jenkins.io/2.0/) and it will
definitely surpass usage of the Job DSL plugin, which is in the same space.
HTH.
-Andrei
On Mon, Sep 19, 2016 at 9:50 PM Robert Munteanu wrote:
> On Mon, 2016-09-19 at 18:02 +0200, Konrad Windszus wrote:
> > Wouldn’t it make more sense to rely on the pipeline plugin instead of
> > the Job DSL Plugin? This is by default shipped in Jenkins since 2.0.0
> > so probably there is not even the necessity to install an additional
> > plugin.
> > For a comparison between those two approaches look at
> > http://stackoverflow.com/questions/37657810/job-dsl-plugin-vs-
> > pipeline-plugin.
>
> AFAIU the pipeline plugin defines a pipeline, but that's not what I'm
> looking for at first. I skipped the documentation and the examples, but
> I could not find an example about how to create jobs using the pipeline
> plugin.
>
> I want to use the Job DSL plugin to define a large number of jobs
> programatically, rather than maintain them by hand.
>
> In the future we can orchestrate these jobs by using the pipeline
> plugin.
>
> Of course, if anyone can point me in the direction of generating jobs
> using the pipeline plugin, I'd be happy to use just that.
>
> Thanks,
>
> Robert
>
> >
> > I don’t have any practical experience with either of those two
> > approaches, but since the Pipeline DSL gained a lot more attention
> > from the Jenkins community in the past I am wondering whether using
> > that that is not enough for Sling.
> > Konrad
> >
> >
> > >
> > > Am 19.09.2016 um 16:15 schrieb Robert Munteanu
> > > :
> > >
> > > On Mon, 2016-09-19 at 15:53 +0200, Carsten Ziegeler wrote:
> > > >
> > > > The benefit of the current solution is clear: below or equal to
> > > > zero.
> > > > Afaik, Robert look already into having per module jobs in
> > > > jenkins.
> > > > Not
> > > > sure how far this is.
> > >
> > > I pinged infra about this and noone opposed :-)
> > >
> > > http://mail-archives.apache.org/mod_mbox/www-builds/201609.mbox/b
> > > rows
> > > er
> > >
> > > I would be happy to look into per-module builds based on Jenkins.
> > > I've
> > > done some work automating similar setups in the past using Job DSL
> > > plugin
> > >
> > > https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
> > >
> > > I can file a request for infra to install this plug-in and start
> > > testing on a limited set of modules if no one objects. We can then
> > > gradually migrate modules off the 'main' job and move them to
> > > individual jobs.
> > >
> > > Thanks,
> > >
> > > Robert
> > >
> > > >
> > > >
> > > > Carsten
> > > >
> > > > >
> > > > >
> > > > > +1 for openly talking about this. I think the most important
> > > > > thing
> > > > > is
> > > > > clearly understanding the benefits and drawbacks for the
> > > > > different
> > > > > ways of
> > > > > building continuously.
> > > > >
> > > > > On Mon, Sep 19, 2016 at 3:43 PM Stefan Seifert > > > > sion
> > > > > .de>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > for quiet some time now, it seems that the IT tests of the
> > > > > > > i18n
> > > > > > > module
> > > > > > fail in reactor build
> > > > > >
> > > > > > i also had a brief look at the tests of the i18n project and
> > > > > > was
> > > > > > not able
> > > > > > to spot the problem easily.
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > I suggest we turn off Jenkins completely
> > > > > >
> > > > > > do we have an alternative?
> > > > > >
> > > > > > ian invested a good amount of time in getting the sling CI
> > > > > > running in
> > > > > > travis, not sure what the status ist.
> > > > > > currently it seems JDK 1.8 builds run fine, JDK 1.7 builds
> > > > > > not
> > > > > > [1].
> > > > > >
> > > > > > in my pov the main problem of our CI builds are that we have
> > > > > > two
> > > > > > really
> > > > > > huge builds that build everything ("production bundles" and
> > > > > > contrib), and
> > > > > > if one of them gets flaky all gets flaky, and it is extremely
> > > > > > time-consuming trying to fix a problem, wait some hours, try
> > > > > > again etc.
> > > > > > plus the fact a lot of problem on the apache Jenkins are
> > > > > > difficult to
> > > > > > reproduce locally.
> > > > > >
> > > > > > most sling bundles (or small group of bundles) are quite
> > > > > > autonomous. when
> > > > > > we had a CI build for each of them individually this would be
> > > > > > much easier.
> > > > > > if one breaks the CI is still functional for the other 99%
> > > > > > bundles. this
> > > > > > leads again into the svn vs. git and huge repo vs. small
> > > > > > repos
> > > > > > debate, and
> > > > > > of course no one wants to maintain a huge set of CI build
> > > > > > configurations
> > > >
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
On Tue, 2016-09-20 at 09:36 +0200, Bertrand Delacretaz wrote: > Hi, > > On Mon, Sep 19, 2016 at 10:26 PM, Robert Munteanu > wrote: > > > > ...Another benefit is that if a contributor wants to reproduce a > > problem > > on Jenkins, all the effort that is required is to run Jenkins via > > docker, install a couple of plugins, create the seed job and all > > the > > Jenkins configuration is replicated... > > This sounds great, with the job creation code in svn as you suggest. > Bonus points for having a Docker file in svn so that we can easily > compare a local Jenkins build with the https://builds.apache.org/ > one. That's a great idea, filed https://issues.apache.org/jira/browse/SLING-6062 Robert
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
Hi, On Mon, Sep 19, 2016 at 10:26 PM, Robert Munteanu wrote: > ...Another benefit is that if a contributor wants to reproduce a problem > on Jenkins, all the effort that is required is to run Jenkins via > docker, install a couple of plugins, create the seed job and all the > Jenkins configuration is replicated... This sounds great, with the job creation code in svn as you suggest. Bonus points for having a Docker file in svn so that we can easily compare a local Jenkins build with the https://builds.apache.org/ one. -Bertrand
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
On Mon, 2016-09-19 at 23:26 +0300, Robert Munteanu wrote: > On Mon, 2016-09-19 at 22:50 +0300, Robert Munteanu wrote: > > > > I want to use the Job DSL plugin to define a large number of jobs > > programatically, rather than maintain them by hand. Here's a more in-depth look at the issue http://marcesher.com/2016/08/04/jenkins-as-code-comparing-job-dsl-and -pipelines/ Which seems to echo my impression that Job DSL and Pipeline are complementary to each other. Robert
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
On Mon, 2016-09-19 at 22:50 +0300, Robert Munteanu wrote:
> I want to use the Job DSL plugin to define a large number of jobs
> programatically, rather than maintain them by hand.
Here's a quick sample of how I intend to use the DSL plugin.
First, I create a 'seed' job, in charge of generating other jobs.
Then, I use the following script:
def svnBase = "https://svn.apache.org/repos/asf/sling/trunk";
def modules = ["bundles/extensions/i18n", "contrib/extensions/sling-
pipes"]
modules.each {
def svnDir = svnBase +"/" + it
def jobName = "sling-" + it.replaceAll('/', '-')
job(jobName) {
scm {
svn(svnDir)
}
triggers {
scm('H/15 * * * *')
}
steps {
maven {
goals("clean")
goals("verify")
mavenInstallation("Maven 3.3.9")
}
}
}
}
This configures two jobs with identical settings. Of course, in the
future we can configure all jobs this way, and the script will be
stored in SVN.
Another benefit is that if a contributor wants to reproduce a problem
on Jenkins, all the effort that is required is to run Jenkins via
docker, install a couple of plugins, create the seed job and all the
Jenkins configuration is replicated.
Robert
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
On Mon, 2016-09-19 at 18:02 +0200, Konrad Windszus wrote:
> Wouldn’t it make more sense to rely on the pipeline plugin instead of
> the Job DSL Plugin? This is by default shipped in Jenkins since 2.0.0
> so probably there is not even the necessity to install an additional
> plugin.
> For a comparison between those two approaches look at
> http://stackoverflow.com/questions/37657810/job-dsl-plugin-vs-
> pipeline-plugin.
AFAIU the pipeline plugin defines a pipeline, but that's not what I'm
looking for at first. I skipped the documentation and the examples, but
I could not find an example about how to create jobs using the pipeline
plugin.
I want to use the Job DSL plugin to define a large number of jobs
programatically, rather than maintain them by hand.
In the future we can orchestrate these jobs by using the pipeline
plugin.
Of course, if anyone can point me in the direction of generating jobs
using the pipeline plugin, I'd be happy to use just that.
Thanks,
Robert
>
> I don’t have any practical experience with either of those two
> approaches, but since the Pipeline DSL gained a lot more attention
> from the Jenkins community in the past I am wondering whether using
> that that is not enough for Sling.
> Konrad
>
>
> >
> > Am 19.09.2016 um 16:15 schrieb Robert Munteanu
> > :
> >
> > On Mon, 2016-09-19 at 15:53 +0200, Carsten Ziegeler wrote:
> > >
> > > The benefit of the current solution is clear: below or equal to
> > > zero.
> > > Afaik, Robert look already into having per module jobs in
> > > jenkins.
> > > Not
> > > sure how far this is.
> >
> > I pinged infra about this and noone opposed :-)
> >
> > http://mail-archives.apache.org/mod_mbox/www-builds/201609.mbox/b
> > rows
> > er
> >
> > I would be happy to look into per-module builds based on Jenkins.
> > I've
> > done some work automating similar setups in the past using Job DSL
> > plugin
> >
> > https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
> >
> > I can file a request for infra to install this plug-in and start
> > testing on a limited set of modules if no one objects. We can then
> > gradually migrate modules off the 'main' job and move them to
> > individual jobs.
> >
> > Thanks,
> >
> > Robert
> >
> > >
> > >
> > > Carsten
> > >
> > > >
> > > >
> > > > +1 for openly talking about this. I think the most important
> > > > thing
> > > > is
> > > > clearly understanding the benefits and drawbacks for the
> > > > different
> > > > ways of
> > > > building continuously.
> > > >
> > > > On Mon, Sep 19, 2016 at 3:43 PM Stefan Seifert > > > sion
> > > > .de>
> > > > wrote:
> > > >
> > > > >
> > > > >
> > > > > >
> > > > > >
> > > > > > for quiet some time now, it seems that the IT tests of the
> > > > > > i18n
> > > > > > module
> > > > > fail in reactor build
> > > > >
> > > > > i also had a brief look at the tests of the i18n project and
> > > > > was
> > > > > not able
> > > > > to spot the problem easily.
> > > > >
> > > > > >
> > > > > >
> > > > > > I suggest we turn off Jenkins completely
> > > > >
> > > > > do we have an alternative?
> > > > >
> > > > > ian invested a good amount of time in getting the sling CI
> > > > > running in
> > > > > travis, not sure what the status ist.
> > > > > currently it seems JDK 1.8 builds run fine, JDK 1.7 builds
> > > > > not
> > > > > [1].
> > > > >
> > > > > in my pov the main problem of our CI builds are that we have
> > > > > two
> > > > > really
> > > > > huge builds that build everything ("production bundles" and
> > > > > contrib), and
> > > > > if one of them gets flaky all gets flaky, and it is extremely
> > > > > time-consuming trying to fix a problem, wait some hours, try
> > > > > again etc.
> > > > > plus the fact a lot of problem on the apache Jenkins are
> > > > > difficult to
> > > > > reproduce locally.
> > > > >
> > > > > most sling bundles (or small group of bundles) are quite
> > > > > autonomous. when
> > > > > we had a CI build for each of them individually this would be
> > > > > much easier.
> > > > > if one breaks the CI is still functional for the other 99%
> > > > > bundles. this
> > > > > leads again into the svn vs. git and huge repo vs. small
> > > > > repos
> > > > > debate, and
> > > > > of course no one wants to maintain a huge set of CI build
> > > > > configurations
> > > > > individually, we would need to automate this.
> > > > >
> > > > > in the past it was difficult to get a consensus on those
> > > > > topics
> > > > > (scm, ci)
> > > > > on this list. perhaps we can have a discussion on this on the
> > > > > adaptTo() in
> > > > > berlin next week.
> > > > >
> > > > > stefan
> > > > >
> > > > > [1] https://travis-ci.org/apache/sling/builds
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > >
> >
>
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
+1 to the pipeline plugin.
Info on the other CI builds.
There is a build that builds pull requests from GitHub in Jenkins, IIRC its
only JDK8 and doesn't have tests enabled as there were too many false
negatives, but I would need to check that.
Also there are Travis CI builds for JDK8 and JDK7, the JDK7 one is disabled
and the JDK8 doesn't run integration tests. The Travis CI builds tend to
come back faster than Jenkins at the moment, and will run all you ask it to
in parallel, so a CI matrix is viable without a pipeline.
Ideally all those builds should be running tests and run on JDK8 and JDK7
but while some of the tests remain unstable a false negative is not
helpful.
If you want to change the travis builds, commit or patch to .travis.yaml.
Try to keep the volume of output down. > 1000 lines causes the build to
fail, and no output for > 10min also fails the build, hence some of the
slightly odd build commands in the .travis.yaml file.
Best Regards
Ian
On 19 September 2016 at 17:04, Andrei Dulvac
wrote:
> Was going to write the exact same thing. The pipeline plugin, now just
> "pipeline" into jenkins 2, is the future and I would recommend using it.
> Except if there are plugins you want to use that are not supported yet.
>
> -Andrei
>
> On Mon, Sep 19, 2016 at 6:02 PM Konrad Windszus wrote:
>
> > Wouldn’t it make more sense to rely on the pipeline plugin instead of the
> > Job DSL Plugin? This is by default shipped in Jenkins since 2.0.0 so
> > probably there is not even the necessity to install an additional plugin.
> > For a comparison between those two approaches look at
> > http://stackoverflow.com/questions/37657810/job-dsl-
> plugin-vs-pipeline-plugin
> > .
> >
> > I don’t have any practical experience with either of those two
> approaches,
> > but since the Pipeline DSL gained a lot more attention from the Jenkins
> > community in the past I am wondering whether using that that is not
> enough
> > for Sling.
> > Konrad
> >
> >
> > > Am 19.09.2016 um 16:15 schrieb Robert Munteanu :
> > >
> > > On Mon, 2016-09-19 at 15:53 +0200, Carsten Ziegeler wrote:
> > >> The benefit of the current solution is clear: below or equal to zero.
> > >> Afaik, Robert look already into having per module jobs in jenkins.
> > >> Not
> > >> sure how far this is.
> > >
> > > I pinged infra about this and noone opposed :-)
> > >
> > > http://mail-archives.apache.org/mod_mbox/www-builds/
> 201609.mbox/brows
> > > er
> > >
> > > I would be happy to look into per-module builds based on Jenkins. I've
> > > done some work automating similar setups in the past using Job DSL
> > > plugin
> > >
> > > https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
> > >
> > > I can file a request for infra to install this plug-in and start
> > > testing on a limited set of modules if no one objects. We can then
> > > gradually migrate modules off the 'main' job and move them to
> > > individual jobs.
> > >
> > > Thanks,
> > >
> > > Robert
> > >
> > >>
> > >> Carsten
> > >>
> > >>>
> > >>> +1 for openly talking about this. I think the most important thing
> > >>> is
> > >>> clearly understanding the benefits and drawbacks for the different
> > >>> ways of
> > >>> building continuously.
> > >>>
> > >>> On Mon, Sep 19, 2016 at 3:43 PM Stefan Seifert > >>> .de>
> > >>> wrote:
> > >>>
> >
> > >
> > > for quiet some time now, it seems that the IT tests of the i18n
> > > module
> > fail in reactor build
> >
> > i also had a brief look at the tests of the i18n project and was
> > not able
> > to spot the problem easily.
> >
> > >
> > > I suggest we turn off Jenkins completely
> >
> > do we have an alternative?
> >
> > ian invested a good amount of time in getting the sling CI
> > running in
> > travis, not sure what the status ist.
> > currently it seems JDK 1.8 builds run fine, JDK 1.7 builds not
> > [1].
> >
> > in my pov the main problem of our CI builds are that we have two
> > really
> > huge builds that build everything ("production bundles" and
> > contrib), and
> > if one of them gets flaky all gets flaky, and it is extremely
> > time-consuming trying to fix a problem, wait some hours, try
> > again etc.
> > plus the fact a lot of problem on the apache Jenkins are
> > difficult to
> > reproduce locally.
> >
> > most sling bundles (or small group of bundles) are quite
> > autonomous. when
> > we had a CI build for each of them individually this would be
> > much easier.
> > if one breaks the CI is still functional for the other 99%
> > bundles. this
> > leads again into the svn vs. git and huge repo vs. small repos
> > debate, and
> > of course no one wants to maintain a huge set of CI build
> > configurations
> > individually, we would need to automate this.
> >
> > in the past it was difficult to get a consen
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
Was going to write the exact same thing. The pipeline plugin, now just
"pipeline" into jenkins 2, is the future and I would recommend using it.
Except if there are plugins you want to use that are not supported yet.
-Andrei
On Mon, Sep 19, 2016 at 6:02 PM Konrad Windszus wrote:
> Wouldn’t it make more sense to rely on the pipeline plugin instead of the
> Job DSL Plugin? This is by default shipped in Jenkins since 2.0.0 so
> probably there is not even the necessity to install an additional plugin.
> For a comparison between those two approaches look at
> http://stackoverflow.com/questions/37657810/job-dsl-plugin-vs-pipeline-plugin
> .
>
> I don’t have any practical experience with either of those two approaches,
> but since the Pipeline DSL gained a lot more attention from the Jenkins
> community in the past I am wondering whether using that that is not enough
> for Sling.
> Konrad
>
>
> > Am 19.09.2016 um 16:15 schrieb Robert Munteanu :
> >
> > On Mon, 2016-09-19 at 15:53 +0200, Carsten Ziegeler wrote:
> >> The benefit of the current solution is clear: below or equal to zero.
> >> Afaik, Robert look already into having per module jobs in jenkins.
> >> Not
> >> sure how far this is.
> >
> > I pinged infra about this and noone opposed :-)
> >
> > http://mail-archives.apache.org/mod_mbox/www-builds/201609.mbox/brows
> > er
> >
> > I would be happy to look into per-module builds based on Jenkins. I've
> > done some work automating similar setups in the past using Job DSL
> > plugin
> >
> > https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
> >
> > I can file a request for infra to install this plug-in and start
> > testing on a limited set of modules if no one objects. We can then
> > gradually migrate modules off the 'main' job and move them to
> > individual jobs.
> >
> > Thanks,
> >
> > Robert
> >
> >>
> >> Carsten
> >>
> >>>
> >>> +1 for openly talking about this. I think the most important thing
> >>> is
> >>> clearly understanding the benefits and drawbacks for the different
> >>> ways of
> >>> building continuously.
> >>>
> >>> On Mon, Sep 19, 2016 at 3:43 PM Stefan Seifert >>> .de>
> >>> wrote:
> >>>
>
> >
> > for quiet some time now, it seems that the IT tests of the i18n
> > module
> fail in reactor build
>
> i also had a brief look at the tests of the i18n project and was
> not able
> to spot the problem easily.
>
> >
> > I suggest we turn off Jenkins completely
>
> do we have an alternative?
>
> ian invested a good amount of time in getting the sling CI
> running in
> travis, not sure what the status ist.
> currently it seems JDK 1.8 builds run fine, JDK 1.7 builds not
> [1].
>
> in my pov the main problem of our CI builds are that we have two
> really
> huge builds that build everything ("production bundles" and
> contrib), and
> if one of them gets flaky all gets flaky, and it is extremely
> time-consuming trying to fix a problem, wait some hours, try
> again etc.
> plus the fact a lot of problem on the apache Jenkins are
> difficult to
> reproduce locally.
>
> most sling bundles (or small group of bundles) are quite
> autonomous. when
> we had a CI build for each of them individually this would be
> much easier.
> if one breaks the CI is still functional for the other 99%
> bundles. this
> leads again into the svn vs. git and huge repo vs. small repos
> debate, and
> of course no one wants to maintain a huge set of CI build
> configurations
> individually, we would need to automate this.
>
> in the past it was difficult to get a consensus on those topics
> (scm, ci)
> on this list. perhaps we can have a discussion on this on the
> adaptTo() in
> berlin next week.
>
> stefan
>
> [1] https://travis-ci.org/apache/sling/builds
>
>
> >>>
> >>
> >>
> >>
> >>
> >
>
>
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
Wouldn’t it make more sense to rely on the pipeline plugin instead of the Job
DSL Plugin? This is by default shipped in Jenkins since 2.0.0 so probably there
is not even the necessity to install an additional plugin.
For a comparison between those two approaches look at
http://stackoverflow.com/questions/37657810/job-dsl-plugin-vs-pipeline-plugin.
I don’t have any practical experience with either of those two approaches, but
since the Pipeline DSL gained a lot more attention from the Jenkins community
in the past I am wondering whether using that that is not enough for Sling.
Konrad
> Am 19.09.2016 um 16:15 schrieb Robert Munteanu :
>
> On Mon, 2016-09-19 at 15:53 +0200, Carsten Ziegeler wrote:
>> The benefit of the current solution is clear: below or equal to zero.
>> Afaik, Robert look already into having per module jobs in jenkins.
>> Not
>> sure how far this is.
>
> I pinged infra about this and noone opposed :-)
>
> http://mail-archives.apache.org/mod_mbox/www-builds/201609.mbox/brows
> er
>
> I would be happy to look into per-module builds based on Jenkins. I've
> done some work automating similar setups in the past using Job DSL
> plugin
>
> https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
>
> I can file a request for infra to install this plug-in and start
> testing on a limited set of modules if no one objects. We can then
> gradually migrate modules off the 'main' job and move them to
> individual jobs.
>
> Thanks,
>
> Robert
>
>>
>> Carsten
>>
>>>
>>> +1 for openly talking about this. I think the most important thing
>>> is
>>> clearly understanding the benefits and drawbacks for the different
>>> ways of
>>> building continuously.
>>>
>>> On Mon, Sep 19, 2016 at 3:43 PM Stefan Seifert >> .de>
>>> wrote:
>>>
>
> for quiet some time now, it seems that the IT tests of the i18n
> module
fail in reactor build
i also had a brief look at the tests of the i18n project and was
not able
to spot the problem easily.
>
> I suggest we turn off Jenkins completely
do we have an alternative?
ian invested a good amount of time in getting the sling CI
running in
travis, not sure what the status ist.
currently it seems JDK 1.8 builds run fine, JDK 1.7 builds not
[1].
in my pov the main problem of our CI builds are that we have two
really
huge builds that build everything ("production bundles" and
contrib), and
if one of them gets flaky all gets flaky, and it is extremely
time-consuming trying to fix a problem, wait some hours, try
again etc.
plus the fact a lot of problem on the apache Jenkins are
difficult to
reproduce locally.
most sling bundles (or small group of bundles) are quite
autonomous. when
we had a CI build for each of them individually this would be
much easier.
if one breaks the CI is still functional for the other 99%
bundles. this
leads again into the svn vs. git and huge repo vs. small repos
debate, and
of course no one wants to maintain a huge set of CI build
configurations
individually, we would need to automate this.
in the past it was difficult to get a consensus on those topics
(scm, ci)
on this list. perhaps we can have a discussion on this on the
adaptTo() in
berlin next week.
stefan
[1] https://travis-ci.org/apache/sling/builds
>>>
>>
>>
>>
>>
>
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
On Mon, 2016-09-19 at 16:39 +0200, Carsten Ziegeler wrote: > > > > On Mon, 2016-09-19 at 15:53 +0200, Carsten Ziegeler wrote: > > > > > > The benefit of the current solution is clear: below or equal to > > > zero. > > > Afaik, Robert look already into having per module jobs in > > > jenkins. > > > Not > > > sure how far this is. > > > > I pinged infra about this and noone opposed :-) > > > > http://mail-archives.apache.org/mod_mbox/www-builds/201609.mbox/b > > rows > > er > > > > I would be happy to look into per-module builds based on Jenkins. > > I've > > done some work automating similar setups in the past using Job DSL > > plugin > > > > https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin > > > > I can file a request for infra to install this plug-in and start > > testing on a limited set of modules if no one objects. We can then > > gradually migrate modules off the 'main' job and move them to > > individual jobs. > > > Sounds great to me, thanks Robert! Filed https://issues.apache.org/jira/browse/SLING-6061 https://issues.apache.org/jira/browse/INFRA-12632 Let's see where this takes us :-) Robert
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
> On Mon, 2016-09-19 at 15:53 +0200, Carsten Ziegeler wrote: >> The benefit of the current solution is clear: below or equal to zero. >> Afaik, Robert look already into having per module jobs in jenkins. >> Not >> sure how far this is. > > I pinged infra about this and noone opposed :-) > > http://mail-archives.apache.org/mod_mbox/www-builds/201609.mbox/brows > er > > I would be happy to look into per-module builds based on Jenkins. I've > done some work automating similar setups in the past using Job DSL > plugin > > https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin > > I can file a request for infra to install this plug-in and start > testing on a limited set of modules if no one objects. We can then > gradually migrate modules off the 'main' job and move them to > individual jobs. > Sounds great to me, thanks Robert! Carsten -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
On Mon, 2016-09-19 at 15:53 +0200, Carsten Ziegeler wrote:
> The benefit of the current solution is clear: below or equal to zero.
> Afaik, Robert look already into having per module jobs in jenkins.
> Not
> sure how far this is.
I pinged infra about this and noone opposed :-)
http://mail-archives.apache.org/mod_mbox/www-builds/201609.mbox/brows
er
I would be happy to look into per-module builds based on Jenkins. I've
done some work automating similar setups in the past using Job DSL
plugin
https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
I can file a request for infra to install this plug-in and start
testing on a limited set of modules if no one objects. We can then
gradually migrate modules off the 'main' job and move them to
individual jobs.
Thanks,
Robert
>
> Carsten
>
> >
> > +1 for openly talking about this. I think the most important thing
> > is
> > clearly understanding the benefits and drawbacks for the different
> > ways of
> > building continuously.
> >
> > On Mon, Sep 19, 2016 at 3:43 PM Stefan Seifert > .de>
> > wrote:
> >
> > >
> > > >
> > > > for quiet some time now, it seems that the IT tests of the i18n
> > > > module
> > > fail in reactor build
> > >
> > > i also had a brief look at the tests of the i18n project and was
> > > not able
> > > to spot the problem easily.
> > >
> > > >
> > > > I suggest we turn off Jenkins completely
> > >
> > > do we have an alternative?
> > >
> > > ian invested a good amount of time in getting the sling CI
> > > running in
> > > travis, not sure what the status ist.
> > > currently it seems JDK 1.8 builds run fine, JDK 1.7 builds not
> > > [1].
> > >
> > > in my pov the main problem of our CI builds are that we have two
> > > really
> > > huge builds that build everything ("production bundles" and
> > > contrib), and
> > > if one of them gets flaky all gets flaky, and it is extremely
> > > time-consuming trying to fix a problem, wait some hours, try
> > > again etc.
> > > plus the fact a lot of problem on the apache Jenkins are
> > > difficult to
> > > reproduce locally.
> > >
> > > most sling bundles (or small group of bundles) are quite
> > > autonomous. when
> > > we had a CI build for each of them individually this would be
> > > much easier.
> > > if one breaks the CI is still functional for the other 99%
> > > bundles. this
> > > leads again into the svn vs. git and huge repo vs. small repos
> > > debate, and
> > > of course no one wants to maintain a huge set of CI build
> > > configurations
> > > individually, we would need to automate this.
> > >
> > > in the past it was difficult to get a consensus on those topics
> > > (scm, ci)
> > > on this list. perhaps we can have a discussion on this on the
> > > adaptTo() in
> > > berlin next week.
> > >
> > > stefan
> > >
> > > [1] https://travis-ci.org/apache/sling/builds
> > >
> > >
> >
>
>
>
>
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
The benefit of the current solution is clear: below or equal to zero.
Afaik, Robert look already into having per module jobs in jenkins. Not
sure how far this is.
Carsten
> +1 for openly talking about this. I think the most important thing is
> clearly understanding the benefits and drawbacks for the different ways of
> building continuously.
>
> On Mon, Sep 19, 2016 at 3:43 PM Stefan Seifert
> wrote:
>
>>> for quiet some time now, it seems that the IT tests of the i18n module
>> fail in reactor build
>>
>> i also had a brief look at the tests of the i18n project and was not able
>> to spot the problem easily.
>>
>>> I suggest we turn off Jenkins completely
>>
>> do we have an alternative?
>>
>> ian invested a good amount of time in getting the sling CI running in
>> travis, not sure what the status ist.
>> currently it seems JDK 1.8 builds run fine, JDK 1.7 builds not [1].
>>
>> in my pov the main problem of our CI builds are that we have two really
>> huge builds that build everything ("production bundles" and contrib), and
>> if one of them gets flaky all gets flaky, and it is extremely
>> time-consuming trying to fix a problem, wait some hours, try again etc.
>> plus the fact a lot of problem on the apache Jenkins are difficult to
>> reproduce locally.
>>
>> most sling bundles (or small group of bundles) are quite autonomous. when
>> we had a CI build for each of them individually this would be much easier.
>> if one breaks the CI is still functional for the other 99% bundles. this
>> leads again into the svn vs. git and huge repo vs. small repos debate, and
>> of course no one wants to maintain a huge set of CI build configurations
>> individually, we would need to automate this.
>>
>> in the past it was difficult to get a consensus on those topics (scm, ci)
>> on this list. perhaps we can have a discussion on this on the adaptTo() in
>> berlin next week.
>>
>> stefan
>>
>> [1] https://travis-ci.org/apache/sling/builds
>>
>>
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziege...@apache.org
Re: CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
+1 for openly talking about this. I think the most important thing is
clearly understanding the benefits and drawbacks for the different ways of
building continuously.
On Mon, Sep 19, 2016 at 3:43 PM Stefan Seifert
wrote:
> > for quiet some time now, it seems that the IT tests of the i18n module
> fail in reactor build
>
> i also had a brief look at the tests of the i18n project and was not able
> to spot the problem easily.
>
> >I suggest we turn off Jenkins completely
>
> do we have an alternative?
>
> ian invested a good amount of time in getting the sling CI running in
> travis, not sure what the status ist.
> currently it seems JDK 1.8 builds run fine, JDK 1.7 builds not [1].
>
> in my pov the main problem of our CI builds are that we have two really
> huge builds that build everything ("production bundles" and contrib), and
> if one of them gets flaky all gets flaky, and it is extremely
> time-consuming trying to fix a problem, wait some hours, try again etc.
> plus the fact a lot of problem on the apache Jenkins are difficult to
> reproduce locally.
>
> most sling bundles (or small group of bundles) are quite autonomous. when
> we had a CI build for each of them individually this would be much easier.
> if one breaks the CI is still functional for the other 99% bundles. this
> leads again into the svn vs. git and huge repo vs. small repos debate, and
> of course no one wants to maintain a huge set of CI build configurations
> individually, we would need to automate this.
>
> in the past it was difficult to get a consensus on those topics (scm, ci)
> on this list. perhaps we can have a discussion on this on the adaptTo() in
> berlin next week.
>
> stefan
>
> [1] https://travis-ci.org/apache/sling/builds
>
>
CI alternatives for Sling (was: [i18n] IT Tests failing in reactor build)
> for quiet some time now, it seems that the IT tests of the i18n module fail
> in reactor build
i also had a brief look at the tests of the i18n project and was not able to
spot the problem easily.
>I suggest we turn off Jenkins completely
do we have an alternative?
ian invested a good amount of time in getting the sling CI running in travis,
not sure what the status ist.
currently it seems JDK 1.8 builds run fine, JDK 1.7 builds not [1].
in my pov the main problem of our CI builds are that we have two really huge
builds that build everything ("production bundles" and contrib), and if one of
them gets flaky all gets flaky, and it is extremely time-consuming trying to
fix a problem, wait some hours, try again etc. plus the fact a lot of problem
on the apache Jenkins are difficult to reproduce locally.
most sling bundles (or small group of bundles) are quite autonomous. when we
had a CI build for each of them individually this would be much easier. if one
breaks the CI is still functional for the other 99% bundles. this leads again
into the svn vs. git and huge repo vs. small repos debate, and of course no one
wants to maintain a huge set of CI build configurations individually, we would
need to automate this.
in the past it was difficult to get a consensus on those topics (scm, ci) on
this list. perhaps we can have a discussion on this on the adaptTo() in berlin
next week.
stefan
[1] https://travis-ci.org/apache/sling/builds
