Hi, so have you really measured the time you are "wasting" ? Furthermore which Maven version and plugin versions do you use? Are you using freestyle or Maven job type...
On Saturday, July 30, 2016 at 8:13:14 PM UTC+2, Jesse Farinacci wrote: > > My team faces similar challenges, and I agree with almost everything > said so far. I definitely echo the sentiment that it feels like there > is a lot of wasted repetition when invoking discrete phases > separately. We currently do this via separate jobs now, which often > run at different frequencies. However, we have been able to configure > Maven such that we really only have two repetitious phases, across > these multiple jobs, which luckily are not costly for us. Maven > generate-resources and compile phases, as we can skip over tests and > deployments which are not part of the logical unit being executed. In > the end, even though we know we are wasting some execution time, it > turns out not to be all that much at all.. and isn't yet worthy of any > kind of optimization or special attention. > > So right now our (~2min) compile+unit test runs every commit and only > runs Junit tests. 2 Minutes..Ok how many time is "wasted" (timestamper plugin and the mojo execution in Jenkins if you you are using maven job type can help here) BTW: How many modules does you project contains of? > Our (~8min) compile+integration test runs almost > every commit+jenkins build window to group multiple commits and only > runs Arquillian tests. You might run jobs in parallel ? > Our (~15min) compile+user acceptance test runs > every day and only runs Selenium tests. Then we have lots of on the > side jobs which ensure legal compliance, code coverage, file encoding > fixes, code style fixes, static analysis, etc etc, which run hourly, > daily, or weekly, and are mostly fast (<5min). > > We are exploring Jenkins Pipeline in order to chain, both sequential > where necessary and then parallel where it makes sense, these discrete > jobs in such a way that we will have a continuous delivery available > at the end which has run the full gauntlet of all our testing and best > practices verification for every commit without doing unnecessary and > expensive steps, failing the pipeline as soon as possible. Each commit > triggers a build which triggers a new git branch which is passed to > subsequent phases, if any stage fails, the branch is deleted and makes > no further progress. Additionally, we are trying to leverage separate > Jenkins jobs for each discrete job so that we can trigger it on demand > in order to do some quick checking, we also find that we can get > easier cross-project reuse using that technique. We're using Git, > Apache Maven, Jenkins CI, Jacoco, Arquillian, SonarQube, Checkstyle, > PMD, FindBugs, and many others. > > There are only two critiques we can find with this style: 1) the > aforementioned modest amount of wasted generate-resources and compile > phases across multiple jobs in the same pipeline, and 2) each phase is > producing artifacts to be tested and not reusing the ones built and > verified at previous stages, thus there may be a hypothetical > difference between what different stages are actually testing, though > we think the risk of this is not consequential. You could create in Jenkins Maven repositories to provided the artifacts to other steps...Or can package them by Jenkins an give them to the next step... So it sounds you don't trust your version control ? If you checkout the same sha1 you will get always the same state? And if you build from that you will always get the same result didn't you ? Or I could say you don't trust the build system to create the same result from the same source state ? > This technique otherwise seems like a holy grail to me and the team. I welcome ideas > about it. > If it is already the hole grail I can't say anything about it... > > On Mon, Jul 25, 2016 at 10:36 AM, Victor Noël <[email protected] > <javascript:>> wrote: > > Thanks for your feedbacks! > > > > So in the end, what is the solution you adopted? more coarse grained > stages? > > > > Le dimanche 24 juillet 2016 14:18:18 UTC+2, [email protected] a > écrit : > >> > >> Hello Victor, > >> > >> I am faced with a similar problem and gave up on the idea rather early > on. > >> Here are some of the characteristics of multiple stages of a pipeline > >> > >> Ability of each stage to be explictly invoked by some command > >> Ability of each stage to be re-run in such a way that it can start off > >> from where the previous stage finished > >> > >> The maven lifecycle reactor does not play well with these concepts. For > >> example, there is no elegant way to tell maven to compile the code > assuming > >> that the generation phase has happened. Likewise, one cannot ask maven > to > >> simply execute the tests assuming that compilation has already taken > place. > >> The short answer that the benefits of breaking something into various > >> pipeline stages is lost when maven. > >> > >> The situation with gradle is not better as far as the relationship with > >> the build tool and jenkins is concerned. However, the fact that gradle > is > >> much better at quickly checking if earlier steps have been completed > makes > >> it only marginally better. > >> > >> > >> On Saturday, July 23, 2016 at 1:10:08 PM UTC+5:30, Victor Noël wrote: > >>> > >>> Hi, > >>> > >>> I'm trying to wrap wy head around moving to pipeline jobs. > >>> > >>> Currently I use maven jobs (but my question applies to freestyle jobs > >>> that execute maven) by executing one single build command that will > validate > >>> constraints (checkstyle and/or other stuffs like that), generate code, > >>> compile, run test, run it test and deploy artefacts. > >>> > >>> What is the best way to convert to pipeline with multiple stages? > >>> > >>> My main concern is that if I simply run each of the step with maven in > >>> each stage, then I feel like some of steps from maven may be executed > >>> multiple times (such as test being executed again for it tests or for > >>> deploy). > >>> > >>> Is there some success stories around here on how to handle such a > >>> migration? > >>> > >>> Thanks! > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "Jenkins Users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to [email protected] <javascript:>. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/jenkinsci-users/e869c7c4-2ea5-4d8b-8268-7a083348b2b7%40googlegroups.com. > > > > > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/bee0d30c-583a-4fbe-88fe-3f29259a6690%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
