JB - for your comparison, please also omit cross-compiling all the Go examples because they are only built using Gradle.
On Tue, May 1, 2018 at 8:59 AM Jean-Baptiste Onofré <[email protected]> wrote: > Thanks for the update Kenn, that makes sense. > > I'm checking the artifacts generated by Gradle right now. > > Regards > JB > > On 01/05/2018 17:42, Kenneth Knowles wrote: > > Raw execution time for tasks from clean is not the only thing to test. I > > would say it is not even important. Try these from clean: > > > > - Gradle: ./gradlew :beam-sdks-java-io-mongodb:test && ./gradlew > > :beam-sdks-java-io-mongodb:test > > - Maven: mvn -pl sdks/java/io/mongodb test -am && mvn -pl > > sdks/java/io/mongodb test -am > > > > Quick run on my laptop: > > > > - Gradle: 66s (65s then 1s) > > - Maven: 317s (173s then 144s) > > > > Of course, the mvn command runs a bunch of useless executions AND it is > > incorrect because it isn't using built jars. That's part of the point - > > there is no way to do what you want with mvn. Let's try to make a > > command that avoids useless work and builds the jars: > > > > - Maven: (mvn -pl sdks/java/io/mongodb install -DskipTests -am && mvn > > -pl sdks/java/io/mongodb test) && (each time) > > > > That takes 102s the first time and 64s the second time. And that is > > about the realistic workflow for someone trying to get something done. > > Even if we touch a file Gradle finishes in 20s. So the best case for mvn > > is this head-to-head: > > > > - Gradle: 65s + 20s + 20s + 20s + 20s + ... > > - Maven: 102s + 64s + 64s + 64s + 64s + ... > > > > Kenn > > > > > > On Tue, May 1, 2018 at 8:09 AM Jean-Baptiste Onofré <[email protected] > > <mailto:[email protected]>> wrote: > > > > Thanks, for me, Maven 3.5.2 takes quite the same time than Gradle > > (using > > the wrapper). It's maybe related to my environment. > > > > Anyway, I'm doing a complete build review both in term of building > > time, > > and equivalence (artifacts publishing, test, plugin execution). > > > > I will provide an update soon. > > > > Regards > > JB > > > > On 01/05/2018 16:57, Reuven Lax wrote: > > > Luke did gather data which showed that on our Jenkins executors > the > > > Gradle build was much faster than the Maven build. Also right now > we > > > have incremental builds turned off, but once we're confident > > enough to > > > enable them (at least for local development) that will often drop > > build > > > times a lot. > > > > > > On Tue, May 1, 2018 at 4:01 AM Jean-Baptiste Onofré > > <[email protected] <mailto:[email protected]> > > > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > > > > > By the way, I'm curious: did someone evaluate the build time > gap > > > between Maven > > > and Gradle ? One of the main reason to migrate to Gradle was > > the inc > > > build and > > > build time. The builds I have launched are quite the same in > > > duration. I will do > > > deeper tests to evaluate the gap. > > > > > > Regards > > > JB > > > > > > On 05/01/2018 12:48 PM, Łukasz Gajowy wrote: > > > > Hi Scott, > > > > > > > > thanks for the update! Just a clarification about IO > > performance > > > tests: those > > > > were fully migrated in Beam and all task necessary for > running > > > them are there > > > > but Jenkins jobs still run mvn commands. This is due the > > fact that > > > > PerfkitBenchmarker code (which is invoked by Jenkins and > > > constructs the commands > > > > by itself) was not updated yet. This should be finished > before > > > fully dropping mvn. > > > > > > > > More on that topic here, in > > > > comments: https://issues.apache.org/jira/browse/BEAM-3942 > > > > PR changing the commands to gradle is waiting for PerfKit > > devs review > > > > here: > > > > https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/1648 > > > > > > > > Best regards, > > > > > > > > 2018-05-01 9:17 GMT+02:00 Romain Manni-Bucau > > > <[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>> > > > > <mailto:[email protected] > > <mailto:[email protected]> <mailto:[email protected] > > <mailto:[email protected]>>>>: > > > > > > > > Hi Scott > > > > > > > > While > > > > https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057 > > > > > > > > > < > https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057> is > > > > open, gradle is a concurrent of maven but maven must > > stay the > > > default build > > > > tool cause gradle breaks users. > > > > > > > > > > > > Le 1 mai 2018 01:59, "Scott Wegner" > > <[email protected] <mailto:[email protected]> > > > <mailto:[email protected] <mailto:[email protected]>> > > > > <mailto:[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>>> a > > > écrit : > > > > > > > > Many many of you have been hacking diligently on > the > > > Gradle build, and > > > > I'm happy to announce that we now have a > > > fully-functioning Gradle build! > > > > There's been a ton of progress since our last > > update [1]: > > > > > > > > * Improved nightly snapshot release [2] > > > > * Improve runner quickstarts [5] [11] > > > > * Python post-commit ported to Gradle [3] > > > > * Update performance testing framework for Gradle > > [4] [12] > > > > * Generate javadocs from Gradle [6] > > > > * Update to latest Gradle version [7] [21] > > > > * Updated documentation [8] [22] > > > > * Tune CI build resource usage for Jenkins [9] [19] > > > > * Improve shading of test jars [10] [13] [14] > > > > * Add 'errorprone' and 'spotless' static analysis > > [15] [24] > > > > * Improve IntelliJ project generation [16] [17] > > > > * Reduce number of ValidatesRunner tests [18] > > > > * Update release documentation for Gradle [20] > > > > * Update docker build scripts for Gradle [23] > > > > > > > > The build process and Jenkins environment have > > stabilized > > > and we've > > > > resolved migration blockers. The final step is to > use > > > Gradle to produce > > > > an official release. The release documentation has > > been > > > updated for > > > > Gradle and I recommend we use these docs for the > 2.5.0 > > > release. Assuming > > > > the release goes well, we can declare the > > migration fully > > > validated and > > > > stop supporting dual build systems. > > > > > > > > During the migration we identified a number of > > > opportunities to improve > > > > the build even further. Feel free to grab one of > the > > > items off of the > > > > JIRA: BEAM-4045 [24] > > > > > > > > Thanks again to all those that contributed. This > has > > > truly been a > > > > community effort! > > > > > > > > [1] > > > > > > https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E > > > > > > > > > < > https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E > > > > > > [2] https://github.com/apache/beam/pull/5142 > > > > <https://github.com/apache/beam/pull/5142> > > > > [3] https://github.com/apache/beam/pull/5146 > > > > <https://github.com/apache/beam/pull/5146> > > > > [4] https://github.com/apache/beam/pull/5003 > > > > <https://github.com/apache/beam/pull/5003> > > > > [5] https://github.com/apache/beam/pull/5151 > > > > <https://github.com/apache/beam/pull/5151> > > > > [6] https://github.com/apache/beam/pull/5121 > > > > <https://github.com/apache/beam/pull/5121> > > > > [7] https://github.com/apache/beam/pull/5104 > > > > <https://github.com/apache/beam/pull/5104> > > > > [8] https://github.com/apache/beam/pull/5183 > > > > <https://github.com/apache/beam/pull/5183> > > > > [9] https://github.com/apache/beam/pull/5171 > > > > <https://github.com/apache/beam/pull/5171> > > > > [10] https://github.com/apache/beam/pull/5117 > > > > <https://github.com/apache/beam/pull/5117> > > > > [11] https://github.com/apache/beam/pull/5200 > > > > <https://github.com/apache/beam/pull/5200> > > > > [12] https://github.com/apache/beam/pull/5051 > > > > <https://github.com/apache/beam/pull/5051> > > > > [13] https://github.com/apache/beam/pull/4740 > > > > <https://github.com/apache/beam/pull/4740> > > > > [14] https://github.com/apache/beam/pull/4702 > > > > <https://github.com/apache/beam/pull/4702> > > > > [15] https://github.com/apache/beam/pull/4701 > > > > <https://github.com/apache/beam/pull/4701> > > > > [16] https://github.com/apache/beam/pull/4626 > > > > <https://github.com/apache/beam/pull/4626> > > > > [17] https://github.com/apache/beam/pull/4625 > > > > <https://github.com/apache/beam/pull/4625> > > > > [18] https://github.com/apache/beam/pull/5193 > > > > <https://github.com/apache/beam/pull/5193> > > > > [19] https://github.com/apache/beam/pull/5222 > > > > <https://github.com/apache/beam/pull/5222> > > > > [20] https://github.com/apache/beam/pull/5187 > > > > <https://github.com/apache/beam/pull/5187> > > > > [21] https://github.com/apache/beam/pull/5217 > > > > <https://github.com/apache/beam/pull/5217> > > > > [22] https://github.com/apache/beam/pull/5115 > > > > <https://github.com/apache/beam/pull/5115> > > > > [23] https://github.com/apache/beam/pull/5252 > > > > <https://github.com/apache/beam/pull/5252> > > > > [24] https://github.com/apache/beam/pull/5161 > > > > <https://github.com/apache/beam/pull/5161> > > > > [25] > https://issues.apache.org/jira/browse/BEAM-4045 > > > > <https://issues.apache.org/jira/browse/BEAM-4045> > > > > -- > > > > > > > > > > > > Got feedback? http://go/swegner-feedback > > > > > > > > > > > > > > -- > > > Jean-Baptiste Onofré > > > [email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>> > > > http://blog.nanthrax.net > > > Talend - http://www.talend.com > > > > > >
