+1 on only specifying the target that you need to build, You should use './gradlew -p path/to/java/project assemble' OR './gradlew :project-artifact-name:assemble' to build the jars that you should need. You can run these commands in a checked out version of your workspace and validate that they produce what you expect.
On Tue, May 8, 2018 at 9:17 AM Scott Wegner <sweg...@google.com> wrote: > A few thoughts: > > 1. Gradle can intelligently build only the dependencies necessary for a > task, so it shouldn't build all of Python if the test suite, if you only > specify the task you're interested in. I'm not sure of the command for > "build all of the dependencies of my tests but don't run my tests"; maybe > "./gradlew mytests -x mytests" ? > > 2. Some tasks in the build are not yet cacheable for various reasons. So > you may see them getting rebuilt on the second execution even on success, > which would then be included in your overall build timing. Information > about which tasks were used from the build cache is available in the Gradle > build scan (--scan). > > Another idea for measuring the execution time of just your tests would be > to pull this out of Gradle's build report. Adding the --profile flag > generates a report in $buildDir/reports/profile, which should have the > timing info for just the task you're interested in: > https://docs.gradle.org/current/userguide/command_line_interface.html > > On Tue, May 8, 2018 at 8:23 AM Łukasz Gajowy <lukasz.gaj...@gmail.com> > wrote: > >> Hi Beam Devs, >> >> currently PerfkitBenchmarker (a tool used to invoke performance tests) >> has two phases that run gradle commands: >> >> - Pre-build phase: this is where all the beam repo is build. This >> phase is to prepare the necessary artifacts so that it doesn't happen when >> executing tests. >> - Actual test running phase. After all necessary code is built we run >> the test and measure it's execution time. The execution time is displayed >> on the PerfKit dashboard [1]. >> >> After the recent mvn - gradle migration we noticed that we are unable to >> "Pre build" the code[2]. Because one of the python related tasks fails, the >> whole "preBuild" phase fails silently and the actual building happens in >> the "test running" phase which increases the execution time (this is >> visible in the plots on the dashboard). >> >> This whole situation made me wonder about several things, and I'd like to >> ask you for opinions. I think: >> >> - we should skip all the python related tasks while building beam for >> java performance tests in PerfKit. Those are not needed anyway when we are >> running java tests. Is it possible to skip them in one go (eg. the same >> fashion we skip all checks using -xcheck option)? >> - the same goes for Python tests: we should skip all java related >> tasks when building beam for python performance tests in PerfKit. Note >> that >> this bullet is something to be developed in the future, as >> beam_PerformanceTests_Python job (the only Python Performance test job) is >> failing for 4 months now and seems abandoned. IMO it should be done when >> someone will bring the test back to life. For now the job should be >> disabled. >> - we should modify Perfkit so that when the prebuild phase fails for >> some reason, the test is not executed. Now we don't do this and the test >> execution time depends on whether "gradle integrationTest" command builds >> something or just runs the test. IMO when the command has to build >> anything >> the execution time should not be included in the Dashboards, because it's >> a >> false result. >> >> What do you think of all this? >> >> [1] >> https://apache-beam-testing.appspot.com/explore?dashboard=5755685136498688 >> [2] https://issues.apache.org/jira/browse/BEAM-4256 >> >> Best regards, >> Łukasz Gajowy >> >> >> >>
