[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=118300=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118300 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 02/Jul/18 17:53 Start Date: 02/Jul/18 17:53 Worklog Time Spent: 10m Work Description: kennknowles commented on issue #5858: [BEAM-3249] Update the build script to be compatible with building Gradle within Google URL: https://github.com/apache/beam/pull/5858#issuecomment-401883855 Nice find. How unfortunate. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 118300) Time Spent: 14h 50m (was: 14h 40m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 14h 50m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=118270=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118270
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 02/Jul/18 17:30
Start Date: 02/Jul/18 17:30
Worklog Time Spent: 10m
Work Description: lukecwik closed pull request #5858: [BEAM-3249] Update
the build script to be compatible with building Gradle within Google
URL: https://github.com/apache/beam/pull/5858
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build.gradle b/build.gradle
index cccb4b9c7b4..ff7a38ae0b6 100644
--- a/build.gradle
+++ b/build.gradle
@@ -33,13 +33,6 @@ buildscript {
repositories {
maven { url offlineRepositoryRoot }
-if (gradle.hasProperty('extraMavenUrls')) {
- gradle.ext.extraMavenUrls.each {
-mavenUrl ->
- return maven { url mavenUrl }
- }
-}
-
// To run gradle in offline mode, one must first invoke
// 'updateOfflineRepository' to create an offline repo
// inside the root project directory. See the application
@@ -57,7 +50,6 @@ buildscript {
}
dependencies {
classpath 'net.researchgate:gradle-release:2.6.0'
// Enable gradle-based release management
-classpath "com.gradle:build-scan-plugin:1.13.1"
// Enable publishing build scans
classpath "net.ltgt.gradle:gradle-apt-plugin:0.13"
// Enable a Java annotation processor
classpath "com.google.protobuf:protobuf-gradle-plugin:0.8.5"
// Enable proto code generation
classpath "io.spring.gradle:propdeps-plugin:0.0.9.RELEASE"
// Enable provided and optional configurations
@@ -72,6 +64,11 @@ buildscript {
classpath
"gradle.plugin.io.pry.gradle.offline_dependencies:gradle-offline-dependencies-plugin:0.3"
// Enable creating an offline repository
classpath "net.ltgt.gradle:gradle-errorprone-plugin:0.0.13"
// Enable errorprone Java static analysis
classpath "com.github.ben-manes:gradle-versions-plugin:0.17.0"
// Enable dependency checks
+
+// Plugins which require online access should not be enabled when running
in offline mode.
+if (!gradle.startParameter.isOffline()) {
+ classpath "com.gradle:build-scan-plugin:1.13.1"
// Enable publishing build scans
+}
}
}
@@ -80,7 +77,11 @@ buildscript {
apply plugin: "base"
-apply plugin: "com.gradle.build-scan"
+// Plugins which require online access should not be enabled when running in
offline mode.
+if (!gradle.startParameter.isOffline()) {
+ apply plugin: "com.gradle.build-scan"
+}
+
apply plugin: 'com.github.ben-manes.versions'
// JENKINS_HOME and BUILD_ID set automatically during Jenkins execution
def isCIBuild = ['JENKINS_HOME', 'BUILD_ID'].every System.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 118270)
Time Spent: 14h 40m (was: 14.5h)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 14h 40m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
>
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=118257=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118257 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 02/Jul/18 17:25 Start Date: 02/Jul/18 17:25 Worklog Time Spent: 10m Work Description: markflyhigh commented on issue #5858: [BEAM-3249] Update the build script to be compatible with building Gradle within Google URL: https://github.com/apache/beam/pull/5858#issuecomment-401876208 Thank you! LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 118257) Time Spent: 14.5h (was: 14h 20m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 14.5h > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=93004=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-93004
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 20/Apr/18 01:10
Start Date: 20/Apr/18 01:10
Worklog Time Spent: 10m
Work Description: aaltay closed pull request #5117: [BEAM-3249] Clean-up
and use shaded test jars, removing evaluationDependsOn
URL: https://github.com/apache/beam/pull/5117
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/examples/java/build.gradle b/examples/java/build.gradle
index 76889cafa59..4abe28b5209 100644
--- a/examples/java/build.gradle
+++ b/examples/java/build.gradle
@@ -26,15 +26,6 @@ ext.summary = """Apache Beam SDK provides a simple,
Java-based
interface for processing virtually any size data. This
artifact includes all Apache Beam Java SDK examples."""
-/*
- * We need to rely on manually specifying these evaluationDependsOn to ensure
that
- * the following projects are evaluated before we evaluate this project. This
is because
- * we are attempting to reference the "sourceSets.test.output" directly.
- * TODO: Swap to generating test artifacts which we can then rely on instead of
- * the test outputs directly.
- */
-evaluationDependsOn(":beam-sdks-java-io-google-cloud-platform")
-
/** Define the list of runners which execute a precommit test. */
// https://issues.apache.org/jira/browse/BEAM-3583
def preCommitRunners = ["dataflowRunner", "dataflowStreamingRunner",
"directRunner", "flinkRunner", "sparkRunner"]
@@ -68,7 +59,7 @@ dependencies {
shadow library.java.slf4j_api
shadow project(path: ":beam-runners-direct-java", configuration: "shadow")
shadow library.java.slf4j_jdk14
- shadowTest
project(":beam-sdks-java-io-google-cloud-platform").sourceSets.test.output
+ shadowTest project(path: ":beam-sdks-java-io-google-cloud-platform",
configuration: "shadowTest")
shadowTest library.java.hamcrest_core
shadowTest library.java.junit
shadowTest library.java.mockito_core
diff --git
a/examples/java/src/main/java/org/apache/beam/examples/complete/game/GameStats.java
b/examples/java/src/main/java/org/apache/beam/examples/complete/game/GameStats.java
index d81aa30892a..f93a24266a1 100644
---
a/examples/java/src/main/java/org/apache/beam/examples/complete/game/GameStats.java
+++
b/examples/java/src/main/java/org/apache/beam/examples/complete/game/GameStats.java
@@ -153,7 +153,7 @@ public void processElement(ProcessContext c, BoundedWindow
window) {
/**
* Options supported by {@link GameStats}.
*/
- interface Options extends LeaderBoard.Options {
+ public interface Options extends LeaderBoard.Options {
@Description("Numeric value of fixed window duration for user analysis, in
minutes")
@Default.Integer(60)
Integer getFixedWindowDuration();
diff --git
a/examples/java/src/main/java/org/apache/beam/examples/complete/game/HourlyTeamScore.java
b/examples/java/src/main/java/org/apache/beam/examples/complete/game/HourlyTeamScore.java
index 86c5196c6cb..0f2912894d3 100644
---
a/examples/java/src/main/java/org/apache/beam/examples/complete/game/HourlyTeamScore.java
+++
b/examples/java/src/main/java/org/apache/beam/examples/complete/game/HourlyTeamScore.java
@@ -82,7 +82,7 @@
/**
* Options supported by {@link HourlyTeamScore}.
*/
- interface Options extends UserScore.Options {
+ public interface Options extends UserScore.Options {
@Description("Numeric value of fixed window duration, in minutes")
@Default.Integer(60)
diff --git
a/examples/java/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java
b/examples/java/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java
index 732840b1a1e..31dc48944e4 100644
---
a/examples/java/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java
+++
b/examples/java/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java
@@ -96,7 +96,7 @@
/**
* Options supported by {@link LeaderBoard}.
*/
- interface Options extends HourlyTeamScore.Options, ExampleOptions,
StreamingOptions {
+ public interface Options extends HourlyTeamScore.Options, ExampleOptions,
StreamingOptions {
@Description("BigQuery Dataset to write tables to. Must already exist.")
@Validation.Required
diff --git a/runners/apex/build.gradle b/runners/apex/build.gradle
index 713abb56cee..d7a57bad30b 100644
--- a/runners/apex/build.gradle
+++ b/runners/apex/build.gradle
@@ -27,11 +27,7 @@ description = "Apache Beam :: Runners :: Apex"
* We need to rely on manually specifying these evaluationDependsOn to ensure
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=92921=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-92921 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 19/Apr/18 23:21 Start Date: 19/Apr/18 23:21 Worklog Time Spent: 10m Work Description: swegner commented on issue #5117: [BEAM-3249] Clean-up and use shaded test jars, removing evaluationDependsOn URL: https://github.com/apache/beam/pull/5117#issuecomment-382910451 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 92921) Time Spent: 14h 10m (was: 14h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 14h 10m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=92661=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-92661 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 19/Apr/18 16:19 Start Date: 19/Apr/18 16:19 Worklog Time Spent: 10m Work Description: swegner commented on issue #5117: [BEAM-3249] Clean-up and use shaded test jars, removing evaluationDependsOn URL: https://github.com/apache/beam/pull/5117#issuecomment-382795830 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 92661) Time Spent: 14h (was: 13h 50m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 14h > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=92294=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-92294 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 18/Apr/18 23:22 Start Date: 18/Apr/18 23:22 Worklog Time Spent: 10m Work Description: swegner commented on issue #5117: [BEAM-3249] Clean-up and use shaded test jars, removing evaluationDependsOn URL: https://github.com/apache/beam/pull/5117#issuecomment-382559541 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 92294) Time Spent: 13h 50m (was: 13h 40m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 13h 50m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=92255=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-92255 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 18/Apr/18 22:03 Start Date: 18/Apr/18 22:03 Worklog Time Spent: 10m Work Description: swegner commented on issue #5117: [BEAM-3249] Clean-up and use shaded test jars, removing evaluationDependsOn URL: https://github.com/apache/beam/pull/5117#issuecomment-382543681 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 92255) Time Spent: 13h 40m (was: 13.5h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 13h 40m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=92170=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-92170 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 18/Apr/18 17:03 Start Date: 18/Apr/18 17:03 Worklog Time Spent: 10m Work Description: swegner commented on issue #5117: [BEAM-3249] Clean-up and use shaded test jars, removing evaluationDependsOn URL: https://github.com/apache/beam/pull/5117#issuecomment-382458187 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 92170) Time Spent: 13.5h (was: 13h 20m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 13.5h > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=92149=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-92149 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 18/Apr/18 15:51 Start Date: 18/Apr/18 15:51 Worklog Time Spent: 10m Work Description: swegner commented on issue #5117: [BEAM-3249] Clean-up and use shaded test jars, removing evaluationDependsOn URL: https://github.com/apache/beam/pull/5117#issuecomment-382435883 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 92149) Time Spent: 13h 20m (was: 13h 10m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 13h 20m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=91487=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91487 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 16/Apr/18 21:03 Start Date: 16/Apr/18 21:03 Worklog Time Spent: 10m Work Description: swegner commented on issue #5117: [BEAM-3249] Clean-up and use shaded test jars, removing evaluationDependsOn URL: https://github.com/apache/beam/pull/5117#issuecomment-381748609 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 91487) Time Spent: 13h 10m (was: 13h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 13h 10m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=90612=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90612 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 12/Apr/18 22:21 Start Date: 12/Apr/18 22:21 Worklog Time Spent: 10m Work Description: swegner commented on issue #5117: [BEAM-3249] Clean-up and use shaded test jars, removing evaluationDependsOn URL: https://github.com/apache/beam/pull/5117#issuecomment-380962180 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 90612) Time Spent: 13h (was: 12h 50m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 13h > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=90575=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90575 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 12/Apr/18 20:54 Start Date: 12/Apr/18 20:54 Worklog Time Spent: 10m Work Description: swegner commented on issue #5117: [BEAM-3249] Clean-up and use shaded test jars, removing evaluationDependsOn URL: https://github.com/apache/beam/pull/5117#issuecomment-380941379 R: @aaltay This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 90575) Time Spent: 12h 50m (was: 12h 40m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 12h 50m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=90574=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90574 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 12/Apr/18 20:54 Start Date: 12/Apr/18 20:54 Worklog Time Spent: 10m Work Description: swegner commented on issue #5117: [BEAM-3249] Clean-up and use shaded test jars, removing evaluationDependsOn URL: https://github.com/apache/beam/pull/5117#issuecomment-380941309 FYI, this is the previous commit from #5107 which seems to have somehow gotten excluded after a force push. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 90574) Time Spent: 12h 40m (was: 12.5h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 12h 40m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=90235=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90235 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 12/Apr/18 00:01 Start Date: 12/Apr/18 00:01 Worklog Time Spent: 10m Work Description: chamikaramj closed pull request #5107: [BEAM-3249] Clean-up and use shaded test jars, removing evaluationDependsOn URL: https://github.com/apache/beam/pull/5107 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 90235) Time Spent: 12.5h (was: 12h 20m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 12.5h > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=90164=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90164 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 11/Apr/18 20:47 Start Date: 11/Apr/18 20:47 Worklog Time Spent: 10m Work Description: lukecwik opened a new pull request #5107: [BEAM-3249] Clean-up and use shaded test jars, removing evaluationDependsOn URL: https://github.com/apache/beam/pull/5107 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand: - [ ] What the pull request does - [ ] Why it does it - [ ] How it does it - [ ] Why this approach - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 90164) Time Spent: 12h 20m (was: 12h 10m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 12h 20m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=90152=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90152
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 11/Apr/18 20:03
Start Date: 11/Apr/18 20:03
Worklog Time Spent: 10m
Work Description: lukecwik closed pull request #5103: [BEAM-3249] Only
generate all artifacts when publishing.
URL: https://github.com/apache/beam/pull/5103
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build_rules.gradle b/build_rules.gradle
index fba441e2677..8f54fe0f478 100644
--- a/build_rules.gradle
+++ b/build_rules.gradle
@@ -325,7 +325,6 @@ ext.DEFAULT_SHADOW_CLOSURE = {
class JavaNatureConfiguration {
double javaVersion = 1.8// Controls the JDK source language and
target compatibility
boolean enableFindbugs = true // Controls whether the findbugs plugin is
enabled and configured
- boolean enableShadow = true // Controls whether the shadow plugin is
enabled and configured
// The shadowJar / shadowTestJar tasks execute the following closure to
configure themselves.
// Users can compose their closure with the default closure via:
// DEFAULT_SHADOW_CLOSURE << {
@@ -349,6 +348,7 @@ class JavaNatureConfiguration {
// * maven
// * net.ltgt.apt (plugin to configure annotation processing tool)
// * propdeps (provide optional and provided dependency configurations)
+// * propdeps-maven
// * checkstyle
// * findbugs
// * shadow
@@ -447,6 +447,7 @@ ext.applyJavaNature = {
// TODO: Either remove these plugins and find another way to generate the
Maven poms
// with the correct dependency scopes configured.
apply plugin: 'propdeps'
+ apply plugin: 'propdeps-maven'
// Configures a checkstyle plugin enforcing a set of rules and also allows
for a set of
// suppressions.
@@ -494,64 +495,62 @@ ext.applyJavaNature = {
//
// TODO: Enforce all relocations are always performed to:
// getJavaRelocatedPath(package_suffix) where package_suffix is something
like "com.google.commmon"
- if (configuration.enableShadow) {
-apply plugin: 'com.github.johnrengelman.shadow'
+ apply plugin: 'com.github.johnrengelman.shadow'
-// Create a new configuration 'shadowTest' like 'shadow' for the test scope
-configurations {
- shadow {
-description = "Dependencies for shaded source set 'main'"
- }
- compile.extendsFrom shadow
- shadowTest {
-description = "Dependencies for shaded source set 'test'"
-extendsFrom shadow
- }
- testCompile.extendsFrom shadowTest
+ // Create a new configuration 'shadowTest' like 'shadow' for the test scope
+ configurations {
+shadow {
+ description = "Dependencies for shaded source set 'main'"
+}
+compile.extendsFrom shadow
+shadowTest {
+ description = "Dependencies for shaded source set 'test'"
+ extendsFrom shadow
}
+testCompile.extendsFrom shadowTest
+ }
-// Always configure the shadowJar classifier and merge service files.
-shadowJar ({
- classifier = "shaded"
- mergeServiceFiles()
+ // Always configure the shadowJar classifier and merge service files.
+ shadowJar ({
+classifier = "shaded"
+mergeServiceFiles()
+ } << configuration.shadowClosure)
-} << configuration.shadowClosure)
+ // Always configure the shadowTestJar classifier and merge service files.
+ tasks.create(name: 'shadowTestJar', type:
com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar, {
+classifier = "shaded-tests"
+from sourceSets.test.output
+configurations = [project.configurations.testRuntime]
+ } << configuration.shadowClosure)
-// Always configure the shadowTestJar classifier and merge service files.
-tasks.create(name: 'shadowTestJar', type:
com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar, {
- classifier = "shaded-tests"
- from sourceSets.test.output
- configurations = [project.configurations.testRuntime]
-} << configuration.shadowClosure)
+ // Ensure that shaded jar and test-jar are part of the their own
configuration artifact sets
+ artifacts.shadow shadowJar
+ artifacts.shadowTest shadowTestJar
-// Ensure that shaded jar and test-jar are part of the their own
cconfiguration artifact sets
-// and the archives configuration set.
-artifacts.shadow shadowJar
-artifacts.shadowTest shadowTestJar
+ if (isRelease() || project.hasProperty('publishing')) {
+apply plugin: "maven-publish"
+
+// Only build artifacts for archives if we are publishing
artifacts.archives shadowJar
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=90098=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90098 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 11/Apr/18 18:13 Start Date: 11/Apr/18 18:13 Worklog Time Spent: 10m Work Description: lukecwik opened a new pull request #5103: [BEAM-3249] Only generate all artifacts when publishing. URL: https://github.com/apache/beam/pull/5103 This removes enableShadow since the user can now pass in the shading configuration closure explicitly. Had to re-add propdeps-maven since it was supplying a task needed for uploadingArtifacts (which is part of the old 'maven' plugin). It seems like we are mixing stuff from the old 'maven' and the new 'maven-publish' tasks which requires further investigation. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand: - [ ] What the pull request does - [ ] Why it does it - [ ] How it does it - [ ] Why this approach - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 90098) Time Spent: 12h (was: 11h 50m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 12h > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=90048=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90048
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 11/Apr/18 17:19
Start Date: 11/Apr/18 17:19
Worklog Time Spent: 10m
Work Description: lukecwik closed pull request #5098: [BEAM-3249] Add
support for translating 'provided' scope dependencies to the generated pom.
URL: https://github.com/apache/beam/pull/5098
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build_rules.gradle b/build_rules.gradle
index 08b578ec724..ed09ee7fe0b 100644
--- a/build_rules.gradle
+++ b/build_rules.gradle
@@ -300,7 +300,6 @@ class JavaNatureConfiguration {
// * maven
// * net.ltgt.apt (plugin to configure annotation processing tool)
// * propdeps (provide optional and provided dependency configurations)
-// * propdeps-maven (configure Maven pom generation to understand optional
and provided dependency configurations)
// * checkstyle
// * findbugs
// * shadow
@@ -395,7 +394,6 @@ ext.applyJavaNature = {
// TODO: Either remove these plugins and find another way to generate the
Maven poms
// with the correct dependency scopes configured.
apply plugin: 'propdeps'
- apply plugin: 'propdeps-maven'
// Configures a checkstyle plugin enforcing a set of rules and also allows
for a set of
// suppressions.
@@ -655,34 +653,39 @@ ext.applyJavaNature = {
// Iterate over the dependencies that would exist post shading,
// adding a node for each
def dependenciesNode = root.appendNode('dependencies')
-configurations.shadow.allDependencies.each {
- def dependencyNode = dependenciesNode.appendNode('dependency')
- dependencyNode.appendNode('groupId', it.group)
- dependencyNode.appendNode('artifactId', it.name)
- dependencyNode.appendNode('version', it.version)
-
- // TODO: Should we use the runtime scope which forces all our
- // consumers to declare what they consume?
- dependencyNode.appendNode('scope', 'compile')
-
- // Start with any exclusions that were added via configuration
exclude rules.
- // Then add all the exclusions that are specific to the
dependency (if any
- // were declared). Finally build the node that represents all
exclusions.
- def exclusions = []
- exclusions += configurations.shadow.excludeRules
- if (it.hasProperty('excludeRules')) {
-exclusions += it.excludeRules
- }
- if (!exclusions.empty) {
-def exclusionsNode = dependencyNode.appendNode('exclusions')
-for (ExcludeRule exclude : exclusions) {
- def exclusionNode = exclusionsNode.appendNode('exclusion')
- exclusionNode.appendNode('groupId', exclude.group)
- exclusionNode.appendNode('artifactId', exclude.module)
+def generateDependenciesFromConfiguration = { param ->
+ configurations."${param.configuration}".allDependencies.each {
+def dependencyNode = dependenciesNode.appendNode('dependency')
+dependencyNode.appendNode('groupId', it.group)
+dependencyNode.appendNode('artifactId', it.name)
+dependencyNode.appendNode('version', it.version)
+
+dependencyNode.appendNode('scope', param.scope)
+
+// Start with any exclusions that were added via configuration
exclude rules.
+// Then add all the exclusions that are specific to the
dependency (if any
+// were declared). Finally build the node that represents all
exclusions.
+def exclusions = []
+exclusions +=
configurations."${param.configuration}".excludeRules
+if (it.hasProperty('excludeRules')) {
+ exclusions += it.excludeRules
+}
+if (!exclusions.empty) {
+ def exclusionsNode = dependencyNode.appendNode('exclusions')
+ for (ExcludeRule exclude : exclusions) {
+def exclusionNode = exclusionsNode.appendNode('exclusion')
+exclusionNode.appendNode('groupId', exclude.group)
+exclusionNode.appendNode('artifactId', exclude.module)
+ }
}
}
}
+// TODO: Should we use the runtime scope instead of the compile
scope
+//
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=90025=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90025 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 11/Apr/18 16:11 Start Date: 11/Apr/18 16:11 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5098: [BEAM-3249] Add support for translating 'provided' scope dependencies to the generated pom. URL: https://github.com/apache/beam/pull/5098#issuecomment-380509090 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 90025) Time Spent: 11h 40m (was: 11.5h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 11h 40m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=90022=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90022
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 11/Apr/18 16:10
Start Date: 11/Apr/18 16:10
Worklog Time Spent: 10m
Work Description: lukecwik closed pull request #4992: [BEAM-3249] Disable
all Jenkins jobs that rely on Maven.
URL: https://github.com/apache/beam/pull/4992
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build_rules.gradle b/build_rules.gradle
index 08b578ec724..1d6ceeca23d 100644
--- a/build_rules.gradle
+++ b/build_rules.gradle
@@ -31,6 +31,32 @@
println "Applying build_rules.gradle to $project.name"
+// Define the set of repositories and dependencies required to
+// fetch and enable plugins.
+buildscript {
+ repositories {
+maven { url file(offlineRepositoryRoot) }
+
+// To run gradle in offline mode, one must first invoke
+// 'updateOfflineRepository' to create an offline repo
+// inside the root project directory. See the application
+// of the offline repo plugin within build_rules.gradle
+// for further details.
+if (gradle.startParameter.isOffline()) {
+ return
+}
+
+mavenLocal()
+mavenCentral()
+jcenter()
+maven { url "https://plugins.gradle.org/m2/; }
+maven { url "http://repo.spring.io/plugins-release; }
+ }
+ dependencies {
+classpath "com.github.jengelman.gradle.plugins:shadow:2.0.1"
// Enable shading Java dependencies
+ }
+}
+
/*/
// Apply common properties/repositories and tasks to all projects.
@@ -281,11 +307,34 @@ ext.getJavaRelocatedPath = { String suffix ->
+ suffix)
}
+ext.DEFAULT_SHADOW_CLOSURE = {
+ dependencies {
+exclude(".*")
+include(dependency(library.java.guava))
+ }
+ relocate("com.google.common", getJavaRelocatedPath("com.google.common")) {
+// com.google.common is too generic, need to exclude guava-testlib
+exclude "com.google.common.collect.testing.**"
+exclude "com.google.common.escape.testing.**"
+exclude "com.google.common.testing.**"
+exclude "com.google.common.util.concurrent.testing.**"
+ }
+}
+
// A class defining the set of configurable properties accepted by
applyJavaNature
class JavaNatureConfiguration {
double javaVersion = 1.8// Controls the JDK source language and
target compatibility
boolean enableFindbugs = true // Controls whether the findbugs plugin is
enabled and configured
boolean enableShadow = true // Controls whether the shadow plugin is
enabled and configured
+ // The shadowJar / shadowTestJar tasks execute the following closure to
configure themselves.
+ // Users can compose their closure with the default closure via:
+ // DEFAULT_SHADOW_CLOSURE << {
+ //dependencies {
+ // include(...)
+ //}
+ //relocate(...)
+ // }
+ Closure shadowClosure;
}
// Configures a project with a default set of plugins that should apply to all
Java projects.
@@ -337,6 +386,10 @@ ext.applyJavaNature = {
println "applyJavaNature with " + (it ? "$it" : "default configuration") + "
for project $project.name"
// Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
JavaNatureConfiguration configuration = it ? it as JavaNatureConfiguration :
new JavaNatureConfiguration()
+ if (!configuration.shadowClosure) {
+configuration.shadowClosure = DEFAULT_SHADOW_CLOSURE
+ }
+
apply plugin: "java"
// Configure the Java compiler source language and target compatibility
levels. Also ensure that
@@ -446,23 +499,6 @@ ext.applyJavaNature = {
if (configuration.enableShadow) {
apply plugin: 'com.github.johnrengelman.shadow'
-// Shade guava in all our dependencies.
-shadowJar {
- classifier = "shaded"
- mergeServiceFiles()
- dependencies {
-exclude(".*")
-include(dependency(library.java.guava))
- }
- relocate("com.google.common", getJavaRelocatedPath("com.google.common"))
{
-// com.google.common is too generic, need to exclude guava-testlib
-exclude "com.google.common.collect.testing.**"
-exclude "com.google.common.escape.testing.**"
-exclude "com.google.common.testing.**"
-exclude "com.google.common.util.concurrent.testing.**"
- }
-}
-
// Create a new configuration 'shadowTest' like 'shadow' for the test scope
configurations {
shadow {
@@ -475,18 +511,27 @@
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=90020=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90020
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 11/Apr/18 16:09
Start Date: 11/Apr/18 16:09
Worklog Time Spent: 10m
Work Description: lukecwik closed pull request #5087: [BEAM-3249,
BEAM-4014] Automatically propagate shading configuration to test-jar
URL: https://github.com/apache/beam/pull/5087
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build_rules.gradle b/build_rules.gradle
index 08b578ec724..1d6ceeca23d 100644
--- a/build_rules.gradle
+++ b/build_rules.gradle
@@ -31,6 +31,32 @@
println "Applying build_rules.gradle to $project.name"
+// Define the set of repositories and dependencies required to
+// fetch and enable plugins.
+buildscript {
+ repositories {
+maven { url file(offlineRepositoryRoot) }
+
+// To run gradle in offline mode, one must first invoke
+// 'updateOfflineRepository' to create an offline repo
+// inside the root project directory. See the application
+// of the offline repo plugin within build_rules.gradle
+// for further details.
+if (gradle.startParameter.isOffline()) {
+ return
+}
+
+mavenLocal()
+mavenCentral()
+jcenter()
+maven { url "https://plugins.gradle.org/m2/; }
+maven { url "http://repo.spring.io/plugins-release; }
+ }
+ dependencies {
+classpath "com.github.jengelman.gradle.plugins:shadow:2.0.1"
// Enable shading Java dependencies
+ }
+}
+
/*/
// Apply common properties/repositories and tasks to all projects.
@@ -281,11 +307,34 @@ ext.getJavaRelocatedPath = { String suffix ->
+ suffix)
}
+ext.DEFAULT_SHADOW_CLOSURE = {
+ dependencies {
+exclude(".*")
+include(dependency(library.java.guava))
+ }
+ relocate("com.google.common", getJavaRelocatedPath("com.google.common")) {
+// com.google.common is too generic, need to exclude guava-testlib
+exclude "com.google.common.collect.testing.**"
+exclude "com.google.common.escape.testing.**"
+exclude "com.google.common.testing.**"
+exclude "com.google.common.util.concurrent.testing.**"
+ }
+}
+
// A class defining the set of configurable properties accepted by
applyJavaNature
class JavaNatureConfiguration {
double javaVersion = 1.8// Controls the JDK source language and
target compatibility
boolean enableFindbugs = true // Controls whether the findbugs plugin is
enabled and configured
boolean enableShadow = true // Controls whether the shadow plugin is
enabled and configured
+ // The shadowJar / shadowTestJar tasks execute the following closure to
configure themselves.
+ // Users can compose their closure with the default closure via:
+ // DEFAULT_SHADOW_CLOSURE << {
+ //dependencies {
+ // include(...)
+ //}
+ //relocate(...)
+ // }
+ Closure shadowClosure;
}
// Configures a project with a default set of plugins that should apply to all
Java projects.
@@ -337,6 +386,10 @@ ext.applyJavaNature = {
println "applyJavaNature with " + (it ? "$it" : "default configuration") + "
for project $project.name"
// Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
JavaNatureConfiguration configuration = it ? it as JavaNatureConfiguration :
new JavaNatureConfiguration()
+ if (!configuration.shadowClosure) {
+configuration.shadowClosure = DEFAULT_SHADOW_CLOSURE
+ }
+
apply plugin: "java"
// Configure the Java compiler source language and target compatibility
levels. Also ensure that
@@ -446,23 +499,6 @@ ext.applyJavaNature = {
if (configuration.enableShadow) {
apply plugin: 'com.github.johnrengelman.shadow'
-// Shade guava in all our dependencies.
-shadowJar {
- classifier = "shaded"
- mergeServiceFiles()
- dependencies {
-exclude(".*")
-include(dependency(library.java.guava))
- }
- relocate("com.google.common", getJavaRelocatedPath("com.google.common"))
{
-// com.google.common is too generic, need to exclude guava-testlib
-exclude "com.google.common.collect.testing.**"
-exclude "com.google.common.escape.testing.**"
-exclude "com.google.common.testing.**"
-exclude "com.google.common.util.concurrent.testing.**"
- }
-}
-
// Create a new configuration 'shadowTest' like 'shadow' for the test scope
configurations {
shadow {
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=89990=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89990 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 11/Apr/18 15:15 Start Date: 11/Apr/18 15:15 Worklog Time Spent: 10m Work Description: lukecwik opened a new pull request #5098: [BEAM-3249] Add support for translating 'provided' scope dependencies to the generated pom. URL: https://github.com/apache/beam/pull/5098 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand: - [ ] What the pull request does - [ ] Why it does it - [ ] How it does it - [ ] Why this approach - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 89990) Time Spent: 11h 10m (was: 11h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 11h 10m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=89947=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89947 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 11/Apr/18 13:30 Start Date: 11/Apr/18 13:30 Worklog Time Spent: 10m Work Description: lukecwik opened a new pull request #5087: [BEAM-3249, BEAM-4014] Automatically propagate shading configuration to test-jar URL: https://github.com/apache/beam/pull/5087 Fix shading for sdks/java/io/google-cloud-platform Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand: - [ ] What the pull request does - [ ] Why it does it - [ ] How it does it - [ ] Why this approach - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 89947) Time Spent: 11h (was: 10h 50m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 11h > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=89945=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89945 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 11/Apr/18 13:22 Start Date: 11/Apr/18 13:22 Worklog Time Spent: 10m Work Description: lukecwik closed pull request #5087: [BEAM-3249, BEAM-4014] Automatically propagate shading configuration to test-jar URL: https://github.com/apache/beam/pull/5087 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 89945) Time Spent: 10h 50m (was: 10h 40m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 10h 50m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=89754=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89754 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 11/Apr/18 02:44 Start Date: 11/Apr/18 02:44 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5087: [BEAM-3249, BEAM-4014] Automatically propagate shading configuration to test-jar URL: https://github.com/apache/beam/pull/5087#issuecomment-380308929 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 89754) Time Spent: 10h 40m (was: 10.5h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 10h 40m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=89588=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89588
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 10/Apr/18 19:56
Start Date: 10/Apr/18 19:56
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5087:
[BEAM-3249, BEAM-4014] Automatically propagate shading configuration to test-jar
URL: https://github.com/apache/beam/pull/5087#discussion_r180548711
##
File path: build_rules.gradle
##
@@ -31,6 +31,32 @@
println "Applying build_rules.gradle to $project.name"
+// Define the set of repositories and dependencies required to
+// fetch and enable plugins.
+buildscript {
+ repositories {
+maven { url file(offlineRepositoryRoot) }
Review comment:
gradle.properties
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89588)
Time Spent: 10.5h (was: 10h 20m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 10.5h
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=89573=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89573 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 10/Apr/18 19:33 Start Date: 10/Apr/18 19:33 Worklog Time Spent: 10m Work Description: lukecwik opened a new pull request #5087: [BEAM-3249, BEAM-4014] Automatically propagate shading configuration to test-jar URL: https://github.com/apache/beam/pull/5087 Fix shading for sdks/java/io/google-cloud-platform Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand: - [ ] What the pull request does - [ ] Why it does it - [ ] How it does it - [ ] Why this approach - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 89573) Time Spent: 10h 10m (was: 10h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 10h 10m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=89081=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89081
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 09/Apr/18 19:36
Start Date: 09/Apr/18 19:36
Worklog Time Spent: 10m
Work Description: lukecwik closed pull request #5062: [BEAM-3249] Minor
spark runner test execution improvements.
URL: https://github.com/apache/beam/pull/5062
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/runners/spark/build.gradle b/runners/spark/build.gradle
index cc103908d28..56d1bb105ba 100644
--- a/runners/spark/build.gradle
+++ b/runners/spark/build.gradle
@@ -49,6 +49,7 @@ test {
]"""
// Only one SparkContext may be running in a JVM (SPARK-2243)
forkEvery 1
+ maxParallelForks 4
useJUnit {
excludeCategories "org.apache.beam.runners.spark.UsesCheckpointRecovery"
excludeCategories "org.apache.beam.runners.spark.StreamingTest"
@@ -120,7 +121,9 @@ task validatesRunnerBatch(type: Test) {
classpath = configurations.validatesRunner
testClassesDirs =
files(project(":beam-sdks-java-core").sourceSets.test.output.classesDirs) +
files(project.sourceSets.test.output.classesDirs)
+ // Only one SparkContext may be running in a JVM (SPARK-2243)
forkEvery 1
+ maxParallelForks 4
useJUnit {
includeCategories 'org.apache.beam.sdk.testing.ValidatesRunner'
includeCategories 'org.apache.beam.runners.spark.UsesCheckpointRecovery'
@@ -141,7 +144,11 @@ task validatesRunnerStreaming(type: Test) {
])
systemProperty "beamTestPipelineOptions", pipelineOptions
+ classpath = configurations.validatesRunner
+ testClassesDirs = files(project.sourceSets.test.output.classesDirs)
+ // Only one SparkContext may be running in a JVM (SPARK-2243)
forkEvery 1
+ maxParallelForks 4
useJUnit {
includeCategories 'org.apache.beam.runners.spark.StreamingTest'
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89081)
Time Spent: 10h (was: 9h 50m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 10h
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=89077=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89077 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 09/Apr/18 19:16 Start Date: 09/Apr/18 19:16 Worklog Time Spent: 10m Work Description: lukecwik opened a new pull request #5062: [BEAM-3249] Minor spark runner test execution improvements. URL: https://github.com/apache/beam/pull/5062 DESCRIPTION HERE Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand: - [ ] What the pull request does - [ ] Why it does it - [ ] How it does it - [ ] Why this approach - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 89077) Time Spent: 9h 50m (was: 9h 40m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 9h 50m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=87616=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87616
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 04/Apr/18 16:15
Start Date: 04/Apr/18 16:15
Worklog Time Spent: 10m
Work Description: aaltay closed pull request #4814: [BEAM-3249] Drop Java
Maven PreCommit.
URL: https://github.com/apache/beam/pull/4814
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/.test-infra/jenkins/job_beam_PreCommit_Java_GradleBuild.groovy
b/.test-infra/jenkins/job_beam_PreCommit_Java_GradleBuild.groovy
index c5edbf1bf67..fe3e1ac76c6 100644
--- a/.test-infra/jenkins/job_beam_PreCommit_Java_GradleBuild.groovy
+++ b/.test-infra/jenkins/job_beam_PreCommit_Java_GradleBuild.groovy
@@ -50,7 +50,7 @@ job('beam_PreCommit_Java_GradleBuild') {
def gradle_command_line = './gradlew ' + gradle_switches.join(' ') + '
:javaPreCommit'
// Sets that this is a PreCommit job.
- common_job_properties.setPreCommit(delegate, gradle_command_line, 'Run Java
Gradle PreCommit')
+ common_job_properties.setPreCommit(delegate, gradle_command_line, 'Run Java
PreCommit')
steps {
gradle {
rootBuildScriptDir(common_job_properties.checkoutDir)
diff --git a/.test-infra/jenkins/job_beam_PreCommit_Java_MavenInstall.groovy
b/.test-infra/jenkins/job_beam_PreCommit_Java_MavenInstall.groovy
deleted file mode 100644
index 38fef725e78..000
--- a/.test-infra/jenkins/job_beam_PreCommit_Java_MavenInstall.groovy
+++ /dev/null
@@ -1,56 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import common_job_properties
-
-// This is the Java precommit which runs a maven install, and the current set
-// of precommit tests.
-mavenJob('beam_PreCommit_Java_MavenInstall') {
- description('Runs an install of the current GitHub Pull Request.')
-
- previousNames('beam_PreCommit_MavenVerify')
-
- // Execute concurrent builds if necessary.
- concurrentBuild()
-
- // Set common parameters.
- common_job_properties.setTopLevelMainJobProperties(
-delegate,
-'master',
-150)
-
- // Set Maven parameters.
- common_job_properties.setMavenConfig(delegate)
-
- // Sets that this is a PreCommit job.
- common_job_properties.setPreCommit(delegate, 'mvn clean install -pl
sdks/java/core,runners/direct-java,sdks/java/fn-execution -am -amd', 'Run Java
PreCommit')
-
- // Maven goals for this job: The Java SDK, its dependencies, and things that
depend on it.
- goals([
-'--batch-mode',
-'--errors',
-'--activate-profiles
release,jenkins-precommit,direct-runner,dataflow-runner,spark-runner,flink-runner,apex-runner',
-'--projects sdks/java/core,runners/direct-java,sdks/java/fn-execution',
-'--also-make',
-'--also-make-dependents',
-'-D pullRequest=$ghprbPullId',
-'help:effective-settings',
-'clean',
-'install'
- ].join(' '))
-}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 87616)
Time Spent: 9h 40m (was: 9.5h)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 9h 40m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=87611=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87611 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 04/Apr/18 16:06 Start Date: 04/Apr/18 16:06 Worklog Time Spent: 10m Work Description: lukecwik opened a new pull request #4814: [BEAM-3249] Drop Java Maven PreCommit. URL: https://github.com/apache/beam/pull/4814 Java Gradle precommit passed 46.34% of the time compared to the Java Maven precommit which passed 46.15%. Also for non timeout builds, the average build times were 35 mins for Gradle and 67 mins for Maven respectively. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand: - [ ] What the pull request does - [ ] Why it does it - [ ] How it does it - [ ] Why this approach - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 87611) Time Spent: 9.5h (was: 9h 20m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 9.5h > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=87272=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87272
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 03/Apr/18 20:56
Start Date: 03/Apr/18 20:56
Worklog Time Spent: 10m
Work Description: lukecwik closed pull request #5009: [BEAM-3249] Add
missing gradle artifact ids
URL: https://github.com/apache/beam/pull/5009
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/runners/core-java/build.gradle b/runners/core-java/build.gradle
index 39c4c0410d0..05e27915380 100644
--- a/runners/core-java/build.gradle
+++ b/runners/core-java/build.gradle
@@ -17,7 +17,7 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature(artifactId: "runners-core-java")
+applyJavaNature(artifactId: "beam-runners-core-java")
description = "Apache Beam :: Runners :: Core Java"
diff --git a/runners/local-artifact-service-java/build.gradle
b/runners/local-artifact-service-java/build.gradle
index 205cd8e494b..dda4137a2e9 100644
--- a/runners/local-artifact-service-java/build.gradle
+++ b/runners/local-artifact-service-java/build.gradle
@@ -17,7 +17,7 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-local-artifact-service-java")
description = "Apache Beam :: Runners :: Java Local Artifact Service"
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 87272)
Time Spent: 9h 20m (was: 9h 10m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 9h 20m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=87265=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87265 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 03/Apr/18 20:39 Start Date: 03/Apr/18 20:39 Worklog Time Spent: 10m Work Description: bsidhom opened a new pull request #5009: [BEAM-3249] Add missing gradle artifact ids URL: https://github.com/apache/beam/pull/5009 This isn't necessarily comprehensive, but I came across some packages that were applying the java nature but not including a proper artifact id. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand: - [ ] What the pull request does - [ ] Why it does it - [ ] How it does it - [ ] Why this approach - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 87265) Time Spent: 9h (was: 8h 50m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 9h > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=87266=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87266 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 03/Apr/18 20:39 Start Date: 03/Apr/18 20:39 Worklog Time Spent: 10m Work Description: bsidhom commented on issue #5009: [BEAM-3249] Add missing gradle artifact ids URL: https://github.com/apache/beam/pull/5009#issuecomment-378391527 R: @lukecwik This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 87266) Time Spent: 9h 10m (was: 9h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 9h 10m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=86945=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-86945 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 03/Apr/18 05:16 Start Date: 03/Apr/18 05:16 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #4992: [BEAM-3249] Disable all Jenkins jobs that rely on Maven. URL: https://github.com/apache/beam/pull/4992#issuecomment-378130860 Updated for PR/4999 since that deleted a bunch of Jenkins jobs. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 86945) Time Spent: 8h 50m (was: 8h 40m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 8h 50m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=86739=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-86739 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 02/Apr/18 20:03 Start Date: 02/Apr/18 20:03 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #4992: [BEAM-3249] Disable all Jenkins jobs that rely on Maven. URL: https://github.com/apache/beam/pull/4992#issuecomment-378027960 R: @reuvenlax This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 86739) Time Spent: 8h 40m (was: 8.5h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 8h 40m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=86737=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-86737 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 02/Apr/18 20:00 Start Date: 02/Apr/18 20:00 Worklog Time Spent: 10m Work Description: lukecwik opened a new pull request #4992: [BEAM-3249] Disable all Jenkins jobs that rely on Maven. URL: https://github.com/apache/beam/pull/4992 This is for the Maven -> Gradle migration fixit day. This change disables all maven based Jenkins jobs and leaves them around so that people working on the migration can update them to be Gradle based. This also ensures that there is plenty of Jenkins capacity for testing migration to Gradle. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand: - [ ] What the pull request does - [ ] Why it does it - [ ] How it does it - [ ] Why this approach - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 86737) Time Spent: 8.5h (was: 8h 20m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 8.5h > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=86201=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-86201
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 30/Mar/18 23:20
Start Date: 30/Mar/18 23:20
Worklog Time Spent: 10m
Work Description: lukecwik closed pull request #4982: [BEAM-3249] Allow
for re-use of dependencies within other projects by placing them within
build_rules.gradle
URL: https://github.com/apache/beam/pull/4982
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build.gradle b/build.gradle
index a977302048d..ad58db61eeb 100644
--- a/build.gradle
+++ b/build.gradle
@@ -23,144 +23,6 @@
apply from: project(":").file("build_rules.gradle")
-// These versions are defined here because they represent
-// a dependency version which should match across multiple
-// Maven artifacts.
-def google_cloud_bigdataoss_version = "1.4.5"
-def bigtable_version = "1.0.0"
-def bigtable_proto_version = "1.0.0-pre3"
-def google_clients_version = "1.22.0"
-def google_auth_version = "0.7.1"
-def grpc_version = "1.2.0"
-def protobuf_version = "3.2.0"
-def guava_version = "20.0"
-def netty_version = "4.1.8.Final"
-def grpc_google_common_protos = "0.1.9"
-def hamcrest_version = "1.3"
-def hadoop_version = "2.7.3"
-def jackson_version = "2.8.9"
-def spark_version = "2.2.1"
-def pubsub_grpc_version = "0.1.18"
-def apex_core_version = "3.6.0"
-def apex_malhar_version = "3.4.0"
-def postgres_version = "9.4.1212.jre7"
-def jaxb_api_version = "2.2.12"
-
-// A map of maps containing common libraries used per language. To use:
-// dependencies {
-// shadow library.java.slf4j_api
-// }
-ext.library = [
- java: [
-activemq_amqp: "org.apache.activemq:activemq-amqp:5.13.1",
-activemq_broker: "org.apache.activemq:activemq-broker:5.13.1",
-activemq_client: "org.apache.activemq:activemq-client:5.13.1",
-activemq_jaas: "org.apache.activemq:activemq-jaas:5.13.1",
-activemq_junit: "org.apache.activemq.tooling:activemq-junit:5.13.1",
-activemq_kahadb_store: "org.apache.activemq:activemq-kahadb-store:5.13.1",
-activemq_mqtt: "org.apache.activemq:activemq-mqtt:5.13.1",
-apex_common: "org.apache.apex:apex-common:$apex_core_version",
-apex_engine: "org.apache.apex:apex-engine:$apex_core_version",
-args4j: "args4j:args4j:2.33",
-avro: "org.apache.avro:avro:1.8.2",
-bigdataoss_gcsio:
"com.google.cloud.bigdataoss:gcsio:$google_cloud_bigdataoss_version",
-bigdataoss_util:
"com.google.cloud.bigdataoss:util:$google_cloud_bigdataoss_version",
-bigtable_client_core:
"com.google.cloud.bigtable:bigtable-client-core:$bigtable_version",
-bigtable_protos:
"com.google.cloud.bigtable:bigtable-protos:$bigtable_proto_version",
-byte_buddy: "net.bytebuddy:byte-buddy:1.7.10",
-commons_compress: "org.apache.commons:commons-compress:1.16.1",
-commons_csv: "org.apache.commons:commons-csv:1.4",
-commons_io_1x: "commons-io:commons-io:1.3.2",
-commons_io_2x: "commons-io:commons-io:2.5",
-commons_lang3: "org.apache.commons:commons-lang3:3.6",
-datastore_v1_proto_client:
"com.google.cloud.datastore:datastore-v1-proto-client:1.4.0",
-datastore_v1_protos:
"com.google.cloud.datastore:datastore-v1-protos:1.3.0",
-error_prone_annotations:
"com.google.errorprone:error_prone_annotations:2.0.15",
-findbugs_annotations:
"com.github.stephenc.findbugs:findbugs-annotations:1.3.9-1",
-findbugs_jsr305: "com.google.code.findbugs:jsr305:3.0.1",
-gax_grpc: "com.google.api:gax-grpc:0.20.0",
-google_api_client:
"com.google.api-client:google-api-client:$google_clients_version",
-google_api_client_jackson2:
"com.google.api-client:google-api-client-jackson2:$google_clients_version",
-google_api_client_java6:
"com.google.api-client:google-api-client-java6:$google_clients_version",
-google_api_common: "com.google.api:api-common:1.0.0-rc2",
-google_api_services_bigquery:
"com.google.apis:google-api-services-bigquery:v2-rev374-$google_clients_version",
-google_api_services_clouddebugger:
"com.google.apis:google-api-services-clouddebugger:v2-rev8-$google_clients_version",
-google_api_services_cloudresourcemanager:
"com.google.apis:google-api-services-cloudresourcemanager:v1-rev6-$google_clients_version",
-google_api_services_dataflow:
"com.google.apis:google-api-services-dataflow:v1b3-rev221-$google_clients_version",
-google_api_services_pubsub:
"com.google.apis:google-api-services-pubsub:v1-rev10-$google_clients_version",
-google_api_services_storage:
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=86144=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-86144 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 30/Mar/18 20:14 Start Date: 30/Mar/18 20:14 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #4982: [BEAM-3249] Allow for re-use of dependencies within other projects by placing them within build_rules.gradle URL: https://github.com/apache/beam/pull/4982#issuecomment-377612262 Kenn, Gradle doesn't work as you expect in this regard. Moving things out of build_rules.gradle into build.gradle would use the root project since it is in scope. This is why we are using apply from within each project so that when we define things like applyJavaNature, it uses the current project and its context. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 86144) Time Spent: 8h 10m (was: 8h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 8h 10m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=86143=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-86143 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 30/Mar/18 20:13 Start Date: 30/Mar/18 20:13 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #4982: [BEAM-3249] Allow for re-use of dependencies within other projects by placing them within build_rules.gradle URL: https://github.com/apache/beam/pull/4982#issuecomment-377612262 Kenn, Gradle doesn't work as you expect in this regard. Moving things out of build_rules.gradle into build.gradle would use the root project since it is in scope. This is why we are using apply from within each project. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 86143) Time Spent: 8h (was: 7h 50m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 8h > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=86140=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-86140 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 30/Mar/18 20:04 Start Date: 30/Mar/18 20:04 Worklog Time Spent: 10m Work Description: kennknowles commented on issue #4982: [BEAM-3249] Allow for re-use of dependencies within other projects by placing them within build_rules.gradle URL: https://github.com/apache/beam/pull/4982#issuecomment-377610364 Just drive-by comment: A similar action that would make the project more idiomatic and result in fewer moving pieces would be to move the nature defs from build_rules.gradle into build.gradle (thus deleting build_rules.gradle). It would also make the code clearer since projects would be passed explicitly instead of implicitly global. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 86140) Time Spent: 7h 50m (was: 7h 40m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 7h 50m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=86113=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-86113 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 30/Mar/18 18:31 Start Date: 30/Mar/18 18:31 Worklog Time Spent: 10m Work Description: lukecwik opened a new pull request #4982: [BEAM-3249] Allow for re-use of dependencies within other projects by placing them within build_rules.gradle URL: https://github.com/apache/beam/pull/4982 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand: - [ ] What the pull request does - [ ] Why it does it - [ ] How it does it - [ ] Why this approach - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 86113) Time Spent: 7h 40m (was: 7.5h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 7h 40m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81689=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81689
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 18/Mar/18 21:02
Start Date: 18/Mar/18 21:02
Worklog Time Spent: 10m
Work Description: lukecwik closed pull request #4812: [BEAM-3249]
Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build.gradle b/build.gradle
index 69502ab77c0..6ffeb57c69c 100644
--- a/build.gradle
+++ b/build.gradle
@@ -150,6 +150,14 @@ ext.library = [
stax2_api: "org.codehaus.woodstox:stax2-api:3.1.4",
woodstox_core_asl: "org.codehaus.woodstox:woodstox-core-asl:4.4.1",
],
+ // For generating pom.xml from archetypes.
+ maven: [
+maven_compiler_plugin: "maven-plugins:maven-compiler-plugin:3.7.0",
+maven_exec_plugin: "maven-plugins:maven-exec-plugin:1.6.0",
+maven_jar_plugin: "maven-plugins:maven-jar-plugin:3.0.2",
+maven_shade_plugin: "maven-plugins:maven-shade-plugin:3.1.0",
+maven_surefire_plugin: "maven-plugins:maven-surefire-plugin:2.20.1",
+ ],
]
// Define the set of repositories and dependencies required to
diff --git a/build_rules.gradle b/build_rules.gradle
index 7d3cc19da25..4d0e7a026c1 100644
--- a/build_rules.gradle
+++ b/build_rules.gradle
@@ -184,6 +184,15 @@ ext.applyJavaNature = {
apply plugin: "maven-publish"
apply plugin: "java"
+ publishing {
+repositories {
+ maven {
+url project.properties["distMgmtSnapshotsUrl"] ?:
"https://repository.apache.org/content/repositories/snapshots;
+ }
+}
+ }
+
+
// Configure the Java compiler source language and target compatibility
levels. Also ensure that
// we configure the Java compiler to use UTF-8.
sourceCompatibility = configuration.javaVersion
@@ -325,6 +334,19 @@ ext.applyJavaNature = {
artifacts.archives shadowJar
if (configuration.artifactId) {
+ task sourcesJar(type: Jar) {
+from sourceSets.main.allSource
+classifier = 'sources'
+ }
+ task testSourcesJar(type: Jar) {
+from sourceSets.test.allSource
+classifier = 'test-sources'
+ }
+ task javadocJar(type: Jar, dependsOn: javadoc) {
+classifier = 'javadoc'
+from javadoc.destinationDir
+ }
+
// If a publication artifact id is supplied, publish the shadow jar.
publishing {
publications {
@@ -339,6 +361,19 @@ ext.applyJavaNature = {
asNode().appendNode('name', description)
}
}
+// TODO: Use the shadow test jar here instead of the test jar.
+artifact packageTests {
+ classifier "tests"
+}
+artifact sourcesJar {
+ classifier "sources"
+}
+artifact testSourcesJar {
+ classifier "test-sources"
+}
+artifact javadocJar {
+ classifier "javadoc"
+}
}
}
}
diff --git a/examples/java/build.gradle b/examples/java/build.gradle
index 72e63dfcb97..cf1dced 100644
--- a/examples/java/build.gradle
+++ b/examples/java/build.gradle
@@ -19,7 +19,7 @@
import groovy.json.JsonOutput
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-examples-java")
description = "Apache Beam :: Examples :: Java"
diff --git a/model/fn-execution/build.gradle b/model/fn-execution/build.gradle
index 140d4d0ca5f..79c47b4712b 100644
--- a/model/fn-execution/build.gradle
+++ b/model/fn-execution/build.gradle
@@ -17,7 +17,7 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature(enableFindbugs: false)
+applyJavaNature(artifactId: "beam-model-fn-execution", enableFindbugs: false)
applyGrpcNature()
description = "Apache Beam :: Model :: Fn Execution"
diff --git a/model/job-management/build.gradle
b/model/job-management/build.gradle
index 55ebf47698d..96a71d96d6a 100644
--- a/model/job-management/build.gradle
+++ b/model/job-management/build.gradle
@@ -17,7 +17,7 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature(enableFindbugs: false)
+applyJavaNature(artifactId: "beam-model-job-management", enableFindbugs: false)
applyGrpcNature()
description = "Apache Beam :: Model :: Job Management"
diff --git a/model/pipeline/build.gradle b/model/pipeline/build.gradle
index 4e8f3339501..b0cff40bffa 100644
--- a/model/pipeline/build.gradle
+++ b/model/pipeline/build.gradle
@@ -17,7 +17,7 @@
*/
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81677=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81677 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 18/Mar/18 18:34 Start Date: 18/Mar/18 18:34 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #4812: [BEAM-3249] Publish java artifacts using gradle URL: https://github.com/apache/beam/pull/4812#issuecomment-374029533 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 81677) Time Spent: 7h 10m (was: 7h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 7h 10m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81678=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81678 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 18/Mar/18 18:34 Start Date: 18/Mar/18 18:34 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #4812: [BEAM-3249] Publish java artifacts using gradle URL: https://github.com/apache/beam/pull/4812#issuecomment-374029559 Run Java Gradle PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 81678) Time Spent: 7h 20m (was: 7h 10m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 7h 20m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81670=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81670
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 18/Mar/18 17:22
Start Date: 18/Mar/18 17:22
Worklog Time Spent: 10m
Work Description: lukecwik closed pull request #4830: [BEAM-3249,
BEAM-3457] Make Apache Rat a dependency of the Java and Gradle precommits.
URL: https://github.com/apache/beam/pull/4830
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build.gradle b/build.gradle
index 69502ab77c0..ac800c40c0b 100644
--- a/build.gradle
+++ b/build.gradle
@@ -278,7 +278,10 @@ rat {
// Proto/grpc generated wrappers
"**/apache_beam/portability/api/*_pb2*.py",
-"**/go/pkg/beam/model/**/*.pb.go",
+"**/go/pkg/beam/**/*.pb.go",
+
+// Ignore Go test data files
+"**/go/data/**",
// VCF test files
"**/apache_beam/testing/data/vcf/*",
@@ -290,20 +293,14 @@ check.dependsOn rat
// to be specified on the commandline when executing locally.
// This indirection also makes Jenkins use the branch of the PR
// for the test definitions.
-
-def javaPreCommitRoots = [
- ":sdks:java:core",
- ":runners:direct-java",
- ":sdks:java:fn-execution",
-]
task javaPreCommit() {
- for (String javaPreCommitRoot : javaPreCommitRoots) {
-dependsOn javaPreCommitRoot + ":buildNeeded"
-dependsOn javaPreCommitRoot + ":buildDependents"
- }
+ dependsOn ":rat"
+ dependsOn ":sdks:java:core:buildNeeded"
+ dependsOn ":sdks:java:core:buildDependents"
dependsOn ":examples:java:preCommit"
}
task goPreCommit() {
+ dependsOn ":rat"
dependsOn ":sdks:go:test"
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 81670)
Time Spent: 7h (was: 6h 50m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 7h
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81586=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81586
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 17/Mar/18 15:35
Start Date: 17/Mar/18 15:35
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on a change in pull request
#4812: [BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r175261312
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,59 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+// Extract the version from a library:version dependency.
+def ver = { moduleName ->
+ def version = ""
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 81586)
Time Spent: 6h 40m (was: 6.5h)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 6h 40m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81587=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81587
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 17/Mar/18 15:35
Start Date: 17/Mar/18 15:35
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on a change in pull request
#4812: [BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r175261314
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,59 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+// Extract the version from a library:version dependency.
+def ver = { moduleName ->
Review comment:
Done, made it inline and deleted the ver() function
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 81587)
Time Spent: 6h 50m (was: 6h 40m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81268=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81268
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 16/Mar/18 17:39
Start Date: 16/Mar/18 17:39
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #4812:
[BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r175158437
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,59 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+// Extract the version from a library:version dependency.
+def ver = { moduleName ->
Review comment:
Note that defining really short names like `ver` can lead to naming
conflicts since ver becomes `global` within this project.
It might be better to just invoke the one liner
`dependencies.create(moduleName).getVersion()` as needed directly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 81268)
Time Spent: 6.5h (was: 6h 20m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81231=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81231
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 16/Mar/18 17:15
Start Date: 16/Mar/18 17:15
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #4812:
[BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r175158437
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,59 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+// Extract the version from a library:version dependency.
+def ver = { moduleName ->
Review comment:
Note that defining really short names like `ver` can lead to naming
conflicts since ver becomes `global` within this project.
I might be better to just invoke the one liner
`dependencies.create(moduleName).getVersion()` as needed directly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 81231)
Time Spent: 6h 20m (was: 6h 10m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81230=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81230
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 16/Mar/18 17:15
Start Date: 16/Mar/18 17:15
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #4812:
[BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r175158046
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,59 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+// Extract the version from a library:version dependency.
+def ver = { moduleName ->
+ def version = ""
Review comment:
simplify to:
`return dependencies.create(moduleName).getVersion()`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 81230)
Time Spent: 6h 20m (was: 6h 10m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81226=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81226
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 16/Mar/18 17:07
Start Date: 16/Mar/18 17:07
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on a change in pull request
#4812: [BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r175156305
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,53 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+// Extract the version from a library:version dependency.
+def ver = { return it.substring(1+it.lastIndexOf(':')) }
+
+processResources {
+ filter org.apache.tools.ant.filters.ReplaceTokens, tokens: [
+'project.version': version,
+'bigquery.version': ver(project.library.java.google_api_services_bigquery),
+'google-clients.version': ver(project.library.java.google_api_client),
+'guava.version': ver(project.library.java.guava),
+'hamcrest.version': ver(project.library.java.hamcrest_library),
+'jackson.version': ver(project.library.java.jackson_core),
+'joda.version': ver(project.library.java.joda_time),
+'junit.version': ver(project.library.java.junit),
+'pubsub.version': ver(project.library.java.google_api_services_pubsub),
+'slf4j.version': ver(project.library.java.slf4j_api),
+'spark.version': ver(project.library.java.spark_core),
+'hadoop.version': ver(project.library.java.hadoop_client),
+'mockito.version': ver(project.library.java.mockito_core),
+'maven-compiler-plugin.version': '3.7.0',
Review comment:
Done. Moved to root build.gradle in project.ext.library.maven
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 81226)
Time Spent: 6h 10m (was: 6h)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81221=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81221
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 16/Mar/18 17:07
Start Date: 16/Mar/18 17:07
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on a change in pull request
#4812: [BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r175156188
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,53 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+// Extract the version from a library:version dependency.
+def ver = { return it.substring(1+it.lastIndexOf(':')) }
Review comment:
Done. Used dependencies { create(moduleName).getVersion() }
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 81221)
Time Spent: 6h (was: 5h 50m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 6h
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81020=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81020
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 15/Mar/18 23:24
Start Date: 15/Mar/18 23:24
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #4812:
[BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r174961399
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,53 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+// Extract the version from a library:version dependency.
+def ver = { return it.substring(1+it.lastIndexOf(':')) }
+
+processResources {
+ filter org.apache.tools.ant.filters.ReplaceTokens, tokens: [
+'project.version': version,
+'bigquery.version': ver(project.library.java.google_api_services_bigquery),
+'google-clients.version': ver(project.library.java.google_api_client),
+'guava.version': ver(project.library.java.guava),
+'hamcrest.version': ver(project.library.java.hamcrest_library),
+'jackson.version': ver(project.library.java.jackson_core),
+'joda.version': ver(project.library.java.joda_time),
+'junit.version': ver(project.library.java.junit),
+'pubsub.version': ver(project.library.java.google_api_services_pubsub),
+'slf4j.version': ver(project.library.java.slf4j_api),
+'spark.version': ver(project.library.java.spark_core),
+'hadoop.version': ver(project.library.java.hadoop_client),
+'mockito.version': ver(project.library.java.mockito_core),
+'maven-compiler-plugin.version': '3.7.0',
Review comment:
It would be useful to elevate all the Maven dependencies to the project root
build.gradle file so that its not duplicated for each archetype.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 81020)
Time Spent: 5h 50m (was: 5h 40m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81019=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81019
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 15/Mar/18 23:23
Start Date: 15/Mar/18 23:23
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #4812:
[BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r174961399
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,53 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+// Extract the version from a library:version dependency.
+def ver = { return it.substring(1+it.lastIndexOf(':')) }
+
+processResources {
+ filter org.apache.tools.ant.filters.ReplaceTokens, tokens: [
+'project.version': version,
+'bigquery.version': ver(project.library.java.google_api_services_bigquery),
+'google-clients.version': ver(project.library.java.google_api_client),
+'guava.version': ver(project.library.java.guava),
+'hamcrest.version': ver(project.library.java.hamcrest_library),
+'jackson.version': ver(project.library.java.jackson_core),
+'joda.version': ver(project.library.java.joda_time),
+'junit.version': ver(project.library.java.junit),
+'pubsub.version': ver(project.library.java.google_api_services_pubsub),
+'slf4j.version': ver(project.library.java.slf4j_api),
+'spark.version': ver(project.library.java.spark_core),
+'hadoop.version': ver(project.library.java.hadoop_client),
+'mockito.version': ver(project.library.java.mockito_core),
+'maven-compiler-plugin.version': '3.7.0',
Review comment:
It would be useful to elevate all the Maven dependencies to the project root
build.gradle file so that its not duplicated or each archetype.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 81019)
Time Spent: 5h 40m (was: 5.5h)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=81018=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81018
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 15/Mar/18 23:23
Start Date: 15/Mar/18 23:23
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #4812:
[BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r174961281
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,53 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+// Extract the version from a library:version dependency.
+def ver = { return it.substring(1+it.lastIndexOf(':')) }
Review comment:
This doesn't work if the version has a classifier attached to it.
Please create a Dependency object and call getVersion on it:
https://docs.gradle.org/current/javadoc/org/gradle/api/artifacts/Dependency.html#getVersion--
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 81018)
Time Spent: 5.5h (was: 5h 20m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=80369=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80369 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 14/Mar/18 15:24 Start Date: 14/Mar/18 15:24 Worklog Time Spent: 10m Work Description: alanmyrvold commented on issue #4812: [BEAM-3249] Publish java artifacts using gradle URL: https://github.com/apache/beam/pull/4812#issuecomment-373061220 @lukecwik PTAL? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 80369) Time Spent: 5h 20m (was: 5h 10m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 5h 20m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=80349=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80349
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 14/Mar/18 15:02
Start Date: 14/Mar/18 15:02
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on a change in pull request
#4812: [BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r174490881
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,44 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+processResources {
+ filter org.apache.tools.ant.filters.ReplaceTokens, tokens: [
+ 'project.version': version,
+ 'bigquery.version': 'v2-rev374-1.22.0',
+ 'google-clients.version': '1.22.0',
+ 'guava.version': '20.0',
+ 'hamcrest.version': '1.3',
+ 'jackson.version': '2.8.9',
+ 'joda.version': '2.4',
+ 'junit.version': '4.12',
+ 'maven-compiler-plugin.version': '3.7.0',
+ 'maven-exec-plugin.version': '1.6.0',
+ 'maven-jar-plugin.version': '3.0.2',
+ 'maven-shade-plugin.version': '3.1.0',
+ 'pubsub.version': 'v1-rev10-1.22.0',
+ 'slf4j.version': '1.7.25',
+ 'spark.version': '2.2.1',
+ 'hadoop.version': '2.7.3',
+ 'maven-surefire-plugin.version': '2.20.1',
+ 'mockito.version': '1.9.5'
+ ]
+}
+
+task generateSources(type: Exec) {
+ println "Generating sources for archetype"
+ commandLine './generate-sources.sh'
+}
+
+sourceSets {
+ main {
+ output.dir('src', builtBy: 'generateSources')
+ }
+}
+
dependencies {
shadow project(path: ":examples:java", configuration: "shadow")
Review comment:
Done. Removed the dependency and added evaluationDependsOn.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 80349)
Time Spent: 4.5h (was: 4h 20m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=80351=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80351
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 14/Mar/18 15:02
Start Date: 14/Mar/18 15:02
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on a change in pull request
#4812: [BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r174490918
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,44 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+processResources {
+ filter org.apache.tools.ant.filters.ReplaceTokens, tokens: [
Review comment:
Fixed
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 80351)
Time Spent: 4h 50m (was: 4h 40m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=80352=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80352
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 14/Mar/18 15:02
Start Date: 14/Mar/18 15:02
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on a change in pull request
#4812: [BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r174490931
##
File path: build_rules.gradle
##
@@ -336,6 +361,18 @@ ext.applyJavaNature = {
asNode().appendNode('name', description)
}
}
+artifact packageTests {
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 80352)
Time Spent: 5h (was: 4h 50m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 5h
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=80353=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80353
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 14/Mar/18 15:03
Start Date: 14/Mar/18 15:03
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on a change in pull request
#4812: [BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r174490963
##
File path: build_rules.gradle
##
@@ -322,6 +334,19 @@ ext.applyJavaNature = {
artifacts.archives shadowJar
if (configuration.artifactId) {
+ task sourcesJar(type: Jar) {
+from 'src/main'
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 80353)
Time Spent: 5h 10m (was: 5h)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=80348=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80348
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 14/Mar/18 15:02
Start Date: 14/Mar/18 15:02
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on a change in pull request
#4812: [BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r174490863
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,44 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+processResources {
+ filter org.apache.tools.ant.filters.ReplaceTokens, tokens: [
+ 'project.version': version,
+ 'bigquery.version': 'v2-rev374-1.22.0',
+ 'google-clients.version': '1.22.0',
+ 'guava.version': '20.0',
+ 'hamcrest.version': '1.3',
+ 'jackson.version': '2.8.9',
+ 'joda.version': '2.4',
+ 'junit.version': '4.12',
+ 'maven-compiler-plugin.version': '3.7.0',
+ 'maven-exec-plugin.version': '1.6.0',
+ 'maven-jar-plugin.version': '3.0.2',
+ 'maven-shade-plugin.version': '3.1.0',
+ 'pubsub.version': 'v1-rev10-1.22.0',
+ 'slf4j.version': '1.7.25',
+ 'spark.version': '2.2.1',
+ 'hadoop.version': '2.7.3',
+ 'maven-surefire-plugin.version': '2.20.1',
+ 'mockito.version': '1.9.5'
+ ]
+}
+
+task generateSources(type: Exec) {
+ println "Generating sources for archetype"
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 80348)
Time Spent: 4h 20m (was: 4h 10m)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=80354=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80354
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 14/Mar/18 15:03
Start Date: 14/Mar/18 15:03
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on a change in pull request
#4812: [BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r174490956
##
File path: build_rules.gradle
##
@@ -322,6 +334,19 @@ ext.applyJavaNature = {
artifacts.archives shadowJar
if (configuration.artifactId) {
+ task sourcesJar(type: Jar) {
+from 'src/main'
+classifier = 'sources'
+ }
+ task testSourcesJar(type: Jar) {
+from 'src/test'
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 80354)
Time Spent: 5h 10m (was: 5h)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=80350=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80350
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 14/Mar/18 15:02
Start Date: 14/Mar/18 15:02
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on a change in pull request
#4812: [BEAM-3249] Publish java artifacts using gradle
URL: https://github.com/apache/beam/pull/4812#discussion_r174490893
##
File path: sdks/java/maven-archetypes/examples/build.gradle
##
@@ -17,10 +17,44 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature()
+applyJavaNature(artifactId: "beam-sdks-java-maven-archetypes-examples")
description = "Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples"
+processResources {
+ filter org.apache.tools.ant.filters.ReplaceTokens, tokens: [
+ 'project.version': version,
+ 'bigquery.version': 'v2-rev374-1.22.0',
Review comment:
Extracting the versions from the project.library.java.* libraries. I did not
see maven plugin versions defined elsewhere, so those are still hardcoded.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 80350)
Time Spent: 4h 40m (was: 4.5h)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven without using Gradle's support for incremental builds.
> Gradle (mins)
> min: 25.04
> max: 160.14
> median: 45.78
> average: 52.19
> stdev: 30.80
> Maven (mins)
> min: 56.86
> max: 216.55
> median: 87.93
> average: 109.10
> stdev: 48.01
> I excluded a few timeouts (240 mins) that happened during the Maven build
> from its numbers but we can see conclusively that Gradle is about twice as
> fast for the build when compared to Maven when run using Jenkins.
> Original dev@ thread:
> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
> The data is available here
> https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[
https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=79010=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-79010
]
ASF GitHub Bot logged work on BEAM-3249:
Author: ASF GitHub Bot
Created on: 09/Mar/18 20:17
Start Date: 09/Mar/18 20:17
Worklog Time Spent: 10m
Work Description: lukecwik closed pull request #4814: [BEAM-3249] Drop
Java Maven PreCommit.
URL: https://github.com/apache/beam/pull/4814
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/.test-infra/jenkins/job_beam_PreCommit_Java_GradleBuild.groovy
b/.test-infra/jenkins/job_beam_PreCommit_Java_GradleBuild.groovy
index f5deef1b552..a1bc9624343 100644
--- a/.test-infra/jenkins/job_beam_PreCommit_Java_GradleBuild.groovy
+++ b/.test-infra/jenkins/job_beam_PreCommit_Java_GradleBuild.groovy
@@ -48,7 +48,7 @@ job('beam_PreCommit_Java_GradleBuild') {
def gradle_command_line = './gradlew ' + gradle_switches.join(' ') + '
:javaPreCommit'
// Sets that this is a PreCommit job.
- common_job_properties.setPreCommit(delegate, gradle_command_line, 'Run Java
Gradle PreCommit')
+ common_job_properties.setPreCommit(delegate, gradle_command_line, 'Run Java
PreCommit')
steps {
gradle {
rootBuildScriptDir(common_job_properties.checkoutDir)
diff --git a/.test-infra/jenkins/job_beam_PreCommit_Java_MavenInstall.groovy
b/.test-infra/jenkins/job_beam_PreCommit_Java_MavenInstall.groovy
deleted file mode 100644
index 0775e2f309e..000
--- a/.test-infra/jenkins/job_beam_PreCommit_Java_MavenInstall.groovy
+++ /dev/null
@@ -1,56 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import common_job_properties
-
-// This is the Java precommit which runs a maven install, and the current set
-// of precommit tests.
-mavenJob('beam_PreCommit_Java_MavenInstall') {
- description('Runs an install of the current GitHub Pull Request.')
-
- previousNames('beam_PreCommit_MavenVerify')
-
- // Execute concurrent builds if necessary.
- concurrentBuild()
-
- // Set common parameters.
- common_job_properties.setTopLevelMainJobProperties(
-delegate,
-'master',
-240)
-
- // Set Maven parameters.
- common_job_properties.setMavenConfig(delegate)
-
- // Sets that this is a PreCommit job.
- common_job_properties.setPreCommit(delegate, 'mvn clean install -pl
sdks/java/core,runners/direct-java,sdks/java/fn-execution -am -amd', 'Run Java
PreCommit')
-
- // Maven goals for this job: The Java SDK, its dependencies, and things that
depend on it.
- goals([
-'--batch-mode',
-'--errors',
-'--activate-profiles
release,jenkins-precommit,direct-runner,dataflow-runner,spark-runner,flink-runner,apex-runner',
-'--projects sdks/java/core,runners/direct-java,sdks/java/fn-execution',
-'--also-make',
-'--also-make-dependents',
-'-D pullRequest=$ghprbPullId',
-'help:effective-settings',
-'clean',
-'install'
- ].join(' '))
-}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 79010)
Time Spent: 4h 10m (was: 4h)
> Use Gradle to build/release project
> ---
>
> Key: BEAM-3249
> URL: https://issues.apache.org/jira/browse/BEAM-3249
> Project: Beam
> Issue Type: Improvement
> Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> I have collected data by running several builds against master using Gradle
> and Maven
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=78984=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-78984 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 09/Mar/18 19:06 Start Date: 09/Mar/18 19:06 Worklog Time Spent: 10m Work Description: kennknowles commented on issue #4830: [BEAM-3249, BEAM-3457] Make Apache Rat a dependency of the Java and Gradle precommits. URL: https://github.com/apache/beam/pull/4830#issuecomment-371914836 Yea, if something does contain tests we should run them. Though if it is the new root, we should just make it the root for Java. And if it is the root for multiple languages (like if there were some tests in `model`) then it would be best to run them once separately and to _not_ run them on any language-specific build Job. Anyhow, we can just evolve as things happen. This change LGTM for sure. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 78984) Time Spent: 4h (was: 3h 50m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 4h > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=78982=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-78982 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 09/Mar/18 19:05 Start Date: 09/Mar/18 19:05 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #4830: [BEAM-3249, BEAM-3457] Make Apache Rat a dependency of the Java and Gradle precommits. URL: https://github.com/apache/beam/pull/4830#issuecomment-371914392 The pre-shaded guava isn't a great example because it wouldn't contain tests but other projects that contain utility/test libraries that the SDK and runners could depend on. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 78982) Time Spent: 3h 40m (was: 3.5h) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 3h 40m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=78983=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-78983 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 09/Mar/18 19:05 Start Date: 09/Mar/18 19:05 Worklog Time Spent: 10m Work Description: kennknowles commented on issue #4830: [BEAM-3249, BEAM-3457] Make Apache Rat a dependency of the Java and Gradle precommits. URL: https://github.com/apache/beam/pull/4830#issuecomment-371914429 Filed https://issues.apache.org/jira/browse/BEAM-3823 about breaking out RAT. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 78983) Time Spent: 3h 50m (was: 3h 40m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=78981=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-78981 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 09/Mar/18 19:01 Start Date: 09/Mar/18 19:01 Worklog Time Spent: 10m Work Description: kennknowles commented on issue #4830: [BEAM-3249, BEAM-3457] Make Apache Rat a dependency of the Java and Gradle precommits. URL: https://github.com/apache/beam/pull/4830#issuecomment-371913408 What do you expect to be run by `sdks:java:core:buildNeeded` if there is a pre-shaded Guava? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 78981) Time Spent: 3.5h (was: 3h 20m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 3.5h > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3249) Use Gradle to build/release project
[ https://issues.apache.org/jira/browse/BEAM-3249?focusedWorklogId=78962=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-78962 ] ASF GitHub Bot logged work on BEAM-3249: Author: ASF GitHub Bot Created on: 09/Mar/18 18:13 Start Date: 09/Mar/18 18:13 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #4830: [BEAM-3249, BEAM-3457] Make Apache Rat a dependency of the Java and Gradle precommits. URL: https://github.com/apache/beam/pull/4830#issuecomment-371898019 Swapped to `:sdks:java:core:buildNeeded` and `:sdks:java:core:buildDependents` to prevent a regression if the SDK core ever depends on another project like the shaded guava module that had been discussed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 78962) Time Spent: 3h 20m (was: 3h 10m) > Use Gradle to build/release project > --- > > Key: BEAM-3249 > URL: https://issues.apache.org/jira/browse/BEAM-3249 > Project: Beam > Issue Type: Improvement > Components: build-system, testing >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 3h 20m > Remaining Estimate: 0h > > I have collected data by running several builds against master using Gradle > and Maven without using Gradle's support for incremental builds. > Gradle (mins) > min: 25.04 > max: 160.14 > median: 45.78 > average: 52.19 > stdev: 30.80 > Maven (mins) > min: 56.86 > max: 216.55 > median: 87.93 > average: 109.10 > stdev: 48.01 > I excluded a few timeouts (240 mins) that happened during the Maven build > from its numbers but we can see conclusively that Gradle is about twice as > fast for the build when compared to Maven when run using Jenkins. > Original dev@ thread: > https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E > The data is available here > https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
