[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-07-13 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=122737=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-122737
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 13/Jul/18 10:09
Start Date: 13/Jul/18 10:09
Worklog Time Spent: 10m 
  Work Description: vectorijk commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-404789589
 
 
   I am proposing a new `tpc` module. It needs this feature. This `run` gradle 
task looks good. Eventually, we could figure out fat jar solution. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 122737)
Time Spent: 10h 50m  (was: 10h 40m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 10h 50m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-25 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=95346=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-95346
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 26/Apr/18 05:04
Start Date: 26/Apr/18 05:04
Worklog Time Spent: 10m 
  Work Description: melap commented on issue #418: [BEAM-3995] Update 
Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#issuecomment-384513986
 
 
   the robot likes me right now @kennknowles  ;)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 95346)
Time Spent: 10h 40m  (was: 10.5h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 10h 40m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-25 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=95345=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-95345
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 26/Apr/18 05:03
Start Date: 26/Apr/18 05:03
Worklog Time Spent: 10m 
  Work Description: asfgit closed pull request #418: [BEAM-3995] Update 
Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/src/documentation/sdks/nexmark.md 
b/src/documentation/sdks/nexmark.md
index 5e86cd782..d73943a4f 100644
--- a/src/documentation/sdks/nexmark.md
+++ b/src/documentation/sdks/nexmark.md
@@ -9,12 +9,14 @@ permalink: /documentation/sdks/java/nexmark/
 ## What it is
 
 Nexmark is a suite of pipelines inspired by the 'continuous data stream'
-queries in [Nexmark research 
paper](http://datalab.cs.pdx.edu/niagaraST/NEXMark/)
+queries in [Nexmark research
+paper](http://datalab.cs.pdx.edu/niagaraST/NEXMark/)
 
-These are multiple queries over a three entities model representing on online 
auction system:
+These are multiple queries over a three entities model representing on online
+auction system:
 
- - **Person** represents a person submitting an item for auction and/or making 
a bid
-on an auction.
+ - **Person** represents a person submitting an item for auction and/or making
+   a bid on an auction.
  - **Auction** represents an item under auction.
  - **Bid** represents a bid for an item under auction.
 
@@ -62,11 +64,14 @@ We have augmented the original queries with five more:
   queries.
 
 ## Benchmark workload configuration
-Here are some of the knobs of the benchmark workload (see 
[NexmarkConfiguration.java](https://github.com/apache/beam/blob/master/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/NexmarkConfiguration.java)).
+
+Here are some of the knobs of the benchmark workload (see
+[NexmarkConfiguration.java](https://github.com/apache/beam/blob/master/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/NexmarkConfiguration.java)).
 
 These configuration items can be passed to the launch command line.
 
 ### Events generation (defaults)
+
 * 100 000 events generated
 * 100 generator threads
 * Event rate in SIN curve
@@ -76,21 +81,26 @@ These configuration items can be passed to the launch 
command line.
 * 1000 concurrent persons bidding / creating auctions
 
 ### Windows (defaults)
+
 * size 10s
 * sliding period 5s
 * watermark hold for 0s
 
 ### Events Proportions (defaults)
+
 * Hot Auctions = ½
 * Hot Bidders =¼
 * Hot Sellers=¼
 
 ### Technical
+
 * Artificial CPU load
 * Artificial IO load
 
 ## Nexmark output
-Here is an example output of the Nexmark benchmark run in streaming mode with 
the SMOKE suite on the (local) direct runner:
+
+Here is an example output of the Nexmark benchmark run in streaming mode with
+the SMOKE suite on the (local) direct runner:
 
 
 Performance:
@@ -112,17 +122,19 @@ Performance:
 
 ## Benchmark launch configuration
 
-We can specify the Beam runner to use with maven profiles, available profiles 
are:
-
-direct-runner
-spark-runner
-flink-runner
-apex-runner
+The Nexmark launcher accepts the `--runner` argument as usual for programs that
+use Beam PipelineOptions to manage their command line arguments. In addition
+to this, the necessary dependencies must be configured.
 
-The runner must also be specified like in any other Beam pipeline using:
+When running via Gradle, the following two parameters control the execution:
 
---runner
+-P nexmark.args
+The command line to pass to the Nexmark main program.
 
+-P nexmark.runner
+   The Gradle project name of the runner, such as 
":beam-runners-direct-java" or
+   ":beam-runners-flink. The project names can be found in the root
+`settings.gradle`.
 
 Test data is deterministically synthesized on demand. The test
 data may be synthesized in the same pipeline as the query itself,
@@ -417,124 +429,192 @@ Yet to come
 
 ### Running SMOKE suite on the DirectRunner (local)
 
+The DirectRunner is default, so it is not required to pass `-Pnexmark.runner`.
+Here we do it for maximum clarity.
+
+The direct runner does not have separate batch and streaming modes, but the
+Nexmark launch does.
+
+These parameters leave on many of the DirectRunner's extra safety checks so the
+SMOKE suite can make sure there is nothing broken in the Nexmark suite.
+
 Batch Mode:
 
-mvn exec:java -Dexec.mainClass=org.apache.beam.sdk.nexmark.Main 
-Pdirect-runner -Dexec.args="--runner=DirectRunner --suite=SMOKE 
--streaming=false 

[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-25 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=95343=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-95343
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 26/Apr/18 05:00
Start Date: 26/Apr/18 05:00
Worklog Time Spent: 10m 
  Work Description: melap commented on issue #418: [BEAM-3995] Update 
Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#issuecomment-384513438
 
 
   @asfgit merge


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 95343)
Time Spent: 10h 20m  (was: 10h 10m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 10h 20m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-25 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=95342=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-95342
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 26/Apr/18 04:59
Start Date: 26/Apr/18 04:59
Worklog Time Spent: 10m 
  Work Description: asfgit commented on issue #418: [BEAM-3995] Update 
Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#issuecomment-384513328
 
 
   Error: PR failed in verification; check the Jenkins job for more information.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 95342)
Time Spent: 10h 10m  (was: 10h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 10h 10m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-25 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=95341=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-95341
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 26/Apr/18 04:55
Start Date: 26/Apr/18 04:55
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #418: [BEAM-3995] Update 
Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#issuecomment-384512907
 
 
   @asfgit merge


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 95341)
Time Spent: 10h  (was: 9h 50m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 10h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-25 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=95338=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-95338
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 26/Apr/18 04:45
Start Date: 26/Apr/18 04:45
Worklog Time Spent: 10m 
  Work Description: melap commented on issue #418: [BEAM-3995] Update 
Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#issuecomment-384511690
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 95338)
Time Spent: 9h 50m  (was: 9h 40m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 9h 50m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-19 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=92568=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-92568
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 19/Apr/18 13:53
Start Date: 19/Apr/18 13:53
Worklog Time Spent: 10m 
  Work Description: asfgit commented on issue #418: [BEAM-3995] Update 
Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#issuecomment-382744330
 
 
   Error: PR failed in verification; check the Jenkins job for more information.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 92568)
Time Spent: 9h 40m  (was: 9.5h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 9h 40m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-19 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=92567=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-92567
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 19/Apr/18 13:49
Start Date: 19/Apr/18 13:49
Worklog Time Spent: 10m 
  Work Description: asfgit commented on issue #418: [BEAM-3995] Update 
Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#issuecomment-382742827
 
 
   Error: PR failed in verification; check the Jenkins job for more information.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 92567)
Time Spent: 9.5h  (was: 9h 20m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 9.5h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-19 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=92560=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-92560
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 19/Apr/18 13:44
Start Date: 19/Apr/18 13:44
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on a change in pull request #418: 
[BEAM-3995] Update Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#discussion_r182749979
 
 

 ##
 File path: src/documentation/sdks/nexmark.md
 ##
 @@ -112,17 +122,19 @@ Performance:
 
 ## Benchmark launch configuration
 
-We can specify the Beam runner to use with maven profiles, available profiles 
are:
-
-direct-runner
-spark-runner
-flink-runner
-apex-runner
+The Nexmark launcher accepts the `--runner` argument as usual for programs that
+use Beam PipelineOptions to manage their command line arguments. In addition
+to this, the necessary dependencies must be configured.
 
-The runner must also be specified like in any other Beam pipeline using:
+When running via Gradle, the following two parameters control the execution:
 
---runner
+-P nexmark.args
+The command line to pass to the Nexmark main program.
 
+-P nexmark.runner
 
 Review comment:
   OK. Filed https://issues.apache.org/jira/browse/BEAM-4139 in the gradle 
improvements to go back to the repo and change it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 92560)
Time Spent: 9h 10m  (was: 9h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 9h 10m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-19 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=92561=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-92561
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 19/Apr/18 13:44
Start Date: 19/Apr/18 13:44
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #418: [BEAM-3995] Update 
Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#issuecomment-382741167
 
 
   @asfgit merge


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 92561)
Time Spent: 9h 20m  (was: 9h 10m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 9h 20m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-16 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=91189=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91189
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 16/Apr/18 07:43
Start Date: 16/Apr/18 07:43
Worklog Time Spent: 10m 
  Work Description: echauchot commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-381508674
 
 
   @kennknowles if you want to schedule spark streaming run you need to skip 
queries 3 and 7. The only way to do this right now is to use --query=x 
--query=y --query=z skipping 3 and 7 or create a custom suite for spark (needs 
an update of the nexmark code) ... I'm not entirely sure this command line 
works. But the main problem is the termination part in spark streaming 
pipelines that I was mentioning above (see related 
https://issues.apache.org/jira/browse/BEAM-2847).
   Also, we can schedule Apex runs for which we need to skip query 3 in both 
batch and streaming modes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 91189)
Time Spent: 9h  (was: 8h 50m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 9h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-14 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=91102=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91102
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 14/Apr/18 23:27
Start Date: 14/Apr/18 23:27
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-381367922
 
 
   @echauchot yes that sounds like a good idea. Let's set things up so we can 
run things that are not 100% supported by every runner. I think Spark streaming 
probably supports some of the suites, just not the joins?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 91102)
Time Spent: 8h 50m  (was: 8h 40m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 8h 50m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-13 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90736=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90736
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 13/Apr/18 08:31
Start Date: 13/Apr/18 08:31
Worklog Time Spent: 10m 
  Work Description: echauchot commented on a change in pull request #418: 
[BEAM-3995] Update Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#discussion_r181319081
 
 

 ##
 File path: src/documentation/sdks/nexmark.md
 ##
 @@ -112,17 +122,19 @@ Performance:
 
 ## Benchmark launch configuration
 
-We can specify the Beam runner to use with maven profiles, available profiles 
are:
-
-direct-runner
-spark-runner
-flink-runner
-apex-runner
+The Nexmark launcher accepts the `--runner` argument as usual for programs that
+use Beam PipelineOptions to manage their command line arguments. In addition
+to this, the necessary dependencies must be configured.
 
-The runner must also be specified like in any other Beam pipeline using:
+When running via Gradle, the following two parameters control the execution:
 
---runner
+-P nexmark.args
+The command line to pass to the Nexmark main program.
 
+-P nexmark.runner
 
 Review comment:
   Yes I think I prefer this explicit version rather than the magic one.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90736)
Time Spent: 8h 40m  (was: 8.5h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 8h 40m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-12 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90568=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90568
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 12/Apr/18 20:37
Start Date: 12/Apr/18 20:37
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on a change in pull request #418: 
[BEAM-3995] Update Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#discussion_r181214223
 
 

 ##
 File path: src/documentation/sdks/nexmark.md
 ##
 @@ -112,17 +122,19 @@ Performance:
 
 ## Benchmark launch configuration
 
-We can specify the Beam runner to use with maven profiles, available profiles 
are:
-
-direct-runner
-spark-runner
-flink-runner
-apex-runner
+The Nexmark launcher accepts the `--runner` argument as usual for programs that
+use Beam PipelineOptions to manage their command line arguments. In addition
+to this, the necessary dependencies must be configured.
 
-The runner must also be specified like in any other Beam pipeline using:
+When running via Gradle, the following two parameters control the execution:
 
---runner
+-P nexmark.args
+The command line to pass to the Nexmark main program.
 
+-P nexmark.runner
 
 Review comment:
   The alternative which I mean is that we could just make it like before where 
you choose a profile and `-P nexmark.runner` is a choice from an enum, not a 
magic Gradle subproject name.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90568)
Time Spent: 8.5h  (was: 8h 20m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 8.5h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-12 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90565=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90565
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 12/Apr/18 20:36
Start Date: 12/Apr/18 20:36
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on a change in pull request #418: 
[BEAM-3995] Update Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#discussion_r181213684
 
 

 ##
 File path: src/documentation/sdks/nexmark.md
 ##
 @@ -417,124 +429,192 @@ Yet to come
 
 ### Running SMOKE suite on the DirectRunner (local)
 
+The DirectRunner is default, so it is not required to pass `-Pnexmark.runner`.
+Here we do it for maximum clarity.
+
+The direct runner does not have separate batch and streaming modes, but the
+Nexmark launch does.
+
+These parameters leave on many of the DirectRunner's extra safety checks so the
+SMOKE suite can make sure there is nothing broken in the Nexmark suite.
+
 Batch Mode:
 
-mvn exec:java -Dexec.mainClass=org.apache.beam.sdk.nexmark.Main 
-Pdirect-runner -Dexec.args="--runner=DirectRunner --suite=SMOKE 
--streaming=false --manageResources=false --monitorJobs=true 
--enforceEncodability=true --enforceImmutability=true"
+./gradlew :beam-sdks-java-nexmark:run \
+-Pnexmark.runner=":beam-runners-direct-java" \
+-Pnexmark.args="
+--runner=DirectRunner
+--streaming=false
+--suite=SMOKE
+--manageResources=false
+--monitorJobs=true
+--enforceEncodability=true
+--enforceImmutability=true"
 
 Streaming Mode:
 
-mvn exec:java -Dexec.mainClass=org.apache.beam.sdk.nexmark.Main 
-Pdirect-runner -Dexec.args="--runner=DirectRunner --suite=SMOKE 
--streaming=true --manageResources=false --monitorJobs=true 
--enforceEncodability=true --enforceImmutability=true"
-
+./gradlew :beam-sdks-java-nexmark:run \
+-Pnexmark.runner=":beam-runners-direct-java" \
+-Pnexmark.args="
+--runner=DirectRunner
+--streaming=true
+--suite=SMOKE
+--manageResources=false
+--monitorJobs=true
+--enforceEncodability=true
+--enforceImmutability=true"
 
 ### Running SMOKE suite on the SparkRunner (local)
 
+The SparkRunner is special-cased in the Nexmark gradle launch. The task will
+provide the version of Spark that the SparkRunner is built against, and
+configure logging.
 
 Review comment:
   This is a comment about the hardcoding above. The nexmark build tweaks the 
dependencies based on special needs of the SparkRunner instead of being just a 
generic suite of pipelines. I will delete the comment here since it "just 
works" for users copying and pasting from here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90565)
Time Spent: 8h 10m  (was: 8h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 8h 10m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-12 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90566=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90566
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 12/Apr/18 20:36
Start Date: 12/Apr/18 20:36
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on a change in pull request #418: 
[BEAM-3995] Update Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#discussion_r181212848
 
 

 ##
 File path: src/documentation/sdks/nexmark.md
 ##
 @@ -112,17 +122,19 @@ Performance:
 
 ## Benchmark launch configuration
 
-We can specify the Beam runner to use with maven profiles, available profiles 
are:
-
-direct-runner
-spark-runner
-flink-runner
-apex-runner
+The Nexmark launcher accepts the `--runner` argument as usual for programs that
+use Beam PipelineOptions to manage their command line arguments. In addition
+to this, the necessary dependencies must be configured.
 
-The runner must also be specified like in any other Beam pipeline using:
+When running via Gradle, the following two parameters control the execution:
 
---runner
+-P nexmark.args
+The command line to pass to the Nexmark main program.
 
+-P nexmark.runner
 
 Review comment:
   Yes, Gradle puts them on the classpath. The one exception is the Spark 
runner. I used to have a `-P nexmark.provideSpark` parameter to keep it a 
little decoupled, but since it is pretty much still "magic" knowledge, I 
followed @lukecwik's suggestion and just hardcoded knowledge of the dependency, 
so I did that 
[here](https://github.com/apache/beam/blob/master/sdks/java/nexmark/build.gradle#L35).
   
   I think when I add the fat jar capability it may require a separate fat jar 
for each known runner config. At that point, pretending that the runner 
dependency can be anything (versus a selection of a hardcoded profile) may not 
be useful.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90566)
Time Spent: 8h 20m  (was: 8h 10m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 8h 20m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-12 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90357=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90357
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 12/Apr/18 09:58
Start Date: 12/Apr/18 09:58
Worklog Time Spent: 10m 
  Work Description: echauchot commented on a change in pull request #418: 
[BEAM-3995] Update Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#discussion_r181024910
 
 

 ##
 File path: src/documentation/sdks/nexmark.md
 ##
 @@ -112,17 +122,19 @@ Performance:
 
 ## Benchmark launch configuration
 
-We can specify the Beam runner to use with maven profiles, available profiles 
are:
-
-direct-runner
-spark-runner
-flink-runner
-apex-runner
+The Nexmark launcher accepts the `--runner` argument as usual for programs that
+use Beam PipelineOptions to manage their command line arguments. In addition
+to this, the necessary dependencies must be configured.
 
-The runner must also be specified like in any other Beam pipeline using:
+When running via Gradle, the following two parameters control the execution:
 
---runner
+-P nexmark.args
+The command line to pass to the Nexmark main program.
 
+-P nexmark.runner
 
 Review comment:
   no more profiles for the runners then? The needed dependencies are all 
included in the runner projet specified by the user?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90357)
Time Spent: 8h  (was: 7h 50m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 8h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-12 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90356=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90356
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 12/Apr/18 09:58
Start Date: 12/Apr/18 09:58
Worklog Time Spent: 10m 
  Work Description: echauchot commented on a change in pull request #418: 
[BEAM-3995] Update Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#discussion_r181026806
 
 

 ##
 File path: src/documentation/sdks/nexmark.md
 ##
 @@ -417,124 +429,192 @@ Yet to come
 
 ### Running SMOKE suite on the DirectRunner (local)
 
+The DirectRunner is default, so it is not required to pass `-Pnexmark.runner`.
+Here we do it for maximum clarity.
+
+The direct runner does not have separate batch and streaming modes, but the
+Nexmark launch does.
+
+These parameters leave on many of the DirectRunner's extra safety checks so the
+SMOKE suite can make sure there is nothing broken in the Nexmark suite.
+
 Batch Mode:
 
-mvn exec:java -Dexec.mainClass=org.apache.beam.sdk.nexmark.Main 
-Pdirect-runner -Dexec.args="--runner=DirectRunner --suite=SMOKE 
--streaming=false --manageResources=false --monitorJobs=true 
--enforceEncodability=true --enforceImmutability=true"
+./gradlew :beam-sdks-java-nexmark:run \
+-Pnexmark.runner=":beam-runners-direct-java" \
+-Pnexmark.args="
+--runner=DirectRunner
+--streaming=false
+--suite=SMOKE
+--manageResources=false
+--monitorJobs=true
+--enforceEncodability=true
+--enforceImmutability=true"
 
 Streaming Mode:
 
-mvn exec:java -Dexec.mainClass=org.apache.beam.sdk.nexmark.Main 
-Pdirect-runner -Dexec.args="--runner=DirectRunner --suite=SMOKE 
--streaming=true --manageResources=false --monitorJobs=true 
--enforceEncodability=true --enforceImmutability=true"
-
+./gradlew :beam-sdks-java-nexmark:run \
+-Pnexmark.runner=":beam-runners-direct-java" \
+-Pnexmark.args="
+--runner=DirectRunner
+--streaming=true
+--suite=SMOKE
+--manageResources=false
+--monitorJobs=true
+--enforceEncodability=true
+--enforceImmutability=true"
 
 ### Running SMOKE suite on the SparkRunner (local)
 
+The SparkRunner is special-cased in the Nexmark gradle launch. The task will
+provide the version of Spark that the SparkRunner is built against, and
+configure logging.
 
 Review comment:
   Sorry but I don't understand this comment: owing to what's bellow, the 
version is automatically provided and no parameter needs to be set by the user. 
So why not just remove this comment if it is not targeted to the user of 
nexmark?
   Is there something I missed? 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90356)
Time Spent: 7h 50m  (was: 7h 40m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 7h 50m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-12 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90354=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90354
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 12/Apr/18 09:53
Start Date: 12/Apr/18 09:53
Worklog Time Spent: 10m 
  Work Description: echauchot commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380745803
 
 
   @kennknowles Do you agree to schedule the list of runs in my above comment?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90354)
Time Spent: 7h 40m  (was: 7.5h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 7h 40m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90126=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90126
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 19:01
Start Date: 11/Apr/18 19:01
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #418: [BEAM-3995] Update 
Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418#issuecomment-380560925
 
 
   @jbonofre @echauchot @akedin @apilloud


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90126)
Time Spent: 7.5h  (was: 7h 20m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 7.5h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90125=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90125
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 19:00
Start Date: 11/Apr/18 19:00
Worklog Time Spent: 10m 
  Work Description: kennknowles opened a new pull request #418: [BEAM-3995] 
Update Nexmark launch instructions for Gradle
URL: https://github.com/apache/beam-site/pull/418
 
 
   This ports the instructions to Gradle. There are a couple of known issues, 
but I thought a PR updating the docs was a great place to make sure the new 
invocations work.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90125)
Time Spent: 7h 20m  (was: 7h 10m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 7h 20m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90085=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90085
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 17:56
Start Date: 11/Apr/18 17:56
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380535731
 
 
   I deliberately considered and rejected the idea that Nexmark should have 
such explicit intimate knowledge of a runner's implementation details. TBH I'm 
also unhappy about the fact that it has to know as much as it does about how to 
provide Spark, or have a special parameter just to do so. Even the presence of 
the parameter is a major smell.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90085)
Time Spent: 7h 10m  (was: 7h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 7h 10m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90084=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90084
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 17:55
Start Date: 11/Apr/18 17:55
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380537905
 
 
   I want this done more than I want to haggle about getting it very high 
quality in a way we both agree on. I will merge your suggestion to take the 
LGTM and move on.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90084)
Time Spent: 7h  (was: 6h 50m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 7h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90079=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90079
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 17:50
Start Date: 11/Apr/18 17:50
Worklog Time Spent: 10m 
  Work Description: kennknowles closed pull request #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/java/nexmark/build.gradle b/sdks/java/nexmark/build.gradle
index 83a3d05f99d..8fcba5fa91b 100644
--- a/sdks/java/nexmark/build.gradle
+++ b/sdks/java/nexmark/build.gradle
@@ -21,6 +21,26 @@ applyJavaNature()
 
 description = "Apache Beam :: SDKs :: Java :: Nexmark"
 
+// When running via Gradle, this property can be used to pass commandline 
arguments
+// to the nexmark launch
+def nexmarkArgsProperty = "nexmark.args"
+def nexmarkArgs = project.hasProperty(nexmarkArgsProperty) ?
+project.getProperty(nexmarkArgsProperty).split() : []
+
+// When running via Gradle, this property sets the runner dependency
+def nexmarkRunnerProperty = "nexmark.runner"
+def nexmarkRunnerDependency = (project.hasProperty(nexmarkRunnerProperty)
+? project.getProperty(nexmarkRunnerProperty)
+: ":beam-runners-direct-java")
+def shouldProvideSpark = ":beam-runners-spark".equals(nexmarkRunnerDependency)
+
+configurations {
+  // A configuration for running the Nexmark launcher directly from Gradle, 
which
+  // uses Gradle to put the appropriate dependencies on the Classpath rather 
than
+  // bundling them into a fat jar
+  gradleRun
+}
+
 dependencies {
   compile library.java.guava
   shadow project(path: ":beam-sdks-java-core", configuration: "shadow")
@@ -43,8 +63,43 @@ dependencies {
   shadow project(path: ":beam-runners-direct-java", configuration: "shadow")
   shadow library.java.slf4j_jdk14
   testCompile library.java.hamcrest_core
+
+  gradleRun project(path: project.path, configuration: "shadow")
+  gradleRun project(path: nexmarkRunnerDependency, configuration: "shadow")
+
+  // The Spark runner requires the user to provide a Spark dependency. For 
self-contained
+  // runs with the Spark runner, we can provide such a dependency. This is 
deliberately phrased
+  // to not hardcode any runner other than :beam-runners-direct-java
+  if (shouldProvideSpark) {
+gradleRun library.java.spark_streaming
+gradleRun library.java.spark_core, {
+  exclude group:"org.slf4j", module:"jul-to-slf4j"
+}
+  }
+}
+
+if (shouldProvideSpark) {
+  configurations.gradleRun {
+// Using Spark runner causes a StackOverflowError if slf4j-jdk14 is on the 
classpath
+exclude group: "org.slf4j", module: "slf4j-jdk14"
+  }
 }
 
 test {
   jvmArgs "-da"
 }
+
+// Execute the Nexmark queries or suites via Gradle.
+//
+// Parameters:
+//   -Pnexmark.runner
+//   Specify a runner subproject, such as ":beam-runners-spark" or 
":beam-runners-flink"
+//   Defaults to ":beam-runners-direct-java"
+//
+//   -Pnexmark.args
+//   Specify the command line for invoking org.apache.beam.sdk.nexmark.Main
+task run(type: JavaExec) {
+  main = "org.apache.beam.sdk.nexmark.Main"
+  classpath = configurations.gradleRun
+  args nexmarkArgs
+}


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90079)
Time Spent: 6h 50m  (was: 6h 40m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90076=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90076
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 17:46
Start Date: 11/Apr/18 17:46
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380537905
 
 
   Unfortunately, I want this done more than I want to haggle about getting it 
to my quality bar. I will merge your suggestion to take the LGTM and move on.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90076)
Time Spent: 6h 40m  (was: 6.5h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 6h 40m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90074=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90074
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 17:44
Start Date: 11/Apr/18 17:44
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380537355
 
 
   An alternative would be to make the `nexmark.runner` parameter an enum 
rather than a free-form dependency, making this _only_ special cases like with 
Maven profiles. That is also internally consistent.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90074)
Time Spent: 6.5h  (was: 6h 20m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90070=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90070
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 17:41
Start Date: 11/Apr/18 17:41
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380535731
 
 
   I deliberately considered and rejected the idea that Nexmark should have 
such explicit intimate knowledge of a runner's implementation details. TBH I'm 
also unhappy about the fact that it has to know as much as it does, but I don't 
want to make it worse.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90070)
Time Spent: 6h 20m  (was: 6h 10m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 6h 20m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90066=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90066
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 17:40
Start Date: 11/Apr/18 17:40
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380535731
 
 
   I deliberately considered and rejected the idea that Nexmark should have 
intimate knowledge of a runner's implementation details.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90066)
Time Spent: 6h  (was: 5h 50m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 6h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90067=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90067
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 17:40
Start Date: 11/Apr/18 17:40
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380535731
 
 
   I deliberately considered and rejected the idea that Nexmark should have 
such explicit intimate knowledge of a runner's implementation details.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90067)
Time Spent: 6h 10m  (was: 6h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90065=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90065
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 17:39
Start Date: 11/Apr/18 17:39
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380535731
 
 
   I deliberately considered and rejected the idea that we should explicitly 
special case based on magic strings or knowledge of a particular runner.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90065)
Time Spent: 5h 50m  (was: 5h 40m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90062=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90062
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 17:38
Start Date: 11/Apr/18 17:38
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380535731
 
 
   I deliberately considered and rejected the idea that we should explicitly 
special case based on magic strings.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90062)
Time Spent: 5h 40m  (was: 5.5h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 5h 40m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90060=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90060
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 17:38
Start Date: 11/Apr/18 17:38
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380535492
 
 
   Your change LGTM.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90060)
Time Spent: 5.5h  (was: 5h 20m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 5.5h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90059=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90059
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 17:37
Start Date: 11/Apr/18 17:37
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380535433
 
 
   It is not credible that the naming revert was technical required. Easy to 
believe it was the shortest path to getting things out the door, though.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90059)
Time Spent: 5h 20m  (was: 5h 10m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 5h 20m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=90036=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90036
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 16:45
Start Date: 11/Apr/18 16:45
Worklog Time Spent: 10m 
  Work Description: akedin commented on issue #5051: [BEAM-3995] Add basic 
Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380519633
 
 
   lgtm


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 90036)
Time Spent: 5h 10m  (was: 5h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 5h 10m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89963=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89963
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 13:52
Start Date: 11/Apr/18 13:52
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on a change in pull request #5051: 
[BEAM-3995] Add basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180760912
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -43,8 +68,53 @@ dependencies {
   shadow project(path: ":beam-runners-direct-java", configuration: "shadow")
   shadow library.java.slf4j_jdk14
   testCompile library.java.hamcrest_core
+
+  gradleRun project(path: project.path, configuration: "shadow")
+  gradleRun project(path: nexmarkRunnerDependency, configuration: "shadow")
+
+  // The Spark runner requires the user to provide a Spark dependency. For 
self-contained
+  // runs with the Spark runner, we can provide such a dependency. This is 
deliberately phrased
+  // to not hardcode any runner other than :runners:direct-java
+  if (shouldProvideSpark) {
+gradleRun library.java.spark_streaming
+gradleRun library.java.spark_core, {
+  exclude group:"org.slf4j", module:"jul-to-slf4j"
+}
+
+// TODO: pre-Gradle this was excluded in the SparkRunner overall, in 
Gradle it is
+// only excluded in the testRuntimeClasspath.
+gradleRun project(path: ":beam-sdks-java-core", configuration: "shadow"), {
 
 Review comment:
   You don't need to specify the project, you can just exclude the values 
configuration wide which your doing below.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89963)
Time Spent: 5h  (was: 4h 50m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89962=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89962
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 13:51
Start Date: 11/Apr/18 13:51
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on a change in pull request #5051: 
[BEAM-3995] Add basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180760912
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -43,8 +68,53 @@ dependencies {
   shadow project(path: ":beam-runners-direct-java", configuration: "shadow")
   shadow library.java.slf4j_jdk14
   testCompile library.java.hamcrest_core
+
+  gradleRun project(path: project.path, configuration: "shadow")
+  gradleRun project(path: nexmarkRunnerDependency, configuration: "shadow")
+
+  // The Spark runner requires the user to provide a Spark dependency. For 
self-contained
+  // runs with the Spark runner, we can provide such a dependency. This is 
deliberately phrased
+  // to not hardcode any runner other than :runners:direct-java
+  if (shouldProvideSpark) {
+gradleRun library.java.spark_streaming
+gradleRun library.java.spark_core, {
+  exclude group:"org.slf4j", module:"jul-to-slf4j"
+}
+
+// TODO: pre-Gradle this was excluded in the SparkRunner overall, in 
Gradle it is
+// only excluded in the testRuntimeClasspath.
+gradleRun project(path: ":beam-sdks-java-core", configuration: "shadow"), {
 
 Review comment:
   You don't need to specify the project, you can just exclude the values 
configuration wide.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89962)
Time Spent: 4h 50m  (was: 4h 40m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89838=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89838
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 08:04
Start Date: 11/Apr/18 08:04
Worklog Time Spent: 10m 
  Work Description: echauchot commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380357483
 
 
   I have recently updated the nexmark support matrix on the website after 
running all the test suites. Runners and modes that support 100% of the queries 
are
   - on direct runner in batch mode
   - on direct runner in streaming mode
   - on flink runner in batch mode
   - on flink runner in streaming mode
   - on spark runner in batch mode
   
   => I think we should schedule all these runs
   
   It is a pity that we can not run spark in streaming mode but we still have 
major issues with it in streaming mode:
   - state support 
   - timer support
   - pcollectionView support
   - spark does not comply to the user given timeout in `waitUntilFinish` 
probably because it forks a lot of micro-batches and, to gracefull stop the 
pipeline, the underlining spark engine needs to wait for all of them leading to 
almost never-ending benchmark suites.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89838)
Time Spent: 4h 40m  (was: 4.5h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89837=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89837
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 08:01
Start Date: 11/Apr/18 08:01
Worklog Time Spent: 10m 
  Work Description: echauchot commented on a change in pull request #5051: 
[BEAM-3995] Add basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180658821
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -46,3 +46,14 @@ dependencies {
 test {
   jvmArgs "-da"
 }
+
+def nexmarkArgsProperty = "nexmark.args"
+
+task run (type: JavaExec) {
 
 Review comment:
   yes you just run Main and it runs all the queries that are specified on the 
benchmark suite (e.g. SMOKE)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89837)
Time Spent: 4.5h  (was: 4h 20m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 4.5h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89834=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89834
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 07:57
Start Date: 11/Apr/18 07:57
Worklog Time Spent: 10m 
  Work Description: echauchot commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380356889
 
 
   @kennknowles I had an ongoing PR (https://github.com/apache/beam/pull/4976) 
for doing exactly that on maven. I was about to convert it to gradle, you were 
quicker :)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89834)
Time Spent: 4h 20m  (was: 4h 10m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89829=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89829
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 07:40
Start Date: 11/Apr/18 07:40
Worklog Time Spent: 10m 
  Work Description: echauchot commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380356889
 
 
   @kennknowles I had an ongoing PR for doing exactly that on maven. I was 
about to convert it to gradle, you were quicker :)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89829)
Time Spent: 4h 10m  (was: 4h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 4h 10m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89826=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89826
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 07:37
Start Date: 11/Apr/18 07:37
Worklog Time Spent: 10m 
  Work Description: echauchot commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380357483
 
 
   I have recently updated the nexmark support matrix on the website. Runners 
and modes that support 100% of the queries are
   - on direct runner in batch mode
   - on direct runner in streaming mode
   - on flink runner in batch mode
   - on flink runner in streaming mode
   - on spark runner in batch mode
   I think we should schedule all these runs


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89826)
Time Spent: 4h  (was: 3h 50m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89825=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89825
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 07:35
Start Date: 11/Apr/18 07:35
Worklog Time Spent: 10m 
  Work Description: echauchot commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380356889
 
 
   @kennknowles I had an ongoing PR for doing exactly that on maven. I was 
about to convert it to gradle.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89825)
Time Spent: 3h 50m  (was: 3h 40m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89824=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89824
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 07:32
Start Date: 11/Apr/18 07:32
Worklog Time Spent: 10m 
  Work Description: echauchot commented on a change in pull request #5051: 
[BEAM-3995] Add basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180658821
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -46,3 +46,14 @@ dependencies {
 test {
   jvmArgs "-da"
 }
+
+def nexmarkArgsProperty = "nexmark.args"
+
+task run (type: JavaExec) {
 
 Review comment:
   yes you just run Main and it runs all the queries that are specified on the 
benchmark profile (e.g. SMOKE)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89824)
Time Spent: 3h 40m  (was: 3.5h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-11 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89823=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89823
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 07:31
Start Date: 11/Apr/18 07:31
Worklog Time Spent: 10m 
  Work Description: echauchot commented on a change in pull request #5051: 
[BEAM-3995] Add basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180658821
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -46,3 +46,14 @@ dependencies {
 test {
   jvmArgs "-da"
 }
+
+def nexmarkArgsProperty = "nexmark.args"
+
+task run (type: JavaExec) {
 
 Review comment:
   yes you just run Main


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89823)
Time Spent: 3.5h  (was: 3h 20m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-10 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89750=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89750
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 11/Apr/18 02:11
Start Date: 11/Apr/18 02:11
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380303801
 
 
   Aligning the naming for projects was needed to generate the correct 
dependency names between projects.
   
   Note that you can always use `-p path/to/project taskWithinProject` to 
execute using directory names instead of `:beam-project-name:taskWithinProject`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89750)
Time Spent: 3h 20m  (was: 3h 10m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-10 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89654=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89654
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 10/Apr/18 22:23
Start Date: 10/Apr/18 22:23
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380266575
 
 
   New commands (will port web page after merge)
   
   DirectRunner SMOKE (default):
   ```
   ./gradlew :beam-sdks-java-nexmark:run -Pnexmark.args="
   --runner=DirectRunner
   --suite=SMOKE
   --streaming=false
   --manageResources=false
   --monitorJobs=true
   --enforceEncodability=true
   --enforceImmutability=true"
   ```
   
   DirectRunner SMOKE (explicit):
   ```
   ./gradlew :beam-sdks-java-nexmark:run \
   -Pnexmark.runner=":beam-runners-direct-java" \
   -Pnexmark.args="
   --runner=DirectRunner
   --suite=SMOKE
   --streaming=false
   --manageResources=false
   --monitorJobs=true
   --enforceEncodability=true
   --enforceImmutability=true"
   ```
   
   Spark (special provided dep demonstration):
   ```
   ./gradlew :beam-sdks-java-nexmark:run \
   -Pnexmark.runner=":beam-runners-spark" \
   -Pnexmark.provideSpark=true \
   -Pnexmark.args="
   --runner=SparkRunner
   --suite=SMOKE
   --streamTimeout=60
   --streaming=false
   --manageResources=false
   --monitorJobs=true"
   ```
   
   Flink:
   ```
   ./gradlew :beam-sdks-java-nexmark:run \
   -Pnexmark.runner=":beam-runners-flink_2.11" \
   -Pnexmark.args="
   --runner=FlinkRunner
   --suite=SMOKE
   --streamTimeout=60
   --streaming=false
   --manageResources=false
   --monitorJobs=true
   --flinkMaster=local"
   ```
   
   CC @akedin @apilloud 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89654)
Time Spent: 3h 10m  (was: 3h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-10 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89652=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89652
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 10/Apr/18 22:10
Start Date: 10/Apr/18 22:10
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380263705
 
 
   OK I've regressed this PR to match.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89652)
Time Spent: 3h  (was: 2h 50m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-10 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89636=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89636
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 10/Apr/18 21:42
Start Date: 10/Apr/18 21:42
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380256699
 
 
   If the renames are mandatory for some obscure reason, I'll get to it in a 
bit. Otherwise we should revert and make the project structure simple and have 
explicit artifact ids.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89636)
Time Spent: 2h 50m  (was: 2h 40m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-10 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89635=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89635
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 10/Apr/18 21:36
Start Date: 10/Apr/18 21:36
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380255022
 
 
   It looks like the automatic correspondence between directories and projects 
was regressed? Is this a maven compatibility thing?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89635)
Time Spent: 2h 40m  (was: 2.5h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-10 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89634=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89634
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 10/Apr/18 21:34
Start Date: 10/Apr/18 21:34
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-380254396
 
 
   Confirmed via:
   
   DirectRunner SMOKE:
   ```
   // Defaults to direct runner
   ./gradlew :sdks:java:nexmark:run -Pnexmark.args="--runner=DirectRunner 
--suite=SMOKE --streaming=false --manageResources=false --monitorJobs=true 
--enforceEncodability=true --enforceImmutability=true"
   
   ./gradlew :sdks:java:nexmark:run -Pnexmark.runner=":runners:direct-java" 
-Pnexmark.args="--runner=DirectRunner --suite=SMOKE --streaming=false 
--manageResources=false --monitorJobs=true --enforceEncodability=true 
--enforceImmutability=true"
   ```
   
   FlinkRunner SMOKE:
   ```
   ./gradlew :sdks:java:nexmark:run -Pnexmark.runner=":runners:flink" 
-Pnexmark.args="--runner=FlinkRunner --suite=SMOKE --streamTimeout=60 
--streaming=false --manageResources=false --monitorJobs=true  
--flinkMaster=local"
   ```
   
   SparkRunner SMOKE:
   ```
   ./gradlew :sdks:java:nexmark:run -Pnexmark.runner=":runners:spark" 
-Pnexmark.provideSpark=true -Pnexmark.args="--runner=SparkRunner --suite=SMOKE 
--streamTimeout=60 --streaming=false --manageResources=false --monitorJobs=true"
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89634)
Time Spent: 2.5h  (was: 2h 20m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-10 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89631=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89631
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 10/Apr/18 21:32
Start Date: 10/Apr/18 21:32
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on a change in pull request 
#5051: [BEAM-3995] Add basic DirectRunner Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180561808
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -46,3 +46,14 @@ dependencies {
 test {
   jvmArgs "-da"
 }
+
+def nexmarkArgsProperty = "nexmark.args"
+
+task run (type: JavaExec) {
 
 Review comment:
   Nope, it is just one executable, configured by command line.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89631)
Time Spent: 2h 10m  (was: 2h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-10 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89632=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89632
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 10/Apr/18 21:32
Start Date: 10/Apr/18 21:32
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on a change in pull request 
#5051: [BEAM-3995] Add basic DirectRunner Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180567338
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -46,3 +46,14 @@ dependencies {
 test {
   jvmArgs "-da"
 }
+
+def nexmarkArgsProperty = "nexmark.args"
+
+task run (type: JavaExec) {
+  main = "org.apache.beam.sdk.nexmark.Main"
+  classpath = sourceSets.main.runtimeClasspath
 
 Review comment:
   Nice idea. Done.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89632)
Time Spent: 2h 20m  (was: 2h 10m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-09 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89113=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89113
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 09/Apr/18 20:31
Start Date: 09/Apr/18 20:31
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on a change in pull request #5051: 
[BEAM-3995] Add basic DirectRunner Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180218636
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -46,3 +46,14 @@ dependencies {
 test {
   jvmArgs "-da"
 }
+
+def nexmarkArgsProperty = "nexmark.args"
+
+task run (type: JavaExec) {
+  main = "org.apache.beam.sdk.nexmark.Main"
+  classpath = sourceSets.main.runtimeClasspath
 
 Review comment:
   The reason why I suggested to run using the shaded artifacts is that it 
would match what users would be running if they depended on the Apache Beam 
nexmark artifact distributed via Maven central.
   
   Also, if you want to integrate nexmark into the perf/integration testing it 
would seem like you would want to run it using gradle and not have to build a 
shell script to first build the bundled jar and then execute it using java.
   
   Having a separate configuration per runner makes sense if a runner needs 
additional overrides (typical of spark) to get it to work with Nexmark because 
of a dependency conflict.
   
   Configuration per runner with dynamic classifier?
   ```
   configuration {
 nexmarkRun
   }
   
   dependencies {
 ...
 nexmarkRun project(name: project.path, configuration: 'shadow')
 nexmarkRun project(name: runnerToUse, configuration: 'shadow')
   }
   
   task run(type: JavaExec) {
 ...
 classpath = configurations.nexmarkRun
   }
   ```
   Run via `./gradlew -p sdks/java/nexmark run 
-PrunnerToUse=':beam-runners-direct-java'`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89113)
Time Spent: 2h  (was: 1h 50m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-09 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89112=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89112
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 09/Apr/18 20:26
Start Date: 09/Apr/18 20:26
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on a change in pull request #5051: 
[BEAM-3995] Add basic DirectRunner Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180219450
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -46,3 +46,14 @@ dependencies {
 test {
   jvmArgs "-da"
 }
+
+def nexmarkArgsProperty = "nexmark.args"
+
+task run (type: JavaExec) {
 
 Review comment:
   I was under the impression that each query was its own executable.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89112)
Time Spent: 1h 50m  (was: 1h 40m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-09 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89108=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89108
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 09/Apr/18 20:23
Start Date: 09/Apr/18 20:23
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on a change in pull request #5051: 
[BEAM-3995] Add basic DirectRunner Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180218636
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -46,3 +46,14 @@ dependencies {
 test {
   jvmArgs "-da"
 }
+
+def nexmarkArgsProperty = "nexmark.args"
+
+task run (type: JavaExec) {
+  main = "org.apache.beam.sdk.nexmark.Main"
+  classpath = sourceSets.main.runtimeClasspath
 
 Review comment:
   The reason why I suggested to run using the shaded artifacts is that it 
would match what users would be running if they depended on the Apache Beam 
nexmark artifact distributed via Maven central.
   
   Having a separate configuration per runner makes sense if a runner needs 
additional overrides (typical of spark) to get it to work with Nexmark because 
of a dependency conflict.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89108)
Time Spent: 1h 40m  (was: 1.5h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-09 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89097=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89097
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 09/Apr/18 20:10
Start Date: 09/Apr/18 20:10
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on a change in pull request 
#5051: [BEAM-3995] Add basic DirectRunner Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180214420
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -46,3 +46,14 @@ dependencies {
 test {
   jvmArgs "-da"
 }
+
+def nexmarkArgsProperty = "nexmark.args"
+
+task run (type: JavaExec) {
+  main = "org.apache.beam.sdk.nexmark.Main"
+  classpath = sourceSets.main.runtimeClasspath
 
 Review comment:
   Nexmark is not a test suite but a main application. Not sure if that changes 
your comment.
   
   I wanted to build a bundled artifact that included all transitive deps and 
was standalone. Needs one per runner, which is unfortunate but a natural 
consequence. The shaded configuration currently fails to include things like 
joda time when I switch it to `compile` so I abandoned the effort. I will 
resume it, actually, as I do like being able to run this way. Configuration per 
runner with dynamic classifier?
   
   Separately, what I am doing here is just using JavaExec to launch the main 
class. It isn't a library but an application so shading should be a noop when 
executed via JavaExec, is that not so? So I figure the non-bundled run should 
be entirely unshaded to save needless build.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89097)
Time Spent: 1h 20m  (was: 1h 10m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-09 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=89098=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89098
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 09/Apr/18 20:10
Start Date: 09/Apr/18 20:10
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on a change in pull request 
#5051: [BEAM-3995] Add basic DirectRunner Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180213366
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -46,3 +46,14 @@ dependencies {
 test {
   jvmArgs "-da"
 }
+
+def nexmarkArgsProperty = "nexmark.args"
+
+task run (type: JavaExec) {
 
 Review comment:
   Well, the reason I chose it is because `run` because Nexmark is an 
application with one specific entry point. It is the same task that that 
`application` plugin uses, and we are doing the same thing here. I decided that 
plugin might not be the right fit, but if you feel differently I'll try it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 89098)
Time Spent: 1.5h  (was: 1h 20m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-09 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=88919=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88919
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 09/Apr/18 13:11
Start Date: 09/Apr/18 13:11
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #5051: [BEAM-3995] Add 
basic DirectRunner Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-379746410
 
 
   You can make the exec dynamic by making a new configuration which takes a 
project property that chooses which runner to include with the default being 
the direct runner.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 88919)
Time Spent: 1h 10m  (was: 1h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-09 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=88916=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88916
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 09/Apr/18 13:09
Start Date: 09/Apr/18 13:09
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on a change in pull request #5051: 
[BEAM-3995] Add basic DirectRunner Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180087769
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -46,3 +46,14 @@ dependencies {
 test {
   jvmArgs "-da"
 }
+
+def nexmarkArgsProperty = "nexmark.args"
+
+task run (type: JavaExec) {
+  main = "org.apache.beam.sdk.nexmark.Main"
+  classpath = sourceSets.main.runtimeClasspath
 
 Review comment:
   This will run on the preshaded artifacts.
   
   Create another configuration similar to the Flink validates runner which 
contains nexmark `shadow` configuration.
   
https://github.com/apache/beam/blob/c2d674448611d0490804533b96c41ba05e071572/runners/flink/build.gradle#L81


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 88916)
Time Spent: 1h  (was: 50m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-09 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=88917=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88917
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 09/Apr/18 13:09
Start Date: 09/Apr/18 13:09
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on a change in pull request #5051: 
[BEAM-3995] Add basic DirectRunner Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#discussion_r180088173
 
 

 ##
 File path: sdks/java/nexmark/build.gradle
 ##
 @@ -46,3 +46,14 @@ dependencies {
 test {
   jvmArgs "-da"
 }
+
+def nexmarkArgsProperty = "nexmark.args"
+
+task run (type: JavaExec) {
 
 Review comment:
   Is there a more meaningful name you could provide instead of `run`?
   
   nit: `run (` -> `run(` to follow the same style used in all the other build 
files and Java


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 88917)
Time Spent: 1h  (was: 50m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-06 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=88649=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88649
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 06/Apr/18 23:48
Start Date: 06/Apr/18 23:48
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5051: [BEAM-3995] Add 
basic DirectRunner Gradle run task to Nexmark
URL: https://github.com/apache/beam/pull/5051#issuecomment-379413203
 
 
   I've gone through a few iterations. First, hardcoded some configs. Second, 
building a fat jar - I still like it best but something about our shadow config 
is already screwed up ;_;. Last, using JavaExec to do like Maven `exec:java`. 
You have to plumb the args as a string then split but otherwise the same. Not 
as bad as I was worried about.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 88649)
Time Spent: 50m  (was: 40m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-04 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=87813=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87813
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 04/Apr/18 23:18
Start Date: 04/Apr/18 23:18
Worklog Time Spent: 10m 
  Work Description: kennknowles closed pull request #5026: [BEAM-3995] 
Build self-contained Nexmark jar
URL: https://github.com/apache/beam/pull/5026
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/java/nexmark/build.gradle b/sdks/java/nexmark/build.gradle
index 078365df375..b2f8c3e4552 100644
--- a/sdks/java/nexmark/build.gradle
+++ b/sdks/java/nexmark/build.gradle
@@ -23,26 +23,32 @@ description = "Apache Beam :: SDKs :: Java :: Nexmark"
 
 dependencies {
   compile library.java.guava
-  shadow project(path: ":sdks:java:core", configuration: "shadow")
-  shadow project(path: ":sdks:java:io:google-cloud-platform", configuration: 
"shadow")
-  shadow project(path: ":sdks:java:extensions:google-cloud-platform-core", 
configuration: "shadow")
-  shadow project(path: ":sdks:java:extensions:sql", configuration: "shadow")
-  shadow library.java.google_api_services_bigquery
-  shadow library.java.jackson_core
-  shadow library.java.jackson_annotations
-  shadow library.java.jackson_databind
-  shadow library.java.avro
-  shadow library.java.joda_time
-  shadow library.java.slf4j_api
-  shadow library.java.findbugs_jsr305
-  shadow library.java.junit
-  shadow library.java.hamcrest_core
-  shadow library.java.commons_lang3
-  shadow project(path: ":runners:direct-java", configuration: "shadow")
-  shadow library.java.slf4j_jdk14
+  compile project(path: ":sdks:java:core")
+  compile project(path: ":sdks:java:io:google-cloud-platform")
+  compile project(path: ":sdks:java:extensions:google-cloud-platform-core")
+  compile project(path: ":sdks:java:extensions:sql")
+  compile library.java.google_api_services_bigquery
+  compile library.java.jackson_core
+  compile library.java.jackson_annotations
+  compile library.java.jackson_databind
+  compile library.java.avro
+  compile library.java.joda_time
+  compile library.java.slf4j_api
+  compile library.java.findbugs_jsr305
+  compile library.java.junit
+  compile library.java.hamcrest_core
+  compile library.java.commons_lang3
+  compile project(path: ":runners:direct-java")
+  runtime library.java.slf4j_jdk14
   testCompile library.java.hamcrest_core
 }
 
 test {
   jvmArgs "-da"
 }
+
+jar {
+  manifest {
+attributes 'Main-Class': 'org.apache.beam.sdk.nexmark.Main'
+  }
+}


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 87813)
Time Spent: 40m  (was: 0.5h)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-04 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=87812=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87812
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 04/Apr/18 23:18
Start Date: 04/Apr/18 23:18
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5026: [BEAM-3995] Build 
self-contained Nexmark jar
URL: https://github.com/apache/beam/pull/5026#issuecomment-378774775
 
 
   Ah, I think this is really not going to be workable. String munging to plumb 
cmdline args may still be what we need to do, in 2018.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 87812)
Time Spent: 0.5h  (was: 20m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-04 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=87810=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87810
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 04/Apr/18 23:14
Start Date: 04/Apr/18 23:14
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #5026: [BEAM-3995] Build 
self-contained Nexmark jar
URL: https://github.com/apache/beam/pull/5026#issuecomment-378774094
 
 
   It is a big hammer; too big. We'll actually need to have a way to get the 
different runners on the classpath without bundling. But Gradle has nothing 
that I would care to use equivalent to `mvn exec:java`. Still poking around but 
the solutions I've found are not reasonable.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 87810)
Time Spent: 20m  (was: 10m)

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-3995) Launch Nexmark suites from gradle and update web page docs

2018-04-04 Thread ASF GitHub Bot (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-3995?focusedWorklogId=87805=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87805
 ]

ASF GitHub Bot logged work on BEAM-3995:


Author: ASF GitHub Bot
Created on: 04/Apr/18 23:12
Start Date: 04/Apr/18 23:12
Worklog Time Spent: 10m 
  Work Description: kennknowles opened a new pull request #5026: 
[BEAM-3995] Build self-contained Nexmark jar
URL: https://github.com/apache/beam/pull/5026
 
 
   To run the Nexmark benchmarks, we will separate the build from the run step. 
The build step is owned by Gradle, while the run step is simply running a 
program. Nexmark is already designed this way.
   
   WIP: the expected full set of dependencies is _not_ contained in the shadow 
jar. I don't know why this is yet, but expect it has to do with how we manage 
the base setup. TBD.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [ ] Make sure there is a [JIRA 
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the 
change (usually before you start working on it).  Trivial changes like typos do 
not require a JIRA issue.  Your pull request should address just this issue, 
without pulling in other changes.
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue.
- [ ] Write a pull request description that is detailed enough to 
understand:
  - [ ] What the pull request does
  - [ ] Why it does it
  - [ ] How it does it
  - [ ] Why this approach
- [ ] Each commit in the pull request should have a meaningful subject line 
and body.
- [ ] Run `mvn clean verify` to make sure basic checks pass. A more 
thorough check will be performed on your pull request automatically.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 87805)
Time Spent: 10m
Remaining Estimate: 0h

> Launch Nexmark suites from gradle and update web page docs
> --
>
> Key: BEAM-3995
> URL: https://issues.apache.org/jira/browse/BEAM-3995
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-nexmark, website
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently our instructions for running Nexmark benchmarks on various runners 
> is pretty tightly tied to Maven. We need a good story for running them with 
> gradle (or just building an executable with gradle and running that 
> standalone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)