[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131941=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131941
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 07/Aug/18 16:46
Start Date: 07/Aug/18 16:46
Worklog Time Spent: 10m 
  Work Description: apilloud closed pull request #6149: [BEAM-4761] Pin 
Dataflow Nexmark Autoscale to 4
URL: https://github.com/apache/beam/pull/6149
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy 
b/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
index 5ff02f91a35..bb9d8d8817c 100644
--- a/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
+++ b/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
@@ -39,6 +39,9 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--numWorkers=4',
+  '--maxNumWorkers=4',
+  '--autoscalingAlgorithm=NONE',
   '--nexmarkParallel=16',
   '--streaming=false',
   '--suite=STRESS',
@@ -56,6 +59,9 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--numWorkers=4',
+  '--maxNumWorkers=4',
+  '--autoscalingAlgorithm=NONE',
   '--nexmarkParallel=16',
   '--streaming=true',
   '--suite=STRESS',
@@ -73,6 +79,9 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--numWorkers=4',
+  '--maxNumWorkers=4',
+  '--autoscalingAlgorithm=NONE',
   '--nexmarkParallel=16',
   '--queryLanguage=sql',
   '--streaming=false',
@@ -91,6 +100,9 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--numWorkers=4',
+  '--maxNumWorkers=4',
+  '--autoscalingAlgorithm=NONE',
   '--nexmarkParallel=16',
   '--queryLanguage=sql',
   '--streaming=true',
diff --git 
a/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/NexmarkSuite.java 
b/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/NexmarkSuite.java
index 7a6d08a1b3e..48423dc3f35 100644
--- 
a/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/NexmarkSuite.java
+++ 
b/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/NexmarkSuite.java
@@ -69,7 +69,7 @@
 List configurations = smoke();
 for (NexmarkConfiguration configuration : configurations) {
   if (configuration.numEvents >= 0) {
-configuration.numEvents *= 1000;
+configuration.numEvents *= 100;
   }
 }
 return configurations;
@@ -79,7 +79,7 @@
 List configurations = smoke();
 for (NexmarkConfiguration configuration : configurations) {
   if (configuration.numEvents >= 0) {
-configuration.numEvents *= 1000;
+configuration.numEvents *= 1;
   }
 }
 return configurations;


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131941)
Time Spent: 10h 50m  (was: 10h 40m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 10h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131930=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131930
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 07/Aug/18 16:23
Start Date: 07/Aug/18 16:23
Worklog Time Spent: 10m 
  Work Description: akedin commented on issue #6149: [BEAM-4761] Pin 
Dataflow Nexmark Autoscale to 4
URL: https://github.com/apache/beam/pull/6149#issuecomment-46860
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131930)
Time Spent: 10h 40m  (was: 10.5h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 10h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131927=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131927
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 07/Aug/18 16:22
Start Date: 07/Aug/18 16:22
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #6149: [BEAM-4761] Pin 
Dataflow Nexmark Autoscale to 4
URL: https://github.com/apache/beam/pull/6149#issuecomment-46667
 
 
   run java precommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131927)
Time Spent: 10.5h  (was: 10h 20m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 10.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131921=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131921
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 07/Aug/18 16:06
Start Date: 07/Aug/18 16:06
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #6149: [BEAM-4761] Pin 
Dataflow Nexmark Autoscale to 4
URL: https://github.com/apache/beam/pull/6149#issuecomment-41185
 
 
   R: @akedin This disables the dataflow autoscaler (same flags we pass on the 
internal nexmark tests) and adjusts the event count so this will complete in a 
reasonable time.
   cc: @echauchot 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131921)
Time Spent: 10h 20m  (was: 10h 10m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 10h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131660=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131660
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 06/Aug/18 23:24
Start Date: 06/Aug/18 23:24
Worklog Time Spent: 10m 
  Work Description: asfgit closed pull request #525: [BEAM-4761] Add 
Dataflow Runner
URL: https://github.com/apache/beam-site/pull/525
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/src/documentation/sdks/nexmark.md 
b/src/documentation/sdks/nexmark.md
index bf96803f29..6dac601f68 100644
--- a/src/documentation/sdks/nexmark.md
+++ b/src/documentation/sdks/nexmark.md
@@ -689,6 +689,8 @@ Each dashboard contains:
 
 [Nexmark performance spark 
runner](https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712)
 
+[Nexmark performance dataflow 
runner](https://apache-beam-testing.appspot.com/explore?dashboard=5670405876482048)
+
 
 ### Output size dashboards links
 
@@ -698,3 +700,5 @@ Each dashboard contains:
 
 [Nexmark output size spark 
runner](https://apache-beam-testing.appspot.com/explore?dashboard=5163657986048000)
 
+[Nexmark output size dataflow 
runner](https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856)
+


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131660)
Time Spent: 10h 10m  (was: 10h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 10h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131658=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131658
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 06/Aug/18 23:20
Start Date: 06/Aug/18 23:20
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #525: [BEAM-4761] Add 
Dataflow Runner
URL: https://github.com/apache/beam-site/pull/525#issuecomment-410883031
 
 
   @asfgit merge


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131658)
Time Spent: 10h  (was: 9h 50m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 10h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131657=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131657
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 06/Aug/18 23:19
Start Date: 06/Aug/18 23:19
Worklog Time Spent: 10m 
  Work Description: akedin commented on issue #525: [BEAM-4761] Add 
Dataflow Runner
URL: https://github.com/apache/beam-site/pull/525#issuecomment-410882899
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131657)
Time Spent: 9h 50m  (was: 9h 40m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 9h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131651=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131651
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 06/Aug/18 23:11
Start Date: 06/Aug/18 23:11
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #525: [BEAM-4761] Add 
Dataflow Runner
URL: https://github.com/apache/beam-site/pull/525#issuecomment-410881372
 
 
   R: @akedin 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131651)
Time Spent: 9h 40m  (was: 9.5h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 9h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131650=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131650
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 06/Aug/18 23:09
Start Date: 06/Aug/18 23:09
Worklog Time Spent: 10m 
  Work Description: apilloud opened a new pull request #525: [BEAM-4761] 
Add Dataflow Runner
URL: https://github.com/apache/beam-site/pull/525
 
 
   This adds the dataflow runner to the perf links.
   
   Once your pull request has been opened and assigned a number, please edit the
   URL below, replacing `PULL_REQUEST_NUMBER` with the number of your pull 
request.
   
   
http://apache-beam-website-pull-requests.storage.googleapis.com/PULL_REQUEST_NUMBER/index.html
   
   Finally, it will help us expedite review of your Pull Request if you tag
   someone (e.g. @username) to look at it.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131650)
Time Spent: 9.5h  (was: 9h 20m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 9.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131571=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131571
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 06/Aug/18 20:30
Start Date: 06/Aug/18 20:30
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #6149: [BEAM-4761] Pin 
Dataflow Nexmark Autoscale to 4
URL: https://github.com/apache/beam/pull/6149#issuecomment-410843226
 
 
   
https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Dataflow/101/


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131571)
Time Spent: 9h 20m  (was: 9h 10m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 9h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131524=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131524
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 06/Aug/18 19:14
Start Date: 06/Aug/18 19:14
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #6149: [BEAM-4761] Pin 
Dataflow Nexmark Autoscale to 4
URL: https://github.com/apache/beam/pull/6149#issuecomment-410821415
 
 
   run seed job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131524)
Time Spent: 9h 10m  (was: 9h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 9h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131507=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131507
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 06/Aug/18 18:12
Start Date: 06/Aug/18 18:12
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #6149: [BEAM-4761] Pin 
Dataflow Nexmark Autoscale to 4
URL: https://github.com/apache/beam/pull/6149#issuecomment-410800901
 
 
   run seed job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131507)
Time Spent: 9h  (was: 8h 50m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 9h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131506=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131506
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 06/Aug/18 18:12
Start Date: 06/Aug/18 18:12
Worklog Time Spent: 10m 
  Work Description: apilloud opened a new pull request #6149: [BEAM-4761] 
Pin Dataflow Nexmark Autoscale to 4
URL: https://github.com/apache/beam/pull/6149
 
 
   Perf tests and the Dataflow Autoscaler don't work well together, pin to 4 
workers.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [x] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131506)
Time Spent: 8h 50m  (was: 8h 40m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 8h 50m
>  Remaining Estimate: 0h
>

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131148=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131148
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 04/Aug/18 00:06
Start Date: 04/Aug/18 00:06
Worklog Time Spent: 10m 
  Work Description: akedin closed pull request #6140: [BEAM-4761] Handle 
skipped tests in parallel Nexmark
URL: https://github.com/apache/beam/pull/6140
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java 
b/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
index ecd7b5b9bf6..5aa68be7ea3 100644
--- a/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
+++ b/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
@@ -147,7 +147,9 @@ void runAll(String[] args) throws IOException {
 
 NexmarkConfiguration configuration = result.configuration;
 NexmarkPerf perf = result.perf;
-if (perf.errors == null || perf.errors.size() > 0) {
+if (perf == null) {
+  continue;
+} else if (perf.errors == null || perf.errors.size() > 0) {
   successful = false;
 }
 appendPerf(options.getPerfFilename(), configuration, perf);


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131148)
Time Spent: 8h 40m  (was: 8.5h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 8h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131144=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131144
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:52
Start Date: 03/Aug/18 23:52
Worklog Time Spent: 10m 
  Work Description: akedin commented on issue #6140: [BEAM-4761] Handle 
skipped tests in parallel Nexmark
URL: https://github.com/apache/beam/pull/6140#issuecomment-410403739
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131144)
Time Spent: 8.5h  (was: 8h 20m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 8.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131140=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131140
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:33
Start Date: 03/Aug/18 23:33
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #6140: [BEAM-4761] Handle 
skipped tests in parallel Nexmark
URL: https://github.com/apache/beam/pull/6140#issuecomment-410401427
 
 
   @akedin I broke nexmark on SQL by dropping the check for null results (which 
indicates a skipped test). This fixes it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131140)
Time Spent: 8h 20m  (was: 8h 10m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 8h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131138=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131138
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:31
Start Date: 03/Aug/18 23:31
Worklog Time Spent: 10m 
  Work Description: apilloud opened a new pull request #6140: [BEAM-4761] 
Handle skipped tests in parallel Nexmark
URL: https://github.com/apache/beam/pull/6140
 
 
   I dropped this check in refactoring, it breaks Nexmark on SQL, which skips 
some tests.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [x] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131138)
Time Spent: 8h 10m  (was: 8h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 8h 10m
>  Remaining Estimate: 

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131104=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131104
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 22:08
Start Date: 03/Aug/18 22:08
Worklog Time Spent: 10m 
  Work Description: apilloud closed pull request #6127: [BEAM-4761] Run 
Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy 
b/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
index 3abf2b16c6f..5ff02f91a35 100644
--- a/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
+++ b/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
@@ -39,6 +39,7 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--nexmarkParallel=16',
   '--streaming=false',
   '--suite=STRESS',
   '--manageResources=false',
@@ -55,6 +56,7 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--nexmarkParallel=16',
   '--streaming=true',
   '--suite=STRESS',
   '--manageResources=false',
@@ -71,6 +73,7 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--nexmarkParallel=16',
   '--queryLanguage=sql',
   '--streaming=false',
   '--suite=STRESS',
@@ -88,6 +91,7 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--nexmarkParallel=16',
   '--queryLanguage=sql',
   '--streaming=true',
   '--suite=STRESS',
diff --git 
a/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java 
b/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
index 537ddd22985..ecd7b5b9bf6 100644
--- a/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
+++ b/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
@@ -33,6 +33,13 @@
 import java.util.LinkedHashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletionService;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ExecutorCompletionService;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
 import javax.annotation.Nullable;
 import org.apache.beam.sdk.Pipeline;
 import org.apache.beam.sdk.coders.CoderException;
@@ -74,30 +81,81 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Result {
+private final NexmarkConfiguration configuration;
+private final NexmarkPerf perf;
+
+private Result(NexmarkConfiguration configuration, NexmarkPerf perf) {
+  this.configuration = configuration;
+  this.perf = perf;
+}
+  }
+
+  private static class Run implements Callable {
+private final NexmarkLauncher nexmarkLauncher;
+private final NexmarkConfiguration configuration;
+
+private Run(String[] args, NexmarkConfiguration configuration) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+}
+
+@Override
+public Result call() throws IOException {
+  NexmarkPerf perf = nexmarkLauncher.run(configuration);
+  return new Result(configuration, perf);
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
 Map baseline = 

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131083=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131083
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 21:38
Start Date: 03/Aug/18 21:38
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #6127: [BEAM-4761] Run 
Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#issuecomment-410382965
 
 
   Run Java PreCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131083)
Time Spent: 7h 50m  (was: 7h 40m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 7h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131074=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131074
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 21:13
Start Date: 03/Aug/18 21:13
Worklog Time Spent: 10m 
  Work Description: apilloud commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207669154
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +82,83 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Result {
+private final NexmarkConfiguration configuration;
+private final NexmarkPerf perf;
+
+private Result(NexmarkConfiguration configuration, NexmarkPerf perf) {
+  this.configuration = configuration;
+  this.perf = perf;
+}
+  }
+
+  private static class Run implements Callable {
+private final NexmarkLauncher nexmarkLauncher;
+private final NexmarkConfiguration configuration;
+
+private Run(String[] args, NexmarkConfiguration configuration) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+}
+
+@Override
+public Result call() throws IOException {
+  NexmarkPerf perf = nexmarkLauncher.run(configuration);
+  return new Result(configuration, perf);
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
 Map baseline = 
loadBaseline(options.getBaselineFilename());
 Map actual = new LinkedHashMap<>();
-Iterable configurations = 
options.getSuite().getConfigurations(options);
+Set configurations = 
options.getSuite().getConfigurations(options);
+int nThreads = Math.min(options.getNexmarkParallel(), 
configurations.size());
+ExecutorService executor = Executors.newFixedThreadPool(nThreads);
+CompletionService completion = new 
ExecutorCompletionService(executor);
 
 boolean successful = true;
 try {
-  // Run all the configurations.
+  // Schedule all the configurations.
+  int scheduled = 0;
   for (NexmarkConfiguration configuration : configurations) {
-NexmarkPerf perf = nexmarkLauncher.run(configuration);
-if (perf != null) {
-  if (perf.errors == null || perf.errors.size() > 0) {
-successful = false;
+completion.submit(new Run(args, configuration));
+scheduled++;
 
 Review comment:
   deleted.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131074)
Time Spent: 7.5h  (was: 7h 20m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 7.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131075=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131075
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 21:13
Start Date: 03/Aug/18 21:13
Worklog Time Spent: 10m 
  Work Description: apilloud commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207669171
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -107,6 +168,14 @@ void runAll(OptionT options, NexmarkLauncher 
nexmarkLauncher) throws IOException
 saveSummary(options.getSummaryFilename(), configurations, actual, 
baseline, start, options);
 saveJavascript(options.getJavascriptFilename(), configurations, 
actual, baseline, start);
   }
+
+  executor.shutdown();
+  try {
+if (!executor.awaitTermination(1L, TimeUnit.SECONDS)) {
+  throw new RuntimeException("Executor did not shutdown");
 
 Review comment:
   deleted.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131075)
Time Spent: 7h 40m  (was: 7.5h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 7h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131072=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131072
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 21:01
Start Date: 03/Aug/18 21:01
Worklog Time Spent: 10m 
  Work Description: akedin commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207664753
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +82,83 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Result {
+private final NexmarkConfiguration configuration;
+private final NexmarkPerf perf;
+
+private Result(NexmarkConfiguration configuration, NexmarkPerf perf) {
+  this.configuration = configuration;
+  this.perf = perf;
+}
+  }
+
+  private static class Run implements Callable {
+private final NexmarkLauncher nexmarkLauncher;
+private final NexmarkConfiguration configuration;
+
+private Run(String[] args, NexmarkConfiguration configuration) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+}
+
+@Override
+public Result call() throws IOException {
+  NexmarkPerf perf = nexmarkLauncher.run(configuration);
+  return new Result(configuration, perf);
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
 Map baseline = 
loadBaseline(options.getBaselineFilename());
 Map actual = new LinkedHashMap<>();
-Iterable configurations = 
options.getSuite().getConfigurations(options);
+Set configurations = 
options.getSuite().getConfigurations(options);
+int nThreads = Math.min(options.getNexmarkParallel(), 
configurations.size());
+ExecutorService executor = Executors.newFixedThreadPool(nThreads);
+CompletionService completion = new 
ExecutorCompletionService(executor);
 
 boolean successful = true;
 try {
-  // Run all the configurations.
+  // Schedule all the configurations.
+  int scheduled = 0;
   for (NexmarkConfiguration configuration : configurations) {
-NexmarkPerf perf = nexmarkLauncher.run(configuration);
-if (perf != null) {
-  if (perf.errors == null || perf.errors.size() > 0) {
-successful = false;
+completion.submit(new Run(args, configuration));
+scheduled++;
 
 Review comment:
   nit: you don't need to keep `scheduled` counter anymore, you have 
`configurations.size()` now


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131072)
Time Spent: 7h 20m  (was: 7h 10m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 7h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131073=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131073
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 21:01
Start Date: 03/Aug/18 21:01
Worklog Time Spent: 10m 
  Work Description: akedin commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r20723
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -107,6 +168,14 @@ void runAll(OptionT options, NexmarkLauncher 
nexmarkLauncher) throws IOException
 saveSummary(options.getSummaryFilename(), configurations, actual, 
baseline, start, options);
 saveJavascript(options.getJavascriptFilename(), configurations, 
actual, baseline, start);
   }
+
+  executor.shutdown();
+  try {
+if (!executor.awaitTermination(1L, TimeUnit.SECONDS)) {
+  throw new RuntimeException("Executor did not shutdown");
 
 Review comment:
   nit: you're waiting for all runs to complete before reaching this point, and 
right after this you exit the program, so I'm not sure if it's worth waiting 
for executor termination


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131073)
Time Spent: 7h 20m  (was: 7h 10m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 7h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131062=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131062
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 20:39
Start Date: 03/Aug/18 20:39
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #6127: [BEAM-4761] Run 
Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#issuecomment-410370261
 
 
   Run Java PreCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131062)
Time Spent: 7h 10m  (was: 7h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 7h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131052=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131052
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 19:50
Start Date: 03/Aug/18 19:50
Worklog Time Spent: 10m 
  Work Description: apilloud commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207649810
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +80,92 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
 
 Review comment:
   Can't use `ExecutorService` because I don't know what order the tests will 
complete in and want to display results as they come in. `CompletionService` 
solves that and eliminates a bunch of code.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131052)
Time Spent: 7h  (was: 6h 50m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 7h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131026=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131026
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 19:29
Start Date: 03/Aug/18 19:29
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207645431
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +80,92 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
 
 Review comment:
   Actually this is a good idea. By doing so you can also avoid that count, 
because you will try to get result from a list of Future one by one.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131026)
Time Spent: 6h 50m  (was: 6h 40m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131022=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131022
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 19:12
Start Date: 03/Aug/18 19:12
Worklog Time Spent: 10m 
  Work Description: akedin commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207639879
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +80,92 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
+}
+
+@Override
+public void run() {
+  try {
+try {
+  perf = nexmarkLauncher.run(configuration);
+} catch (IOException | RuntimeException e) {
+  exception = e;
+} finally {
+  results.put(this);
+}
+  } catch (InterruptedException ie) {
+return;
+  }
+}
+
+private NexmarkPerf get() throws IOException {
+  if (exception != null) {
+if (exception instanceof IOException) {
+  throw (IOException) exception;
+} else if (exception instanceof RuntimeException) {
+  throw (RuntimeException) exception;
 
 Review comment:
   nit: rethrowing without wrapping is an antipattern


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131022)
Time Spent: 6h 40m  (was: 6.5h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 6h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131021=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131021
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 19:12
Start Date: 03/Aug/18 19:12
Worklog Time Spent: 10m 
  Work Description: akedin commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207641674
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +80,92 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
 
 Review comment:
   Can you avoid using global blocking state by using 
`ExecutorService.submit(Callable)` override? 
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html#submit(java.util.concurrent.Callable)
   
   With `Callable` you are allowed to return the value which you then extract 
from the `Future`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131021)
Time Spent: 6.5h  (was: 6h 20m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131012=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131012
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 18:44
Start Date: 03/Aug/18 18:44
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #6127: [BEAM-4761] Run 
Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#issuecomment-410342598
 
 
   LGTM thanks!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131012)
Time Spent: 6h 20m  (was: 6h 10m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 6h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131010=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131010
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 18:42
Start Date: 03/Aug/18 18:42
Worklog Time Spent: 10m 
  Work Description: apilloud commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207634029
 
 

 ##
 File path: .test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
 ##
 @@ -61,6 +62,7 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--nexmarkParallel=16',
 
 Review comment:
   These are idle threads for most of the test. We have similar tests that run 
with several times as many.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131010)
Time Spent: 6h  (was: 5h 50m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 6h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131011=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131011
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 18:42
Start Date: 03/Aug/18 18:42
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #6127: [BEAM-4761] Run 
Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#issuecomment-410342121
 
 
   @amaliujia Should have fixed everything you found. Please take another look.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131011)
Time Spent: 6h 10m  (was: 6h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131009=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131009
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 18:41
Start Date: 03/Aug/18 18:41
Worklog Time Spent: 10m 
  Work Description: apilloud commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207633742
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +78,97 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
+}
+
+@Override
+public void run() {
+  try {
+try {
+  perf = nexmarkLauncher.run(configuration);
+} catch (IOException | RuntimeException e) {
+  exception = e;
+} finally {
+  results.put(this);
+}
+  } catch (InterruptedException ie) {
+return;
+  }
+}
+
+private NexmarkPerf get() throws IOException {
+  if (exception != null) {
+if (exception instanceof IOException) {
+  throw (IOException) exception;
+} else if (exception instanceof RuntimeException) {
+  throw (RuntimeException) exception;
+}
+// This should be unreachable.
+  }
+  assert perf != null;
+  return perf;
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
+ExecutorService executor = 
Executors.newFixedThreadPool(options.getNexmarkParallel());
 
 Review comment:
   Had to make `configurations` a Set. Done.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131009)
Time Spent: 5h 50m  (was: 5h 40m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131007=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131007
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 18:36
Start Date: 03/Aug/18 18:36
Worklog Time Spent: 10m 
  Work Description: Ardagan commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207632576
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +78,97 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
+}
+
+@Override
+public void run() {
+  try {
+try {
+  perf = nexmarkLauncher.run(configuration);
+} catch (IOException | RuntimeException e) {
+  exception = e;
+} finally {
+  results.put(this);
+}
+  } catch (InterruptedException ie) {
+return;
+  }
+}
+
+private NexmarkPerf get() throws IOException {
+  if (exception != null) {
+if (exception instanceof IOException) {
+  throw (IOException) exception;
+} else if (exception instanceof RuntimeException) {
+  throw (RuntimeException) exception;
+}
+// This should be unreachable.
+  }
+  assert perf != null;
+  return perf;
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
+ExecutorService executor = 
Executors.newFixedThreadPool(options.getNexmarkParallel());
+BlockingQueue results = new LinkedBlockingQueue();
 Map baseline = 
loadBaseline(options.getBaselineFilename());
 Map actual = new LinkedHashMap<>();
 Iterable configurations = 
options.getSuite().getConfigurations(options);
 
 boolean successful = true;
 try {
-  // Run all the configurations.
+  // Schedule all the configurations.
   for (NexmarkConfiguration configuration : configurations) {
-NexmarkPerf perf = nexmarkLauncher.run(configuration);
-if (perf != null) {
-  if (perf.errors == null || perf.errors.size() > 0) {
-successful = false;
+executor.execute(new Run(args, configuration, results));
+if (options.getNexmarkParallel() > 1) {
+  // Stagger start
+  try {
+Thread.sleep(2000);
 
 Review comment:
   This doesn't really address quota issue directly. I believe the problem was 
with running too many post-commit tests in parallel and I've addressed that 
issue in https://github.com/apache/beam/pull/6125.
   
   What we can do instead is add retry in case of quota issue. But again, that 
is not a panacea and does not address issue directly.
   
   My opinion is that we should remove this check as of now.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131007)
Time Spent: 5h 40m  (was: 5.5h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 5h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131004=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131004
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 18:32
Start Date: 03/Aug/18 18:32
Worklog Time Spent: 10m 
  Work Description: Ardagan commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207631195
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +78,97 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
+}
+
+@Override
+public void run() {
+  try {
+try {
+  perf = nexmarkLauncher.run(configuration);
+} catch (IOException | RuntimeException e) {
+  exception = e;
+} finally {
+  results.put(this);
+}
+  } catch (InterruptedException ie) {
+return;
+  }
+}
+
+private NexmarkPerf get() throws IOException {
+  if (exception != null) {
+if (exception instanceof IOException) {
+  throw (IOException) exception;
+} else if (exception instanceof RuntimeException) {
+  throw (RuntimeException) exception;
+}
+// This should be unreachable.
+  }
+  assert perf != null;
+  return perf;
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
+ExecutorService executor = 
Executors.newFixedThreadPool(options.getNexmarkParallel());
+BlockingQueue results = new LinkedBlockingQueue();
 Map baseline = 
loadBaseline(options.getBaselineFilename());
 Map actual = new LinkedHashMap<>();
 Iterable configurations = 
options.getSuite().getConfigurations(options);
 
 boolean successful = true;
 try {
-  // Run all the configurations.
+  // Schedule all the configurations.
   for (NexmarkConfiguration configuration : configurations) {
-NexmarkPerf perf = nexmarkLauncher.run(configuration);
-if (perf != null) {
-  if (perf.errors == null || perf.errors.size() > 0) {
-successful = false;
+executor.execute(new Run(args, configuration, results));
+if (options.getNexmarkParallel() > 1) {
+  // Stagger start
+  try {
+Thread.sleep(2000);
+  } catch (InterruptedException e) {
+return;
   }
-  appendPerf(options.getPerfFilename(), configuration, perf);
-  actual.put(configuration, perf);
-  // Summarize what we've run so far.
-  saveSummary(null, configurations, actual, baseline, start, options);
 }
   }
+
+  // Collect all the results.
+  for (Object i : configurations) {
+Run result;
+try {
+  result = results.take();
 
 Review comment:
   Synced with Andrew. We have test timeout configured in test framework. Right 
now we rely on that one to kill the test if it hangs for too long.
   We can add cleanup on timout logic in another PR.
   I believe this summary can be applied to other timeout-related comments on 
this PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131004)
Time Spent: 5.5h  (was: 5h 20m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: 

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131003=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131003
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 18:32
Start Date: 03/Aug/18 18:32
Worklog Time Spent: 10m 
  Work Description: apilloud commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207631284
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +78,97 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
+}
+
+@Override
+public void run() {
+  try {
+try {
+  perf = nexmarkLauncher.run(configuration);
+} catch (IOException | RuntimeException e) {
+  exception = e;
+} finally {
+  results.put(this);
+}
+  } catch (InterruptedException ie) {
+return;
+  }
+}
+
+private NexmarkPerf get() throws IOException {
+  if (exception != null) {
+if (exception instanceof IOException) {
+  throw (IOException) exception;
+} else if (exception instanceof RuntimeException) {
+  throw (RuntimeException) exception;
+}
+// This should be unreachable.
+  }
+  assert perf != null;
+  return perf;
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
+ExecutorService executor = 
Executors.newFixedThreadPool(options.getNexmarkParallel());
+BlockingQueue results = new LinkedBlockingQueue();
 Map baseline = 
loadBaseline(options.getBaselineFilename());
 Map actual = new LinkedHashMap<>();
 Iterable configurations = 
options.getSuite().getConfigurations(options);
 
 boolean successful = true;
 try {
-  // Run all the configurations.
+  // Schedule all the configurations.
   for (NexmarkConfiguration configuration : configurations) {
-NexmarkPerf perf = nexmarkLauncher.run(configuration);
-if (perf != null) {
-  if (perf.errors == null || perf.errors.size() > 0) {
-successful = false;
+executor.execute(new Run(args, configuration, results));
+if (options.getNexmarkParallel() > 1) {
+  // Stagger start
+  try {
+Thread.sleep(2000);
+  } catch (InterruptedException e) {
+return;
   }
-  appendPerf(options.getPerfFilename(), configuration, perf);
-  actual.put(configuration, perf);
-  // Summarize what we've run so far.
-  saveSummary(null, configurations, actual, baseline, start, options);
 }
   }
+
+  // Collect all the results.
+  for (Object i : configurations) {
+Run result;
+try {
+  result = results.take();
 
 Review comment:
   This is a wait, but there is no reasonable timeout that can be applied at 
this layer. This is waiting for a test to complete. These tests are variable 
runtime and are run for multiple hours in some places.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131003)
Time Spent: 5h 20m  (was: 5h 10m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: 

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131002=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131002
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 18:31
Start Date: 03/Aug/18 18:31
Worklog Time Spent: 10m 
  Work Description: Ardagan commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207631195
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +78,97 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
+}
+
+@Override
+public void run() {
+  try {
+try {
+  perf = nexmarkLauncher.run(configuration);
+} catch (IOException | RuntimeException e) {
+  exception = e;
+} finally {
+  results.put(this);
+}
+  } catch (InterruptedException ie) {
+return;
+  }
+}
+
+private NexmarkPerf get() throws IOException {
+  if (exception != null) {
+if (exception instanceof IOException) {
+  throw (IOException) exception;
+} else if (exception instanceof RuntimeException) {
+  throw (RuntimeException) exception;
+}
+// This should be unreachable.
+  }
+  assert perf != null;
+  return perf;
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
+ExecutorService executor = 
Executors.newFixedThreadPool(options.getNexmarkParallel());
+BlockingQueue results = new LinkedBlockingQueue();
 Map baseline = 
loadBaseline(options.getBaselineFilename());
 Map actual = new LinkedHashMap<>();
 Iterable configurations = 
options.getSuite().getConfigurations(options);
 
 boolean successful = true;
 try {
-  // Run all the configurations.
+  // Schedule all the configurations.
   for (NexmarkConfiguration configuration : configurations) {
-NexmarkPerf perf = nexmarkLauncher.run(configuration);
-if (perf != null) {
-  if (perf.errors == null || perf.errors.size() > 0) {
-successful = false;
+executor.execute(new Run(args, configuration, results));
+if (options.getNexmarkParallel() > 1) {
+  // Stagger start
+  try {
+Thread.sleep(2000);
+  } catch (InterruptedException e) {
+return;
   }
-  appendPerf(options.getPerfFilename(), configuration, perf);
-  actual.put(configuration, perf);
-  // Summarize what we've run so far.
-  saveSummary(null, configurations, actual, baseline, start, options);
 }
   }
+
+  // Collect all the results.
+  for (Object i : configurations) {
+Run result;
+try {
+  result = results.take();
 
 Review comment:
   Synced with Andrew. We have test timeout configured in test framework. Right 
now we rely on that one to kill the test if it hangs for too long.
   We can add cleanup on timout logic in another PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131002)
Time Spent: 5h 10m  (was: 5h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131001=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131001
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 18:31
Start Date: 03/Aug/18 18:31
Worklog Time Spent: 10m 
  Work Description: apilloud commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207631071
 
 

 ##
 File path: .test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
 ##
 @@ -45,6 +45,7 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
 
 Review comment:
   There are several files like this. I can address it in another PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131001)
Time Spent: 5h  (was: 4h 50m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131000=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131000
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 18:31
Start Date: 03/Aug/18 18:31
Worklog Time Spent: 10m 
  Work Description: apilloud commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207630943
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -107,6 +178,9 @@ void runAll(OptionT options, NexmarkLauncher 
nexmarkLauncher) throws IOException
 saveSummary(options.getSummaryFilename(), configurations, actual, 
baseline, start, options);
 saveJavascript(options.getJavascriptFilename(), configurations, 
actual, baseline, start);
   }
+
+  executor.shutdown();
+  while (!executor.isTerminated()) {}
 
 Review comment:
   It should shutdown immediately, but I agree that a timeout asserting that is 
good. Added.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131000)
Time Spent: 4h 50m  (was: 4h 40m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=130994=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-130994
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 18:12
Start Date: 03/Aug/18 18:12
Worklog Time Spent: 10m 
  Work Description: apilloud commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207625618
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +78,97 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
+}
+
+@Override
+public void run() {
+  try {
+try {
+  perf = nexmarkLauncher.run(configuration);
+} catch (IOException | RuntimeException e) {
+  exception = e;
+} finally {
+  results.put(this);
+}
+  } catch (InterruptedException ie) {
+return;
+  }
+}
+
+private NexmarkPerf get() throws IOException {
+  if (exception != null) {
+if (exception instanceof IOException) {
+  throw (IOException) exception;
+} else if (exception instanceof RuntimeException) {
+  throw (RuntimeException) exception;
+}
+// This should be unreachable.
+  }
+  assert perf != null;
+  return perf;
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
+ExecutorService executor = 
Executors.newFixedThreadPool(options.getNexmarkParallel());
+BlockingQueue results = new LinkedBlockingQueue();
 Map baseline = 
loadBaseline(options.getBaselineFilename());
 Map actual = new LinkedHashMap<>();
 Iterable configurations = 
options.getSuite().getConfigurations(options);
 
 boolean successful = true;
 try {
-  // Run all the configurations.
+  // Schedule all the configurations.
   for (NexmarkConfiguration configuration : configurations) {
-NexmarkPerf perf = nexmarkLauncher.run(configuration);
-if (perf != null) {
-  if (perf.errors == null || perf.errors.size() > 0) {
-successful = false;
+executor.execute(new Run(args, configuration, results));
+if (options.getNexmarkParallel() > 1) {
+  // Stagger start
+  try {
+Thread.sleep(2000);
+  } catch (InterruptedException e) {
+return;
   }
-  appendPerf(options.getPerfFilename(), configuration, perf);
-  actual.put(configuration, perf);
-  // Summarize what we've run so far.
-  saveSummary(null, configurations, actual, baseline, start, options);
 }
   }
+
+  // Collect all the results.
+  for (Object i : configurations) {
 
 Review comment:
   I like that better too, but `Iterable` doesn't have `size()`. I'm looping 
through it above, I'll add a count there.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 130994)
Time Spent: 4h 40m  (was: 4.5h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=130992=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-130992
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 18:07
Start Date: 03/Aug/18 18:07
Worklog Time Spent: 10m 
  Work Description: apilloud commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207624442
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +78,97 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
+}
+
+@Override
+public void run() {
+  try {
+try {
+  perf = nexmarkLauncher.run(configuration);
+} catch (IOException | RuntimeException e) {
+  exception = e;
+} finally {
+  results.put(this);
+}
+  } catch (InterruptedException ie) {
+return;
+  }
+}
+
+private NexmarkPerf get() throws IOException {
+  if (exception != null) {
+if (exception instanceof IOException) {
+  throw (IOException) exception;
+} else if (exception instanceof RuntimeException) {
+  throw (RuntimeException) exception;
+}
+// This should be unreachable.
+  }
+  assert perf != null;
+  return perf;
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
+ExecutorService executor = 
Executors.newFixedThreadPool(options.getNexmarkParallel());
+BlockingQueue results = new LinkedBlockingQueue();
 Map baseline = 
loadBaseline(options.getBaselineFilename());
 Map actual = new LinkedHashMap<>();
 Iterable configurations = 
options.getSuite().getConfigurations(options);
 
 boolean successful = true;
 try {
-  // Run all the configurations.
+  // Schedule all the configurations.
   for (NexmarkConfiguration configuration : configurations) {
-NexmarkPerf perf = nexmarkLauncher.run(configuration);
-if (perf != null) {
-  if (perf.errors == null || perf.errors.size() > 0) {
-successful = false;
+executor.execute(new Run(args, configuration, results));
+if (options.getNexmarkParallel() > 1) {
+  // Stagger start
+  try {
+Thread.sleep(2000);
 
 Review comment:
   I heard we were hitting quota issues on spinup. Happy to drop it if @Ardagan 
thinks it is unneeded.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 130992)
Time Spent: 4.5h  (was: 4h 20m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 4.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=130697=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-130697
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 01:53
Start Date: 03/Aug/18 01:53
Worklog Time Spent: 10m 
  Work Description: Ardagan commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207422153
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +78,97 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
+}
+
+@Override
+public void run() {
+  try {
+try {
+  perf = nexmarkLauncher.run(configuration);
+} catch (IOException | RuntimeException e) {
+  exception = e;
+} finally {
+  results.put(this);
+}
+  } catch (InterruptedException ie) {
+return;
+  }
+}
+
+private NexmarkPerf get() throws IOException {
+  if (exception != null) {
+if (exception instanceof IOException) {
+  throw (IOException) exception;
+} else if (exception instanceof RuntimeException) {
+  throw (RuntimeException) exception;
+}
+// This should be unreachable.
+  }
+  assert perf != null;
+  return perf;
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
+ExecutorService executor = 
Executors.newFixedThreadPool(options.getNexmarkParallel());
+BlockingQueue results = new LinkedBlockingQueue();
 Map baseline = 
loadBaseline(options.getBaselineFilename());
 Map actual = new LinkedHashMap<>();
 Iterable configurations = 
options.getSuite().getConfigurations(options);
 
 boolean successful = true;
 try {
-  // Run all the configurations.
+  // Schedule all the configurations.
   for (NexmarkConfiguration configuration : configurations) {
-NexmarkPerf perf = nexmarkLauncher.run(configuration);
-if (perf != null) {
-  if (perf.errors == null || perf.errors.size() > 0) {
-successful = false;
+executor.execute(new Run(args, configuration, results));
+if (options.getNexmarkParallel() > 1) {
+  // Stagger start
+  try {
+Thread.sleep(2000);
+  } catch (InterruptedException e) {
+return;
   }
-  appendPerf(options.getPerfFilename(), configuration, perf);
-  actual.put(configuration, perf);
-  // Summarize what we've run so far.
-  saveSummary(null, configurations, actual, baseline, start, options);
 }
   }
+
+  // Collect all the results.
+  for (Object i : configurations) {
+Run result;
+try {
+  result = results.take();
 
 Review comment:
   This looks like a wait to me. If I'm correct, please add timeout.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 130697)
Time Spent: 4h 10m  (was: 4h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 4h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian 

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=130698=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-130698
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 01:53
Start Date: 03/Aug/18 01:53
Worklog Time Spent: 10m 
  Work Description: Ardagan commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207421887
 
 

 ##
 File path: .test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
 ##
 @@ -45,6 +45,7 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
 
 Review comment:
   Most of these switches are same across file. You can move them into a 
constant and specify only different switches explicitly.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 130698)
Time Spent: 4h 20m  (was: 4h 10m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=130642=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-130642
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 02/Aug/18 21:24
Start Date: 02/Aug/18 21:24
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207374139
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +78,97 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
+}
+
+@Override
+public void run() {
+  try {
+try {
+  perf = nexmarkLauncher.run(configuration);
+} catch (IOException | RuntimeException e) {
+  exception = e;
+} finally {
+  results.put(this);
+}
+  } catch (InterruptedException ie) {
+return;
+  }
+}
+
+private NexmarkPerf get() throws IOException {
+  if (exception != null) {
+if (exception instanceof IOException) {
+  throw (IOException) exception;
+} else if (exception instanceof RuntimeException) {
+  throw (RuntimeException) exception;
+}
+// This should be unreachable.
+  }
+  assert perf != null;
+  return perf;
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
+ExecutorService executor = 
Executors.newFixedThreadPool(options.getNexmarkParallel());
+BlockingQueue results = new LinkedBlockingQueue();
 Map baseline = 
loadBaseline(options.getBaselineFilename());
 Map actual = new LinkedHashMap<>();
 Iterable configurations = 
options.getSuite().getConfigurations(options);
 
 boolean successful = true;
 try {
-  // Run all the configurations.
+  // Schedule all the configurations.
   for (NexmarkConfiguration configuration : configurations) {
-NexmarkPerf perf = nexmarkLauncher.run(configuration);
-if (perf != null) {
-  if (perf.errors == null || perf.errors.size() > 0) {
-successful = false;
+executor.execute(new Run(args, configuration, results));
+if (options.getNexmarkParallel() > 1) {
+  // Stagger start
+  try {
+Thread.sleep(2000);
 
 Review comment:
   Is there a special reason to wait for 2 sec between two `pool.execute`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 130642)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=130644=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-130644
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 02/Aug/18 21:24
Start Date: 02/Aug/18 21:24
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207375507
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +78,97 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
+}
+
+@Override
+public void run() {
+  try {
+try {
+  perf = nexmarkLauncher.run(configuration);
+} catch (IOException | RuntimeException e) {
+  exception = e;
+} finally {
+  results.put(this);
+}
+  } catch (InterruptedException ie) {
+return;
+  }
+}
+
+private NexmarkPerf get() throws IOException {
+  if (exception != null) {
+if (exception instanceof IOException) {
+  throw (IOException) exception;
+} else if (exception instanceof RuntimeException) {
+  throw (RuntimeException) exception;
+}
+// This should be unreachable.
+  }
+  assert perf != null;
+  return perf;
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
+ExecutorService executor = 
Executors.newFixedThreadPool(options.getNexmarkParallel());
 
 Review comment:
   Nit: create a thread pool with `size = min(options.getNexmarkParallel(), 
configurations.size())`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 130644)
Time Spent: 3h 50m  (was: 3h 40m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=130645=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-130645
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 02/Aug/18 21:24
Start Date: 02/Aug/18 21:24
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207378341
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -107,6 +178,9 @@ void runAll(OptionT options, NexmarkLauncher 
nexmarkLauncher) throws IOException
 saveSummary(options.getSummaryFilename(), configurations, actual, 
baseline, start, options);
 saveJavascript(options.getJavascriptFilename(), configurations, 
actual, baseline, start);
   }
+
+  executor.shutdown();
+  while (!executor.isTerminated()) {}
 
 Review comment:
   I tend to avoid waiting for one thing without timeout. In case by any chance 
it never returns (or most likely takes too long time to return).  
[awaitTermination](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ExecutorService.html#awaitTermination-long-java.util.concurrent.TimeUnit-)
 is a better choice.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 130645)
Time Spent: 4h  (was: 3h 50m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 4h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=130641=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-130641
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 02/Aug/18 21:24
Start Date: 02/Aug/18 21:24
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207378999
 
 

 ##
 File path: .test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
 ##
 @@ -61,6 +62,7 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--nexmarkParallel=16',
 
 Review comment:
   Assuming 16 threads are ok for a Beam Jenkins host. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 130641)
Time Spent: 3h 40m  (was: 3.5h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=130643=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-130643
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 02/Aug/18 21:24
Start Date: 02/Aug/18 21:24
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on a change in pull request #6127: 
[BEAM-4761] Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#discussion_r207375069
 
 

 ##
 File path: 
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
 ##
 @@ -74,30 +78,97 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Run implements Runnable {
+final NexmarkLauncher nexmarkLauncher;
+final NexmarkConfiguration configuration;
+final BlockingQueue results;
+NexmarkPerf perf = null;
+Exception exception = null;
+
+private Run(String[] args, NexmarkConfiguration configuration, 
BlockingQueue results) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+  this.results = results;
+}
+
+@Override
+public void run() {
+  try {
+try {
+  perf = nexmarkLauncher.run(configuration);
+} catch (IOException | RuntimeException e) {
+  exception = e;
+} finally {
+  results.put(this);
+}
+  } catch (InterruptedException ie) {
+return;
+  }
+}
+
+private NexmarkPerf get() throws IOException {
+  if (exception != null) {
+if (exception instanceof IOException) {
+  throw (IOException) exception;
+} else if (exception instanceof RuntimeException) {
+  throw (RuntimeException) exception;
+}
+// This should be unreachable.
+  }
+  assert perf != null;
+  return perf;
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
+ExecutorService executor = 
Executors.newFixedThreadPool(options.getNexmarkParallel());
+BlockingQueue results = new LinkedBlockingQueue();
 Map baseline = 
loadBaseline(options.getBaselineFilename());
 Map actual = new LinkedHashMap<>();
 Iterable configurations = 
options.getSuite().getConfigurations(options);
 
 boolean successful = true;
 try {
-  // Run all the configurations.
+  // Schedule all the configurations.
   for (NexmarkConfiguration configuration : configurations) {
-NexmarkPerf perf = nexmarkLauncher.run(configuration);
-if (perf != null) {
-  if (perf.errors == null || perf.errors.size() > 0) {
-successful = false;
+executor.execute(new Run(args, configuration, results));
+if (options.getNexmarkParallel() > 1) {
+  // Stagger start
+  try {
+Thread.sleep(2000);
+  } catch (InterruptedException e) {
+return;
   }
-  appendPerf(options.getPerfFilename(), configuration, perf);
-  actual.put(configuration, perf);
-  // Summarize what we've run so far.
-  saveSummary(null, configurations, actual, baseline, start, options);
 }
   }
+
+  // Collect all the results.
+  for (Object i : configurations) {
 
 Review comment:
   Seems like `Object i` is not used? I prefer `for( i = 0; i < 
configurations.size(); i++)` to count the number of results.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 130643)
Time Spent: 3h 50m  (was: 3h 40m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=130611=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-130611
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 02/Aug/18 20:17
Start Date: 02/Aug/18 20:17
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #6127: [BEAM-4761] Run 
Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127#issuecomment-410054540
 
 
   Slight refactor to Nexmark so tests run in a thread pool.
   R: @amaliujia (You appear to know a lot about java thread pools.)
   cc: @akedin 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 130611)
Time Spent: 3.5h  (was: 3h 20m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=130608=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-130608
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 02/Aug/18 20:15
Start Date: 02/Aug/18 20:15
Worklog Time Spent: 10m 
  Work Description: apilloud opened a new pull request #6127: [BEAM-4761] 
Run Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127
 
 
   Dataflow runs jobs remotely and they are slow to spin up, so we should run 
multiple at the same time. This should fix the timeouts in the Nexmark Dataflow 
postcommit.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [x] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 130608)
Time Spent: 3h 20m  (was: 3h 10m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>   

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=125382=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-125382
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 20/Jul/18 08:03
Start Date: 20/Jul/18 08:03
Worklog Time Spent: 10m 
  Work Description: echauchot closed pull request #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/.test-infra/jenkins/NexmarkBigqueryProperties.groovy 
b/.test-infra/jenkins/NexmarkBigqueryProperties.groovy
index c7b2265ae71..4126b4c63ba 100644
--- a/.test-infra/jenkins/NexmarkBigqueryProperties.groovy
+++ b/.test-infra/jenkins/NexmarkBigqueryProperties.groovy
@@ -24,5 +24,5 @@ class NexmarkBigqueryProperties {
  '--project=apache-beam-testing',
  
'--resourceNameMode=QUERY_RUNNER_AND_MODE',
  '--exportSummaryToBigQuery=true',
- 
'--tempLocation=gs://temp-storage-for-perf-tests/'].join(' ')
-}
\ No newline at end of file
+ 
'--tempLocation=gs://temp-storage-for-perf-tests/nexmark'].join(' ')
+}
diff --git a/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy 
b/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
new file mode 100644
index 000..f30c85feb68
--- /dev/null
+++ b/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+import NexmarkBigqueryProperties
+import NoPhraseTriggeringPostCommitBuilder
+
+// This job runs the suite of ValidatesRunner tests against the Dataflow 
runner.
+NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_Dataflow',
+'Dataflow Runner Nexmark Tests', this) {
+  description('Runs the Nexmark suite on the Dataflow runner.')
+
+  // Execute concurrent builds if necessary.
+  concurrentBuild()
+  throttleConcurrentBuilds {
+maxTotal(10)
+  }
+
+  // Set common parameters.
+  commonJobProperties.setTopLevelMainJobProperties(delegate, 'master', 240)
+
+  // Gradle goals for this job.
+  steps {
+shell('echo *** RUN NEXMARK IN BATCH MODE USING DATAFLOW RUNNER ***')
+gradle {
+  rootBuildScriptDir(commonJobProperties.checkoutDir)
+  tasks(':beam-sdks-java-nexmark:run')
+  commonJobProperties.setGradleSwitches(delegate)
+  switches('-Pnexmark.runner=":beam-runners-google-cloud-dataflow-java"' +
+  ' -Pnexmark.args="' +
+  [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
+  '--runner=DataflowRunner',
+  '--streaming=false',
+  '--suite=STRESS',
+  '--manageResources=false',
+  '--monitorJobs=true',
+  '--enforceEncodability=true',
+  '--enforceImmutability=true"'].join(' '))
+}
+shell('echo *** RUN NEXMARK IN STREAMING MODE USING DATAFLOW RUNNER ***')
+gradle {
+  rootBuildScriptDir(commonJobProperties.checkoutDir)
+  tasks(':beam-sdks-java-nexmark:run')
+  commonJobProperties.setGradleSwitches(delegate)
+  switches('-Pnexmark.runner=":beam-runners-google-cloud-dataflow-java"' +
+  ' -Pnexmark.args="' +
+  [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
+  '--runner=DataflowRunner',
+  '--streaming=true',
+  '--suite=STRESS',
+  '--manageResources=false',
+  '--monitorJobs=true',
+  '--enforceEncodability=true',
+  '--enforceImmutability=true"'].join(' '))
+}
+shell('echo *** RUN NEXMARK IN SQL BATCH MODE USING DATAFLOW RUNNER ***')
+

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=125381=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-125381
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 20/Jul/18 08:03
Start Date: 20/Jul/18 08:03
Worklog Time Spent: 10m 
  Work Description: echauchot commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-406521173
 
 
   Thanks for your work Andrew ! I'm glad to see Nexmax run on Dataflow ! Can 
you add the DF dashboards to perfkit? You can duplicate performance and output 
size dahsboards of another runner and change the table name on each graph. 
Beware when you duplicate a dashboard there is a bug on perfkit (I opened a 
ticket on perfkit): use save as otherwise the dashboards ids mess up.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 125381)
Time Spent: 3h  (was: 2h 50m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 3h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=125376=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-125376
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 20/Jul/18 07:56
Start Date: 20/Jul/18 07:56
Worklog Time Spent: 10m 
  Work Description: echauchot commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-406519626
 
 
   go from SMOKE test suite to STRESS regarding what you explained above looks 
good to me because Nexmark postCommits are not made to compare the runners / 
engines between each other.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 125376)
Time Spent: 2h 50m  (was: 2h 40m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=125221=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-125221
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 19/Jul/18 20:14
Start Date: 19/Jul/18 20:14
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-406400188
 
 
   This appears to be working now: 
https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Dataflow/3/
 I made one more change and bumped the run size up, as dataflow takes ~2 
minutes to spin up (which counts towards Nexmark run time). We need more than 2 
seconds of work to get any sort of test.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 125221)
Time Spent: 2.5h  (was: 2h 20m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=125222=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-125222
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 19/Jul/18 20:14
Start Date: 19/Jul/18 20:14
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-406400224
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 125222)
Time Spent: 2h 40m  (was: 2.5h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=125001=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-125001
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 19/Jul/18 14:14
Start Date: 19/Jul/18 14:14
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-406290990
 
 
   The dataflow runner won't write to the root of a GCS bucket, so I changed 
the config.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 125001)
Time Spent: 2h 10m  (was: 2h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=125003=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-125003
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 19/Jul/18 14:14
Start Date: 19/Jul/18 14:14
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-406291008
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 125003)
Time Spent: 2h 20m  (was: 2h 10m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=124782=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-124782
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 19/Jul/18 03:58
Start Date: 19/Jul/18 03:58
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-406147299
 
 
   Still failing. Some of the NexmarkBigqueryProperties appear to be causing 
problems. I'll get it working locally with the publish.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 124782)
Time Spent: 2h  (was: 1h 50m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=124555=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-124555
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 18/Jul/18 19:02
Start Date: 18/Jul/18 19:02
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-406039903
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 124555)
Time Spent: 1h 50m  (was: 1h 40m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=124542=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-124542
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 18/Jul/18 18:12
Start Date: 18/Jul/18 18:12
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-406024789
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 124542)
Time Spent: 1h 40m  (was: 1.5h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=124541=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-124541
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 18/Jul/18 18:12
Start Date: 18/Jul/18 18:12
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-406024769
 
 
   Seems `--project` is specified in BigQuery args, so I added it a second 
time. Fixed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 124541)
Time Spent: 1.5h  (was: 1h 20m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=124501=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-124501
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 18/Jul/18 17:21
Start Date: 18/Jul/18 17:21
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-406009245
 
 
   I've updated with the limit. None of those parameters are required, they all 
have default values. These pass locally, but the seed job got reset before 
anything ran. I will run it again.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 124501)
Time Spent: 1h 10m  (was: 1h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=124502=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-124502
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 18/Jul/18 17:21
Start Date: 18/Jul/18 17:21
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-406009263
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 124502)
Time Spent: 1h 20m  (was: 1h 10m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=124358=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-124358
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 18/Jul/18 08:41
Start Date: 18/Jul/18 08:41
Worklog Time Spent: 10m 
  Work Description: echauchot commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-405856256
 
 
   @apilloud, are these dataflow specific parameters needed ? 
   ```
   --zone
   --workerMachineType
   --stagingLocation
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 124358)
Time Spent: 1h  (was: 50m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 1h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=124353=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-124353
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 18/Jul/18 08:29
Start Date: 18/Jul/18 08:29
Worklog Time Spent: 10m 
  Work Description: echauchot commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-405852733
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 124353)
Time Spent: 50m  (was: 40m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=124242=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-124242
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 17/Jul/18 22:43
Start Date: 17/Jul/18 22:43
Worklog Time Spent: 10m 
  Work Description: ThatRfernand commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-405751879
 
 
   We can concurrently run up to 100 Dataflow jobs. I've seen a peak of 70
   under heavy ValidatesRunner pressure (three of those running concurrently).
   I think you're safe running < 10 or so.
   
   
   
   On Tue, Jul 17, 2018 at 3:00 PM Andrew Pilloud 
   wrote:
   
   > @ThatRfernand  I've been told you are
   > the quota expert. This runs one pipeline at a time on Dataflow. How many
   > concurrent jobs should I cap it at?
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > , or mute
   > the thread
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 124242)
Time Spent: 40m  (was: 0.5h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=124227=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-124227
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 17/Jul/18 22:00
Start Date: 17/Jul/18 22:00
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-405742423
 
 
   @ThatRfernand I've been told you are the quota expert. This runs one 
pipeline at a time on Dataflow. How many concurrent jobs should I cap it at?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 124227)
Time Spent: 0.5h  (was: 20m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=124100=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-124100
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 17/Jul/18 15:13
Start Date: 17/Jul/18 15:13
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5966: [BEAM-4761] Run 
Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966#issuecomment-405618662
 
 
   @jasonkuster I'm guessing this is going to run into some quota issues? Do 
you know what limits I need to set?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 124100)
Time Spent: 20m  (was: 10m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=124099=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-124099
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 17/Jul/18 15:12
Start Date: 17/Jul/18 15:12
Worklog Time Spent: 10m 
  Work Description: apilloud opened a new pull request #5966: [BEAM-4761] 
Run Nexmark on Dataflow
URL: https://github.com/apache/beam/pull/5966
 
 
   This adds a Nexmark postcommit job for Dataflow. 
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [X] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 124099)
Time Spent: 10m
Remaining Estimate: 0h

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was