Jenkins build is back to normal : beam_PerformanceTests_Python #1268

[Apache Jenkins Server]

See 

Jenkins build is back to normal : beam_PostCommit_Java_Nexmark_Direct #191

[Apache Jenkins Server]

See 

[jira] [Work logged] (BEAM-4276) Implement the portable lifted Combiner transforms in Go SDK

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4276?focusedWorklogId=131162=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131162
 ]

ASF GitHub Bot logged work on BEAM-4276:


Author: ASF GitHub Bot
Created on: 04/Aug/18 01:32
Start Date: 04/Aug/18 01:32
Worklog Time Spent: 10m 
  Work Description: lostluck commented on issue #6143: [BEAM-4276] Combiner 
lifting for Dataflow
URL: https://github.com/apache/beam/pull/6143#issuecomment-410412665
 
 
   R: @herohde @youngoli 
   I'm pretty sure I've got this working properly, but the integration tests 
are too small for a meaningful difference when combiners are lifted. Oddly, the 
wordcount Dataflow jobs seems to like taking ~3m3s to complete.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131162)
Time Spent: 9.5h  (was: 9h 20m)

> Implement the portable lifted Combiner transforms in Go SDK
> ---
>
> Key: BEAM-4276
> URL: https://issues.apache.org/jira/browse/BEAM-4276
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-go
>Reporter: Henning Rohde
>Assignee: Robert Burke
>Priority: Major
>  Time Spent: 9.5h
>  Remaining Estimate: 0h
>
> Specifically add the necessary code to produce a Combine Composite with the 
> correct URN, and permit the SDK harness to understand the lifted parts when 
> receiving a bundle plan from the worker.
> Not expected as part of this issue is:
> Additional performance tweaks to the in memory cache (See 
> [BEAM-4468|https://issues.apache.org/jira/browse/BEAM-4468])



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4276) Implement the portable lifted Combiner transforms in Go SDK

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4276?focusedWorklogId=131161=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131161
 ]

ASF GitHub Bot logged work on BEAM-4276:


Author: ASF GitHub Bot
Created on: 04/Aug/18 01:20
Start Date: 04/Aug/18 01:20
Worklog Time Spent: 10m 
  Work Description: lostluck commented on issue #6143: [BEAM-4276] Combiner 
lifting for Dataflow
URL: https://github.com/apache/beam/pull/6143#issuecomment-410411914
 
 
   Run Go PostCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131161)
Time Spent: 9h 20m  (was: 9h 10m)

> Implement the portable lifted Combiner transforms in Go SDK
> ---
>
> Key: BEAM-4276
> URL: https://issues.apache.org/jira/browse/BEAM-4276
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-go
>Reporter: Henning Rohde
>Assignee: Robert Burke
>Priority: Major
>  Time Spent: 9h 20m
>  Remaining Estimate: 0h
>
> Specifically add the necessary code to produce a Combine Composite with the 
> correct URN, and permit the SDK harness to understand the lifted parts when 
> receiving a bundle plan from the worker.
> Not expected as part of this issue is:
> Additional performance tweaks to the in memory cache (See 
> [BEAM-4468|https://issues.apache.org/jira/browse/BEAM-4468])



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PerformanceTests_Python #1267

[Apache Jenkins Server]

See 


--
[...truncated 100.18 KB...]
2018-08-04 01:09:27,927 0baf5535 MainThread INFO Retrying exception running 
IssueRetryableCommand: Command returned a non-zero exit code.

2018-08-04 01:09:46,098 0baf5535 MainThread INFO Running: bq load 
--autodetect --source_format=NEWLINE_DELIMITED_JSON 
beam_performance.pkb_results 

2018-08-04 01:09:49,974 0baf5535 MainThread INFO Ran: {bq load --autodetect 
--source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results 

  ReturnCode:1
STDOUT: 

BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r6d1b59a3091e4529_0165027bb68d_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT

STDERR: Upload complete.Waiting on bqjob_r6d1b59a3091e4529_0165027bb68d_1 
... (0s) Current status: RUNNING
  Waiting on 
bqjob_r6d1b59a3091e4529_0165027bb68d_1 ... (0s) Current status: DONE   
2018-08-04 01:09:49,975 0baf5535 MainThread INFO Retrying exception running 
IssueRetryableCommand: Command returned a non-zero exit code.

2018-08-04 01:10:12,981 0baf5535 MainThread INFO Running: bq load 
--autodetect --source_format=NEWLINE_DELIMITED_JSON 
beam_performance.pkb_results 

2018-08-04 01:10:16,990 0baf5535 MainThread INFO Ran: {bq load --autodetect 
--source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results 

  ReturnCode:1
STDOUT: 

BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4840b2454a8e3494_0165027c1fc7_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT

STDERR: Upload complete.Waiting on bqjob_r4840b2454a8e3494_0165027c1fc7_1 
... (0s) Current status: RUNNING
  Waiting on 
bqjob_r4840b2454a8e3494_0165027c1fc7_1 ... (0s) Current status: DONE   
2018-08-04 01:10:16,991 0baf5535 MainThread INFO Retrying exception running 
IssueRetryableCommand: Command returned a non-zero exit code.

2018-08-04 01:10:44,182 0baf5535 MainThread INFO Running: bq load 
--autodetect --source_format=NEWLINE_DELIMITED_JSON 
beam_performance.pkb_results 

2018-08-04 01:10:48,193 0baf5535 MainThread INFO Ran: {bq load --autodetect 
--source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results 

  ReturnCode:1
STDOUT: 

BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r178283251e23bd57_0165027c99aa_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT

STDERR: Upload complete.Waiting on bqjob_r178283251e23bd57_0165027c99aa_1 
... (0s) Current status: RUNNING
  Waiting on 
bqjob_r178283251e23bd57_0165027c99aa_1 ... (0s) Current status: DONE   
2018-08-04 01:10:48,194 0baf5535 MainThread INFO Retrying exception running 
IssueRetryableCommand: Command returned a non-zero exit code.

2018-08-04 01:11:09,452 0baf5535 MainThread INFO Running: bq load 
--autodetect --source_format=NEWLINE_DELIMITED_JSON 
beam_performance.pkb_results 

2018-08-04 01:11:13,329 0baf5535 MainThread INFO Ran: {bq load --autodetect 
--source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results 

  ReturnCode:1
STDOUT: 

BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4b3530b4d72d9f35_0165027cfc3b_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT

STDERR: Upload complete.Waiting on bqjob_r4b3530b4d72d9f35_0165027cfc3b_1 
... (0s) Current status: RUNNING
  Waiting on 
bqjob_r4b3530b4d72d9f35_0165027cfc3b_1 ... (0s) Current status: DONE   
2018-08-04 01:11:13,330 0baf5535 MainThread INFO Retrying exception running 
IssueRetryableCommand: Command returned a non-zero exit code.

2018-08-04 

[jira] [Work logged] (BEAM-4000) Futurize and fix python 2 compatibility for io subpackage

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4000?focusedWorklogId=131159=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131159
 ]

ASF GitHub Bot logged work on BEAM-4000:


Author: ASF GitHub Bot
Created on: 04/Aug/18 01:11
Start Date: 04/Aug/18 01:11
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on a change in pull request #5715: 
[BEAM-4000] Futurize io subpackage
URL: https://github.com/apache/beam/pull/5715#discussion_r207695556
 
 

 ##
 File path: sdks/python/apache_beam/io/gcp/pubsub.py
 ##
 @@ -33,8 +33,9 @@
 from __future__ import absolute_import
 
 import re
+from builtins import object
 
-from six import text_type
+from past.builtins import basestring
 
 Review comment:
   Correction: `past.builtins.basestring` is configured to mimic a superclass 
of both `str` and `unicode` in Python 2. So possibly there is no significant 
change in behavior but I am not 100% sure.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131159)
Time Spent: 6.5h  (was: 6h 20m)

> Futurize and fix python 2 compatibility for io subpackage
> -
>
> Key: BEAM-4000
> URL: https://issues.apache.org/jira/browse/BEAM-4000
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Robbe
>Assignee: Matthias Feys
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4276) Implement the portable lifted Combiner transforms in Go SDK

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4276?focusedWorklogId=131158=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131158
 ]

ASF GitHub Bot logged work on BEAM-4276:


Author: ASF GitHub Bot
Created on: 04/Aug/18 01:08
Start Date: 04/Aug/18 01:08
Worklog Time Spent: 10m 
  Work Description: lostluck commented on issue #6143: [BEAM-4276] Combiner 
lifting for Dataflow
URL: https://github.com/apache/beam/pull/6143#issuecomment-410410983
 
 
   Run Go PostCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131158)
Time Spent: 9h 10m  (was: 9h)

> Implement the portable lifted Combiner transforms in Go SDK
> ---
>
> Key: BEAM-4276
> URL: https://issues.apache.org/jira/browse/BEAM-4276
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-go
>Reporter: Henning Rohde
>Assignee: Robert Burke
>Priority: Major
>  Time Spent: 9h 10m
>  Remaining Estimate: 0h
>
> Specifically add the necessary code to produce a Combine Composite with the 
> correct URN, and permit the SDK harness to understand the lifted parts when 
> receiving a bundle plan from the worker.
> Not expected as part of this issue is:
> Additional performance tweaks to the in memory cache (See 
> [BEAM-4468|https://issues.apache.org/jira/browse/BEAM-4468])



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostCommit_Java_GradleBuild #1173

[Apache Jenkins Server]

See 


Changes:

[apilloud] [BEAM-4761] Handle skipped tests in parallel Nexmark

--
[...truncated 12.77 MB...]
bq show -j --format=prettyjson --project_id=apache-beam-testing 
beam_load_bigqueryreadwriteit0testsqltypesjenkins0804010209cc624cd6_f5310eaa13d84901ace5c9a24c02535f_182b37520fc744796eea8617d303aaac_1_0-0

org.apache.beam.sdk.extensions.sql.PubsubToBigqueryIT > testSimpleInsert 
STANDARD_ERROR
Aug 04, 2018 1:02:12 AM 
org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSource createRandomSubscription
WARNING: Created subscription 
projects/apache-beam-testing/subscriptions/integ-test-PubsubToBigqueryIT-testSimpleInsert-2018-08-04-01-02-04-623-events--5949498161138093095_beam_9009087704051826521
 to topic 
projects/apache-beam-testing/topics/integ-test-PubsubToBigqueryIT-testSimpleInsert-2018-08-04-01-02-04-623-events--5949498161138093095.
 Note this subscription WILL NOT be deleted when the pipeline terminates

org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT > 
testSelectsPayloadContent STANDARD_ERROR
Aug 04, 2018 1:02:13 AM 
org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSource createRandomSubscription
WARNING: Created subscription 
projects/apache-beam-testing/subscriptions/integ-test-PubsubJsonIT-testSelectsPayloadContent-2018-08-04-01-02-05-841-events--7815642782319627272_beam_-8355858822348088535
 to topic 
projects/apache-beam-testing/topics/integ-test-PubsubJsonIT-testSelectsPayloadContent-2018-08-04-01-02-05-841-events--7815642782319627272.
 Note this subscription WILL NOT be deleted when the pipeline terminates

org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryReadWriteIT > 
testSQLTypes STANDARD_ERROR
Aug 04, 2018 1:02:14 AM 
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job 
{jobId=beam_load_bigqueryreadwriteit0testsqltypesjenkins0804010209cc624cd6_f5310eaa13d84901ace5c9a24c02535f_182b37520fc744796eea8617d303aaac_1_0-0,
 location=US, projectId=apache-beam-testing} completed in state DONE
Aug 04, 2018 1:02:14 AM org.apache.beam.sdk.io.gcp.bigquery.WriteTables load
INFO: Load job 
{jobId=beam_load_bigqueryreadwriteit0testsqltypesjenkins0804010209cc624cd6_f5310eaa13d84901ace5c9a24c02535f_182b37520fc744796eea8617d303aaac_1_0-0,
 location=US, projectId=apache-beam-testing} succeeded. Statistics: 
{"creationTime":"1533344530763","endTime":"1533344532024","load":{"badRecords":"0","inputFileBytes":"243","inputFiles":"1","outputBytes":"82","outputRows":"1"},"startTime":"1533344531062"}

org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryReadWriteIT > 
testSQLRead STANDARD_ERROR
Aug 04, 2018 1:02:14 AM 
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl 
createTable
INFO: Trying to create BigQuery table: 
apache-beam-testing:integ_test.BigQueryReadWriteIT_testSQLRead_2018_08_04_01_02_14_894_6873646947722189803
Aug 04, 2018 1:02:15 AM 
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl 
createTable
INFO: Trying to create BigQuery table: 
apache-beam-testing:integ_test.BigQueryReadWriteIT_testSQLRead_2018_08_04_01_02_15_082_6675018824432359136
Aug 04, 2018 1:02:15 AM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
INSERT INTO `beam`.`TEST`
VALUES ROW(9223372036854775807, 127, 32767, 2147483647, 1.0, 1.0, TRUE, 
TIMESTAMP '2018-05-28 20:17:40.123', 'varchar', 'char', ARRAY['123', '456'])
Aug 04, 2018 1:02:15 AM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
BeamIOSinkRel(table=[[beam, TEST]], operation=[INSERT], flattened=[true])
  LogicalProject(c_bigint=[9223372036854775807], c_tinyint=[127], 
c_smallint=[32767], c_integer=[2147483647], c_float=[1.0], c_double=[1.0], 
c_boolean=[true], c_timestamp=[2018-05-28 20:17:40.123], c_varchar=['varchar'], 
c_char=['char'], c_arr=[ARRAY('123', '456')])
LogicalValues(tuples=[[{ 0 }]])

Aug 04, 2018 1:02:15 AM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamIOSinkRel(table=[[beam, TEST]], operation=[INSERT], flattened=[true])
  BeamCalcRel(expr#0=[{inputs}], expr#1=[9223372036854775807], 
expr#2=[127], expr#3=[32767], expr#4=[2147483647], expr#5=[1.0], expr#6=[true], 
expr#7=[2018-05-28 20:17:40.123], expr#8=['varchar'], expr#9=['char'], 
expr#10=['123'], expr#11=['456'], expr#12=[ARRAY($t10, $t11)], c_bigint=[$t1], 
c_tinyint=[$t2], c_smallint=[$t3], c_integer=[$t4], c_float=[$t5], 
c_double=[$t5], c_boolean=[$t6], c_timestamp=[$t7], c_varchar=[$t8], 
c_char=[$t9], c_arr=[$t12])
BeamValuesRel(tuples=[[{ 0 }]])

Aug 04, 2018 1:02:15 AM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner 

[jira] [Work logged] (BEAM-4276) Implement the portable lifted Combiner transforms in Go SDK

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4276?focusedWorklogId=131157=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131157
 ]

ASF GitHub Bot logged work on BEAM-4276:


Author: ASF GitHub Bot
Created on: 04/Aug/18 01:04
Start Date: 04/Aug/18 01:04
Worklog Time Spent: 10m 
  Work Description: lostluck commented on issue #6143: [BEAM-4276] Combiner 
lifting for Dataflow
URL: https://github.com/apache/beam/pull/6143#issuecomment-410410696
 
 
   Run Go PreCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131157)
Time Spent: 9h  (was: 8h 50m)

> Implement the portable lifted Combiner transforms in Go SDK
> ---
>
> Key: BEAM-4276
> URL: https://issues.apache.org/jira/browse/BEAM-4276
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-go
>Reporter: Henning Rohde
>Assignee: Robert Burke
>Priority: Major
>  Time Spent: 9h
>  Remaining Estimate: 0h
>
> Specifically add the necessary code to produce a Combine Composite with the 
> correct URN, and permit the SDK harness to understand the lifted parts when 
> receiving a bundle plan from the worker.
> Not expected as part of this issue is:
> Additional performance tweaks to the in memory cache (See 
> [BEAM-4468|https://issues.apache.org/jira/browse/BEAM-4468])



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4276) Implement the portable lifted Combiner transforms in Go SDK

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4276?focusedWorklogId=131156=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131156
 ]

ASF GitHub Bot logged work on BEAM-4276:


Author: ASF GitHub Bot
Created on: 04/Aug/18 01:02
Start Date: 04/Aug/18 01:02
Worklog Time Spent: 10m 
  Work Description: lostluck opened a new pull request #6143: [BEAM-4276] 
Combiner lifting for Dataflow
URL: https://github.com/apache/beam/pull/6143
 
 
   The previous changes set up combiner lifting for the portable 
representation, but Dataflow still uses its own configuration. This change 
closes the gap.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [X] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131156)
Time Spent: 8h 50m  (was: 8h 40m)

> Implement the portable lifted Combiner transforms in Go SDK
> ---
>
> Key: BEAM-4276
> URL: https://issues.apache.org/jira/browse/BEAM-4276
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-go
>Reporter: Henning Rohde
>Assignee: Robert Burke
>Priority: Major
>  Time Spent: 8h 50m
>  

Build failed in Jenkins: beam_PostCommit_Py_VR_Dataflow #718

[Apache Jenkins Server]

See 


Changes:

[apilloud] [BEAM-4761] Handle skipped tests in parallel Nexmark

--
[...truncated 214.09 KB...]
  File was already downloaded 
/tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
  File was already downloaded 
/tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
  File was already downloaded 
/tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
  File was already downloaded 
/tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
  File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
Collecting mock (from -r postcommit_requirements.txt (line 2))
  File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
  File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
  File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
  File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
  File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
  File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
  File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
  File was already downloaded 
/tmp/dataflow-requirements-cache/setuptools-40.0.0.zip
  File was already downloaded 
/tmp/dataflow-requirements-cache/setuptools-40.0.0.zip
  File was already downloaded 
/tmp/dataflow-requirements-cache/setuptools-40.0.0.zip
  File was already downloaded 
/tmp/dataflow-requirements-cache/setuptools-40.0.0.zip
  File was already downloaded 
/tmp/dataflow-requirements-cache/setuptools-40.0.0.zip
  File was already downloaded 
/tmp/dataflow-requirements-cache/setuptools-40.0.0.zip
  File was already downloaded 
/tmp/dataflow-requirements-cache/setuptools-40.0.0.zip
  File was already downloaded 
/tmp/dataflow-requirements-cache/setuptools-40.0.0.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
  File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz
  File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
  File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
  File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz
  File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
  File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz
  File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz
  File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
  File was already downloaded 
/tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
  File was already downloaded 
/tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
  File was already downloaded 
/tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
  File was already downloaded 

[jira] [Work logged] (BEAM-4000) Futurize and fix python 2 compatibility for io subpackage

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4000?focusedWorklogId=131153=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131153
 ]

ASF GitHub Bot logged work on BEAM-4000:


Author: ASF GitHub Bot
Created on: 04/Aug/18 00:52
Start Date: 04/Aug/18 00:52
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on a change in pull request #5715: 
[BEAM-4000] Futurize io subpackage
URL: https://github.com/apache/beam/pull/5715#discussion_r207694773
 
 

 ##
 File path: sdks/python/apache_beam/io/gcp/pubsub.py
 ##
 @@ -33,8 +33,9 @@
 from __future__ import absolute_import
 
 import re
+from builtins import object
 
-from six import text_type
+from past.builtins import basestring
 
 Review comment:
   It seems that on Python 2 this change translates to changing the 
`element_type` type from `unicode` to `bytes`/`str` in line 176. 
   @chamikaramj Do you have an intuition if this could bite us?
   CC: @altay


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131153)
Time Spent: 6h 20m  (was: 6h 10m)

> Futurize and fix python 2 compatibility for io subpackage
> -
>
> Key: BEAM-4000
> URL: https://issues.apache.org/jira/browse/BEAM-4000
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Robbe
>Assignee: Matthias Feys
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 6h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostCommit_Java_GradleBuild #1172

[Apache Jenkins Server]

See 


--
[...truncated 19.68 MB...]
INFO: 2018-08-04T00:47:01.144Z: Fusing consumer SpannerIO.Write/Write 
mutations to Cloud Spanner/Keys sample as 
view/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into 
SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as 
view/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Aug 04, 2018 12:47:07 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-08-04T00:47:01.184Z: Fusing consumer SpannerIO.Write/Write 
mutations to Cloud Spanner/Keys sample as 
view/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into 
SpannerIO.Write/Write mutations to Cloud Spanner/Sample 
keys/Combine.GroupedValues/Extract
Aug 04, 2018 12:47:07 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-08-04T00:47:01.222Z: Fusing consumer SpannerIO.Write/Write 
mutations to Cloud Spanner/Sample keys/GroupByKey/Write into 
SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/GroupByKey/Reify
Aug 04, 2018 12:47:07 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-08-04T00:47:01.260Z: Fusing consumer SpannerIO.Write/Write 
mutations to Cloud Spanner/Sample keys/Combine.GroupedValues/Extract into 
SpannerIO.Write/Write mutations to Cloud Spanner/Sample 
keys/Combine.GroupedValues
Aug 04, 2018 12:47:07 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-08-04T00:47:01.298Z: Fusing consumer SpannerIO.Write/Write 
mutations to Cloud Spanner/Sample keys/GroupByKey/Reify into 
SpannerIO.Write/Write mutations to Cloud Spanner/Sample 
keys/GroupByKey+SpannerIO.Write/Write mutations to Cloud Spanner/Sample 
keys/Combine.GroupedValues/Partial
Aug 04, 2018 12:47:07 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-08-04T00:47:01.335Z: Fusing consumer SpannerIO.Write/Write 
mutations to Cloud Spanner/Sample keys/GroupByKey+SpannerIO.Write/Write 
mutations to Cloud Spanner/Sample keys/Combine.GroupedValues/Partial into 
SpannerIO.Write/Write mutations to Cloud Spanner/Extract keys
Aug 04, 2018 12:47:07 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-08-04T00:47:01.373Z: Fusing consumer SpannerIO.Write/Write 
mutations to Cloud Spanner/Wait.OnSignal/To wait view 
0/Sample.Any/Combine.globally(SampleAny)/Combine.perKey(SampleAny)/GroupByKey/Write
 into SpannerIO.Write/Write mutations to Cloud Spanner/Wait.OnSignal/To wait 
view 
0/Sample.Any/Combine.globally(SampleAny)/Combine.perKey(SampleAny)/GroupByKey/Reify
Aug 04, 2018 12:47:07 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-08-04T00:47:01.406Z: Fusing consumer SpannerIO.Write/Write 
mutations to Cloud Spanner/Keys sample as view/GBKaSVForSize/Write into 
SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as 
view/ParMultiDo(ToIsmRecordForMapLike)
Aug 04, 2018 12:47:07 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-08-04T00:47:01.447Z: Fusing consumer SpannerIO.Write/Write 
mutations to Cloud Spanner/Wait.OnSignal/To wait view 
0/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into SpannerIO.Write/Write 
mutations to Cloud Spanner/Wait.OnSignal/To wait view 
0/Sample.Any/Flatten.Iterables/FlattenIterables/FlatMap
Aug 04, 2018 12:47:07 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-08-04T00:47:01.470Z: Fusing consumer SpannerIO.Write/Write 
mutations to Cloud Spanner/Keys sample as view/GBKaSVForKeys/Write into 
SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as 
view/ParMultiDo(ToIsmRecordForMapLike)
Aug 04, 2018 12:47:07 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-08-04T00:47:01.499Z: Fusing consumer SpannerIO.Write/Write 
mutations to Cloud Spanner/Wait.OnSignal/To wait view 
0/Sample.Any/Flatten.Iterables/FlattenIterables/FlatMap into 
SpannerIO.Write/Write mutations to Cloud Spanner/Wait.OnSignal/To wait view 
0/Sample.Any/Combine.globally(SampleAny)/Values/Values/Map
Aug 04, 2018 12:47:07 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-08-04T00:47:01.536Z: Fusing consumer SpannerIO.Write/Write 
mutations to Cloud Spanner/Wait.OnSignal/To wait view 
0/Sample.Any/Combine.globally(SampleAny)/Values/Values/Map into 
SpannerIO.Write/Write mutations to Cloud Spanner/Wait.OnSignal/To wait view 
0/Sample.Any/Combine.globally(SampleAny)/Combine.perKey(SampleAny)/Combine.GroupedValues/Extract
Aug 04, 2018 12:47:07 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 

Jenkins build is back to normal : beam_PostCommit_Java_Nexmark_Spark #188

[Apache Jenkins Server]

See 

Jenkins build is back to normal : beam_PostCommit_Java_Nexmark_Flink #189

[Apache Jenkins Server]

See 

Build failed in Jenkins: beam_PerformanceTests_Compressed_TextIOIT #817

[Apache Jenkins Server]

See 


Changes:

[yifanzou] [BEAM-4963] update org.apache.httpcomponents

[amaliujia] BEAM-4808: add e2e test for BeamSqlLine.

[yifanzou] [BEAM-4906] upgrade org.apache.derby

[relax] Add Select and Filter transforms for schema types.

[relax] Add missing Experimental flag.

[apilloud] [BEAM-4804] Use beam-perf label for local perf tests

[gleb] [BEAM-5056] [SQL] Fix nullability in output schema

[valentyn] Use the same language semantics as defined by version of the 
interpreter

[relax] Address code-review comments.

[aaltay] [BEAM-4006] Futurize transforms subpackage (#5729)

[apilloud] [BEAM-4761] Run Nexmark in thread pool

[aaltay] Automate "Run validation tests" step (#6081)

[apilloud] [BEAM-4761] Handle skipped tests in parallel Nexmark

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 2d3ce5181ad144de9c5cc6f945a6d40f9c0ae795 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 2d3ce5181ad144de9c5cc6f945a6d40f9c0ae795
Commit message: "Merge pull request #6140 from apilloud/parallel-nexmark"
 > git rev-list --no-walk e8abafe360e126818fe80ae0f6075e71f0fc227d # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins2491955978350748721.sh
+ rm -rf 

[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins8516885899891596596.sh
+ rm -rf 

[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins3034736620175587348.sh
+ virtualenv 

 --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins2924464321377298061.sh
+ virtualenv 

 --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins4701011671469182578.sh
+ 

 install --upgrade setuptools pip grpcio-tools==1.3.5
Requirement already up-to-date: setuptools in 
./src/.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in 
./src/.beam_env/lib/python2.7/site-packages (18.0)
Collecting grpcio-tools==1.3.5
  Using cached 
https://files.pythonhosted.org/packages/05/f6/0296e29b1bac6f85d2a8556d48adf825307f73109a3c2c17fb734292db0a/grpcio_tools-1.3.5-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied, skipping upgrade: grpcio>=1.3.5 in 
/home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) 
(1.13.0)
Requirement already satisfied, skipping upgrade: protobuf>=3.2.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) 
(3.6.0)
Requirement already satisfied, skipping upgrade: six>=1.5.2 in 

Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #830

[Apache Jenkins Server]

See 


Changes:

[yifanzou] [BEAM-4963] update org.apache.httpcomponents

[amaliujia] BEAM-4808: add e2e test for BeamSqlLine.

[yifanzou] [BEAM-4906] upgrade org.apache.derby

[relax] Add Select and Filter transforms for schema types.

[relax] Add missing Experimental flag.

[apilloud] [BEAM-4804] Use beam-perf label for local perf tests

[gleb] [BEAM-5056] [SQL] Fix nullability in output schema

[valentyn] Use the same language semantics as defined by version of the 
interpreter

[relax] Address code-review comments.

[aaltay] [BEAM-4006] Futurize transforms subpackage (#5729)

[apilloud] [BEAM-4761] Run Nexmark in thread pool

[aaltay] Automate "Run validation tests" step (#6081)

[apilloud] [BEAM-4761] Handle skipped tests in parallel Nexmark

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam14 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 2d3ce5181ad144de9c5cc6f945a6d40f9c0ae795 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 2d3ce5181ad144de9c5cc6f945a6d40f9c0ae795
Commit message: "Merge pull request #6140 from apilloud/parallel-nexmark"
 > git rev-list --no-walk e8abafe360e126818fe80ae0f6075e71f0fc227d # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins8553005361432005655.sh
+ rm -rf 

[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins683668545266251496.sh
+ rm -rf 

[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins5995534741714420800.sh
+ virtualenv 
 
--system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins158478919711646949.sh
+ virtualenv 

 --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins9052118540087899599.sh
+ 

 install --upgrade setuptools pip grpcio-tools==1.3.5
Requirement already up-to-date: setuptools in 
./src/.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in 
./src/.beam_env/lib/python2.7/site-packages (18.0)
Collecting grpcio-tools==1.3.5
  Using cached 
https://files.pythonhosted.org/packages/05/f6/0296e29b1bac6f85d2a8556d48adf825307f73109a3c2c17fb734292db0a/grpcio_tools-1.3.5-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied, skipping upgrade: grpcio>=1.3.5 in 
/home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) 
(1.13.0)
Requirement already satisfied, skipping upgrade: protobuf>=3.2.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) 
(3.6.0)
Requirement already satisfied, skipping upgrade: six>=1.5.2 in 
/usr/local/lib/python2.7/dist-packages (from 
grpcio>=1.3.5->grpcio-tools==1.3.5) (1.11.0)
Requirement already satisfied, skipping upgrade: enum34>=1.0.4 in 

Build failed in Jenkins: beam_PerformanceTests_Python #1266

[Apache Jenkins Server]

See 


--
GitHub pull request #6142 of commit c48115558d7af37b518f69463dba5bd7612054a5, 
no merge conflicts.
Setting status of c48115558d7af37b518f69463dba5bd7612054a5 to PENDING with url 
https://builds.apache.org/job/beam_PerformanceTests_Python/1266/ and message: 
'Build started for merge commit.'
Using context: Python SDK Performance Test
[EnvInject] - Loading node environment variables.
Building remotely on beam15 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/6142/*:refs/remotes/origin/pr/6142/*
 > git rev-parse refs/remotes/origin/pr/6142/merge^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/pr/6142/merge^{commit} # timeout=10
Checking out Revision abc1bce29360cd14fc5fb6f62c1c52193d112fa4 
(refs/remotes/origin/pr/6142/merge)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f abc1bce29360cd14fc5fb6f62c1c52193d112fa4
Commit message: "Merge c48115558d7af37b518f69463dba5bd7612054a5 into 
2d3ce5181ad144de9c5cc6f945a6d40f9c0ae795"
First time build. Skipping changelog.
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins806446748541054.sh
+ rm -rf 

[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins3957460423414083293.sh
+ rm -rf 

[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins2172882893396186781.sh
+ virtualenv 
 
--system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins5967812430427483923.sh
+ virtualenv 

 --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins5622511714784926622.sh
+ 

 install --upgrade setuptools pip grpcio-tools==1.3.5
Requirement already up-to-date: setuptools in 
./src/.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in 
./src/.beam_env/lib/python2.7/site-packages (18.0)
Collecting grpcio-tools==1.3.5
  Using cached 
https://files.pythonhosted.org/packages/05/f6/0296e29b1bac6f85d2a8556d48adf825307f73109a3c2c17fb734292db0a/grpcio_tools-1.3.5-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied, skipping upgrade: grpcio>=1.3.5 in 
/home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) 
(1.13.0)
Requirement already satisfied, skipping upgrade: protobuf>=3.2.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) 
(3.6.0)
Requirement already satisfied, skipping upgrade: six>=1.5.2 in 
/usr/local/lib/python2.7/dist-packages (from 
grpcio>=1.3.5->grpcio-tools==1.3.5) (1.11.0)
Requirement already satisfied, skipping upgrade: enum34>=1.0.4 in 
/usr/local/lib/python2.7/dist-packages (from 
grpcio>=1.3.5->grpcio-tools==1.3.5) (1.1.6)
Requirement already satisfied, skipping upgrade: futures>=2.2.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
grpcio>=1.3.5->grpcio-tools==1.3.5) (3.2.0)
Installing collected packages: grpcio-tools
Successfully installed grpcio-tools-1.3.5
[beam_PerformanceTests_Python] $ /bin/bash -xe 

Build failed in Jenkins: beam_PerformanceTests_HadoopInputFormat #600

[Apache Jenkins Server]

See 


Changes:

[yifanzou] [BEAM-4963] update org.apache.httpcomponents

[amaliujia] BEAM-4808: add e2e test for BeamSqlLine.

[yifanzou] [BEAM-4906] upgrade org.apache.derby

[relax] Add Select and Filter transforms for schema types.

[relax] Add missing Experimental flag.

[apilloud] [BEAM-4804] Use beam-perf label for local perf tests

[gleb] [BEAM-5056] [SQL] Fix nullability in output schema

[valentyn] Use the same language semantics as defined by version of the 
interpreter

[relax] Address code-review comments.

[aaltay] [BEAM-4006] Futurize transforms subpackage (#5729)

[apilloud] [BEAM-4761] Run Nexmark in thread pool

[aaltay] Automate "Run validation tests" step (#6081)

[apilloud] [BEAM-4761] Handle skipped tests in parallel Nexmark

--
[...truncated 448.20 KB...]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.postgresql.util.PSQLException: 
The connection attempt failed.
at 
org.apache.hadoop.mapreduce.lib.db.DBInputFormat.createConnection(DBInputFormat.java:205)
at 
org.apache.hadoop.mapreduce.lib.db.DBInputFormat.setConf(DBInputFormat.java:164)
... 18 more
Caused by: org.postgresql.util.PSQLException: The connection attempt failed.
at 
org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:257)
at 
org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.(PgConnection.java:195)
at org.postgresql.Driver.makeConnection(Driver.java:452)
at org.postgresql.Driver.connect(Driver.java:254)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at 
org.apache.hadoop.mapreduce.lib.db.DBConfiguration.getConnection(DBConfiguration.java:154)
at 
org.apache.hadoop.mapreduce.lib.db.DBInputFormat.createConnection(DBInputFormat.java:198)
... 19 more
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.postgresql.core.PGStream.(PGStream.java:69)
at 
org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:156)
... 27 more
java.lang.RuntimeException: java.lang.RuntimeException: 
org.postgresql.util.PSQLException: The connection attempt failed.
at 
org.apache.hadoop.mapreduce.lib.db.DBInputFormat.setConf(DBInputFormat.java:171)
at 
org.apache.beam.sdk.io.hadoop.inputformat.HadoopInputFormatIO$HadoopInputFormatBoundedSource.createInputFormatInstance(HadoopInputFormatIO.java:571)
at 
org.apache.beam.sdk.io.hadoop.inputformat.HadoopInputFormatIO$HadoopInputFormatBoundedSource.computeSplitsIfNecessary(HadoopInputFormatIO.java:527)
at 
org.apache.beam.sdk.io.hadoop.inputformat.HadoopInputFormatIO$HadoopInputFormatBoundedSource.split(HadoopInputFormatIO.java:487)
at 
com.google.cloud.dataflow.worker.WorkerCustomSources.splitAndValidate(WorkerCustomSources.java:275)
at 
com.google.cloud.dataflow.worker.WorkerCustomSources.performSplitTyped(WorkerCustomSources.java:197)
at 
com.google.cloud.dataflow.worker.WorkerCustomSources.performSplitWithApiLimit(WorkerCustomSources.java:181)
at 
com.google.cloud.dataflow.worker.WorkerCustomSources.performSplit(WorkerCustomSources.java:160)
at 
com.google.cloud.dataflow.worker.WorkerCustomSourceOperationExecutor.execute(WorkerCustomSourceOperationExecutor.java:77)
at 
com.google.cloud.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:393)
at 
com.google.cloud.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:362)
at 
com.google.cloud.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:290)
at 
com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:134)
at 
com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:114)
at 

Build failed in Jenkins: beam_PreCommit_Java_Cron #183

[Apache Jenkins Server]

See 


Changes:

[yifanzou] [BEAM-4963] update org.apache.httpcomponents

[amaliujia] BEAM-4808: add e2e test for BeamSqlLine.

[garrettjonesgoogle] Switching to correct datastore proto package

[yifanzou] [BEAM-4906] upgrade org.apache.derby

[relax] Add Select and Filter transforms for schema types.

[relax] Add missing Experimental flag.

[apilloud] [BEAM-4804] Use beam-perf label for local perf tests

[gleb] [BEAM-5056] [SQL] Fix nullability in output schema

[valentyn] Use the same language semantics as defined by version of the 
interpreter

[relax] Address code-review comments.

[aaltay] [BEAM-4006] Futurize transforms subpackage (#5729)

[apilloud] [BEAM-4761] Run Nexmark in thread pool

[aaltay] Automate "Run validation tests" step (#6081)

--
[...truncated 17.46 MB...]
LogicalProject(auction=[$0], num=[$1])
  LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
  LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000, 1)])
  BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
  LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$0])
  LogicalProject(num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
  LogicalProject(auction=[$0], $f1=[HOP($3, 5000, 1)])
BeamIOSourceRel(table=[[beam, Bid]])

Aug 04, 2018 12:21:42 AM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
  BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], 
starttime=[$t1])
  BeamAggregationRel(group=[{0, 1}], num=[COUNT()])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[5000], expr#6=[1], 
expr#7=[HOP($t3, $t5, $t6)], auction=[$t0], $f1=[$t7])
  BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
  BeamAggregationRel(group=[{1}], maxnum=[MAX($0)])
BeamCalcRel(expr#0..2=[{inputs}], num=[$t2], starttime=[$t1])
  BeamAggregationRel(group=[{0, 1}], num=[COUNT()])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[5000], 
expr#6=[1], expr#7=[HOP($t3, $t5, $t6)], auction=[$t0], $f1=[$t7])
  BeamIOSourceRel(table=[[beam, Bid]])


org.apache.beam.sdk.nexmark.queries.sql.SqlQuery3Test > 
testJoinsPeopleWithAuctions STANDARD_ERROR
Aug 04, 2018 12:21:43 AM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `P`.`name`, `P`.`city`, `P`.`state`, `A`.`id`
FROM `beam`.`Auction` AS `A`
INNER JOIN `beam`.`Person` AS `P` ON `A`.`seller` = `P`.`id`
WHERE `A`.`category` = 10 AND (`P`.`state` = 'OR' OR `P`.`state` = 'ID' OR 
`P`.`state` = 'CA')
Aug 04, 2018 12:21:43 AM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(name=[$11], city=[$14], state=[$15], id=[$0])
  LogicalFilter(condition=[AND(=($8, 10), OR(=($15, 'OR'), =($15, 'ID'), 
=($15, 'CA')))])
LogicalJoin(condition=[=($7, $10)], joinType=[inner])
  BeamIOSourceRel(table=[[beam, Auction]])
  BeamIOSourceRel(table=[[beam, Person]])

Aug 04, 2018 12:21:43 AM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..17=[{inputs}], name=[$t11], city=[$t14], state=[$t15], 
id=[$t0])
  BeamJoinRel(condition=[=($7, $10)], joinType=[inner])
BeamCalcRel(expr#0..9=[{inputs}], expr#10=[10], expr#11=[=($t8, $t10)], 
proj#0..9=[{exprs}], $condition=[$t11])
  BeamIOSourceRel(table=[[beam, Auction]])
BeamCalcRel(expr#0..7=[{inputs}], expr#8=['OR'], expr#9=[=($t5, $t8)], 
expr#10=['ID'], expr#11=[=($t5, $t10)], expr#12=['CA'], expr#13=[=($t5, $t12)], 
expr#14=[OR($t9, $t11, $t13)], proj#0..7=[{exprs}], $condition=[$t14])
  BeamIOSourceRel(table=[[beam, Person]])


org.apache.beam.sdk.nexmark.queries.sql.SqlQuery7Test > testBids STANDARD_ERROR
Aug 04, 2018 12:21:43 AM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, 
`B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, 
`B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS 

[jira] [Commented] (BEAM-4148) Local server api descriptors contain urls that work on Mac and Linux

[Ben Sidhom (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568949#comment-16568949
 ] 

Ben Sidhom commented on BEAM-4148:
--

Basically, if the reference runner is manually creating its own Docker 
instances, it should stop doing that and start using the Docker factory.

> Local server api descriptors contain urls that work on Mac and Linux
> 
>
> Key: BEAM-4148
> URL: https://issues.apache.org/jira/browse/BEAM-4148
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-core
>Reporter: Ben Sidhom
>Priority: Minor
>
> Docker for Mac does not allow host networking and thus will not allow SDK 
> harnesses to access runner services via `localhost`. Instead, a special DNS 
> name is used to refer to the host machine: docker.for.mac.host.internal. 
> (Note that this value sometimes changes between Docker releases).
> We should attempt to detect the host operating system and return different 
> API descriptors based on this.
> See 
> [https://github.com/bsidhom/beam/commit/3adaeb0d33dc26f0910c1f8af2821cce4ee0b965]
>  for how this might be done.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-4148) Local server api descriptors contain urls that work on Mac and Linux

[Ben Sidhom (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568948#comment-16568948
 ] 

Ben Sidhom commented on BEAM-4148:
--

The linked PR should have actually solved this for the Docker environment 
factory (https://github.com/apache/beam/pull/5392). I actually meant to close 
this since this is ideally the only place where Docker environments will be 
created.

 

Where are you still seeing this?

> Local server api descriptors contain urls that work on Mac and Linux
> 
>
> Key: BEAM-4148
> URL: https://issues.apache.org/jira/browse/BEAM-4148
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-core
>Reporter: Ben Sidhom
>Priority: Minor
>
> Docker for Mac does not allow host networking and thus will not allow SDK 
> harnesses to access runner services via `localhost`. Instead, a special DNS 
> name is used to refer to the host machine: docker.for.mac.host.internal. 
> (Note that this value sometimes changes between Docker releases).
> We should attempt to detect the host operating system and return different 
> API descriptors based on this.
> See 
> [https://github.com/bsidhom/beam/commit/3adaeb0d33dc26f0910c1f8af2821cce4ee0b965]
>  for how this might be done.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4000) Futurize and fix python 2 compatibility for io subpackage

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4000?focusedWorklogId=131152=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131152
 ]

ASF GitHub Bot logged work on BEAM-4000:


Author: ASF GitHub Bot
Created on: 04/Aug/18 00:18
Start Date: 04/Aug/18 00:18
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on a change in pull request #5715: 
[BEAM-4000] Futurize io subpackage
URL: https://github.com/apache/beam/pull/5715#discussion_r207692947
 
 

 ##
 File path: sdks/python/apache_beam/io/gcp/pubsub.py
 ##
 @@ -33,8 +33,9 @@
 from __future__ import absolute_import
 
 import re
+from builtins import object
 
-from six import text_type
+from past.builtins import basestring
 
 Review comment:
   @Fematich 
   Do you remember, why did we decide to use `past.builtins.basestring` here 
instead of `past.builtins.unicode`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131152)
Time Spent: 6h 10m  (was: 6h)

> Futurize and fix python 2 compatibility for io subpackage
> -
>
> Key: BEAM-4000
> URL: https://issues.apache.org/jira/browse/BEAM-4000
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Robbe
>Assignee: Matthias Feys
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #190

[Apache Jenkins Server]

See 


Changes:

[yifanzou] [BEAM-4963] update org.apache.httpcomponents

[yifanzou] [BEAM-4906] upgrade org.apache.derby

[apilloud] [BEAM-4761] Run Nexmark in thread pool

[aaltay] Automate "Run validation tests" step (#6081)

--
[...truncated 673.03 KB...]
    24.6  4065.0   92000
  
  0001  20.4  4895.2   92000
  
  0002  14.1  7077.6 351
  
  0003  *** not run ***
  0004  *** not run ***
  0005  *** not run ***
  0006  *** not run ***
  0007  *** not run ***
  0008  *** not run ***
  0009  *** not run ***
  0010  *** not run ***
  0011  *** not run ***
  0012  *** not run ***
==

2018-08-04T00:14:36.864Z Generating 10 events in batch mode
Aug 04, 2018 12:14:36 AM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `P`.`name`, `P`.`city`, `P`.`state`, `A`.`id`
FROM `beam`.`Auction` AS `A`
INNER JOIN `beam`.`Person` AS `P` ON `A`.`seller` = `P`.`id`
WHERE `A`.`category` = 10 AND (`P`.`state` = 'OR' OR `P`.`state` = 'ID' OR 
`P`.`state` = 'CA')
Aug 04, 2018 12:14:36 AM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(name=[$11], city=[$14], state=[$15], id=[$0])
  LogicalFilter(condition=[AND(=($8, 10), OR(=($15, 'OR'), =($15, 'ID'), =($15, 
'CA')))])
LogicalJoin(condition=[=($7, $10)], joinType=[inner])
  BeamIOSourceRel(table=[[beam, Auction]])
  BeamIOSourceRel(table=[[beam, Person]])

Aug 04, 2018 12:14:37 AM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..17=[{inputs}], name=[$t11], city=[$t14], state=[$t15], 
id=[$t0])
  BeamJoinRel(condition=[=($7, $10)], joinType=[inner])
BeamCalcRel(expr#0..9=[{inputs}], expr#10=[10], expr#11=[=($t8, $t10)], 
proj#0..9=[{exprs}], $condition=[$t11])
  BeamIOSourceRel(table=[[beam, Auction]])
BeamCalcRel(expr#0..7=[{inputs}], expr#8=['OR'], expr#9=[=($t5, $t8)], 
expr#10=['ID'], expr#11=[=($t5, $t10)], expr#12=['CA'], expr#13=[=($t5, $t12)], 
expr#14=[OR($t9, $t11, $t13)], proj#0..7=[{exprs}], $condition=[$t14])
  BeamIOSourceRel(table=[[beam, Person]])

2018-08-04T00:14:51.883Z Waiting for main pipeline to 'finish'
2018-08-04T00:14:51.883Z DONE SqlQuery3
2018-08-04T00:14:51.884Z Running query:4; exportSummaryToBigQuery:true; 
numEvents:1
2018-08-04T00:14:51.884Z skipping since configuration is not implemented
2018-08-04T00:14:51.884Z Running query:5; exportSummaryToBigQuery:true

==
Run started 2018-08-04T00:13:31.860Z and ran for PT80.024S

Default configuration:
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":10,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":1,"nextEventRate":1,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":10,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}

Configurations:
  Conf  Description
    query:0; exportSummaryToBigQuery:true
  0001  query:1; exportSummaryToBigQuery:true
  0002  query:2; exportSummaryToBigQuery:true
  0003  query:3; exportSummaryToBigQuery:true
  0004  query:4; exportSummaryToBigQuery:true; numEvents:1
  0005  query:5; exportSummaryToBigQuery:true
  0006  query:6; exportSummaryToBigQuery:true; numEvents:1
  0007  query:7; exportSummaryToBigQuery:true
  0008  query:8; exportSummaryToBigQuery:true
  0009  query:9; exportSummaryToBigQuery:true; numEvents:1
  0010  query:10; exportSummaryToBigQuery:true
  0011  query:11; exportSummaryToBigQuery:true
  0012  query:12; exportSummaryToBigQuery:true

Performance:
  Conf  Runtime(sec)(Baseline)  Events(/sec)(Baseline)   Results
(Baseline)
    24.6  4065.0   92000
  
  0001  20.4  4895.2   92000
  
  0002  14.1  7077.6 351
  
  0003  14.5 

[jira] [Work logged] (BEAM-4176) Java: Portable batch runner passes all ValidatesRunner tests that non-portable runner passes

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4176?focusedWorklogId=131150=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131150
 ]

ASF GitHub Bot logged work on BEAM-4176:


Author: ASF GitHub Bot
Created on: 04/Aug/18 00:08
Start Date: 04/Aug/18 00:08
Worklog Time Spent: 10m 
  Work Description: youngoli commented on a change in pull request #6073: 
[BEAM-4176] Validate Runner Tests generalization and enable for local reference 
runner
URL: https://github.com/apache/beam/pull/6073#discussion_r207692273
 
 

 ##
 File path: 
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
 ##
 @@ -1389,5 +1405,42 @@ artifactId=${project.name}
 args argsNeeded
   }
 }
+
+
+/** 
***/
+
+// Method to create the PortableValidatesRunnerTask.
+project.ext.createPortableValidatesRunnerTask = {
 
 Review comment:
   Comment sounds good to me.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131150)
Time Spent: 10h 20m  (was: 10h 10m)

> Java: Portable batch runner passes all ValidatesRunner tests that 
> non-portable runner passes
> 
>
> Key: BEAM-4176
> URL: https://issues.apache.org/jira/browse/BEAM-4176
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Ben Sidhom
>Priority: Major
>  Time Spent: 10h 20m
>  Remaining Estimate: 0h
>
> We need this as a sanity check that runner execution is correct.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4679) Support portable combiner lifting in Java Reference Runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4679?focusedWorklogId=131149=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131149
 ]

ASF GitHub Bot logged work on BEAM-4679:


Author: ASF GitHub Bot
Created on: 04/Aug/18 00:07
Start Date: 04/Aug/18 00:07
Worklog Time Spent: 10m 
  Work Description: youngoli commented on issue #6141: [BEAM-4679] 
Re-adding Flatten roots in ULR.
URL: https://github.com/apache/beam/pull/6141#issuecomment-410405236
 
 
   R: @angoenka 
   This fixes a few of the failing tests that popped up in the ValidatesRunner 
tests for the ULR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131149)
Time Spent: 20m  (was: 10m)

> Support portable combiner lifting in Java Reference Runner
> --
>
> Key: BEAM-4679
> URL: https://issues.apache.org/jira/browse/BEAM-4679
> Project: Beam
>  Issue Type: Sub-task
>  Components: runner-direct
>Reporter: Daniel Oliveira
>Assignee: Daniel Oliveira
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Adjust the Reference Runner in Java to support portable combiner lifting as 
> described in the following doc:
> https://s.apache.org/beam-runner-api-combine-model



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam] 01/01: Merge pull request #6140 from apilloud/parallel-nexmark

[anton]

This is an automated email from the ASF dual-hosted git repository.

anton pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 2d3ce5181ad144de9c5cc6f945a6d40f9c0ae795
Merge: 4cb7dab b00a03d
Author: Anton Kedin <33067037+ake...@users.noreply.github.com>
AuthorDate: Fri Aug 3 17:06:10 2018 -0700

Merge pull request #6140 from apilloud/parallel-nexmark

[BEAM-4761] Handle skipped tests in parallel Nexmark

 sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131148=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131148
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 04/Aug/18 00:06
Start Date: 04/Aug/18 00:06
Worklog Time Spent: 10m 
  Work Description: akedin closed pull request #6140: [BEAM-4761] Handle 
skipped tests in parallel Nexmark
URL: https://github.com/apache/beam/pull/6140
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java 
b/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
index ecd7b5b9bf6..5aa68be7ea3 100644
--- a/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
+++ b/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
@@ -147,7 +147,9 @@ void runAll(String[] args) throws IOException {
 
 NexmarkConfiguration configuration = result.configuration;
 NexmarkPerf perf = result.perf;
-if (perf.errors == null || perf.errors.size() > 0) {
+if (perf == null) {
+  continue;
+} else if (perf.errors == null || perf.errors.size() > 0) {
   successful = false;
 }
 appendPerf(options.getPerfFilename(), configuration, perf);


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131148)
Time Spent: 8h 40m  (was: 8.5h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 8h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam] branch master updated (4cb7dab -> 2d3ce51)

[anton]

This is an automated email from the ASF dual-hosted git repository.

anton pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 4cb7dab  Merge pull request #6115: [BEAM-4936] update 
org.apache.httpcomponents
 add b00a03d  [BEAM-4761] Handle skipped tests in parallel Nexmark
 new 2d3ce51  Merge pull request #6140 from apilloud/parallel-nexmark

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

[jira] [Work logged] (BEAM-4679) Support portable combiner lifting in Java Reference Runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4679?focusedWorklogId=131147=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131147
 ]

ASF GitHub Bot logged work on BEAM-4679:


Author: ASF GitHub Bot
Created on: 04/Aug/18 00:05
Start Date: 04/Aug/18 00:05
Worklog Time Spent: 10m 
  Work Description: youngoli opened a new pull request #6141: [BEAM-4679] 
Re-adding Flatten roots in ULR.
URL: https://github.com/apache/beam/pull/6141
 
 
   This was causing issues while testing pipelines with the ULR, so this
   is a quick fix for that.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131147)
Time Spent: 10m
Remaining Estimate: 0h

> Support portable combiner lifting in Java Reference Runner
> --
>
> Key: BEAM-4679
> URL: https://issues.apache.org/jira/browse/BEAM-4679
> Project: Beam
>  Issue Type: Sub-task
>  Components: runner-direct
>Reporter: Daniel Oliveira
>Assignee: Daniel Oliveira
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> 

[jira] [Commented] (BEAM-4148) Local server api descriptors contain urls that work on Mac and Linux

[Daniel Oliveira (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568933#comment-16568933
 ] 

Daniel Oliveira commented on BEAM-4148:
---

I ran into this issue while trying to get Validates Runner tests working for 
the Reference Runner. This issue prevents running those on Macs, and most 
likely will also affect the Flink Validates Runner tests since they use Docker 
similarly. I'm considering taking this bug, but a bit hesitant since I have 
very little experience with Docker. What's missing from your example commit 
that's holding it back from being submitted as a PR?

PS: For reference, I also found this link relating to the issue:
https://docs.docker.com/docker-for-mac/networking/#use-cases-and-workarounds

> Local server api descriptors contain urls that work on Mac and Linux
> 
>
> Key: BEAM-4148
> URL: https://issues.apache.org/jira/browse/BEAM-4148
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-core
>Reporter: Ben Sidhom
>Priority: Minor
>
> Docker for Mac does not allow host networking and thus will not allow SDK 
> harnesses to access runner services via `localhost`. Instead, a special DNS 
> name is used to refer to the host machine: docker.for.mac.host.internal. 
> (Note that this value sometimes changes between Docker releases).
> We should attempt to detect the host operating system and return different 
> API descriptors based on this.
> See 
> [https://github.com/bsidhom/beam/commit/3adaeb0d33dc26f0910c1f8af2821cce4ee0b965]
>  for how this might be done.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131144=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131144
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:52
Start Date: 03/Aug/18 23:52
Worklog Time Spent: 10m 
  Work Description: akedin commented on issue #6140: [BEAM-4761] Handle 
skipped tests in parallel Nexmark
URL: https://github.com/apache/beam/pull/6140#issuecomment-410403739
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131144)
Time Spent: 8.5h  (was: 8h 20m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 8.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131140=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131140
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:33
Start Date: 03/Aug/18 23:33
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #6140: [BEAM-4761] Handle 
skipped tests in parallel Nexmark
URL: https://github.com/apache/beam/pull/6140#issuecomment-410401427
 
 
   @akedin I broke nexmark on SQL by dropping the check for null results (which 
indicates a skipped test). This fixes it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131140)
Time Spent: 8h 20m  (was: 8h 10m)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 8h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5072) SortRelNode is wrongly removed

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5072?focusedWorklogId=131139=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131139
 ]

ASF GitHub Bot logged work on BEAM-5072:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:32
Start Date: 03/Aug/18 23:32
Worklog Time Spent: 10m 
  Work Description: akedin commented on issue #6139: [BEAM-5072] Remove 
RelCollationTraitDef configuration in BeamQueryPlanner
URL: https://github.com/apache/beam/pull/6139#issuecomment-410401401
 
 
   + @XuMingmin @xumingming 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131139)
Time Spent: 1.5h  (was: 1h 20m)

> SortRelNode is wrongly removed
> --
>
> Key: BEAM-5072
> URL: https://issues.apache.org/jira/browse/BEAM-5072
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> ORDER BY query generates LogicalSort in logical plan, but it will be removed 
> wrongly during optimization by the SortRemoveRule



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131138=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131138
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:31
Start Date: 03/Aug/18 23:31
Worklog Time Spent: 10m 
  Work Description: apilloud opened a new pull request #6140: [BEAM-4761] 
Handle skipped tests in parallel Nexmark
URL: https://github.com/apache/beam/pull/6140
 
 
   I dropped this check in refactoring, it breaks Nexmark on SQL, which skips 
some tests.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [x] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131138)
Time Spent: 8h 10m  (was: 8h)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 8h 10m
>  Remaining Estimate: 

[jira] [Commented] (BEAM-4908) Beam Dependency Update Request: org.apache.derby:derbyclient 10.14.2.0

[yifan zou (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568916#comment-16568916
 ] 

yifan zou commented on BEAM-4908:
-

https://github.com/apache/beam/pull/6129

> Beam Dependency Update Request: org.apache.derby:derbyclient 10.14.2.0
> --
>
> Key: BEAM-4908
> URL: https://issues.apache.org/jira/browse/BEAM-4908
> Project: Beam
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.7.0
>
>
> 2018-07-25 20:24:15.830071
> Please review and upgrade the org.apache.derby:derbyclient to the 
> latest version 10.14.2.0 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-4907) Beam Dependency Update Request: org.apache.derby:derby 10.14.2.0

[yifan zou (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568915#comment-16568915
 ] 

yifan zou commented on BEAM-4907:
-

https://github.com/apache/beam/pull/6129

> Beam Dependency Update Request: org.apache.derby:derby 10.14.2.0
> 
>
> Key: BEAM-4907
> URL: https://issues.apache.org/jira/browse/BEAM-4907
> Project: Beam
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.7.0
>
>
> 2018-07-25 20:24:07.339681
> Please review and upgrade the org.apache.derby:derby to the latest 
> version 10.14.2.0 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5041) Java Fn SDK Harness skips unprocessed pCollections

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5041?focusedWorklogId=131136=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131136
 ]

ASF GitHub Bot logged work on BEAM-5041:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:29
Start Date: 03/Aug/18 23:29
Worklog Time Spent: 10m 
  Work Description: bsidhom commented on a change in pull request #6093: 
[BEAM-5041] Java Fn SDK Harness use pTransform to track processed graph
URL: https://github.com/apache/beam/pull/6093#discussion_r207688736
 
 

 ##
 File path: 
sdks/java/harness/src/main/java/org/apache/beam/fn/harness/control/ProcessBundleHandler.java
 ##
 @@ -185,23 +184,26 @@ private void 
createRunnerAndConsumersForPTransformRecursively(
   String.format(
   "Cannot process composite transform: %s", 
TextFormat.printToString(pTransform)));
 }
-
-urnToPTransformRunnerFactoryMap
-.getOrDefault(pTransform.getSpec().getUrn(), 
defaultPTransformRunnerFactory)
-.createRunnerForPTransform(
-options,
-beamFnDataClient,
-beamFnStateClient,
-pTransformId,
-pTransform,
-processBundleInstructionId,
-processBundleDescriptor.getPcollectionsMap(),
-processBundleDescriptor.getCodersMap(),
-processBundleDescriptor.getWindowingStrategiesMap(),
-pCollectionIdsToConsumers,
-addStartFunction,
-addFinishFunction,
-splitListener);
+// Skip reprocessing processed pTransforms.
+if (!processedPTransformIds.contains(pTransformId)) {
+  urnToPTransformRunnerFactoryMap
+  .getOrDefault(pTransform.getSpec().getUrn(), 
defaultPTransformRunnerFactory)
+  .createRunnerForPTransform(
+  options,
+  beamFnDataClient,
+  beamFnStateClient,
+  pTransformId,
+  pTransform,
+  processBundleInstructionId,
+  processBundleDescriptor.getPcollectionsMap(),
+  processBundleDescriptor.getCodersMap(),
+  processBundleDescriptor.getWindowingStrategiesMap(),
+  pCollectionIdsToConsumers,
 
 Review comment:
   Sorry, I meant when were elements added?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131136)
Time Spent: 1h 20m  (was: 1h 10m)

> Java Fn SDK Harness skips unprocessed pCollections
> --
>
> Key: BEAM-5041
> URL: https://issues.apache.org/jira/browse/BEAM-5041
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-harness
>Reporter: Ankur Goenka
>Assignee: Ankur Goenka
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Java Sdk Harness used pCollections to keep track of computed consumers 
> [here|https://github.com/apache/beam/blob/ff95a82e461bd8319d9733be60e75992ba90cd7c/sdks/java/harness/src/main/java/org/apache/beam/fn/harness/control/ProcessBundleHandler.java#L158].
>  This is incorrect as consumers are based on pTransforms so pTransforms 
> should be used to keep track of computed consumers.
> In case of Flatten, this creates an issue where pTransforms having same input 
> as that to flatten are not executed. This causes 
> [https://github.com/apache/beam/blob/ff95a82e461bd8319d9733be60e75992ba90cd7c/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/FlattenTest.java#L316]
>  to fail.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5072) SortRelNode is wrongly removed

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5072?focusedWorklogId=131137=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131137
 ]

ASF GitHub Bot logged work on BEAM-5072:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:29
Start Date: 03/Aug/18 23:29
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #6139: [BEAM-5072] Remove 
RelCollationTraitDef configuration in BeamQueryPlanner
URL: https://github.com/apache/beam/pull/6139#issuecomment-410401045
 
 
   Green


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131137)
Time Spent: 1h 20m  (was: 1h 10m)

> SortRelNode is wrongly removed
> --
>
> Key: BEAM-5072
> URL: https://issues.apache.org/jira/browse/BEAM-5072
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> ORDER BY query generates LogicalSort in logical plan, but it will be removed 
> wrongly during optimization by the SortRemoveRule



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-4909) Beam Dependency Update Request: org.apache.derby:derbynet 10.14.2.0

[yifan zou (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568917#comment-16568917
 ] 

yifan zou commented on BEAM-4909:
-

https://github.com/apache/beam/pull/6129

> Beam Dependency Update Request: org.apache.derby:derbynet 10.14.2.0
> ---
>
> Key: BEAM-4909
> URL: https://issues.apache.org/jira/browse/BEAM-4909
> Project: Beam
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.7.0
>
>
> 2018-07-25 20:24:23.915649
> Please review and upgrade the org.apache.derby:derbynet to the latest 
> version 10.14.2.0 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4967) Beam Dependency Update Request: org.apache.httpcomponents:httpcore-nio 4.4.10

[yifan zou (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yifan zou resolved BEAM-4967.
-
   Resolution: Fixed
Fix Version/s: 2.7.0

https://github.com/apache/beam/pull/6115

> Beam Dependency Update Request: org.apache.httpcomponents:httpcore-nio 4.4.10
> -
>
> Key: BEAM-4967
> URL: https://issues.apache.org/jira/browse/BEAM-4967
> Project: Beam
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.7.0
>
>
> 2018-07-25 20:30:18.840197
> Please review and upgrade the org.apache.httpcomponents:httpcore-nio 
> to the latest version 4.4.10 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4966) Beam Dependency Update Request: org.apache.httpcomponents:httpcore 4.4.10

[yifan zou (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yifan zou resolved BEAM-4966.
-
   Resolution: Fixed
Fix Version/s: 2.7.0

https://github.com/apache/beam/pull/6115

> Beam Dependency Update Request: org.apache.httpcomponents:httpcore 4.4.10
> -
>
> Key: BEAM-4966
> URL: https://issues.apache.org/jira/browse/BEAM-4966
> Project: Beam
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.7.0
>
>
> 2018-07-25 20:30:07.304991
> Please review and upgrade the org.apache.httpcomponents:httpcore to 
> the latest version 4.4.10 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4964) Beam Dependency Update Request: org.apache.httpcomponents:httpasyncclient 4.1.4

[yifan zou (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yifan zou resolved BEAM-4964.
-
   Resolution: Fixed
Fix Version/s: 2.7.0

https://github.com/apache/beam/pull/6115

> Beam Dependency Update Request: org.apache.httpcomponents:httpasyncclient 
> 4.1.4
> ---
>
> Key: BEAM-4964
> URL: https://issues.apache.org/jira/browse/BEAM-4964
> Project: Beam
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.7.0
>
>
> 2018-07-25 20:29:48.187034
> Please review and upgrade the 
> org.apache.httpcomponents:httpasyncclient to the latest version 4.1.4 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4965) Beam Dependency Update Request: org.apache.httpcomponents:httpclient 4.5.6

[yifan zou (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yifan zou resolved BEAM-4965.
-
   Resolution: Fixed
Fix Version/s: 2.7.0

https://github.com/apache/beam/pull/6115

> Beam Dependency Update Request: org.apache.httpcomponents:httpclient 4.5.6
> --
>
> Key: BEAM-4965
> URL: https://issues.apache.org/jira/browse/BEAM-4965
> Project: Beam
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.7.0
>
>
> 2018-07-25 20:29:57.630622
> Please review and upgrade the org.apache.httpcomponents:httpclient to 
> the latest version 4.5.6 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4907) Beam Dependency Update Request: org.apache.derby:derby 10.14.2.0

[yifan zou (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yifan zou resolved BEAM-4907.
-
   Resolution: Fixed
Fix Version/s: 2.7.0

> Beam Dependency Update Request: org.apache.derby:derby 10.14.2.0
> 
>
> Key: BEAM-4907
> URL: https://issues.apache.org/jira/browse/BEAM-4907
> Project: Beam
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.7.0
>
>
> 2018-07-25 20:24:07.339681
> Please review and upgrade the org.apache.derby:derby to the latest 
> version 10.14.2.0 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4909) Beam Dependency Update Request: org.apache.derby:derbynet 10.14.2.0

[yifan zou (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yifan zou resolved BEAM-4909.
-
   Resolution: Fixed
Fix Version/s: 2.7.0

> Beam Dependency Update Request: org.apache.derby:derbynet 10.14.2.0
> ---
>
> Key: BEAM-4909
> URL: https://issues.apache.org/jira/browse/BEAM-4909
> Project: Beam
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.7.0
>
>
> 2018-07-25 20:24:23.915649
> Please review and upgrade the org.apache.derby:derbynet to the latest 
> version 10.14.2.0 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4908) Beam Dependency Update Request: org.apache.derby:derbyclient 10.14.2.0

[yifan zou (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yifan zou resolved BEAM-4908.
-
   Resolution: Fixed
Fix Version/s: 2.7.0

> Beam Dependency Update Request: org.apache.derby:derbyclient 10.14.2.0
> --
>
> Key: BEAM-4908
> URL: https://issues.apache.org/jira/browse/BEAM-4908
> Project: Beam
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.7.0
>
>
> 2018-07-25 20:24:15.830071
> Please review and upgrade the org.apache.derby:derbyclient to the 
> latest version 10.14.2.0 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4936) Beam Dependency Update Request: org.codehaus.groovy

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4936?focusedWorklogId=131134=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131134
 ]

ASF GitHub Bot logged work on BEAM-4936:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:25
Start Date: 03/Aug/18 23:25
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #6115: [BEAM-4936] update 
org.apache.httpcomponents
URL: https://github.com/apache/beam/pull/6115#issuecomment-410400408
 
 
   Thanks~


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131134)
Time Spent: 1.5h  (was: 1h 20m)

> Beam Dependency Update Request: org.codehaus.groovy
> ---
>
> Key: BEAM-4936
> URL: https://issues.apache.org/jira/browse/BEAM-4936
> Project: Beam
>  Issue Type: Bug
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Priority: Major
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> 2018-07-25 20:26:44.528984
> Please review and upgrade the org.codehaus.groovy to the latest 
> version None 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Spark #187

[Apache Jenkins Server]

See 


Changes:

[yifanzou] [BEAM-4963] update org.apache.httpcomponents

[yifanzou] [BEAM-4906] upgrade org.apache.derby

[apilloud] [BEAM-4761] Run Nexmark in thread pool

[aaltay] Automate "Run validation tests" step (#6081)

--
[...truncated 670.54 KB...]
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 Transform to: rel#411 via BeamEnumerableConverterRule
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 call#2007 generated 1 successors: [BeamEnumerableConverter#411]
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 PLANNER = 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.volcano.VolcanoPlanner@85aabb4;
 TICK = 8/6; PHASE = OPTIMIZE; COST = {inf}
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 Pop match: rule [JoinPushExpressionsRule] rels 
[rel#410:BeamJoinRel.BEAM_LOGICAL.[](left=rel#408:Subset#3.BEAM_LOGICAL.[],right=rel#409:Subset#6.BEAM_LOGICAL.[],condition=AND(=($5,
 $7), =($1, $6)),joinType=inner)]
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 call#2010: Apply rule [JoinPushExpressionsRule] to 
[rel#410:BeamJoinRel.BEAM_LOGICAL.[](left=rel#408:Subset#3.BEAM_LOGICAL.[],right=rel#409:Subset#6.BEAM_LOGICAL.[],condition=AND(=($5,
 $7), =($1, $6)),joinType=inner)]
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 call#2010 generated 0 successors.
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 PLANNER = 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.volcano.VolcanoPlanner@85aabb4;
 TICK = 9/7; PHASE = OPTIMIZE; COST = {inf}
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 Pop match: rule [FilterJoinRule:FilterJoinRule:no-filter] rels 
[rel#410:BeamJoinRel.BEAM_LOGICAL.[](left=rel#408:Subset#3.BEAM_LOGICAL.[],right=rel#409:Subset#6.BEAM_LOGICAL.[],condition=AND(=($5,
 $7), =($1, $6)),joinType=inner)]
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 call#2000: Apply rule [FilterJoinRule:FilterJoinRule:no-filter] to 
[rel#410:BeamJoinRel.BEAM_LOGICAL.[](left=rel#408:Subset#3.BEAM_LOGICAL.[],right=rel#409:Subset#6.BEAM_LOGICAL.[],condition=AND(=($5,
 $7), =($1, $6)),joinType=inner)]
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 call#2000 generated 0 successors.
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 PLANNER = 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.volcano.VolcanoPlanner@85aabb4;
 TICK = 10/8; PHASE = OPTIMIZE; COST = {inf}
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 Pop match: rule [JoinPushExpressionsRule] rels 
[rel#396:LogicalJoin.NONE.[](left=rel#389:Subset#3.NONE.[],right=rel#395:Subset#6.NONE.[],condition=AND(=($5,
 $7), =($1, $6)),joinType=inner)]
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 call#1924: Apply rule [JoinPushExpressionsRule] to 
[rel#396:LogicalJoin.NONE.[](left=rel#389:Subset#3.NONE.[],right=rel#395:Subset#6.NONE.[],condition=AND(=($5,
 $7), =($1, $6)),joinType=inner)]
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 call#1924 generated 0 successors.
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 PLANNER = 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.volcano.VolcanoPlanner@85aabb4;
 TICK = 11/9; PHASE = OPTIMIZE; COST = {inf}
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 Pop match: rule [FilterJoinRule:FilterJoinRule:no-filter] rels 
[rel#396:LogicalJoin.NONE.[](left=rel#389:Subset#3.NONE.[],right=rel#395:Subset#6.NONE.[],condition=AND(=($5,
 $7), =($1, $6)),joinType=inner)]
18/08/03 23:26:27 DEBUG 
org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.plan.RelOptPlanner:
 call#1913: Apply rule [FilterJoinRule:FilterJoinRule:no-filter] to 

[jira] [Work logged] (BEAM-4906) Beam Dependency Update Request: org.apache.derby

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4906?focusedWorklogId=131133=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131133
 ]

ASF GitHub Bot logged work on BEAM-4906:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:25
Start Date: 03/Aug/18 23:25
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #6129: [BEAM-4906] upgrade 
org.apache.derby
URL: https://github.com/apache/beam/pull/6129#issuecomment-410400394
 
 
   Thanks~


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131133)
Time Spent: 50m  (was: 40m)

> Beam Dependency Update Request: org.apache.derby
> 
>
> Key: BEAM-4906
> URL: https://issues.apache.org/jira/browse/BEAM-4906
> Project: Beam
>  Issue Type: Bug
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> 2018-07-25 20:24:06.236084
> Please review and upgrade the org.apache.derby to the latest version 
> None 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

svn commit: r28549 - /dev/beam/2.6.0/python/

[pabloem]

Author: pabloem
Date: Fri Aug  3 23:24:35 2018
New Revision: 28549

Log:
Adding signatures and hashes for wheel files.

Added:

dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl.asc

dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl.sha512

dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_i686.whl.asc

dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_i686.whl.sha512

dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_x86_64.whl.asc

dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_x86_64.whl.sha512

dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27mu-manylinux1_i686.whl.asc

dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27mu-manylinux1_i686.whl.sha512

dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27mu-manylinux1_x86_64.whl.asc

dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27mu-manylinux1_x86_64.whl.sha512

Added: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl.asc
==
--- 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl.asc
 (added)
+++ 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl.asc
 Fri Aug  3 23:24:35 2018
@@ -0,0 +1,16 @@
+-BEGIN PGP SIGNATURE-
+
+iQIzBAABCgAdFiEELx/tzfbdeZBCL0gvZSJOApLdilEFAltk4/0ACgkQZSJOApLd
+ilEXNA//aa6kVff5rSxWMgTkH00O4Qm3/fdd2GbCif5ZnRIDGyurogaOHLIlb4lK
+lzLV0+oDWOJbZ6Wi9Kg4K503hR58El+4+YXvYO3VMqN1BsKBvKK8y8nZghQ6oVxi
+bb+YOfTUL7ffk/IMYgeCzymknqbHWWdBnltb5rxzlLuKFjxXzTCp/if5osJ+Ej1u
+hDvh8bFc9lN7vTIhXf3xIe1YVMIqz8LTGlgIFk9iPhAIrnRkTtcKpqMZYth81Y9F
+k+gG3vAOGjLSGeUckedlQrzjRY25xJu6g6hp3PSH4LT15dyM9t0ndKIEcroJ+5rY
+JvFxZtZyXf0htFtOyVsim0iD2W87GJGRQKLoPeFc/nldF2kE0R8gx1EBnmQCllwY
+3oFFDl7kV0tyPhs7TCMnuNmaTbFlvsMjXSrrUBcciGfl/2jvIMou5gs52PTfMWCx
+9EGtTdY3/80HK/LwNkvhJB4aGcyiO35KTu6vtNlrHuYWP4gWRH13qASvA/AaniXJ
+gJLkf9wrWS/knNqFv9LgY5lWNZQ/cqLqHZZK7GhOeMsERv7aEuZfCY0boNshRByY
+THmE81gAnVLS/VyPN9LU3rwi/KSUudBAU46nfan4+b54OVSXIkuz5T+d7/D/d4ES
+u5QljP6ppFeAvx+8SbxPsKSo0Ziwk7t9Zno6MWr13noIuPjDxPU=
+=wk7R
+-END PGP SIGNATURE-

Added: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl.sha512
==
--- 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl.sha512
 (added)
+++ 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl.sha512
 Fri Aug  3 23:24:35 2018
@@ -0,0 +1 @@
+e1c21adb85e512f91e1cfa07e8d9f3101236aee43844577173afca9d8eb197b70278614212013dc13e454e8ca78dd9da46a7c63fc09b0e59f740e14f80c57e14
  
./apache_beam-2.6.0.rc2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl

Added: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_i686.whl.asc
==
--- 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_i686.whl.asc 
(added)
+++ 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_i686.whl.asc 
Fri Aug  3 23:24:35 2018
@@ -0,0 +1,16 @@
+-BEGIN PGP SIGNATURE-
+
+iQIzBAABCgAdFiEELx/tzfbdeZBCL0gvZSJOApLdilEFAltk4/wACgkQZSJOApLd
+ilFnvg//Svz3NmrdQfLpJsE7dbBzxTwEGw93TUL63625VTSIudxga7EFjG3XPiRn
+UwTQsr2qQHLGhedHN/LXpN4IqkY6+gXANuF4XwfjTmPUTGAiAovucDsUaWAmNyTP
+Ju4NDn+rOFgFSxZekT9B7Bj5ONTzdRG5nB8ntsGF7avUw+czIjrv+/0GwZHZJSIP
+K75mQZj+rpnc1AOVhROQ6q8Xrxr7oClIvsvRNcgI81kq5LVN7RDY9LNtgEyWb9Do
+ZFz0kNBgVt/aOwmeFQbohqGqZPKhnZ+8MmdelgNyZmFCVAH38gBkMrVpo/TBDvWH
+aJfagZ1kNNwszlEQOSX8DAKb+cBDEmDmFapamltgIyironRX+jx0MJvGIVUJTLph
+pIdG0SiXhnfdiOjVEFFRhUhX2W4FmiRNqU+GC6eQjRWNbnPSkp/YjreRy/NVEN6B
+3Cl/nf10+XiMyG/2DMg29iCAy664EqZgci56puJvVgx6ur53CQhlXDI/IJ5mLQUn
+YZkW/lH98bZ/Np9WBARinPUYHFZff4tzUeHQBCItvU5u/Kaf8nKcuOE0OkE74hcl
+d8Yzq1fGmM7qk8z/H1Z0IlXyY8raIXEo8mP+1f9HDzWhqKsJOfL09YpxpEzKXJNf
+zS0LcEcMSLl4trL8KeIcK5/losUunV9VcmS4QstHmQY6fYFrsgg=
+=ylES
+-END PGP SIGNATURE-

Added: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_i686.whl.sha512
==
--- 

[jira] [Work logged] (BEAM-5068) Get Java mobile-gaming auto validations run on any local environments.

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5068?focusedWorklogId=131131=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131131
 ]

ASF GitHub Bot logged work on BEAM-5068:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:22
Start Date: 03/Aug/18 23:22
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #6135: [BEAM-5068] 
switch mobile gaming input file to a public storage
URL: https://github.com/apache/beam/pull/6135#issuecomment-410386031
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131131)
Time Spent: 3h  (was: 2h 50m)

> Get Java mobile-gaming auto validations run on any local environments.
> --
>
> Key: BEAM-5068
> URL: https://issues.apache.org/jira/browse/BEAM-5068
> Project: Beam
>  Issue Type: Sub-task
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 3h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5068) Get Java mobile-gaming auto validations run on any local environments.

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5068?focusedWorklogId=131129=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131129
 ]

ASF GitHub Bot logged work on BEAM-5068:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:22
Start Date: 03/Aug/18 23:22
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #6135: [BEAM-5068] 
switch mobile gaming input file to a public storage
URL: https://github.com/apache/beam/pull/6135#issuecomment-410360536
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131129)
Time Spent: 2h 40m  (was: 2.5h)

> Get Java mobile-gaming auto validations run on any local environments.
> --
>
> Key: BEAM-5068
> URL: https://issues.apache.org/jira/browse/BEAM-5068
> Project: Beam
>  Issue Type: Sub-task
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5068) Get Java mobile-gaming auto validations run on any local environments.

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5068?focusedWorklogId=131130=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131130
 ]

ASF GitHub Bot logged work on BEAM-5068:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:22
Start Date: 03/Aug/18 23:22
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #6135: [BEAM-5068] 
switch mobile gaming input file to a public storage
URL: https://github.com/apache/beam/pull/6135#issuecomment-410361701
 
 
   Run Dataflow PostRelease


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131130)
Time Spent: 2h 50m  (was: 2h 40m)

> Get Java mobile-gaming auto validations run on any local environments.
> --
>
> Key: BEAM-5068
> URL: https://issues.apache.org/jira/browse/BEAM-5068
> Project: Beam
>  Issue Type: Sub-task
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5068) Get Java mobile-gaming auto validations run on any local environments.

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5068?focusedWorklogId=131125=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131125
 ]

ASF GitHub Bot logged work on BEAM-5068:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:22
Start Date: 03/Aug/18 23:22
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #6135: [BEAM-5068] 
switch mobile gaming input file to a public storage
URL: https://github.com/apache/beam/pull/6135#issuecomment-410321156
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131125)
Time Spent: 2h  (was: 1h 50m)

> Get Java mobile-gaming auto validations run on any local environments.
> --
>
> Key: BEAM-5068
> URL: https://issues.apache.org/jira/browse/BEAM-5068
> Project: Beam
>  Issue Type: Sub-task
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5068) Get Java mobile-gaming auto validations run on any local environments.

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5068?focusedWorklogId=131127=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131127
 ]

ASF GitHub Bot logged work on BEAM-5068:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:22
Start Date: 03/Aug/18 23:22
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #6135: [BEAM-5068] 
switch mobile gaming input file to a public storage
URL: https://github.com/apache/beam/pull/6135#issuecomment-410335629
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131127)
Time Spent: 2h 20m  (was: 2h 10m)

> Get Java mobile-gaming auto validations run on any local environments.
> --
>
> Key: BEAM-5068
> URL: https://issues.apache.org/jira/browse/BEAM-5068
> Project: Beam
>  Issue Type: Sub-task
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5068) Get Java mobile-gaming auto validations run on any local environments.

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5068?focusedWorklogId=131128=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131128
 ]

ASF GitHub Bot logged work on BEAM-5068:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:22
Start Date: 03/Aug/18 23:22
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #6135: [BEAM-5068] 
switch mobile gaming input file to a public storage
URL: https://github.com/apache/beam/pull/6135#issuecomment-410337752
 
 
   Run Dataflow PostRelease


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131128)
Time Spent: 2.5h  (was: 2h 20m)

> Get Java mobile-gaming auto validations run on any local environments.
> --
>
> Key: BEAM-5068
> URL: https://issues.apache.org/jira/browse/BEAM-5068
> Project: Beam
>  Issue Type: Sub-task
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5068) Get Java mobile-gaming auto validations run on any local environments.

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5068?focusedWorklogId=131132=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131132
 ]

ASF GitHub Bot logged work on BEAM-5068:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:22
Start Date: 03/Aug/18 23:22
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #6135: [BEAM-5068] 
switch mobile gaming input file to a public storage
URL: https://github.com/apache/beam/pull/6135#issuecomment-410387474
 
 
   Run Dataflow PostRelease


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131132)
Time Spent: 3h 10m  (was: 3h)

> Get Java mobile-gaming auto validations run on any local environments.
> --
>
> Key: BEAM-5068
> URL: https://issues.apache.org/jira/browse/BEAM-5068
> Project: Beam
>  Issue Type: Sub-task
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5068) Get Java mobile-gaming auto validations run on any local environments.

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5068?focusedWorklogId=131126=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131126
 ]

ASF GitHub Bot logged work on BEAM-5068:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:22
Start Date: 03/Aug/18 23:22
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #6135: [BEAM-5068] 
switch mobile gaming input file to a public storage
URL: https://github.com/apache/beam/pull/6135#issuecomment-410322603
 
 
   Run Dataflow PostRelease


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131126)
Time Spent: 2h 10m  (was: 2h)

> Get Java mobile-gaming auto validations run on any local environments.
> --
>
> Key: BEAM-5068
> URL: https://issues.apache.org/jira/browse/BEAM-5068
> Project: Beam
>  Issue Type: Sub-task
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Flink #188

[Apache Jenkins Server]

See 


Changes:

[yifanzou] [BEAM-4963] update org.apache.httpcomponents

[yifanzou] [BEAM-4906] upgrade org.apache.derby

[apilloud] [BEAM-4761] Run Nexmark in thread pool

[aaltay] Automate "Run validation tests" step (#6081)

--
[...truncated 6.60 MB...]
INFO: Creating FileSystem stream leak safety net for task CHAIN MapPartition 
(MapPartition at SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor)) -> FlatMap 
(FlatMap at SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor).output) (1/1) 
(ad04af95910bd35b3a5568c198226ab4) [DEPLOYING]
Aug 03, 2018 11:17:58 PM org.apache.flink.runtime.taskmanager.Task run
INFO: Loading JAR files for task CHAIN MapPartition (MapPartition at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor)) -> FlatMap (FlatMap at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor).output) (1/1) 
(ad04af95910bd35b3a5568c198226ab4) [DEPLOYING].
Aug 03, 2018 11:17:58 PM org.apache.flink.runtime.taskmanager.Task run
INFO: Registering task at network: CHAIN MapPartition (MapPartition at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor)) -> FlatMap (FlatMap at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor).output) (1/1) 
(ad04af95910bd35b3a5568c198226ab4) [DEPLOYING].
Aug 03, 2018 11:17:58 PM org.apache.flink.runtime.taskmanager.Task 
transitionState
INFO: CHAIN MapPartition (MapPartition at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor)) -> FlatMap (FlatMap at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor).output) (1/1) 
(ad04af95910bd35b3a5568c198226ab4) switched from DEPLOYING to RUNNING.
Aug 03, 2018 11:17:58 PM org.apache.flink.runtime.executiongraph.Execution 
transitionState
INFO: CHAIN MapPartition (MapPartition at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor)) -> FlatMap (FlatMap at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor).output) (1/1) 
(ad04af95910bd35b3a5568c198226ab4) switched from DEPLOYING to RUNNING.
Aug 03, 2018 11:17:58 PM org.apache.flink.runtime.taskmanager.Task 
transitionState
INFO: CHAIN MapPartition (MapPartition at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor)) -> FlatMap (FlatMap at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor).output) (1/1) 
(ad04af95910bd35b3a5568c198226ab4) switched from RUNNING to FINISHED.
Aug 03, 2018 11:17:58 PM org.apache.flink.runtime.taskmanager.Task run
INFO: Freeing task resources for CHAIN MapPartition (MapPartition at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor)) -> FlatMap (FlatMap at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor).output) (1/1) 
(ad04af95910bd35b3a5568c198226ab4).
Aug 03, 2018 11:17:58 PM org.apache.flink.runtime.executiongraph.Execution 
transitionState
INFO: CHAIN MapPartition (MapPartition at 
SqlQuery7/SqlQuery7.Stamp/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at 
SqlQuery7/SqlQuery7.Stamp/ParMultiDo(Anonymous).output) (1/1) 
(6f22982de8a6ddaa44f0155e5fafc355) switched from CREATED to SCHEDULED.
Aug 03, 2018 11:17:58 PM org.apache.flink.runtime.taskmanager.Task run
INFO: Ensuring all FileSystem streams are closed for task CHAIN MapPartition 
(MapPartition at SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor)) -> FlatMap 
(FlatMap at SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor).output) (1/1) 
(ad04af95910bd35b3a5568c198226ab4) [FINISHED]
Aug 03, 2018 11:17:58 PM org.apache.flink.runtime.taskexecutor.TaskExecutor 
unregisterTaskAndNotifyFinalState
INFO: Un-registering task and sending final execution state FINISHED to 
JobManager for task CHAIN MapPartition (MapPartition at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor)) -> FlatMap (FlatMap at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor).output) 
ad04af95910bd35b3a5568c198226ab4.
Aug 03, 2018 11:17:58 PM org.apache.flink.runtime.executiongraph.Execution 
transitionState
INFO: CHAIN MapPartition (MapPartition at 
SqlQuery7/SqlQuery7.Stamp/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at 
SqlQuery7/SqlQuery7.Stamp/ParMultiDo(Anonymous).output) (1/1) 
(6f22982de8a6ddaa44f0155e5fafc355) switched from SCHEDULED to DEPLOYING.
Aug 03, 2018 11:17:58 PM org.apache.flink.runtime.executiongraph.Execution 
deploy
INFO: Deploying CHAIN MapPartition (MapPartition at 
SqlQuery7/SqlQuery7.Stamp/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at 
SqlQuery7/SqlQuery7.Stamp/ParMultiDo(Anonymous).output) (1/1) (attempt #0) to 
localhost
Aug 03, 2018 11:17:58 PM org.apache.flink.runtime.executiongraph.Execution 
transitionState
INFO: CHAIN MapPartition (MapPartition at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor)) -> FlatMap (FlatMap at 
SqlQuery7/SqlQuery7.Debug/ParMultiDo(Monitor).output) (1/1) 
(ad04af95910bd35b3a5568c198226ab4) switched from RUNNING to FINISHED.
Aug 03, 2018 11:17:58 PM org.apache.flink.runtime.taskexecutor.TaskExecutor 
submitTask
INFO: Received task CHAIN MapPartition (MapPartition at 
SqlQuery7/SqlQuery7.Stamp/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at 
SqlQuery7/SqlQuery7.Stamp/ParMultiDo(Anonymous).output) (1/1).
Aug 03, 2018 11:17:58 PM 

Jenkins build is back to normal : beam_PostRelease_NightlySnapshot #326

[Apache Jenkins Server]

See 

[jira] [Work logged] (BEAM-5072) SortRelNode is wrongly removed

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5072?focusedWorklogId=131124=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131124
 ]

ASF GitHub Bot logged work on BEAM-5072:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:08
Start Date: 03/Aug/18 23:08
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #6139: [BEAM-5072] Remove 
RelCollationTraitDef configuration in BeamQueryPlanner
URL: https://github.com/apache/beam/pull/6139#issuecomment-410398056
 
 
   I also removed/commented `SortRemoveRule` because it does nothing without 
`CollationTrait`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131124)
Time Spent: 1h 10m  (was: 1h)

> SortRelNode is wrongly removed
> --
>
> Key: BEAM-5072
> URL: https://issues.apache.org/jira/browse/BEAM-5072
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> ORDER BY query generates LogicalSort in logical plan, but it will be removed 
> wrongly during optimization by the SortRemoveRule



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5072) SortRelNode is wrongly removed

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5072?focusedWorklogId=131122=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131122
 ]

ASF GitHub Bot logged work on BEAM-5072:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:06
Start Date: 03/Aug/18 23:06
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on a change in pull request #6139: 
[BEAM-5072] Remove RelCollationTraitDef configuration in BeamQueryPlanner
URL: https://github.com/apache/beam/pull/6139#discussion_r207686487
 
 

 ##
 File path: 
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java
 ##
 @@ -76,8 +75,7 @@
 final SchemaPlus schema = connection.getRootSchema();
 final SchemaPlus defaultSchema = JdbcDriver.getDefaultSchema(connection);
 
-final ImmutableList traitDefs =
-ImmutableList.of(ConventionTraitDef.INSTANCE, 
RelCollationTraitDef.INSTANCE);
+final ImmutableList traitDefs = 
ImmutableList.of(ConventionTraitDef.INSTANCE);
 
 Review comment:
   I modified JDBC path here: 
https://github.com/amaliujia/beam/blob/5ab3ed8da33ead0cd3b43da1af8893d761c45a47/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcDriver.java#L100


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131122)
Time Spent: 1h  (was: 50m)

> SortRelNode is wrongly removed
> --
>
> Key: BEAM-5072
> URL: https://issues.apache.org/jira/browse/BEAM-5072
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> ORDER BY query generates LogicalSort in logical plan, but it will be removed 
> wrongly during optimization by the SortRemoveRule



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

svn commit: r28546 - /dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl

[pabloem]

Author: pabloem
Date: Fri Aug  3 23:00:33 2018
New Revision: 28546

Log:
Upload python wheels

Added:

dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
   (with props)

Added: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
==
Binary file - no diff available.

Propchange: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
--
svn:mime-type = application/octet-stream

svn commit: r28543 - /dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27mu-manylinux1_i686.whl

[pabloem]

Author: pabloem
Date: Fri Aug  3 23:00:09 2018
New Revision: 28543

Log:
Upload python wheels

Added:
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27mu-manylinux1_i686.whl 
  (with props)

Added: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27mu-manylinux1_i686.whl
==
Binary file - no diff available.

Propchange: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27mu-manylinux1_i686.whl
--
svn:mime-type = application/octet-stream

svn commit: r28547 - /dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_i686.whl

[pabloem]

Author: pabloem
Date: Fri Aug  3 23:03:23 2018
New Revision: 28547

Log:
Upload python wheels

Added:
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_i686.whl  
 (with props)

Added: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_i686.whl
==
Binary file - no diff available.

Propchange: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_i686.whl
--
svn:mime-type = application/octet-stream

[jira] [Work logged] (BEAM-5072) SortRelNode is wrongly removed

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5072?focusedWorklogId=131121=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131121
 ]

ASF GitHub Bot logged work on BEAM-5072:


Author: ASF GitHub Bot
Created on: 03/Aug/18 23:00
Start Date: 03/Aug/18 23:00
Worklog Time Spent: 10m 
  Work Description: akedin commented on a change in pull request #6139: 
[BEAM-5072] Remove RelCollationTraitDef configuration in BeamQueryPlanner
URL: https://github.com/apache/beam/pull/6139#discussion_r207685756
 
 

 ##
 File path: 
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java
 ##
 @@ -76,8 +75,7 @@
 final SchemaPlus schema = connection.getRootSchema();
 final SchemaPlus defaultSchema = JdbcDriver.getDefaultSchema(connection);
 
-final ImmutableList traitDefs =
-ImmutableList.of(ConventionTraitDef.INSTANCE, 
RelCollationTraitDef.INSTANCE);
+final ImmutableList traitDefs = 
ImmutableList.of(ConventionTraitDef.INSTANCE);
 
 Review comment:
   This is only valid for `BeamSqlEnv` path. Does the JDBC path apply the 
CollationTrait?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131121)
Time Spent: 50m  (was: 40m)

> SortRelNode is wrongly removed
> --
>
> Key: BEAM-5072
> URL: https://issues.apache.org/jira/browse/BEAM-5072
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> ORDER BY query generates LogicalSort in logical plan, but it will be removed 
> wrongly during optimization by the SortRemoveRule



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

svn commit: r28545 - /dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27mu-manylinux1_x86_64.whl

[pabloem]

Author: pabloem
Date: Fri Aug  3 23:00:16 2018
New Revision: 28545

Log:
Upload python wheels

Added:

dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27mu-manylinux1_x86_64.whl   
(with props)

Added: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27mu-manylinux1_x86_64.whl
==
Binary file - no diff available.

Propchange: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27mu-manylinux1_x86_64.whl
--
svn:mime-type = application/octet-stream

svn commit: r28544 - /dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_x86_64.whl

[pabloem]

Author: pabloem
Date: Fri Aug  3 23:00:13 2018
New Revision: 28544

Log:
Upload python wheels

Added:

dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_x86_64.whl   
(with props)

Added: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_x86_64.whl
==
Binary file - no diff available.

Propchange: 
dev/beam/2.6.0/python/apache_beam-2.6.0.rc2-cp27-cp27m-manylinux1_x86_64.whl
--
svn:mime-type = application/octet-stream

[jira] [Work logged] (BEAM-4936) Beam Dependency Update Request: org.codehaus.groovy

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4936?focusedWorklogId=131119=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131119
 ]

ASF GitHub Bot logged work on BEAM-4936:


Author: ASF GitHub Bot
Created on: 03/Aug/18 22:54
Start Date: 03/Aug/18 22:54
Worklog Time Spent: 10m 
  Work Description: chamikaramj closed pull request #6115: [BEAM-4936] 
update org.apache.httpcomponents
URL: https://github.com/apache/beam/pull/6115
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/java/io/amazon-web-services/build.gradle 
b/sdks/java/io/amazon-web-services/build.gradle
index d8bb1065d52..cbcc97df0ce 100644
--- a/sdks/java/io/amazon-web-services/build.gradle
+++ b/sdks/java/io/amazon-web-services/build.gradle
@@ -34,7 +34,7 @@ dependencies {
   shadow library.java.jackson_databind
   shadow library.java.slf4j_api
   runtime 'commons-codec:commons-codec:1.9'
-  runtime "org.apache.httpcomponents:httpclient:4.5.2"
+  runtime "org.apache.httpcomponents:httpclient:4.5.6"
   testCompile project(path: ":beam-runners-direct-java", configuration: 
"shadow")
   shadowTest library.java.guava_testlib
   shadowTest library.java.hamcrest_core
diff --git 
a/sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/build.gradle 
b/sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/build.gradle
index 6c0d3bb52e8..bf4fac8d5ac 100644
--- a/sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/build.gradle
+++ b/sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/build.gradle
@@ -27,10 +27,10 @@ def log4j_version = "2.6.2"
 
 dependencies {
   testCompile library.java.jackson_databind
-  testCompile "org.apache.httpcomponents:httpasyncclient:4.1.2"
-  testCompile "org.apache.httpcomponents:httpcore-nio:4.4.5"
-  testCompile "org.apache.httpcomponents:httpcore:4.4.5"
-  testCompile "org.apache.httpcomponents:httpclient:4.5.2"
+  testCompile "org.apache.httpcomponents:httpasyncclient:4.1.4"
+  testCompile "org.apache.httpcomponents:httpcore-nio:4.4.10"
+  testCompile "org.apache.httpcomponents:httpcore:4.4.10"
+  testCompile "org.apache.httpcomponents:httpclient:4.5.6"
 
   testCompile project(path: ":beam-sdks-java-core", configuration: "shadow")
   testCompile project(path: ":beam-sdks-java-io-elasticsearch", configuration: 
"shadow")
diff --git a/sdks/java/io/elasticsearch/build.gradle 
b/sdks/java/io/elasticsearch/build.gradle
index 758c2647d72..b6a06c882a8 100644
--- a/sdks/java/io/elasticsearch/build.gradle
+++ b/sdks/java/io/elasticsearch/build.gradle
@@ -28,9 +28,9 @@ dependencies {
   shadow library.java.jackson_databind
   shadow library.java.jackson_annotations
   shadow "org.elasticsearch.client:elasticsearch-rest-client:5.6.3"
-  shadow "org.apache.httpcomponents:httpasyncclient:4.1.2"
-  shadow "org.apache.httpcomponents:httpcore-nio:4.4.5"
-  shadow "org.apache.httpcomponents:httpcore:4.4.5"
-  shadow "org.apache.httpcomponents:httpclient:4.5.2"
+  shadow "org.apache.httpcomponents:httpasyncclient:4.1.4"
+  shadow "org.apache.httpcomponents:httpcore-nio:4.4.10"
+  shadow "org.apache.httpcomponents:httpcore:4.4.10"
+  shadow "org.apache.httpcomponents:httpclient:4.5.6"
   testCompile project(path: ":beam-sdks-java-io-common", configuration: 
"shadow")
 }
diff --git a/sdks/java/io/solr/build.gradle b/sdks/java/io/solr/build.gradle
index bc2f9087210..807f85e5d52 100644
--- a/sdks/java/io/solr/build.gradle
+++ b/sdks/java/io/solr/build.gradle
@@ -27,7 +27,7 @@ dependencies {
   shadow project(path: ":beam-sdks-java-core", configuration: "shadow")
   shadow library.java.commons_compress
   shadow "org.apache.solr:solr-solrj:5.5.4"
-  compileOnly "org.apache.httpcomponents:httpclient:4.4.1"
+  compileOnly "org.apache.httpcomponents:httpclient:4.5.6"
   testCompile project(path: ":beam-sdks-java-core", configuration: 
"shadowTest")
   testCompile project(path: ":beam-sdks-java-io-common", configuration: 
"shadow")
   testCompile project(path: ":beam-runners-direct-java", configuration: 
"shadow")


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131119)
Time Spent: 1h 20m  (was: 1h 10m)

> Beam Dependency Update Request: org.codehaus.groovy
> ---
>
> Key: BEAM-4936
> URL: https://issues.apache.org/jira/browse/BEAM-4936
>  

[beam] 01/01: Merge pull request #6115: [BEAM-4936] update org.apache.httpcomponents

[chamikara]

This is an automated email from the ASF dual-hosted git repository.

chamikara pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 4cb7dab4052b23c0d84e51b1ac1fdc6002b5b502
Merge: bf22dec 79880fb
Author: Chamikara Jayalath 
AuthorDate: Fri Aug 3 15:54:04 2018 -0700

Merge pull request #6115: [BEAM-4936] update org.apache.httpcomponents

 sdks/java/io/amazon-web-services/build.gradle | 2 +-
 .../elasticsearch-tests/elasticsearch-tests-common/build.gradle   | 8 
 sdks/java/io/elasticsearch/build.gradle   | 8 
 sdks/java/io/solr/build.gradle| 2 +-
 4 files changed, 10 insertions(+), 10 deletions(-)

[jira] [Work logged] (BEAM-4906) Beam Dependency Update Request: org.apache.derby

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4906?focusedWorklogId=131118=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131118
 ]

ASF GitHub Bot logged work on BEAM-4906:


Author: ASF GitHub Bot
Created on: 03/Aug/18 22:53
Start Date: 03/Aug/18 22:53
Worklog Time Spent: 10m 
  Work Description: chamikaramj closed pull request #6129: [BEAM-4906] 
upgrade org.apache.derby
URL: https://github.com/apache/beam/pull/6129
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/java/io/jdbc/build.gradle b/sdks/java/io/jdbc/build.gradle
index 5e5593ad240..8035677cbc0 100644
--- a/sdks/java/io/jdbc/build.gradle
+++ b/sdks/java/io/jdbc/build.gradle
@@ -38,7 +38,7 @@ dependencies {
   testCompile library.java.slf4j_api
   testCompile library.java.slf4j_jdk14
   testCompile library.java.postgres
-  testCompile group: "org.apache.derby", name: "derby", version:"10.12.1.1"
-  testCompile group: "org.apache.derby", name: "derbyclient", 
version:"10.12.1.1"
-  testCompile group: "org.apache.derby", name: "derbynet", version:"10.12.1.1"
+  testCompile group: "org.apache.derby", name: "derby", version:"10.14.2.0"
+  testCompile group: "org.apache.derby", name: "derbyclient", 
version:"10.14.2.0"
+  testCompile group: "org.apache.derby", name: "derbynet", version:"10.14.2.0"
 }


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131118)
Time Spent: 40m  (was: 0.5h)

> Beam Dependency Update Request: org.apache.derby
> 
>
> Key: BEAM-4906
> URL: https://issues.apache.org/jira/browse/BEAM-4906
> Project: Beam
>  Issue Type: Bug
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> 2018-07-25 20:24:06.236084
> Please review and upgrade the org.apache.derby to the latest version 
> None 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam] branch master updated (bf22dec -> 4cb7dab)

[chamikara]

This is an automated email from the ASF dual-hosted git repository.

chamikara pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from bf22dec  Merge pull request #6129: [BEAM-4906] upgrade org.apache.derby
 add 79880fb  [BEAM-4963] update org.apache.httpcomponents
 new 4cb7dab  Merge pull request #6115: [BEAM-4936] update 
org.apache.httpcomponents

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 sdks/java/io/amazon-web-services/build.gradle | 2 +-
 .../elasticsearch-tests/elasticsearch-tests-common/build.gradle   | 8 
 sdks/java/io/elasticsearch/build.gradle   | 8 
 sdks/java/io/solr/build.gradle| 2 +-
 4 files changed, 10 insertions(+), 10 deletions(-)

[beam] 01/01: Merge pull request #6129: [BEAM-4906] upgrade org.apache.derby

[chamikara]

This is an automated email from the ASF dual-hosted git repository.

chamikara pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit bf22dec4ff4fd6ac27d7432794fcaa7a22aa24d2
Merge: b944f66 444005f
Author: Chamikara Jayalath 
AuthorDate: Fri Aug 3 15:53:24 2018 -0700

Merge pull request #6129: [BEAM-4906] upgrade org.apache.derby

 sdks/java/io/jdbc/build.gradle | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

[beam] branch master updated (b944f66 -> bf22dec)

[chamikara]

This is an automated email from the ASF dual-hosted git repository.

chamikara pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from b944f66  Automate "Run validation tests" step (#6081)
 add 444005f  [BEAM-4906] upgrade org.apache.derby
 new bf22dec  Merge pull request #6129: [BEAM-4906] upgrade org.apache.derby

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 sdks/java/io/jdbc/build.gradle | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

svn commit: r28542 - in /dev/beam/2.6.0/python: ./ apache-beam-2.6.0.zip apache-beam-2.6.0.zip.asc apache-beam-2.6.0.zip.sha512

[pabloem]

Author: pabloem
Date: Fri Aug  3 22:52:53 2018
New Revision: 28542

Log:
Pushing Python artifacts for 2.6.0 RC2.

Added:
dev/beam/2.6.0/python/
dev/beam/2.6.0/python/apache-beam-2.6.0.zip   (with props)
dev/beam/2.6.0/python/apache-beam-2.6.0.zip.asc
dev/beam/2.6.0/python/apache-beam-2.6.0.zip.sha512

Added: dev/beam/2.6.0/python/apache-beam-2.6.0.zip
==
Binary file - no diff available.

Propchange: dev/beam/2.6.0/python/apache-beam-2.6.0.zip
--
svn:mime-type = application/octet-stream

Added: dev/beam/2.6.0/python/apache-beam-2.6.0.zip.asc
==
--- dev/beam/2.6.0/python/apache-beam-2.6.0.zip.asc (added)
+++ dev/beam/2.6.0/python/apache-beam-2.6.0.zip.asc Fri Aug  3 22:52:53 2018
@@ -0,0 +1,16 @@
+-BEGIN PGP SIGNATURE-
+
+iQIzBAABCgAdFiEELx/tzfbdeZBCL0gvZSJOApLdilEFAltk3FYACgkQZSJOApLd
+ilHnbRAA4kFbaY7jvR4mGuD+QuZze3zldoBvtKfpGRb0yomrlCEovA+WzLE7Thy+
+QCwoY7D83l+g5AnM06HzaTbWNXUcinqY/yTDITvkInBBgqBxmLx+VifseIIww/TW
+6p1uhDlgbGXkL45cOfzYfKdaqLtNMxMC3NGFK0VcSPVAFTlbtRpmNW/r7VjokbqY
+byANW1LMQ8e0tOQlKLG8ZsOJmmGuSKad1mhZ9fwE+WiJqHHi4cqpEFymv6A05C99
+NY3kDfvz3hGX90/F7qvzyZnCdUjtvwoEyDMpvUU4XPtiZ9HCcvr0iwz7hmpkcdMT
+tsOeF/4aNNWIAT1m0XfeBXLryhgNu9GeHsyTBYnziq/cyVVSCia6MYU5Wv0PoWOp
+IWS6k3T96KbyfQ95XtbIJfosOWM0mJhuOtFiuNMeiAh7xoEqZvo/O1ViH91WnoCA
+zrq42IZb0N8E+55Q+NO3z3LJ2ujtfNKZL65eYt8ww0nk900/sGAyu8aOeTZqpy8V
+26GMM14sQgZU5xAp6g54OTvnJE1FvUXCfICS3MdxrX8o+wJ/f63fYk6UaaK2q2rI
+1XLgO0kyDucXdrggV92JmNgViUUC01Vke7tVR80UGGLUEqvjGP8QVOrQNX5XrM5l
+nLKEVjfkIH9ZQQ0sOaZKrnHY+pmjMOfqnEkE3/Jeqw2513LuLBs=
+=uChM
+-END PGP SIGNATURE-

Added: dev/beam/2.6.0/python/apache-beam-2.6.0.zip.sha512
==
--- dev/beam/2.6.0/python/apache-beam-2.6.0.zip.sha512 (added)
+++ dev/beam/2.6.0/python/apache-beam-2.6.0.zip.sha512 Fri Aug  3 22:52:53 2018
@@ -0,0 +1 @@
+9a70336973d7b782c986bac1085bb61b87a412eeb0a34f9c993045a0c9d2382ea4751a47dc1ce7a4d35bbd072a959fa02c89fe2612658cd527466f5913d13ffa
  apache-beam-2.6.0.zip

svn commit: r28541 - in /dev/beam/2.6.0: ./ apache-beam-2.6.0-source-release.zip apache-beam-2.6.0-source-release.zip.asc apache-beam-2.6.0-source-release.zip.sha512

[pabloem]

Author: pabloem
Date: Fri Aug  3 22:50:15 2018
New Revision: 28541

Log:
Adding artifacts for 2.6.0 RC2.

Added:
dev/beam/2.6.0/
dev/beam/2.6.0/apache-beam-2.6.0-source-release.zip   (with props)
dev/beam/2.6.0/apache-beam-2.6.0-source-release.zip.asc
dev/beam/2.6.0/apache-beam-2.6.0-source-release.zip.sha512

Added: dev/beam/2.6.0/apache-beam-2.6.0-source-release.zip
==
Binary file - no diff available.

Propchange: dev/beam/2.6.0/apache-beam-2.6.0-source-release.zip
--
svn:mime-type = application/octet-stream

Added: dev/beam/2.6.0/apache-beam-2.6.0-source-release.zip.asc
==
--- dev/beam/2.6.0/apache-beam-2.6.0-source-release.zip.asc (added)
+++ dev/beam/2.6.0/apache-beam-2.6.0-source-release.zip.asc Fri Aug  3 22:50:15 
2018
@@ -0,0 +1,16 @@
+-BEGIN PGP SIGNATURE-
+
+iQIzBAABCgAdFiEELx/tzfbdeZBCL0gvZSJOApLdilEFAltk3AkACgkQZSJOApLd
+ilFxyhAArzEb6MbBSbCgp51uL+2+KAVEAgzBcXERlaNrmqRjhtUGZFQp7UltBF8A
+8duQlUrFRErmn3Dtw9xJvCNKqdljhs9TQkueehBFA9Q9f0g5eLPLigJz78CQiJzN
+knigZ6AWZSGGFHeQlPQMJEGg2/JzF7z0RVkQbMjlUQ4YilE/EfpWTmv4Tcs11/2H
+YDjA5JHHvSKrodO76bgDjvNQjkUYDUJ/DTjdB29kWIJtFQa8U54B+9oDjSl89hzP
+WzzLZu8EBGSMdJesEKMdcQbK+EaWJ0MtfPTTM/te8gol5nJ/vJ3okLjRLvoATYnI
+6TnuhBC94pxtSFIYTZfzJk9mJhpVxXI5Ar9durPNknggHdproqYuV2CMXDuNfnYB
+tU2YNLnISlaLxkYA+pvBX5K1pfRRTTKX39Bc+P0R/58AS3lMXTOd0x/NH6xd9ArC
+SiK5F0k/s7u+TPfelyDqEt/62B2wp4AMkn/IMWj/ixsi6u/j4qHE4pME+waHFHPo
+NUOA5YILqYf/qlJGWlXnS70DyMsQ+bqlbYfaGa8WqHoaXhnb4l4PX+Q9awPKG51R
+XFunrNcxP9HV13A3CE3qjBJo4EYG8crnFsgGdm1oLJ3sm21aYY81HZA0MwrUSIiv
+xbQ1BVI8JxfaJaNgAmqgsx2Ncx90bojEhtyLHDiPRb+V9yPby/E=
+=EnXH
+-END PGP SIGNATURE-

Added: dev/beam/2.6.0/apache-beam-2.6.0-source-release.zip.sha512
==
--- dev/beam/2.6.0/apache-beam-2.6.0-source-release.zip.sha512 (added)
+++ dev/beam/2.6.0/apache-beam-2.6.0-source-release.zip.sha512 Fri Aug  3 
22:50:15 2018
@@ -0,0 +1 @@
+3f34db6cea8db4565c3365744c530dba0726f75b76b01299d2589c7b9e028852a38b77253001f7568158bde0180b355345ccfa32e3a7400dd59f6f58051c5bb7
  apache-beam-2.6.0-source-release.zip

[jira] [Work logged] (BEAM-5068) Get Java mobile-gaming auto validations run on any local environments.

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5068?focusedWorklogId=131115=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131115
 ]

ASF GitHub Bot logged work on BEAM-5068:


Author: ASF GitHub Bot
Created on: 03/Aug/18 22:47
Start Date: 03/Aug/18 22:47
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #6135: [BEAM-5068] switch 
mobile gaming input file to a public storage
URL: https://github.com/apache/beam/pull/6135#issuecomment-410395223
 
 
   Run Dataflow PostRelease


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131115)
Time Spent: 1h 50m  (was: 1h 40m)

> Get Java mobile-gaming auto validations run on any local environments.
> --
>
> Key: BEAM-5068
> URL: https://issues.apache.org/jira/browse/BEAM-5068
> Project: Beam
>  Issue Type: Sub-task
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

svn commit: r28540 - /dev/beam/2.6.0/

[pabloem]

Author: pabloem
Date: Fri Aug  3 22:45:48 2018
New Revision: 28540

Log:
Removing artifacts from 2.6.0 RC1 to make place for RC2.

Removed:
dev/beam/2.6.0/

[jira] [Work logged] (BEAM-5072) SortRelNode is wrongly removed

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5072?focusedWorklogId=131112=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131112
 ]

ASF GitHub Bot logged work on BEAM-5072:


Author: ASF GitHub Bot
Created on: 03/Aug/18 22:40
Start Date: 03/Aug/18 22:40
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #6139: [BEAM-5072] Remove 
RelCollationTraitDef configuration in BeamQueryPlanner
URL: https://github.com/apache/beam/pull/6139#issuecomment-410394073
 
 
   What is here so far looks good, you also need to remove it from the JDBC 
planner here: 
https://github.com/apache/beam/blob/7ad96d33105d7addc0d956d448635f71a2cb1b85/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcDriver.java#L88
   
   Planner gives you `getRelTraitDefs`, `clearRelTraitDefs`, and 
`addRelTraitDef`, but nothing to remove a single trait. You'll have to get the 
list, clear them all, then add back the other ones.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131112)
Time Spent: 40m  (was: 0.5h)

> SortRelNode is wrongly removed
> --
>
> Key: BEAM-5072
> URL: https://issues.apache.org/jira/browse/BEAM-5072
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> ORDER BY query generates LogicalSort in logical plan, but it will be removed 
> wrongly during optimization by the SortRemoveRule



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5068) Get Java mobile-gaming auto validations run on any local environments.

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5068?focusedWorklogId=13=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-13
 ]

ASF GitHub Bot logged work on BEAM-5068:


Author: ASF GitHub Bot
Created on: 03/Aug/18 22:39
Start Date: 03/Aug/18 22:39
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #6135: [BEAM-5068] switch 
mobile gaming input file to a public storage
URL: https://github.com/apache/beam/pull/6135#issuecomment-410393986
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 13)
Time Spent: 1h 40m  (was: 1.5h)

> Get Java mobile-gaming auto validations run on any local environments.
> --
>
> Key: BEAM-5068
> URL: https://issues.apache.org/jira/browse/BEAM-5068
> Project: Beam
>  Issue Type: Sub-task
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam] annotated tag v2.6.0-RC2 updated (82fc380 -> 4bf7df2)

[pabloem]

This is an automated email from the ASF dual-hosted git repository.

pabloem pushed a change to annotated tag v2.6.0-RC2
in repository https://gitbox.apache.org/repos/asf/beam.git.


*** WARNING: tag v2.6.0-RC2 was modified! ***

from 82fc380  (commit)
  to 4bf7df2  (tag)
 tagging 82fc3801423c051fafe1ea6ad9ae7891b40b442b (commit)
 replaces v2.6.0-RC1
  by Pablo
  on Fri Aug 3 15:38:38 2018 -0700

- Log -
[Gradle Release Plugin] - creating tag:  'v2.6.0-RC2'.
---


No new revisions were added by this update.

Summary of changes:

[beam] branch release-2.6.0 updated: [Gradle Release Plugin] - new version commit: 'v2.6.0-SNAPSHOT'.

[pabloem]

This is an automated email from the ASF dual-hosted git repository.

pabloem pushed a commit to branch release-2.6.0
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/release-2.6.0 by this push:
 new e817ad3  [Gradle Release Plugin] - new version commit:  
'v2.6.0-SNAPSHOT'.
e817ad3 is described below

commit e817ad38137e3a5b4052dfdefde8adc3ab6be8c2
Author: Pablo 
AuthorDate: Fri Aug 3 15:38:41 2018 -0700

[Gradle Release Plugin] - new version commit:  'v2.6.0-SNAPSHOT'.
---
 gradle.properties | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gradle.properties b/gradle.properties
index 34a5344..bf2e45d 100644
--- a/gradle.properties
+++ b/gradle.properties
@@ -22,4 +22,4 @@ offlineRepositoryRoot=offline-repository
 signing.gnupg.executable=gpg
 signing.gnupg.useLegacyGpg=true
 
-version=2.6.0-RC2
+version=2.6.0-SNAPSHOT

[beam] branch release-2.6.0 updated: [Gradle Release Plugin] - pre tag commit: 'v2.6.0-RC2'.

[pabloem]

This is an automated email from the ASF dual-hosted git repository.

pabloem pushed a commit to branch release-2.6.0
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/release-2.6.0 by this push:
 new 82fc380  [Gradle Release Plugin] - pre tag commit:  'v2.6.0-RC2'.
82fc380 is described below

commit 82fc3801423c051fafe1ea6ad9ae7891b40b442b
Author: Pablo 
AuthorDate: Fri Aug 3 15:38:34 2018 -0700

[Gradle Release Plugin] - pre tag commit:  'v2.6.0-RC2'.
---
 gradle.properties | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gradle.properties b/gradle.properties
index bf2e45d..34a5344 100644
--- a/gradle.properties
+++ b/gradle.properties
@@ -22,4 +22,4 @@ offlineRepositoryRoot=offline-repository
 signing.gnupg.executable=gpg
 signing.gnupg.useLegacyGpg=true
 
-version=2.6.0-SNAPSHOT
+version=2.6.0-RC2

[jira] [Work logged] (BEAM-5072) SortRelNode is wrongly removed

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5072?focusedWorklogId=131110=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131110
 ]

ASF GitHub Bot logged work on BEAM-5072:


Author: ASF GitHub Bot
Created on: 03/Aug/18 22:35
Start Date: 03/Aug/18 22:35
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #6139: [BEAM-5072] Remove 
RelCollationTraitDef configuration in BeamQueryPlanner
URL: https://github.com/apache/beam/pull/6139#issuecomment-410393205
 
 
   Related calcite code that shows how `SortRemoteRule` works: 
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/rules/SortRemoveRule.java#L47


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131110)
Time Spent: 0.5h  (was: 20m)

> SortRelNode is wrongly removed
> --
>
> Key: BEAM-5072
> URL: https://issues.apache.org/jira/browse/BEAM-5072
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> ORDER BY query generates LogicalSort in logical plan, but it will be removed 
> wrongly during optimization by the SortRemoveRule



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam] 01/01: Use the same language semantics as defined by version of the interpreter for Cythonizing. Use `past.builtins.basestring` consistently to type-check strings.

[pabloem]

This is an automated email from the ASF dual-hosted git repository.

pabloem pushed a commit to branch release-2.6.0
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 163e0f093c9d4d29b33b998802b277d4f6956f5e
Author: Valentyn Tymofieiev 
AuthorDate: Fri Aug 3 11:32:07 2018 -0700

Use the same language semantics as defined by version of the interpreter for
Cythonizing. Use `past.builtins.basestring` consistently to type-check 
strings.
---
 sdks/python/apache_beam/runners/common.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/sdks/python/apache_beam/runners/common.py 
b/sdks/python/apache_beam/runners/common.py
index aa43559..38ae666 100644
--- a/sdks/python/apache_beam/runners/common.py
+++ b/sdks/python/apache_beam/runners/common.py
@@ -15,7 +15,6 @@
 # limitations under the License.
 #
 
-# cython: language_level=3
 # cython: profile=True
 
 """Worker operations executor.
@@ -33,7 +32,6 @@ from builtins import zip
 
 from future.utils import raise_
 from past.builtins import basestring
-from past.builtins import unicode
 
 from apache_beam.internal import util
 from apache_beam.options.value_provider import RuntimeValueProvider
@@ -726,7 +724,7 @@ class _OutputProcessor(OutputProcessor):
   tag = None
   if isinstance(result, TaggedOutput):
 tag = result.tag
-if not isinstance(tag, (str, unicode)):
+if not isinstance(tag, basestring):
   raise TypeError('In %s, tag %s is not a string' % (self, tag))
 result = result.value
 

[beam] branch release-2.6.0 updated (dd06e9a -> 163e0f0)

[pabloem]

This is an automated email from the ASF dual-hosted git repository.

pabloem pushed a change to branch release-2.6.0
in repository https://gitbox.apache.org/repos/asf/beam.git.


 discard dd06e9a  Use the same language semantics as defined by version of the 
interpreter for Cythonizing. Use `past.builtins.basestring` consistently to 
type-check strings.
omit 62a668f  [Gradle Release Plugin] - pre tag commit:  'v2.6.0-RC2'.
omit 4fc6f99  [Gradle Release Plugin] - new version commit:  
'v2.6.0-SNAPSHOT'.
omit 5b90329  [Gradle Release Plugin] - pre tag commit:  'v2.6.0-RC2'.
 new 163e0f0  Use the same language semantics as defined by version of the 
interpreter for Cythonizing. Use `past.builtins.basestring` consistently to 
type-check strings.

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (dd06e9a)
\
 N -- N -- N   refs/heads/release-2.6.0 (163e0f0)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 gradle.properties | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Build failed in Jenkins: beam_PostRelease_NightlySnapshot #325

[Apache Jenkins Server]

See 


--
[...truncated 1.63 MB...]
Aug 03, 2018 10:25:22 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
moveToOutputFiles
INFO: Will copy temporary file 
FileResult{tempFilename=/tmp/groovy-generated-3451101599613145920-tmpdir/word-count-beam/.temp-beam-2018-08-03_22-25-19-1/850f0724-f77a-469f-a646-1851567d249b,
 shard=2, 
window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@4ee1d19f, 
paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, 
onTimeIndex=0}} to final location 
/tmp/groovy-generated-3451101599613145920-tmpdir/word-count-beam/counts-2-of-5
Aug 03, 2018 10:25:22 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
moveToOutputFiles
INFO: Will copy temporary file 
FileResult{tempFilename=/tmp/groovy-generated-3451101599613145920-tmpdir/word-count-beam/.temp-beam-2018-08-03_22-25-19-1/34050bc9-cc45-4eec-ad11-1e2884d77659,
 shard=1, 
window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@4ee1d19f, 
paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, 
onTimeIndex=0}} to final location 
/tmp/groovy-generated-3451101599613145920-tmpdir/word-count-beam/counts-1-of-5
Aug 03, 2018 10:25:22 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
removeTemporaryFiles
INFO: Will remove known temporary file 
/tmp/groovy-generated-3451101599613145920-tmpdir/word-count-beam/.temp-beam-2018-08-03_22-25-19-1/850f0724-f77a-469f-a646-1851567d249b
Aug 03, 2018 10:25:22 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
removeTemporaryFiles
INFO: Will remove known temporary file 
/tmp/groovy-generated-3451101599613145920-tmpdir/word-count-beam/.temp-beam-2018-08-03_22-25-19-1/9bb8fbd6-a669-4639-bbae-9c013eef22a8
Aug 03, 2018 10:25:22 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
removeTemporaryFiles
INFO: Will remove known temporary file 
/tmp/groovy-generated-3451101599613145920-tmpdir/word-count-beam/.temp-beam-2018-08-03_22-25-19-1/34050bc9-cc45-4eec-ad11-1e2884d77659
Aug 03, 2018 10:25:22 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
removeTemporaryFiles
INFO: Will remove known temporary file 
/tmp/groovy-generated-3451101599613145920-tmpdir/word-count-beam/.temp-beam-2018-08-03_22-25-19-1/0e9fc825-aa49-4bac-88c5-5bc6f1784622
Aug 03, 2018 10:25:22 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
removeTemporaryFiles
INFO: Will remove known temporary file 
/tmp/groovy-generated-3451101599613145920-tmpdir/word-count-beam/.temp-beam-2018-08-03_22-25-19-1/db48b7a5-5ebb-4ab8-a32e-6e50efc6ef43
grep Foundation counts*
counts-3-of-5:Foundation: 1
Verified Foundation: 1
[SUCCESS]
:beam-runners-direct-java:runQuickstartJavaDirect (Thread[Task worker for ':' 
Thread 11,5,main]) completed. Took 1 mins 35.613 secs.

> Task :beam-runners-google-cloud-dataflow-java:runQuickstartJavaDataflow
Aug 03, 2018 10:26:31 PM 
org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory
 create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 03, 2018 10:26:31 PM org.apache.beam.runners.dataflow.DataflowRunner 
fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from 
the classpath: will stage 114 files. Enable logging at DEBUG level to see which 
files will be staged.
Aug 03, 2018 10:26:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing 
implications related to Google Compute Engine usage and other Google Cloud 
Services.
Aug 03, 2018 10:26:32 PM org.apache.beam.runners.dataflow.util.PackageUtil 
stageClasspathElements
INFO: Uploading 114 files from PipelineOptions.filesToStage to staging location 
to prepare for execution.
Aug 03, 2018 10:26:32 PM org.apache.beam.runners.dataflow.util.PackageUtil 
tryStagePackage
INFO: Uploading 
/tmp/groovy-generated-6803234455974965812-tmpdir/word-count-beam/target/classes 
to 
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/classes-BwaAk5s21ZqaF2XnW0VZ4Q.jar
Aug 03, 2018 10:26:33 PM org.apache.beam.runners.dataflow.util.PackageUtil 
stageClasspathElements
INFO: Staging files complete: 113 files cached, 1 files newly uploaded
Aug 03, 2018 10:26:33 PM 
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ReadLines/Read as step s1
Aug 03, 2018 10:26:33 PM org.apache.beam.sdk.io.FileBasedSource 
getEstimatedSizeBytes
INFO: Filepattern gs://apache-beam-samples/shakespeare/* matched 43 files with 
total size 5284696
Aug 03, 2018 10:26:33 PM 
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WordCount.CountWords/ParDo(ExtractWords) as step s2
Aug 03, 2018 10:26:33 PM 
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding 

[beam] branch release-2.6.0 updated: Use the same language semantics as defined by version of the interpreter for Cythonizing. Use `past.builtins.basestring` consistently to type-check strings.

[pabloem]

This is an automated email from the ASF dual-hosted git repository.

pabloem pushed a commit to branch release-2.6.0
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/release-2.6.0 by this push:
 new dd06e9a  Use the same language semantics as defined by version of the 
interpreter for Cythonizing. Use `past.builtins.basestring` consistently to 
type-check strings.
dd06e9a is described below

commit dd06e9a5df10fde43d33c112c0ebf2fb1d020018
Author: Valentyn Tymofieiev 
AuthorDate: Fri Aug 3 11:32:07 2018 -0700

Use the same language semantics as defined by version of the interpreter for
Cythonizing. Use `past.builtins.basestring` consistently to type-check 
strings.
---
 sdks/python/apache_beam/runners/common.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/sdks/python/apache_beam/runners/common.py 
b/sdks/python/apache_beam/runners/common.py
index aa43559..38ae666 100644
--- a/sdks/python/apache_beam/runners/common.py
+++ b/sdks/python/apache_beam/runners/common.py
@@ -15,7 +15,6 @@
 # limitations under the License.
 #
 
-# cython: language_level=3
 # cython: profile=True
 
 """Worker operations executor.
@@ -33,7 +32,6 @@ from builtins import zip
 
 from future.utils import raise_
 from past.builtins import basestring
-from past.builtins import unicode
 
 from apache_beam.internal import util
 from apache_beam.options.value_provider import RuntimeValueProvider
@@ -726,7 +724,7 @@ class _OutputProcessor(OutputProcessor):
   tag = None
   if isinstance(result, TaggedOutput):
 tag = result.tag
-if not isinstance(tag, (str, unicode)):
+if not isinstance(tag, basestring):
   raise TypeError('In %s, tag %s is not a string' % (self, tag))
 result = result.value
 

[jira] [Work logged] (BEAM-5072) SortRelNode is wrongly removed

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5072?focusedWorklogId=131107=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131107
 ]

ASF GitHub Bot logged work on BEAM-5072:


Author: ASF GitHub Bot
Created on: 03/Aug/18 22:27
Start Date: 03/Aug/18 22:27
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #6139: [BEAM-5072] Remove 
RelCollationTraitDef configuration in BeamQueryPlanner
URL: https://github.com/apache/beam/pull/6139#issuecomment-410391984
 
 
   R: @apilloud @akedin 
   
   Also created JIRA https://issues.apache.org/jira/browse/BEAM-5073 to track 
enabling SortRemoveRule.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131107)
Time Spent: 20m  (was: 10m)

> SortRelNode is wrongly removed
> --
>
> Key: BEAM-5072
> URL: https://issues.apache.org/jira/browse/BEAM-5072
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> ORDER BY query generates LogicalSort in logical plan, but it will be removed 
> wrongly during optimization by the SortRemoveRule



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5073) Enable SortRemoveRule

[Rui Wang (JIRA)]

Rui Wang created BEAM-5073:
--

 Summary: Enable SortRemoveRule
 Key: BEAM-5073
 URL: https://issues.apache.org/jira/browse/BEAM-5073
 Project: Beam
  Issue Type: Improvement
  Components: dsl-sql
Reporter: Rui Wang
Assignee: Rui Wang


Enable SortRemoveRule by using RelCollationTraitDef.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam] branch master updated: Automate "Run validation tests" step (#6081)

[altay]

This is an automated email from the ASF dual-hosted git repository.

altay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new b944f66  Automate "Run validation tests" step (#6081)
b944f66 is described below

commit b944f6631de05625b21226af4e448378826aaf89
Author: Boyuan Zhang <36090911+boyua...@users.noreply.github.com>
AuthorDate: Fri Aug 3 15:26:15 2018 -0700

Automate "Run validation tests" step (#6081)

* Automate java quickstart validation tests
* Automate java mobile game validations
* Automate "Create a PR to trigger python validations"
---
 release/src/main/scripts/run_rc_validation.sh | 541 ++
 1 file changed, 541 insertions(+)

diff --git a/release/src/main/scripts/run_rc_validation.sh 
b/release/src/main/scripts/run_rc_validation.sh
new file mode 100755
index 000..0bd3e23
--- /dev/null
+++ b/release/src/main/scripts/run_rc_validation.sh
@@ -0,0 +1,541 @@
+#!/bin/bash
+#
+#Licensed to the Apache Software Foundation (ASF) under one or more
+#contributor license agreements.  See the NOTICE file distributed with
+#this work for additional information regarding copyright ownership.
+#The ASF licenses this file to You under the Apache License, Version 2.0
+#(the "License"); you may not use this file except in compliance with
+#the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+#Unless required by applicable law or agreed to in writing, software
+#distributed under the License is distributed on an "AS IS" BASIS,
+#WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#See the License for the specific language governing permissions and
+#limitations under the License.
+#
+
+function clean_up(){
+  echo "==Stopping Pubsub Java 
Injector="
+  echo "Please stop java injector manually."
+  echo "===Signing up 
Spreadsheet==="
+  echo "Please open this spreadsheet: 
https://s.apache.org/beam-release-validation;
+  echo "Please sign up your name in the tests you have ran."
+
+  echo "===Final Cleanup==="
+  echo "* Restoring ~/.m2/settings.xml"
+  if [[ ! -z `ls -a ~/.m2/settings_backup.xml` ]]; then
+rm ~/.m2/settings.xml
+cp ~/.m2/settings_backup.xml ~/.m2/settings.xml
+  fi
+  echo "* Restoring ~/.bashrc"
+  if [[ ! -z `ls -a ~/.bashrc_backup` ]]; then
+rm ~/.bashrc
+cp ~/.bashrc_backup ~/.bashrc
+  fi
+
+  rm -rf ~/{LOCAL_CLONE_DIR}
+}
+
+RELEASE=
+REPO_URL=
+RC_NUM=
+RELEASE_BRANCH=
+WORKING_BRANCH=
+LOCAL_CLONE_DIR=rc_validations
+BEAM_ROOT_DIR=beam
+GIT_REPO_URL=https://github.com/apache/beam.git
+PYTHON_RC_DOWNLOAD_URL=https://dist.apache.org/repos/dist/dev/beam
+
+echo "[Input Required] Please enter the release version: "
+read RELEASE
+RELEASE_BRANCH=release-${RELEASE}
+WORKING_BRANCH=release-${RELEASE}-RC${RC_NUM}_validations
+echo "[Input Required] Please enter the release candidate number(e.g. 1): "
+read RC_NUM
+echo "[Input Required] Please copy the repo URL from the vote email sent out 
by Release Manager:"
+echo "The URL should look like: 
https://repository.apache.org/content/repositories/orgapachebeam-;
+read REPO_URL
+
+echo "Checking Environment Variables="
+echo "running validations on release ${RELEASE} RC${RC_NUM}."
+echo "repo URL for this RC: ${REPO_URL}"
+echo "[Confirmation Required] Do you confirm all information above are 
correct? [y|N]"
+read confirmation
+if [[ $confirmation != "y" ]]; then
+  echo "Please rerun this script and make sure you have the right inputs."
+  exit
+fi
+
+echo "Cloning Beam Release Branch"
+cd ~
+mkdir ${LOCAL_CLONE_DIR}
+cd ${LOCAL_CLONE_DIR}
+git clone ${GIT_REPO_URL}
+cd ${BEAM_ROOT_DIR}
+git checkout ${RELEASE_BRANCH}
+git checkout -b ${WORKING_BRANCH}
+
+echo "Starting Java Quickstart==="
+echo "[Current task] Java quickstart with direct runner"
+echo "[Confirmation Required] Do you want to start this task? [y|N]"
+read confirmation
+if [[ $confirmation = "y" ]]; then
+  echo "*"
+  echo "* Running Java Quickstart with DirectRunner"
+  echo "*"
+  ./gradlew :beam-runners-direct-java:runQuickstartJavaDirect \
+  -Prepourl=${REPO_URL} \
+  -Pver=${RELEASE}
+fi
+
+echo "[Current task] Java quickstart with Apex local runner"
+echo "[Confirmation Required] Do you want to start this task? [y|N]"
+read confirmation
+if [[ $confirmation = "y" ]]; then
+  echo "*"
+  echo "* Running Java Quickstart with Apex local 

[jira] [Work logged] (BEAM-5072) SortRelNode is wrongly removed

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5072?focusedWorklogId=131106=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131106
 ]

ASF GitHub Bot logged work on BEAM-5072:


Author: ASF GitHub Bot
Created on: 03/Aug/18 22:25
Start Date: 03/Aug/18 22:25
Worklog Time Spent: 10m 
  Work Description: amaliujia opened a new pull request #6139: [BEAM-5072] 
Remove RelCollationTraitDef configuration in BeamQueryPlanner
URL: https://github.com/apache/beam/pull/6139
 
 
   Right now we configure `RelCollationTraitDef` in `BeamQueryPlanner` but 
seems like we don't use/implement it. However this configuration causes 
`SortRemoveRule` to apply sort to SortRel's input and then drop the SorRel 
(which does not really sort the SortRel's input). 
   
   Remove `RelCollationTraitDef` for now.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 131106)
Time Spent: 10m
Remaining Estimate: 0h

> SortRelNode is wrongly removed
> --
>
> Key: BEAM-5072
> URL: https://issues.apache.org/jira/browse/BEAM-5072
> Project: Beam
>  Issue Type: Bug
>  

[beam] branch release-2.6.0 updated: [Gradle Release Plugin] - pre tag commit: 'v2.6.0-RC2'.

[pabloem]

This is an automated email from the ASF dual-hosted git repository.

pabloem pushed a commit to branch release-2.6.0
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/release-2.6.0 by this push:
 new 62a668f  [Gradle Release Plugin] - pre tag commit:  'v2.6.0-RC2'.
62a668f is described below

commit 62a668fcd8166c9da94e14c355efc49c3f54a33f
Author: Pablo 
AuthorDate: Fri Aug 3 15:23:52 2018 -0700

[Gradle Release Plugin] - pre tag commit:  'v2.6.0-RC2'.
---
 gradle.properties | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gradle.properties b/gradle.properties
index bf2e45d..34a5344 100644
--- a/gradle.properties
+++ b/gradle.properties
@@ -22,4 +22,4 @@ offlineRepositoryRoot=offline-repository
 signing.gnupg.executable=gpg
 signing.gnupg.useLegacyGpg=true
 
-version=2.6.0-SNAPSHOT
+version=2.6.0-RC2

[jira] [Created] (BEAM-5072) SortRelNode is wrongly removed

[Rui Wang (JIRA)]

Rui Wang created BEAM-5072:
--

 Summary: SortRelNode is wrongly removed
 Key: BEAM-5072
 URL: https://issues.apache.org/jira/browse/BEAM-5072
 Project: Beam
  Issue Type: Bug
  Components: dsl-sql
Reporter: Rui Wang
Assignee: Rui Wang


ORDER BY query generates LogicalSort in logical plan, but it will be removed 
wrongly during optimization by the SortRemoveRule



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Jenkins build is back to normal : beam_PostCommit_Java_GradleBuild #1169

[Apache Jenkins Server]

See 

[jira] [Work logged] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?focusedWorklogId=131104=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-131104
 ]

ASF GitHub Bot logged work on BEAM-4761:


Author: ASF GitHub Bot
Created on: 03/Aug/18 22:08
Start Date: 03/Aug/18 22:08
Worklog Time Spent: 10m 
  Work Description: apilloud closed pull request #6127: [BEAM-4761] Run 
Nexmark in thread pool
URL: https://github.com/apache/beam/pull/6127
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy 
b/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
index 3abf2b16c6f..5ff02f91a35 100644
--- a/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
+++ b/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow.groovy
@@ -39,6 +39,7 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--nexmarkParallel=16',
   '--streaming=false',
   '--suite=STRESS',
   '--manageResources=false',
@@ -55,6 +56,7 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--nexmarkParallel=16',
   '--streaming=true',
   '--suite=STRESS',
   '--manageResources=false',
@@ -71,6 +73,7 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--nexmarkParallel=16',
   '--queryLanguage=sql',
   '--streaming=false',
   '--suite=STRESS',
@@ -88,6 +91,7 @@ 
NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_
   ' -Pnexmark.args="' +
   [NexmarkBigqueryProperties.nexmarkBigQueryArgs,
   '--runner=DataflowRunner',
+  '--nexmarkParallel=16',
   '--queryLanguage=sql',
   '--streaming=true',
   '--suite=STRESS',
diff --git 
a/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java 
b/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
index 537ddd22985..ecd7b5b9bf6 100644
--- a/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
+++ b/sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/Main.java
@@ -33,6 +33,13 @@
 import java.util.LinkedHashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletionService;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ExecutorCompletionService;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
 import javax.annotation.Nullable;
 import org.apache.beam.sdk.Pipeline;
 import org.apache.beam.sdk.coders.CoderException;
@@ -74,30 +81,81 @@
  * See http://datalab.cs.pdx.edu/niagaraST/NEXMark/;>
  * http://datalab.cs.pdx.edu/niagaraST/NEXMark/
  */
-public class Main {
+public class Main {
+
+  private static class Result {
+private final NexmarkConfiguration configuration;
+private final NexmarkPerf perf;
+
+private Result(NexmarkConfiguration configuration, NexmarkPerf perf) {
+  this.configuration = configuration;
+  this.perf = perf;
+}
+  }
+
+  private static class Run implements Callable {
+private final NexmarkLauncher nexmarkLauncher;
+private final NexmarkConfiguration configuration;
+
+private Run(String[] args, NexmarkConfiguration configuration) {
+  NexmarkOptions options = 
PipelineOptionsFactory.fromArgs(args).as(NexmarkOptions.class);
+  this.nexmarkLauncher = new NexmarkLauncher<>(options);
+  this.configuration = configuration;
+}
+
+@Override
+public Result call() throws IOException {
+  NexmarkPerf perf = nexmarkLauncher.run(configuration);
+  return new Result(configuration, perf);
+}
+  }
 
   /** Entry point. */
-  void runAll(OptionT options, NexmarkLauncher nexmarkLauncher) throws 
IOException {
+  void runAll(String[] args) throws IOException {
 Instant start = Instant.now();
+NexmarkOptions options =
+
PipelineOptionsFactory.fromArgs(args).withValidation().as(NexmarkOptions.class);
 Map baseline = 

[beam] 01/01: Merge pull request #6127 from apilloud/parallel-nexmark

[apilloud]

This is an automated email from the ASF dual-hosted git repository.

apilloud pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 01caec8beacc204203eb8c278ef464e2664f255a
Merge: 4490cb6 8b478c5
Author: Andrew Pilloud 
AuthorDate: Fri Aug 3 15:08:06 2018 -0700

Merge pull request #6127 from apilloud/parallel-nexmark

[BEAM-4761] Run Nexmark in thread pool

 .../job_PostCommit_Java_Nexmark_Dataflow.groovy|  4 +
 .../java/org/apache/beam/sdk/nexmark/Main.java | 89 ++
 .../apache/beam/sdk/nexmark/NexmarkOptions.java|  6 ++
 .../org/apache/beam/sdk/nexmark/NexmarkSuite.java  |  2 +-
 4 files changed, 84 insertions(+), 17 deletions(-)

[beam] branch master updated (4490cb6 -> 01caec8)

[apilloud]

This is an automated email from the ASF dual-hosted git repository.

apilloud pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 4490cb6  Merge pull request #6006 from 
amaliujia/rui_wang-e2e_test_beamsqlline
 add 8b478c5  [BEAM-4761] Run Nexmark in thread pool
 new 01caec8  Merge pull request #6127 from apilloud/parallel-nexmark

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../job_PostCommit_Java_Nexmark_Dataflow.groovy|  4 +
 .../java/org/apache/beam/sdk/nexmark/Main.java | 89 ++
 .../apache/beam/sdk/nexmark/NexmarkOptions.java|  6 ++
 .../org/apache/beam/sdk/nexmark/NexmarkSuite.java  |  2 +-
 4 files changed, 84 insertions(+), 17 deletions(-)

[jira] [Created] (BEAM-5071) Using the restful API in Beam dependency check system, get rid of bigquery

[yifan zou (JIRA)]

yifan zou created BEAM-5071:
---

 Summary: Using the restful API in Beam dependency check system, 
get rid of bigquery
 Key: BEAM-5071
 URL: https://issues.apache.org/jira/browse/BEAM-5071
 Project: Beam
  Issue Type: Bug
  Components: dependencies
Reporter: yifan zou
Assignee: yifan zou






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)