[jira] [Updated] (BEAM-5376) Row interface doesn't support nullability on all fields.

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-5376:
-
Fix Version/s: (was: Not applicable)
   2.8.0

> Row interface doesn't support nullability on all fields.
> 
>
> Key: BEAM-5376
> URL: https://issues.apache.org/jira/browse/BEAM-5376
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> For example: 
> {code:java}
> public boolean getBoolean(int idx);{code}
> Should be:
> {code:java}
> public Boolean getBoolean(int idx);{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-5572) Beam Dependency Update Request: jline:jline

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-5572.
--
   Resolution: Invalid
Fix Version/s: Not applicable

> Beam Dependency Update Request: jline:jline
> ---
>
> Key: BEAM-5572
> URL: https://issues.apache.org/jira/browse/BEAM-5572
> Project: Beam
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Priority: Major
> Fix For: Not applicable
>
>
>  - 2018-10-01 19:31:43.549653 
> -
> Please consider upgrading the dependency jline:jline. 
> The current version is 2.14.6. The latest version is 3.0.0.M1 
> cc: [~kenn], [~kedin], [~apilloud], [~amaliujia], 
>  Please refer to [Beam Dependency Guide 
> |https://beam.apache.org/contribute/dependencies/]for more information. 
> Do Not Modify The Description Above. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5491) Beam Site build broken by deleted jenkins jobs

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5491:


 Summary: Beam Site build broken by deleted jenkins jobs
 Key: BEAM-5491
 URL: https://issues.apache.org/jira/browse/BEAM-5491
 Project: Beam
  Issue Type: Bug
  Components: website
Reporter: Andrew Pilloud
Assignee: Andrew Pilloud


{code:java}
16:10:09 - ./.testcontent/contribute/release-guide/index.html
16:10:09   *  External link 
https://builds.apache.org/view/A-D/view/Beam/job/beam_Release_NightlySnapshot/ 
failed: 404 No error
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5416) Split Aggregate and Window rel nodes in SQL

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5416:


 Summary: Split Aggregate and Window rel nodes in SQL
 Key: BEAM-5416
 URL: https://issues.apache.org/jira/browse/BEAM-5416
 Project: Beam
  Issue Type: Improvement
  Components: dsl-sql
Reporter: Andrew Pilloud


Currently BeamAggregationRel impliments both Aggregate and Window operations. 
We should split this into two rel nodes if possible.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5411) Separate BeamUnnest and BeamCalc

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5411:


 Summary: Separate BeamUnnest and BeamCalc
 Key: BEAM-5411
 URL: https://issues.apache.org/jira/browse/BEAM-5411
 Project: Beam
  Issue Type: Improvement
  Components: dsl-sql
Reporter: Andrew Pilloud
Assignee: Andrew Pilloud


Currently Correlated Uncollect (BeamUnnest) embeds a fork of BeamCalc. This 
isn't actually needed, simplifying this node enables easier replacement of Calc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5376) Row interface doesn't support nullability on all fields.

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5376:


 Summary: Row interface doesn't support nullability on all fields.
 Key: BEAM-5376
 URL: https://issues.apache.org/jira/browse/BEAM-5376
 Project: Beam
  Issue Type: Improvement
  Components: dsl-sql
Reporter: Andrew Pilloud
Assignee: Andrew Pilloud


For example: 
{code:java}
public boolean getBoolean(int idx);{code}
Should be:
{code:java}
public Boolean getBoolean(int idx);{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (BEAM-5279) Metadata is lost in CalciteUtils schema conversion

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud closed BEAM-5279.

   Resolution: Fixed
Fix Version/s: 2.8.0

> Metadata is lost in CalciteUtils schema conversion
> --
>
> Key: BEAM-5279
> URL: https://issues.apache.org/jira/browse/BEAM-5279
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> We just delete it:
> {code:java}
> return CALCITE_TO_BEAM_TYPE_MAPPING.get(sqlTypeName).withMetadata((byte[]) 
> null);{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (BEAM-5285) Post Commit failed: org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud closed BEAM-5285.

   Resolution: Duplicate
Fix Version/s: Not applicable

> Post Commit failed: 
> org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq
> 
>
> Key: BEAM-5285
> URL: https://issues.apache.org/jira/browse/BEAM-5285
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Boyuan Zhang
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: Not applicable
>
>
> Test log: 
> https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1375/testReport/junit/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-5261) Sql array indexing should be 1-based

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-5261.
--
Resolution: Fixed

> Sql array indexing should be 1-based
> 
>
> Key: BEAM-5261
> URL: https://issues.apache.org/jira/browse/BEAM-5261
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Affects Versions: 2.5.0, 2.6.0, 2.7.0
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Beam SQL uses zero base array indexing. Calcite, Flink, and the SQL Standard 
> are 1 based. See 
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (BEAM-5261) Sql array indexing should be 1-based

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud closed BEAM-5261.


> Sql array indexing should be 1-based
> 
>
> Key: BEAM-5261
> URL: https://issues.apache.org/jira/browse/BEAM-5261
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Affects Versions: 2.5.0, 2.6.0, 2.7.0
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Beam SQL uses zero base array indexing. Calcite, Flink, and the SQL Standard 
> are 1 based. See 
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-5261) Sql array indexing should be 1-based

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-5261:
-
Fix Version/s: 2.8.0

> Sql array indexing should be 1-based
> 
>
> Key: BEAM-5261
> URL: https://issues.apache.org/jira/browse/BEAM-5261
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Affects Versions: 2.5.0, 2.6.0, 2.7.0
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Beam SQL uses zero base array indexing. Calcite, Flink, and the SQL Standard 
> are 1 based. See 
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5279) Metadata is lost in CalciteUtils schema conversion

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5279:


 Summary: Metadata is lost in CalciteUtils schema conversion
 Key: BEAM-5279
 URL: https://issues.apache.org/jira/browse/BEAM-5279
 Project: Beam
  Issue Type: Bug
  Components: dsl-sql
Reporter: Andrew Pilloud
Assignee: Andrew Pilloud


We just delete it:
{code:java}
return CALCITE_TO_BEAM_TYPE_MAPPING.get(sqlTypeName).withMetadata((byte[]) 
null);{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5275) Beam SQL RAND(seed) function isn't compatible with dofn

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5275:


 Summary: Beam SQL RAND(seed) function isn't compatible with dofn
 Key: BEAM-5275
 URL: https://issues.apache.org/jira/browse/BEAM-5275
 Project: Beam
  Issue Type: Bug
  Components: dsl-sql
Reporter: Andrew Pilloud
Assignee: Xu Mingmin


We currently wrap the RAND(seed) operator with a normal dofn, but that isn't 
actually valid because we don't process the rows in a single instance of that 
dofn. We should revisit if this is something we even want to support.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-4704) String operations yield incorrect results when executed through SQL shell

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597973#comment-16597973
 ] 

Andrew Pilloud commented on BEAM-4704:
--

We can't override calcite's implementation if calcite is generating our code. 
Fixing is in calcite blocks BEAM-5112

> String operations yield incorrect results when executed through SQL shell
> -
>
> Key: BEAM-4704
> URL: https://issues.apache.org/jira/browse/BEAM-4704
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Priority: Major
>
> {{TRIM}} is defined to trim _all_ the characters in the first string from the 
> string-to-be-trimmed. Calcite has an incorrect implementation of this. We use 
> our own fixed implementation. But when executed through the SQL shell, the 
> results do not match what we get from the PTransform path. Here two test 
> cases that pass on {{master}} but are incorrect in the shell:
> {code:sql}
> BeamSQL> select TRIM(LEADING 'eh' FROM 'hehe__hehe');
> ++
> | EXPR$0 |
> ++
> | hehe__hehe |
> ++
> {code}
> {code:sql}
> BeamSQL> select TRIM(TRAILING 'eh' FROM 'hehe__hehe');
> ++
> |   EXPR$0   |
> ++
> | hehe__heh  |
> ++
> {code}
> {code:sql}
> BeamSQL> select TRIM(BOTH 'eh' FROM 'hehe__hehe');
> ++
> |   EXPR$0   |
> ++
> | hehe__heh  |
> ++
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-5261) Sql array indexing should be 1-based

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-5261:
-
Fix Version/s: (was: 2.7.0)

> Sql array indexing should be 1-based
> 
>
> Key: BEAM-5261
> URL: https://issues.apache.org/jira/browse/BEAM-5261
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Affects Versions: 2.5.0, 2.6.0, 2.7.0
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
>
> Beam SQL uses zero base array indexing. Calcite, Flink, and the SQL Standard 
> are 1 based. See 
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-5261) Sql array indexing should be 1-based

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-5261:
-
Fix Version/s: 2.7.0

> Sql array indexing should be 1-based
> 
>
> Key: BEAM-5261
> URL: https://issues.apache.org/jira/browse/BEAM-5261
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Affects Versions: 2.5.0, 2.6.0, 2.7.0
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
>
> Beam SQL uses zero base array indexing. Calcite, Flink, and the SQL Standard 
> are 1 based. See 
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-5261) Sql array indexing should be 1-based

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-5261:
-
Affects Version/s: 2.7.0
   2.5.0
   2.6.0

> Sql array indexing should be 1-based
> 
>
> Key: BEAM-5261
> URL: https://issues.apache.org/jira/browse/BEAM-5261
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Affects Versions: 2.5.0, 2.6.0, 2.7.0
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
>
> Beam SQL uses zero base array indexing. Calcite, Flink, and the SQL Standard 
> are 1 based. See 
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5261) Sql array indexing should be 1-based

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596876#comment-16596876
 ] 

Andrew Pilloud commented on BEAM-5261:
--

[https://github.com/apache/beam/commit/21a6c7862bc382bbb55fa7fa85f98600ca669345]
 implimented this, so it is new.

> Sql array indexing should be 1-based
> 
>
> Key: BEAM-5261
> URL: https://issues.apache.org/jira/browse/BEAM-5261
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Affects Versions: 2.5.0, 2.6.0, 2.7.0
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
>
> Beam SQL uses zero base array indexing. Calcite, Flink, and the SQL Standard 
> are 1 based. See 
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-5261) Sql array indexing should be 1-based

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud reassigned BEAM-5261:


Assignee: Andrew Pilloud  (was: Xu Mingmin)

> Sql array indexing should be 1-based
> 
>
> Key: BEAM-5261
> URL: https://issues.apache.org/jira/browse/BEAM-5261
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
>
> Beam SQL uses zero base array indexing. Calcite, Flink, and the SQL Standard 
> are 1 based. See 
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5261) Sql array indexing should be 1-based

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5261:


 Summary: Sql array indexing should be 1-based
 Key: BEAM-5261
 URL: https://issues.apache.org/jira/browse/BEAM-5261
 Project: Beam
  Issue Type: Bug
  Components: dsl-sql
Reporter: Andrew Pilloud
Assignee: Xu Mingmin


Beam SQL uses zero base array indexing. Calcite, Flink, and the SQL Standard 
are 1 based. See 
https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-5249) org.apache.beam.runners.flink.streaming.UnboundedSourceWrapperTest ParameterizedUnboundedSourceWrapperTest hangs

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-5249:
-
Component/s: (was: build-system)
 runner-flink

> org.apache.beam.runners.flink.streaming.UnboundedSourceWrapperTest 
> ParameterizedUnboundedSourceWrapperTest hangs
> 
>
> Key: BEAM-5249
> URL: https://issues.apache.org/jira/browse/BEAM-5249
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Alan Myrvold
>Assignee: Alan Myrvold
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> beam_Release_Gradle_NightlySnapshot sometimes times out at 100 minutes
> [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/155/]
> [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/152/]
> https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/142/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5249) beam_Release_Gradle_NightlySnapshot timeouts

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595335#comment-16595335
 ] 

Andrew Pilloud commented on BEAM-5249:
--

This just hit on PostCommit, it is a failure in the flink runner.

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1363/]
*17:52:45* [Test worker] INFO 
org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapper
 - Unbounded Flink Source 0/4 is reading from sources: 
[org.apache.beam.runners.flink.streaming.TestCountingSource@13904536]*21:11:45* 
Build timed out (after 240 minutes). Marking the build as aborted.

> beam_Release_Gradle_NightlySnapshot timeouts
> 
>
> Key: BEAM-5249
> URL: https://issues.apache.org/jira/browse/BEAM-5249
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Reporter: Alan Myrvold
>Assignee: Alan Myrvold
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> beam_Release_Gradle_NightlySnapshot sometimes times out at 100 minutes
> [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/155/]
> [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/152/]
> https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/142/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-5249) org.apache.beam.runners.flink.streaming.UnboundedSourceWrapperTest ParameterizedUnboundedSourceWrapperTest hangs

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-5249:
-
Summary: org.apache.beam.runners.flink.streaming.UnboundedSourceWrapperTest 
ParameterizedUnboundedSourceWrapperTest hangs  (was: 
beam_Release_Gradle_NightlySnapshot timeouts)

> org.apache.beam.runners.flink.streaming.UnboundedSourceWrapperTest 
> ParameterizedUnboundedSourceWrapperTest hangs
> 
>
> Key: BEAM-5249
> URL: https://issues.apache.org/jira/browse/BEAM-5249
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Reporter: Alan Myrvold
>Assignee: Alan Myrvold
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> beam_Release_Gradle_NightlySnapshot sometimes times out at 100 minutes
> [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/155/]
> [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/152/]
> https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/142/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5213) sdks/go/test/regression failed with no results

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594312#comment-16594312
 ] 

Andrew Pilloud commented on BEAM-5213:
--

There look to be more logs this time: 
https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/805/

 
*15:34:43* FAIL github.com/apache/beam/sdks/go/test/regression [build 
failed]*15:34:43* # 
github.com/apache/beam/sdks/go/test/vendor/google.golang.org/grpc*15:34:43* 
.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/google.golang.org/grpc/clientconn.go:86:14:
 undefined: UnaryClientInterceptor*15:34:43* 
.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/google.golang.org/grpc/clientconn.go:87:14:
 undefined: StreamClientInterceptor*15:34:43* 
.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/google.golang.org/grpc/clientconn.go:88:14:
 undefined: Compressor*15:34:43* 
.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/google.golang.org/grpc/clientconn.go:89:14:
 undefined: Decompressor*15:34:43* 
.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/google.golang.org/grpc/clientconn.go:90:14:
 undefined: backoffStrategy*15:34:43* 
.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/google.golang.org/grpc/clientconn.go:96:16:
 undefined: CallOption*15:34:43* *15:34:43* > Task :beam-sdks-go:test*15:34:43* 
Build cache key for task ':beam-sdks-go:test' is 
ac76939d9fb11c3256723170d73c2570*15:34:43* Caching disabled for task 
':beam-sdks-go:test': Caching has not been enabled for the task*15:34:43* Task 
':beam-sdks-go:test' is not up-to-date because:*15:34:43*   No history is 
available.*15:34:43* Result of package 
github.com/apache/beam/sdks/go/test/regression:*15:34:43* Test for 
github.com/apache/beam/sdks/go/test/regression finished, 1 completed, 1 failed

> sdks/go/test/regression failed with no results
> --
>
> Key: BEAM-5213
> URL: https://issues.apache.org/jira/browse/BEAM-5213
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-go
>Reporter: Andrew Pilloud
>Assignee: Henning Rohde
>Priority: Major
>
> The details on which test failed are missing, but something failed. 
> https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/784/
>  
> *11:26:20* Result of package 
> github.com/apache/beam/sdks/go/test/regression:*11:26:20* Test for 
> github.com/apache/beam/sdks/go/test/regression finished, 1 completed, 1 failed



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5244) beam_PostCommit_Py_VR_Dataflow test_as_list_and_as_dict_side_inputs CalledProcessError

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5244:


 Summary: beam_PostCommit_Py_VR_Dataflow 
test_as_list_and_as_dict_side_inputs CalledProcessError
 Key: BEAM-5244
 URL: https://issues.apache.org/jira/browse/BEAM-5244
 Project: Beam
  Issue Type: Bug
  Components: sdk-py-core
Reporter: Andrew Pilloud
Assignee: Ahmet Altay


https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/896/

 
*13:30:43* 
==*13:30:43*
 ERROR: test_as_list_and_as_dict_side_inputs 
(apache_beam.transforms.sideinputs_test.SideInputsTest)*13:30:43* 
--*13:30:43*
 Traceback (most recent call last):*13:30:43*   File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/sideinputs_test.py",
 line 205, in test_as_list_and_as_dict_side_inputs*13:30:43* 
pipeline.run()*13:30:43*   File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py",
 line 104, in run*13:30:43* result = super(TestPipeline, 
self).run(test_runner_api)*13:30:43*   File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
 line 394, in run*13:30:43* self.to_runner_api(), self.runner, 
self._options).run(False)*13:30:43*   File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
 line 407, in run*13:30:43* return self.runner.run_pipeline(self)*13:30:43* 
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
 line 50, in run_pipeline*13:30:43* self.result = super(TestDataflowRunner, 
self).run_pipeline(pipeline)*13:30:43*   File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
 line 371, in run_pipeline*13:30:43* 
self.dataflow_client.create_job(self.job), self)*13:30:43*   File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py",
 line 184, in wrapper*13:30:43* return fun(*args, **kwargs)*13:30:43*   
File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 490, in create_job*13:30:43* 
self.create_job_description(job)*13:30:43*   File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 519, in create_job_description*13:30:43* resources = 
self._stage_resources(job.options)*13:30:43*   File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 452, in _stage_resources*13:30:43* 
staging_location=google_cloud_options.staging_location)*13:30:43*   File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
 line 161, in stage_job_resources*13:30:43* 
requirements_cache_path)*13:30:43*   File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
 line 411, in _populate_requirements_cache*13:30:43* 
processes.check_call(cmd_args)*13:30:43*   File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py",
 line 46, in check_call*13:30:43* return subprocess.check_call(*args, 
**kwargs)*13:30:43*   File "/usr/lib/python2.7/subprocess.py", line 541, in 
check_call*13:30:43* raise CalledProcessError(retcode, cmd)*13:30:43* 
CalledProcessError: Command 
'['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python',
 '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 
'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', ':all:']' 
returned non-zero exit status 1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-5243) beam_Release_Gradle_NightlySnapshot InvocationError py27-cython/bin/python setup.py nosetests

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-5243:
-
Description: 
It isn't clear to me what exactly failed, logs are full of stack traces.
 [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/151/]

[https://builds.apache.org/job/beam_PostCommit_Python_Verify/5844/]

 
 *01:00:38* ERROR: InvocationError for command 
'/home/jenkins/jenkins-slave/workspace/beam_Release_Gradle_NightlySnapshot/src/sdks/python/target/.tox/py27-cython/bin/python
 setup.py nosetests' (exited with code -11)*01:00:38* 
___ summary 
*01:00:38* ERROR: py27-cython: commands 
failed*01:00:38*

  was:
It isn't clear to me what exactly failed, logs are full of stack traces.
[https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/151/]

 
*01:00:38* ERROR: InvocationError for command 
'/home/jenkins/jenkins-slave/workspace/beam_Release_Gradle_NightlySnapshot/src/sdks/python/target/.tox/py27-cython/bin/python
 setup.py nosetests' (exited with code -11)*01:00:38* 
___ summary 
*01:00:38* ERROR:   py27-cython: commands 
failed*01:00:38*


> beam_Release_Gradle_NightlySnapshot InvocationError py27-cython/bin/python 
> setup.py nosetests
> -
>
> Key: BEAM-5243
> URL: https://issues.apache.org/jira/browse/BEAM-5243
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Andrew Pilloud
>Assignee: Ahmet Altay
>Priority: Major
>
> It isn't clear to me what exactly failed, logs are full of stack traces.
>  [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/151/]
> [https://builds.apache.org/job/beam_PostCommit_Python_Verify/5844/]
>  
>  *01:00:38* ERROR: InvocationError for command 
> '/home/jenkins/jenkins-slave/workspace/beam_Release_Gradle_NightlySnapshot/src/sdks/python/target/.tox/py27-cython/bin/python
>  setup.py nosetests' (exited with code -11)*01:00:38* 
> ___ summary 
> *01:00:38* ERROR: py27-cython: commands 
> failed*01:00:38*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5243) beam_Release_Gradle_NightlySnapshot InvocationError py27-cython/bin/python setup.py nosetests

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5243:


 Summary: beam_Release_Gradle_NightlySnapshot InvocationError 
py27-cython/bin/python setup.py nosetests
 Key: BEAM-5243
 URL: https://issues.apache.org/jira/browse/BEAM-5243
 Project: Beam
  Issue Type: Bug
  Components: sdk-py-core
Reporter: Andrew Pilloud
Assignee: Ahmet Altay


It isn't clear to me what exactly failed, logs are full of stack traces.
[https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/151/]

 
*01:00:38* ERROR: InvocationError for command 
'/home/jenkins/jenkins-slave/workspace/beam_Release_Gradle_NightlySnapshot/src/sdks/python/target/.tox/py27-cython/bin/python
 setup.py nosetests' (exited with code -11)*01:00:38* 
___ summary 
*01:00:38* ERROR:   py27-cython: commands 
failed*01:00:38*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5213) sdks/go/test/regression failed with no results

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594287#comment-16594287
 ] 

Andrew Pilloud commented on BEAM-5213:
--

https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/789/

> sdks/go/test/regression failed with no results
> --
>
> Key: BEAM-5213
> URL: https://issues.apache.org/jira/browse/BEAM-5213
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-go
>Reporter: Andrew Pilloud
>Assignee: Henning Rohde
>Priority: Major
>
> The details on which test failed are missing, but something failed. 
> https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/784/
>  
> *11:26:20* Result of package 
> github.com/apache/beam/sdks/go/test/regression:*11:26:20* Test for 
> github.com/apache/beam/sdks/go/test/regression finished, 1 completed, 1 failed



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-5242) runMobileGamingJavaDirect Aborted after 100 minutes

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-5242:
-
Description: 
[https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/350/]

[https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/342/console]

[https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/340/]

[https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/337/]

 

 
 bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
 Build timed out (after 100 minutes). Marking the build as aborted.
 Build was aborted
 Finished: ABORTED

  was:
[https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/350/]

[https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/342/console]

 
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Build timed out (after 100 minutes). Marking the build as aborted.
Build was aborted
Finished: ABORTED


> runMobileGamingJavaDirect Aborted after 100 minutes
> ---
>
> Key: BEAM-5242
> URL: https://issues.apache.org/jira/browse/BEAM-5242
> Project: Beam
>  Issue Type: Bug
>  Components: examples-java
>Reporter: Andrew Pilloud
>Assignee: Reuven Lax
>Priority: Major
>
> [https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/350/]
> [https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/342/console]
> [https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/340/]
> [https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/337/]
>  
>  
>  bq query SELECT table_id FROM 
> beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
>  Build timed out (after 100 minutes). Marking the build as aborted.
>  Build was aborted
>  Finished: ABORTED



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5242) runMobileGamingJavaDirect Aborted after 100 minutes

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5242:


 Summary: runMobileGamingJavaDirect Aborted after 100 minutes
 Key: BEAM-5242
 URL: https://issues.apache.org/jira/browse/BEAM-5242
 Project: Beam
  Issue Type: Bug
  Components: examples-java
Reporter: Andrew Pilloud
Assignee: Reuven Lax


[https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/350/]

[https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/342/console]

 
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Build timed out (after 100 minutes). Marking the build as aborted.
Build was aborted
Finished: ABORTED



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-5241) runMobileGamingJavaDirect FAILED SocketException: Connection reset

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-5241:
-
Description: 
[https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/349/]

[https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/347/]


 *04:19:41* > Task :beam-runners-direct-java:runMobileGamingJavaDirect 
FAILED*04:19:41* Aug 23, 2018 11:19:32 AM 
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl 
createTable*04:19:41* INFO: Trying to create BigQuery table: 
apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DirectRunner_user*04:19:41*
 bq query SELECT table_id FROM 
beam_postrelease_mobile_gaming.__TABLES_SUMMARY__*04:19:41* [ERROR] Failed to 
execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on 
project word-count-beam: An exception occured while executing the Java class. 
java.net.SocketException: Connection reset -> [Help 1]

  was:
https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/349/
*04:19:41* > Task :beam-runners-direct-java:runMobileGamingJavaDirect 
FAILED*04:19:41* Aug 23, 2018 11:19:32 AM 
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl 
createTable*04:19:41* INFO: Trying to create BigQuery table: 
apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DirectRunner_user*04:19:41*
 bq query SELECT table_id FROM 
beam_postrelease_mobile_gaming.__TABLES_SUMMARY__*04:19:41* [ERROR] Failed to 
execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on 
project word-count-beam: An exception occured while executing the Java class. 
java.net.SocketException: Connection reset -> [Help 1]


> runMobileGamingJavaDirect FAILED SocketException: Connection reset
> --
>
> Key: BEAM-5241
> URL: https://issues.apache.org/jira/browse/BEAM-5241
> Project: Beam
>  Issue Type: Bug
>  Components: examples-java
>Reporter: Andrew Pilloud
>Assignee: Reuven Lax
>Priority: Major
>
> [https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/349/]
> [https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/347/]
>  *04:19:41* > Task :beam-runners-direct-java:runMobileGamingJavaDirect 
> FAILED*04:19:41* Aug 23, 2018 11:19:32 AM 
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl 
> createTable*04:19:41* INFO: Trying to create BigQuery table: 
> apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DirectRunner_user*04:19:41*
>  bq query SELECT table_id FROM 
> beam_postrelease_mobile_gaming.__TABLES_SUMMARY__*04:19:41* [ERROR] Failed to 
> execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on 
> project word-count-beam: An exception occured while executing the Java class. 
> java.net.SocketException: Connection reset -> [Help 1]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5241) runMobileGamingJavaDirect FAILED SocketException: Connection reset

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5241:


 Summary: runMobileGamingJavaDirect FAILED SocketException: 
Connection reset
 Key: BEAM-5241
 URL: https://issues.apache.org/jira/browse/BEAM-5241
 Project: Beam
  Issue Type: Bug
  Components: examples-java
Reporter: Andrew Pilloud
Assignee: Reuven Lax


https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/349/
*04:19:41* > Task :beam-runners-direct-java:runMobileGamingJavaDirect 
FAILED*04:19:41* Aug 23, 2018 11:19:32 AM 
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl 
createTable*04:19:41* INFO: Trying to create BigQuery table: 
apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DirectRunner_user*04:19:41*
 bq query SELECT table_id FROM 
beam_postrelease_mobile_gaming.__TABLES_SUMMARY__*04:19:41* [ERROR] Failed to 
execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on 
project word-count-beam: An exception occured while executing the Java class. 
java.net.SocketException: Connection reset -> [Help 1]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (BEAM-5035) beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava FAILED

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593944#comment-16593944
 ] 

Andrew Pilloud edited comment on BEAM-5035 at 8/27/18 5:25 PM:
---

Fixed by disabling parallel runs.


was (Author: apilloud):
Fixed by disabling paralell runs.

> beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava 
> FAILED
> 
>
> Key: BEAM-5035
> URL: https://issues.apache.org/jira/browse/BEAM-5035
> Project: Beam
>  Issue Type: Improvement
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Andrew Pilloud
>Priority: Critical
> Fix For: 2.7.0
>
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> Compilation failed for
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1105/]
>  > Task :beam-examples-java:compileTestJava FAILED
>  
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java:22:
>  error: cannot access BigqueryMatcher
>  import org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher;
>  ^
>  bad class file: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.7.0-SNAPSHOT-tests.jar(/org/apache/beam/sdk/io/gcp/testing/BigqueryMatcher.class)
>  unable to access file: java.util.zip.ZipException: invalid stored block 
> lengths
>  Please remove or make sure it appears in the correct subdirectory of the 
> classpath.
>  1 error
>  
> https://github.com/apache/beam/blame/328129bf033bc6be16bc8e09af905f37b7516412/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-5035) beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava FAILED

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-5035.
--
   Resolution: Fixed
Fix Version/s: 2.7.0

Fixed by disabling paralell runs.

> beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava 
> FAILED
> 
>
> Key: BEAM-5035
> URL: https://issues.apache.org/jira/browse/BEAM-5035
> Project: Beam
>  Issue Type: Improvement
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Andrew Pilloud
>Priority: Critical
> Fix For: 2.7.0
>
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> Compilation failed for
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1105/]
>  > Task :beam-examples-java:compileTestJava FAILED
>  
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java:22:
>  error: cannot access BigqueryMatcher
>  import org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher;
>  ^
>  bad class file: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.7.0-SNAPSHOT-tests.jar(/org/apache/beam/sdk/io/gcp/testing/BigqueryMatcher.class)
>  unable to access file: java.util.zip.ZipException: invalid stored block 
> lengths
>  Please remove or make sure it appears in the correct subdirectory of the 
> classpath.
>  1 error
>  
> https://github.com/apache/beam/blame/328129bf033bc6be16bc8e09af905f37b7516412/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5035) beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava FAILED

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593906#comment-16593906
 ] 

Andrew Pilloud commented on BEAM-5035:
--

[https://builds.apache.org/job/beam_PreCommit_Java_Cron/270/]

[https://builds.apache.org/job/beam_PreCommit_Java_Cron/272/]

[https://builds.apache.org/job/beam_PreCommit_Java_Cron/273/]

[https://builds.apache.org/job/beam_PreCommit_Java_Cron/275/]

 

> beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava 
> FAILED
> 
>
> Key: BEAM-5035
> URL: https://issues.apache.org/jira/browse/BEAM-5035
> Project: Beam
>  Issue Type: Improvement
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Alan Myrvold
>Priority: Critical
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> Compilation failed for
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1105/]
>  > Task :beam-examples-java:compileTestJava FAILED
>  
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java:22:
>  error: cannot access BigqueryMatcher
>  import org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher;
>  ^
>  bad class file: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.7.0-SNAPSHOT-tests.jar(/org/apache/beam/sdk/io/gcp/testing/BigqueryMatcher.class)
>  unable to access file: java.util.zip.ZipException: invalid stored block 
> lengths
>  Please remove or make sure it appears in the correct subdirectory of the 
> classpath.
>  1 error
>  
> https://github.com/apache/beam/blame/328129bf033bc6be16bc8e09af905f37b7516412/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5035) beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava FAILED

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593900#comment-16593900
 ] 

Andrew Pilloud commented on BEAM-5035:
--

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1346/]

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1347/]

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1348/]

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1349/]

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1351/]

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1354/]

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1356/]

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1357/]

 

> beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava 
> FAILED
> 
>
> Key: BEAM-5035
> URL: https://issues.apache.org/jira/browse/BEAM-5035
> Project: Beam
>  Issue Type: Improvement
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Alan Myrvold
>Priority: Critical
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> Compilation failed for
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1105/]
>  > Task :beam-examples-java:compileTestJava FAILED
>  
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java:22:
>  error: cannot access BigqueryMatcher
>  import org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher;
>  ^
>  bad class file: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.7.0-SNAPSHOT-tests.jar(/org/apache/beam/sdk/io/gcp/testing/BigqueryMatcher.class)
>  unable to access file: java.util.zip.ZipException: invalid stored block 
> lengths
>  Please remove or make sure it appears in the correct subdirectory of the 
> classpath.
>  1 error
>  
> https://github.com/apache/beam/blame/328129bf033bc6be16bc8e09af905f37b7516412/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-5116) beam_PostCommit_Java_GradleBuild :beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest Failed build. System failure.

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-5116.
--
   Resolution: Duplicate
Fix Version/s: Not applicable

> beam_PostCommit_Java_GradleBuild 
> :beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest Failed 
> build. System failure.
> ---
>
> Key: BEAM-5116
> URL: https://issues.apache.org/jira/browse/BEAM-5116
> Project: Beam
>  Issue Type: Bug
>  Components: build-system, test-failures
>Reporter: Mikhail Gryzykhin
>Priority: Major
> Fix For: Not applicable
>
>
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1214/]
> Find log subset below. Suspect is fail is due to memory limitations or zip 
> source availability/integrity.
> > Task :beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest 
> > FAILED
> Build cache key for task 
> ':beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest' is 
> 12c45ad8abd6cf3734c37d0c2592d429
> Task ':beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest' 
> is not up-to-date because:
>   No history is available.
> Starting process 'Gradle Test Executor 129'. Working directory: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/runners/google-cloud-dataflow-java
>  Command: /usr/local/asfpackages/java/jdk1.8.0_172/bin/java 
> -DbeamTestPipelineOptions=["--runner=TestDataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-end-to-end-tests"]
>  
> -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager
>  -Dorg.gradle.native=false -Dfile.encoding=UTF-8 -Duser.country=US 
> -Duser.language=en -Duser.variant -ea -cp 
> /home/jenkins/.gradle/caches/4.8/workerMain/gradle-worker.jar 
> worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test 
> Executor 129'
> Successfully started process 'Gradle Test Executor 129'
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGBUS (0x7) at pc=0x7fc14674a4e2, pid=7054, tid=0x7fc0c1150700
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_172-b11) (build 
> 1.8.0_172-b11)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.172-b11 mixed mode 
> linux-amd64 compressed oops)
> # Problematic frame:
> # C  [libzip.so+0x124e2]  newEntry+0x62
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core 
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/runners/google-cloud-dataflow-java/hs_err_pid7054.log
> Compiled method (nm)1382  211 n 0   
> java.util.zip.ZipFile::getEntry (native)
>  total in heap  [0x7fc1313495d0,0x7fc131349940] = 880
>  relocation [0x7fc1313496f8,0x7fc131349740] = 72
>  main code  [0x7fc131349740,0x7fc131349940] = 512
> #
> # If you would like to submit a bug report, please visit:
> #   
> [http://bugreport.java.com/bugreport/crash.jsp]
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5122) beam_PostCommit_Java_GradleBuild org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593892#comment-16593892
 ] 

Andrew Pilloud commented on BEAM-5122:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1358/testReport/junit/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/

> beam_PostCommit_Java_GradleBuild 
> org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq
> -
>
> Key: BEAM-5122
> URL: https://issues.apache.org/jira/browse/BEAM-5122
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Anton Kedin
>Priority: Critical
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1216/testReport/junit/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/history/]
> Test flakes with timeout of getting update on pubsub:
> java.lang.AssertionError: Did not receive signal on 
> projects/apache-beam-testing/subscriptions/result-subscription--6677803195159868432
>  in 60s at 
> org.apache.beam.sdk.io.gcp.pubsub.TestPubsubSignal.pollForResultForDuration(TestPubsubSignal.java:269)
>  at 
> org.apache.beam.sdk.io.gcp.pubsub.TestPubsubSignal.waitForSuccess(TestPubsubSignal.java:237)
>  at 
> org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq(PubsubJsonIT.java:206)
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1216/testReport/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/]
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5122) beam_PostCommit_Java_GradleBuild org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593889#comment-16593889
 ] 

Andrew Pilloud commented on BEAM-5122:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1353/testReport/junit/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/

> beam_PostCommit_Java_GradleBuild 
> org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq
> -
>
> Key: BEAM-5122
> URL: https://issues.apache.org/jira/browse/BEAM-5122
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Anton Kedin
>Priority: Critical
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1216/testReport/junit/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/history/]
> Test flakes with timeout of getting update on pubsub:
> java.lang.AssertionError: Did not receive signal on 
> projects/apache-beam-testing/subscriptions/result-subscription--6677803195159868432
>  in 60s at 
> org.apache.beam.sdk.io.gcp.pubsub.TestPubsubSignal.pollForResultForDuration(TestPubsubSignal.java:269)
>  at 
> org.apache.beam.sdk.io.gcp.pubsub.TestPubsubSignal.waitForSuccess(TestPubsubSignal.java:237)
>  at 
> org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq(PubsubJsonIT.java:206)
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1216/testReport/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/]
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5122) beam_PostCommit_Java_GradleBuild org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593887#comment-16593887
 ] 

Andrew Pilloud commented on BEAM-5122:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1350/testReport/junit/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/

> beam_PostCommit_Java_GradleBuild 
> org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq
> -
>
> Key: BEAM-5122
> URL: https://issues.apache.org/jira/browse/BEAM-5122
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Anton Kedin
>Priority: Critical
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1216/testReport/junit/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/history/]
> Test flakes with timeout of getting update on pubsub:
> java.lang.AssertionError: Did not receive signal on 
> projects/apache-beam-testing/subscriptions/result-subscription--6677803195159868432
>  in 60s at 
> org.apache.beam.sdk.io.gcp.pubsub.TestPubsubSignal.pollForResultForDuration(TestPubsubSignal.java:269)
>  at 
> org.apache.beam.sdk.io.gcp.pubsub.TestPubsubSignal.waitForSuccess(TestPubsubSignal.java:237)
>  at 
> org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq(PubsubJsonIT.java:206)
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1216/testReport/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/]
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-5207) :beam-runners-apex:compileTestJava dependency issue

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-5207.
--
   Resolution: Duplicate
Fix Version/s: Not applicable

I'm convinced this is BEAM-5035 with logs lost in the race

> :beam-runners-apex:compileTestJava dependency issue
> ---
>
> Key: BEAM-5207
> URL: https://issues.apache.org/jira/browse/BEAM-5207
> Project: Beam
>  Issue Type: New Feature
>  Components: build-system
>Reporter: Andrew Pilloud
>Assignee: Luke Cwik
>Priority: Major
> Fix For: Not applicable
>
>
> Build failed without much in the way of errors, it appears to be some sort of 
> race condition. 
> https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1321
>  
> All input files are considered out-of-date for incremental task 
> ':beam-runners-apex:compileTestJava'



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5035) beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava FAILED

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592309#comment-16592309
 ] 

Andrew Pilloud commented on BEAM-5035:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1344/

> beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava 
> FAILED
> 
>
> Key: BEAM-5035
> URL: https://issues.apache.org/jira/browse/BEAM-5035
> Project: Beam
>  Issue Type: Improvement
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Alan Myrvold
>Priority: Critical
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> Compilation failed for
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1105/]
>  > Task :beam-examples-java:compileTestJava FAILED
>  
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java:22:
>  error: cannot access BigqueryMatcher
>  import org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher;
>  ^
>  bad class file: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.7.0-SNAPSHOT-tests.jar(/org/apache/beam/sdk/io/gcp/testing/BigqueryMatcher.class)
>  unable to access file: java.util.zip.ZipException: invalid stored block 
> lengths
>  Please remove or make sure it appears in the correct subdirectory of the 
> classpath.
>  1 error
>  
> https://github.com/apache/beam/blame/328129bf033bc6be16bc8e09af905f37b7516412/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (BEAM-3604) MqttIOTest testReadNoClientId failure timeout

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592217#comment-16592217
 ] 

Andrew Pilloud edited comment on BEAM-3604 at 8/24/18 9:52 PM:
---

Nevermind, different issue.


was (Author: apilloud):
https://builds.apache.org/job/beam_PreCommit_Java_Cron/253/testReport/junit/org.apache.beam.sdk.io.mqtt/MqttIOTest/testReadNoClientId/

> MqttIOTest testReadNoClientId failure timeout
> -
>
> Key: BEAM-3604
> URL: https://issues.apache.org/jira/browse/BEAM-3604
> Project: Beam
>  Issue Type: Bug
>  Components: io-java-mqtt
>Reporter: Kenneth Knowles
>Assignee: Jean-Baptiste Onofré
>Priority: Critical
>  Labels: flake
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> I've seen failures a bit today. Here is one:
> [https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/1758/testReport/junit/org.apache.beam.sdk.io.mqtt/MqttIOTest/testReadNoClientId/]
> Filing all flakes as "Critical" priority so we can sickbay or fix.
> Since that build will get GC'd, here is the Standard Error. It looks like 
> from that perspective everything went as planned, but perhaps the test has a 
> race condition or something?
> {code}
> Feb 01, 2018 11:28:01 PM org.apache.beam.sdk.io.mqtt.MqttIOTest startBroker
> INFO: Finding free network port
> Feb 01, 2018 11:28:01 PM org.apache.beam.sdk.io.mqtt.MqttIOTest startBroker
> INFO: Starting ActiveMQ brokerService on 57986
> Feb 01, 2018 11:28:03 PM org.apache.activemq.broker.BrokerService 
> doStartPersistenceAdapter
> INFO: Using Persistence Adapter: MemoryPersistenceAdapter
> Feb 01, 2018 11:28:04 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) is 
> starting
> Feb 01, 2018 11:28:04 PM 
> org.apache.activemq.transport.TransportServerThreadSupport doStart
> INFO: Listening for connections at: mqtt://localhost:57986
> Feb 01, 2018 11:28:04 PM org.apache.activemq.broker.TransportConnector start
> INFO: Connector mqtt://localhost:57986 started
> Feb 01, 2018 11:28:04 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) started
> Feb 01, 2018 11:28:04 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: For help or more information please see: http://activemq.apache.org
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService stop
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) is 
> shutting down
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.TransportConnector stop
> INFO: Connector mqtt://localhost:57986 stopped
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService stop
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) uptime 
> 24.039 seconds
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService stop
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) is 
> shutdown
> Feb 01, 2018 11:28:26 PM org.apache.beam.sdk.io.mqtt.MqttIOTest startBroker
> INFO: Finding free network port
> Feb 01, 2018 11:28:26 PM org.apache.beam.sdk.io.mqtt.MqttIOTest startBroker
> INFO: Starting ActiveMQ brokerService on 46799
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService 
> doStartPersistenceAdapter
> INFO: Using Persistence Adapter: MemoryPersistenceAdapter
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:2) is 
> starting
> Feb 01, 2018 11:28:26 PM 
> org.apache.activemq.transport.TransportServerThreadSupport doStart
> INFO: Listening for connections at: mqtt://localhost:46799
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.TransportConnector start
> INFO: Connector mqtt://localhost:46799 started
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:2) started
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: For help or more information please see: http://activemq.apache.org
> Feb 01, 2018 11:28:28 PM org.apache.beam.sdk.io.mqtt.MqttIOTest 
> lambda$testRead$1
> INFO: Waiting pipeline connected to the MQTT broker before sending messages 
> ...
> Feb 01, 2018 11:28:35 PM org.apache.activemq.broker.BrokerService stop
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> 

[jira] [Issue Comment Deleted] (BEAM-3604) MqttIOTest testReadNoClientId failure timeout

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-3604:
-
Comment: was deleted

(was: Nevermind, different issue.)

> MqttIOTest testReadNoClientId failure timeout
> -
>
> Key: BEAM-3604
> URL: https://issues.apache.org/jira/browse/BEAM-3604
> Project: Beam
>  Issue Type: Bug
>  Components: io-java-mqtt
>Reporter: Kenneth Knowles
>Assignee: Jean-Baptiste Onofré
>Priority: Critical
>  Labels: flake
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> I've seen failures a bit today. Here is one:
> [https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/1758/testReport/junit/org.apache.beam.sdk.io.mqtt/MqttIOTest/testReadNoClientId/]
> Filing all flakes as "Critical" priority so we can sickbay or fix.
> Since that build will get GC'd, here is the Standard Error. It looks like 
> from that perspective everything went as planned, but perhaps the test has a 
> race condition or something?
> {code}
> Feb 01, 2018 11:28:01 PM org.apache.beam.sdk.io.mqtt.MqttIOTest startBroker
> INFO: Finding free network port
> Feb 01, 2018 11:28:01 PM org.apache.beam.sdk.io.mqtt.MqttIOTest startBroker
> INFO: Starting ActiveMQ brokerService on 57986
> Feb 01, 2018 11:28:03 PM org.apache.activemq.broker.BrokerService 
> doStartPersistenceAdapter
> INFO: Using Persistence Adapter: MemoryPersistenceAdapter
> Feb 01, 2018 11:28:04 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) is 
> starting
> Feb 01, 2018 11:28:04 PM 
> org.apache.activemq.transport.TransportServerThreadSupport doStart
> INFO: Listening for connections at: mqtt://localhost:57986
> Feb 01, 2018 11:28:04 PM org.apache.activemq.broker.TransportConnector start
> INFO: Connector mqtt://localhost:57986 started
> Feb 01, 2018 11:28:04 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) started
> Feb 01, 2018 11:28:04 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: For help or more information please see: http://activemq.apache.org
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService stop
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) is 
> shutting down
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.TransportConnector stop
> INFO: Connector mqtt://localhost:57986 stopped
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService stop
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) uptime 
> 24.039 seconds
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService stop
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) is 
> shutdown
> Feb 01, 2018 11:28:26 PM org.apache.beam.sdk.io.mqtt.MqttIOTest startBroker
> INFO: Finding free network port
> Feb 01, 2018 11:28:26 PM org.apache.beam.sdk.io.mqtt.MqttIOTest startBroker
> INFO: Starting ActiveMQ brokerService on 46799
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService 
> doStartPersistenceAdapter
> INFO: Using Persistence Adapter: MemoryPersistenceAdapter
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:2) is 
> starting
> Feb 01, 2018 11:28:26 PM 
> org.apache.activemq.transport.TransportServerThreadSupport doStart
> INFO: Listening for connections at: mqtt://localhost:46799
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.TransportConnector start
> INFO: Connector mqtt://localhost:46799 started
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:2) started
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: For help or more information please see: http://activemq.apache.org
> Feb 01, 2018 11:28:28 PM org.apache.beam.sdk.io.mqtt.MqttIOTest 
> lambda$testRead$1
> INFO: Waiting pipeline connected to the MQTT broker before sending messages 
> ...
> Feb 01, 2018 11:28:35 PM org.apache.activemq.broker.BrokerService stop
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:2) is 
> shutting down
> Feb 01, 2018 11:28:35 PM org.apache.activemq.broker.TransportConnector stop
> INFO: Connector mqtt://localhost:46799 stopped
> 

[jira] [Commented] (BEAM-3604) MqttIOTest testReadNoClientId failure timeout

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592217#comment-16592217
 ] 

Andrew Pilloud commented on BEAM-3604:
--

https://builds.apache.org/job/beam_PreCommit_Java_Cron/253/testReport/junit/org.apache.beam.sdk.io.mqtt/MqttIOTest/testReadNoClientId/

> MqttIOTest testReadNoClientId failure timeout
> -
>
> Key: BEAM-3604
> URL: https://issues.apache.org/jira/browse/BEAM-3604
> Project: Beam
>  Issue Type: Bug
>  Components: io-java-mqtt
>Reporter: Kenneth Knowles
>Assignee: Jean-Baptiste Onofré
>Priority: Critical
>  Labels: flake
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> I've seen failures a bit today. Here is one:
> [https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/1758/testReport/junit/org.apache.beam.sdk.io.mqtt/MqttIOTest/testReadNoClientId/]
> Filing all flakes as "Critical" priority so we can sickbay or fix.
> Since that build will get GC'd, here is the Standard Error. It looks like 
> from that perspective everything went as planned, but perhaps the test has a 
> race condition or something?
> {code}
> Feb 01, 2018 11:28:01 PM org.apache.beam.sdk.io.mqtt.MqttIOTest startBroker
> INFO: Finding free network port
> Feb 01, 2018 11:28:01 PM org.apache.beam.sdk.io.mqtt.MqttIOTest startBroker
> INFO: Starting ActiveMQ brokerService on 57986
> Feb 01, 2018 11:28:03 PM org.apache.activemq.broker.BrokerService 
> doStartPersistenceAdapter
> INFO: Using Persistence Adapter: MemoryPersistenceAdapter
> Feb 01, 2018 11:28:04 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) is 
> starting
> Feb 01, 2018 11:28:04 PM 
> org.apache.activemq.transport.TransportServerThreadSupport doStart
> INFO: Listening for connections at: mqtt://localhost:57986
> Feb 01, 2018 11:28:04 PM org.apache.activemq.broker.TransportConnector start
> INFO: Connector mqtt://localhost:57986 started
> Feb 01, 2018 11:28:04 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) started
> Feb 01, 2018 11:28:04 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: For help or more information please see: http://activemq.apache.org
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService stop
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) is 
> shutting down
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.TransportConnector stop
> INFO: Connector mqtt://localhost:57986 stopped
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService stop
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) uptime 
> 24.039 seconds
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService stop
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:1) is 
> shutdown
> Feb 01, 2018 11:28:26 PM org.apache.beam.sdk.io.mqtt.MqttIOTest startBroker
> INFO: Finding free network port
> Feb 01, 2018 11:28:26 PM org.apache.beam.sdk.io.mqtt.MqttIOTest startBroker
> INFO: Starting ActiveMQ brokerService on 46799
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService 
> doStartPersistenceAdapter
> INFO: Using Persistence Adapter: MemoryPersistenceAdapter
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:2) is 
> starting
> Feb 01, 2018 11:28:26 PM 
> org.apache.activemq.transport.TransportServerThreadSupport doStart
> INFO: Listening for connections at: mqtt://localhost:46799
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.TransportConnector start
> INFO: Connector mqtt://localhost:46799 started
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:2) started
> Feb 01, 2018 11:28:26 PM org.apache.activemq.broker.BrokerService 
> doStartBroker
> INFO: For help or more information please see: http://activemq.apache.org
> Feb 01, 2018 11:28:28 PM org.apache.beam.sdk.io.mqtt.MqttIOTest 
> lambda$testRead$1
> INFO: Waiting pipeline connected to the MQTT broker before sending messages 
> ...
> Feb 01, 2018 11:28:35 PM org.apache.activemq.broker.BrokerService stop
> INFO: Apache ActiveMQ 5.13.1 (localhost, 
> ID:115.98.154.104.bc.googleusercontent.com-38646-1517527683931-0:2) is 
> shutting down
> Feb 01, 2018 

[jira] [Assigned] (BEAM-5215) GrpcDataServiceTest.testMessageReceivedBySingleClientWhenThereAreMultipleClients is flaky

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud reassigned BEAM-5215:


Assignee: Luke Cwik  (was: Kenneth Knowles)

> GrpcDataServiceTest.testMessageReceivedBySingleClientWhenThereAreMultipleClients
>  is flaky
> -
>
> Key: BEAM-5215
> URL: https://issues.apache.org/jira/browse/BEAM-5215
> Project: Beam
>  Issue Type: Bug
>  Components: runner-core
>Reporter: Andrew Pilloud
>Assignee: Luke Cwik
>Priority: Major
>
> [https://builds.apache.org/job/beam_PreCommit_Java_Cron/266/testReport/junit/org.apache.beam.runners.fnexecution.data/GrpcDataServiceTest/testMessageReceivedBySingleClientWhenThereAreMultipleClients/]
> {code:java}
> Stacktrace
> java.lang.AssertionError: 
> Expected: iterable over [   instruction_reference: "0"
>   target {
> primitive_transform_reference: "888"
> name: "test"
>   }
>   data: "\002A0\002B0\002C0"
> }
> data {
>   instruction_reference: "0"
>   target {
> primitive_transform_reference: "888"
> name: "test"
>   }
> }
> >,instruction_reference: "1"
>   target {
> primitive_transform_reference: "888"
> name: "test"
>   }
>   data: "\002A1\002B1\002C1"
> }
> data {
>   instruction_reference: "1"
>   target {
> primitive_transform_reference: "888"
> name: "test"
>   }
> }
> >,instruction_reference: "2"
>   target {
> primitive_transform_reference: "888"
> name: "test"
>   }
>   data: "\002A2\002B2\002C2"
> }
> data {
>   instruction_reference: "2"
>   target {
> primitive_transform_reference: "888"
> name: "test"
>   }
> }
> >] in any order
>  but: 
>   at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
>   at org.junit.Assert.assertThat(Assert.java:956)
>   at org.junit.Assert.assertThat(Assert.java:923)
>   at 
> org.apache.beam.runners.fnexecution.data.GrpcDataServiceTest.testMessageReceivedBySingleClientWhenThereAreMultipleClients(GrpcDataServiceTest.java:106)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at 
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106)
>   at 
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
>   at 
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
>   at 
> org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66)
>   at 
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
>   at 
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
>   at 
> org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
>   at 
> 

[jira] [Assigned] (BEAM-5215) GrpcDataServiceTest.testMessageReceivedBySingleClientWhenThereAreMultipleClients is flaky

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud reassigned BEAM-5215:


Assignee: Kenneth Knowles  (was: Eugene Kirpichov)

> GrpcDataServiceTest.testMessageReceivedBySingleClientWhenThereAreMultipleClients
>  is flaky
> -
>
> Key: BEAM-5215
> URL: https://issues.apache.org/jira/browse/BEAM-5215
> Project: Beam
>  Issue Type: Bug
>  Components: runner-core
>Reporter: Andrew Pilloud
>Assignee: Kenneth Knowles
>Priority: Major
>
> [https://builds.apache.org/job/beam_PreCommit_Java_Cron/266/testReport/junit/org.apache.beam.runners.fnexecution.data/GrpcDataServiceTest/testMessageReceivedBySingleClientWhenThereAreMultipleClients/]
> {code:java}
> Stacktrace
> java.lang.AssertionError: 
> Expected: iterable over [   instruction_reference: "0"
>   target {
> primitive_transform_reference: "888"
> name: "test"
>   }
>   data: "\002A0\002B0\002C0"
> }
> data {
>   instruction_reference: "0"
>   target {
> primitive_transform_reference: "888"
> name: "test"
>   }
> }
> >,instruction_reference: "1"
>   target {
> primitive_transform_reference: "888"
> name: "test"
>   }
>   data: "\002A1\002B1\002C1"
> }
> data {
>   instruction_reference: "1"
>   target {
> primitive_transform_reference: "888"
> name: "test"
>   }
> }
> >,instruction_reference: "2"
>   target {
> primitive_transform_reference: "888"
> name: "test"
>   }
>   data: "\002A2\002B2\002C2"
> }
> data {
>   instruction_reference: "2"
>   target {
> primitive_transform_reference: "888"
> name: "test"
>   }
> }
> >] in any order
>  but: 
>   at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
>   at org.junit.Assert.assertThat(Assert.java:956)
>   at org.junit.Assert.assertThat(Assert.java:923)
>   at 
> org.apache.beam.runners.fnexecution.data.GrpcDataServiceTest.testMessageReceivedBySingleClientWhenThereAreMultipleClients(GrpcDataServiceTest.java:106)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at 
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106)
>   at 
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
>   at 
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
>   at 
> org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66)
>   at 
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
>   at 
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
>   at 
> org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
>   at 
> 

[jira] [Created] (BEAM-5215) GrpcDataServiceTest.testMessageReceivedBySingleClientWhenThereAreMultipleClients is flaky

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5215:


 Summary: 
GrpcDataServiceTest.testMessageReceivedBySingleClientWhenThereAreMultipleClients
 is flaky
 Key: BEAM-5215
 URL: https://issues.apache.org/jira/browse/BEAM-5215
 Project: Beam
  Issue Type: Bug
  Components: runner-core
Reporter: Andrew Pilloud
Assignee: Eugene Kirpichov


[https://builds.apache.org/job/beam_PreCommit_Java_Cron/266/testReport/junit/org.apache.beam.runners.fnexecution.data/GrpcDataServiceTest/testMessageReceivedBySingleClientWhenThereAreMultipleClients/]
{code:java}
Stacktrace
java.lang.AssertionError: 
Expected: iterable over [, , ] in any order
 but: 
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
at org.junit.Assert.assertThat(Assert.java:956)
at org.junit.Assert.assertThat(Assert.java:923)
at 
org.apache.beam.runners.fnexecution.data.GrpcDataServiceTest.testMessageReceivedBySingleClientWhenThereAreMultipleClients(GrpcDataServiceTest.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at 
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106)
at 
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
at 
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
at 
org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66)
at 
org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at 
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at 
org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
at 
org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
at 
org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at 
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at 
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:155)
at 
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:137)
at 
org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
at 

[jira] [Created] (BEAM-5213) sdks/go/test/regression failed with no results

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5213:


 Summary: sdks/go/test/regression failed with no results
 Key: BEAM-5213
 URL: https://issues.apache.org/jira/browse/BEAM-5213
 Project: Beam
  Issue Type: Bug
  Components: sdk-go
Reporter: Andrew Pilloud
Assignee: Henning Rohde


The details on which test failed are missing, but something failed. 
https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/784/

 
*11:26:20* Result of package 
github.com/apache/beam/sdks/go/test/regression:*11:26:20* Test for 
github.com/apache/beam/sdks/go/test/regression finished, 1 completed, 1 failed



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5207) :beam-runners-apex:compileTestJava dependency issue

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591897#comment-16591897
 ] 

Andrew Pilloud commented on BEAM-5207:
--

[https://builds.apache.org/job/beam_PreCommit_Java_Cron/263/]

https://builds.apache.org/job/beam_PreCommit_Java_Cron/264/

> :beam-runners-apex:compileTestJava dependency issue
> ---
>
> Key: BEAM-5207
> URL: https://issues.apache.org/jira/browse/BEAM-5207
> Project: Beam
>  Issue Type: New Feature
>  Components: build-system
>Reporter: Andrew Pilloud
>Assignee: Luke Cwik
>Priority: Major
>
> Build failed without much in the way of errors, it appears to be some sort of 
> race condition. 
> https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1321
>  
> All input files are considered out-of-date for incremental task 
> ':beam-runners-apex:compileTestJava'



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5035) beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava FAILED

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591898#comment-16591898
 ] 

Andrew Pilloud commented on BEAM-5035:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1341/

> beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava 
> FAILED
> 
>
> Key: BEAM-5035
> URL: https://issues.apache.org/jira/browse/BEAM-5035
> Project: Beam
>  Issue Type: Improvement
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Alan Myrvold
>Priority: Critical
>
> Compilation failed for
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1105/]
>  > Task :beam-examples-java:compileTestJava FAILED
>  
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java:22:
>  error: cannot access BigqueryMatcher
>  import org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher;
>  ^
>  bad class file: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.7.0-SNAPSHOT-tests.jar(/org/apache/beam/sdk/io/gcp/testing/BigqueryMatcher.class)
>  unable to access file: java.util.zip.ZipException: invalid stored block 
> lengths
>  Please remove or make sure it appears in the correct subdirectory of the 
> classpath.
>  1 error
>  
> https://github.com/apache/beam/blame/328129bf033bc6be16bc8e09af905f37b7516412/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5035) beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava FAILED

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591887#comment-16591887
 ] 

Andrew Pilloud commented on BEAM-5035:
--

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1331/]

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1335/]

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1337/]

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1338/]

 

> beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava 
> FAILED
> 
>
> Key: BEAM-5035
> URL: https://issues.apache.org/jira/browse/BEAM-5035
> Project: Beam
>  Issue Type: Improvement
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Alan Myrvold
>Priority: Critical
>
> Compilation failed for
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1105/]
>  > Task :beam-examples-java:compileTestJava FAILED
>  
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java:22:
>  error: cannot access BigqueryMatcher
>  import org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher;
>  ^
>  bad class file: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.7.0-SNAPSHOT-tests.jar(/org/apache/beam/sdk/io/gcp/testing/BigqueryMatcher.class)
>  unable to access file: java.util.zip.ZipException: invalid stored block 
> lengths
>  Please remove or make sure it appears in the correct subdirectory of the 
> classpath.
>  1 error
>  
> https://github.com/apache/beam/blame/328129bf033bc6be16bc8e09af905f37b7516412/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5207) :beam-runners-apex:compileTestJava dependency issue

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591886#comment-16591886
 ] 

Andrew Pilloud commented on BEAM-5207:
--

This is probably the same as https://issues.apache.org/jira/browse/BEAM-5035

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1332/]

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1334/]

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1339/]

 

> :beam-runners-apex:compileTestJava dependency issue
> ---
>
> Key: BEAM-5207
> URL: https://issues.apache.org/jira/browse/BEAM-5207
> Project: Beam
>  Issue Type: New Feature
>  Components: build-system
>Reporter: Andrew Pilloud
>Assignee: Luke Cwik
>Priority: Major
>
> Build failed without much in the way of errors, it appears to be some sort of 
> race condition. 
> https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1321
>  
> All input files are considered out-of-date for incremental task 
> ':beam-runners-apex:compileTestJava'



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5171) org.apache.beam.sdk.io.CountingSourceTest.test[Un]boundedSourceSplits tests are flaky in Spark runner

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591876#comment-16591876
 ] 

Andrew Pilloud commented on BEAM-5171:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1333/testReport/junit/org.apache.beam.runners.spark.aggregators.metrics.sink/SparkMetricsSinkTest/testInBatchMode/

> org.apache.beam.sdk.io.CountingSourceTest.test[Un]boundedSourceSplits tests 
> are flaky in Spark runner
> -
>
> Key: BEAM-5171
> URL: https://issues.apache.org/jira/browse/BEAM-5171
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Valentyn Tymofieiev
>Assignee: Etienne Chauchot
>Priority: Major
>
> Two tests: 
>  org.apache.beam.sdk.io.CountingSourceTest.testUnboundedSourceSplits 
>  org.apache.beam.sdk.io.CountingSourceTest.testBoundedSourceSplits
> failed in a PostCommit [Spark Validates Runner test 
> suite|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/1277/testReport/]
>  with an error that seems to be common for Spark. Could this be due to 
> misconfiguration of Spark cluster? 
> Task serialization failed: java.io.IOException: Failed to create local dir in 
> /tmp/blockmgr-de91f449-e5d1-4be4-acaa-3ee06fdfa95b/1d.
>  java.io.IOException: Failed to create local dir in 
> /tmp/blockmgr-de91f449-e5d1-4be4-acaa-3ee06fdfa95b/1d.
>  at 
> org.apache.spark.storage.DiskBlockManager.getFile(DiskBlockManager.scala:70)
>  at org.apache.spark.storage.DiskStore.remove(DiskStore.scala:116)
>  at 
> org.apache.spark.storage.BlockManager.removeBlockInternal(BlockManager.scala:1511)
>  at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:1045)
>  at 
> org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1083)
>  at org.apache.spark.storage.BlockManager.putIterator(BlockManager.scala:841)
>  at org.apache.spark.storage.BlockManager.putSingle(BlockManager.scala:1404)
>  at 
> org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:123)
>  at 
> org.apache.spark.broadcast.TorrentBroadcast.(TorrentBroadcast.scala:88)
>  at 
> org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
>  at 
> org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:62)
>  at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1482)
>  at 
> org.apache.spark.scheduler.DAGScheduler.submitMissingTasks(DAGScheduler.scala:1039)
>  at 
> org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$submitStage(DAGScheduler.scala:947)
>  at 
> org.apache.spark.scheduler.DAGScheduler.handleJobSubmitted(DAGScheduler.scala:891)
>  at 
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1780)
>  at 
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1772)
>  at 
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1761)
>  at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5207) :beam-runners-apex:compileTestJava dependency issue

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590838#comment-16590838
 ] 

Andrew Pilloud commented on BEAM-5207:
--

[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1327/] 
[https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1328/
https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1329/
|https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1328/]

> :beam-runners-apex:compileTestJava dependency issue
> ---
>
> Key: BEAM-5207
> URL: https://issues.apache.org/jira/browse/BEAM-5207
> Project: Beam
>  Issue Type: New Feature
>  Components: build-system
>Reporter: Andrew Pilloud
>Assignee: Luke Cwik
>Priority: Major
>
> Build failed without much in the way of errors, it appears to be some sort of 
> race condition. 
> https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1321
>  
> All input files are considered out-of-date for incremental task 
> ':beam-runners-apex:compileTestJava'



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5207) :beam-runners-apex:compileTestJava dependency issue

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590537#comment-16590537
 ] 

Andrew Pilloud commented on BEAM-5207:
--

https://builds.apache.org/job/beam_PreCommit_Java_Cron/259/

> :beam-runners-apex:compileTestJava dependency issue
> ---
>
> Key: BEAM-5207
> URL: https://issues.apache.org/jira/browse/BEAM-5207
> Project: Beam
>  Issue Type: New Feature
>  Components: build-system
>Reporter: Andrew Pilloud
>Assignee: Luke Cwik
>Priority: Major
>
> Build failed without much in the way of errors, it appears to be some sort of 
> race condition. 
> https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1321
>  
> All input files are considered out-of-date for incremental task 
> ':beam-runners-apex:compileTestJava'



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5035) beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava FAILED

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590535#comment-16590535
 ] 

Andrew Pilloud commented on BEAM-5035:
--

https://builds.apache.org/job/beam_PreCommit_Java_Cron/260/

> beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava 
> FAILED
> 
>
> Key: BEAM-5035
> URL: https://issues.apache.org/jira/browse/BEAM-5035
> Project: Beam
>  Issue Type: Improvement
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Alan Myrvold
>Priority: Critical
>
> Compilation failed for
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1105/]
>  > Task :beam-examples-java:compileTestJava FAILED
>  
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java:22:
>  error: cannot access BigqueryMatcher
>  import org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher;
>  ^
>  bad class file: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.7.0-SNAPSHOT-tests.jar(/org/apache/beam/sdk/io/gcp/testing/BigqueryMatcher.class)
>  unable to access file: java.util.zip.ZipException: invalid stored block 
> lengths
>  Please remove or make sure it appears in the correct subdirectory of the 
> classpath.
>  1 error
>  
> https://github.com/apache/beam/blame/328129bf033bc6be16bc8e09af905f37b7516412/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5035) beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava FAILED

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590534#comment-16590534
 ] 

Andrew Pilloud commented on BEAM-5035:
--

https://builds.apache.org/job/beam_PreCommit_Java_Cron/261/

> beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava 
> FAILED
> 
>
> Key: BEAM-5035
> URL: https://issues.apache.org/jira/browse/BEAM-5035
> Project: Beam
>  Issue Type: Improvement
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Alan Myrvold
>Priority: Critical
>
> Compilation failed for
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1105/]
>  > Task :beam-examples-java:compileTestJava FAILED
>  
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java:22:
>  error: cannot access BigqueryMatcher
>  import org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher;
>  ^
>  bad class file: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.7.0-SNAPSHOT-tests.jar(/org/apache/beam/sdk/io/gcp/testing/BigqueryMatcher.class)
>  unable to access file: java.util.zip.ZipException: invalid stored block 
> lengths
>  Please remove or make sure it appears in the correct subdirectory of the 
> classpath.
>  1 error
>  
> https://github.com/apache/beam/blame/328129bf033bc6be16bc8e09af905f37b7516412/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5116) beam_PostCommit_Java_GradleBuild :beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest Failed build. System failure.

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590524#comment-16590524
 ] 

Andrew Pilloud commented on BEAM-5116:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1313/

> beam_PostCommit_Java_GradleBuild 
> :beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest Failed 
> build. System failure.
> ---
>
> Key: BEAM-5116
> URL: https://issues.apache.org/jira/browse/BEAM-5116
> Project: Beam
>  Issue Type: Bug
>  Components: build-system, test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Mikhail Gryzykhin
>Priority: Major
>
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1214/]
> Find log subset below. Suspect is fail is due to memory limitations or zip 
> source availability/integrity.
> > Task :beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest 
> > FAILED
> Build cache key for task 
> ':beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest' is 
> 12c45ad8abd6cf3734c37d0c2592d429
> Task ':beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest' 
> is not up-to-date because:
>   No history is available.
> Starting process 'Gradle Test Executor 129'. Working directory: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/runners/google-cloud-dataflow-java
>  Command: /usr/local/asfpackages/java/jdk1.8.0_172/bin/java 
> -DbeamTestPipelineOptions=["--runner=TestDataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-end-to-end-tests"]
>  
> -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager
>  -Dorg.gradle.native=false -Dfile.encoding=UTF-8 -Duser.country=US 
> -Duser.language=en -Duser.variant -ea -cp 
> /home/jenkins/.gradle/caches/4.8/workerMain/gradle-worker.jar 
> worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test 
> Executor 129'
> Successfully started process 'Gradle Test Executor 129'
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGBUS (0x7) at pc=0x7fc14674a4e2, pid=7054, tid=0x7fc0c1150700
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_172-b11) (build 
> 1.8.0_172-b11)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.172-b11 mixed mode 
> linux-amd64 compressed oops)
> # Problematic frame:
> # C  [libzip.so+0x124e2]  newEntry+0x62
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core 
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/runners/google-cloud-dataflow-java/hs_err_pid7054.log
> Compiled method (nm)1382  211 n 0   
> java.util.zip.ZipFile::getEntry (native)
>  total in heap  [0x7fc1313495d0,0x7fc131349940] = 880
>  relocation [0x7fc1313496f8,0x7fc131349740] = 72
>  main code  [0x7fc131349740,0x7fc131349940] = 512
> #
> # If you would like to submit a bug report, please visit:
> #   
> [http://bugreport.java.com/bugreport/crash.jsp]
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5122) beam_PostCommit_Java_GradleBuild org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590526#comment-16590526
 ] 

Andrew Pilloud commented on BEAM-5122:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1312/testReport/junit/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/

> beam_PostCommit_Java_GradleBuild 
> org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq
> -
>
> Key: BEAM-5122
> URL: https://issues.apache.org/jira/browse/BEAM-5122
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Anton Kedin
>Priority: Critical
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1216/testReport/junit/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/history/]
> Test flakes with timeout of getting update on pubsub:
> java.lang.AssertionError: Did not receive signal on 
> projects/apache-beam-testing/subscriptions/result-subscription--6677803195159868432
>  in 60s at 
> org.apache.beam.sdk.io.gcp.pubsub.TestPubsubSignal.pollForResultForDuration(TestPubsubSignal.java:269)
>  at 
> org.apache.beam.sdk.io.gcp.pubsub.TestPubsubSignal.waitForSuccess(TestPubsubSignal.java:237)
>  at 
> org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq(PubsubJsonIT.java:206)
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1216/testReport/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/]
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-5116) beam_PostCommit_Java_GradleBuild :beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest Failed build. System failure.

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-5116:
-
Priority: Major  (was: Minor)

> beam_PostCommit_Java_GradleBuild 
> :beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest Failed 
> build. System failure.
> ---
>
> Key: BEAM-5116
> URL: https://issues.apache.org/jira/browse/BEAM-5116
> Project: Beam
>  Issue Type: Bug
>  Components: build-system, test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Mikhail Gryzykhin
>Priority: Major
>
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1214/]
> Find log subset below. Suspect is fail is due to memory limitations or zip 
> source availability/integrity.
> > Task :beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest 
> > FAILED
> Build cache key for task 
> ':beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest' is 
> 12c45ad8abd6cf3734c37d0c2592d429
> Task ':beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest' 
> is not up-to-date because:
>   No history is available.
> Starting process 'Gradle Test Executor 129'. Working directory: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/runners/google-cloud-dataflow-java
>  Command: /usr/local/asfpackages/java/jdk1.8.0_172/bin/java 
> -DbeamTestPipelineOptions=["--runner=TestDataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-end-to-end-tests"]
>  
> -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager
>  -Dorg.gradle.native=false -Dfile.encoding=UTF-8 -Duser.country=US 
> -Duser.language=en -Duser.variant -ea -cp 
> /home/jenkins/.gradle/caches/4.8/workerMain/gradle-worker.jar 
> worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test 
> Executor 129'
> Successfully started process 'Gradle Test Executor 129'
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGBUS (0x7) at pc=0x7fc14674a4e2, pid=7054, tid=0x7fc0c1150700
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_172-b11) (build 
> 1.8.0_172-b11)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.172-b11 mixed mode 
> linux-amd64 compressed oops)
> # Problematic frame:
> # C  [libzip.so+0x124e2]  newEntry+0x62
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core 
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/runners/google-cloud-dataflow-java/hs_err_pid7054.log
> Compiled method (nm)1382  211 n 0   
> java.util.zip.ZipFile::getEntry (native)
>  total in heap  [0x7fc1313495d0,0x7fc131349940] = 880
>  relocation [0x7fc1313496f8,0x7fc131349740] = 72
>  main code  [0x7fc131349740,0x7fc131349940] = 512
> #
> # If you would like to submit a bug report, please visit:
> #   
> [http://bugreport.java.com/bugreport/crash.jsp]
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-5116) beam_PostCommit_Java_GradleBuild :beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest Failed build. System failure.

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud reassigned BEAM-5116:


Assignee: Mikhail Gryzykhin

> beam_PostCommit_Java_GradleBuild 
> :beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest Failed 
> build. System failure.
> ---
>
> Key: BEAM-5116
> URL: https://issues.apache.org/jira/browse/BEAM-5116
> Project: Beam
>  Issue Type: Bug
>  Components: build-system, test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Mikhail Gryzykhin
>Priority: Minor
>
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1214/]
> Find log subset below. Suspect is fail is due to memory limitations or zip 
> source availability/integrity.
> > Task :beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest 
> > FAILED
> Build cache key for task 
> ':beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest' is 
> 12c45ad8abd6cf3734c37d0c2592d429
> Task ':beam-runners-google-cloud-dataflow-java:examplesJavaIntegrationTest' 
> is not up-to-date because:
>   No history is available.
> Starting process 'Gradle Test Executor 129'. Working directory: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/runners/google-cloud-dataflow-java
>  Command: /usr/local/asfpackages/java/jdk1.8.0_172/bin/java 
> -DbeamTestPipelineOptions=["--runner=TestDataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-end-to-end-tests"]
>  
> -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager
>  -Dorg.gradle.native=false -Dfile.encoding=UTF-8 -Duser.country=US 
> -Duser.language=en -Duser.variant -ea -cp 
> /home/jenkins/.gradle/caches/4.8/workerMain/gradle-worker.jar 
> worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test 
> Executor 129'
> Successfully started process 'Gradle Test Executor 129'
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGBUS (0x7) at pc=0x7fc14674a4e2, pid=7054, tid=0x7fc0c1150700
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_172-b11) (build 
> 1.8.0_172-b11)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.172-b11 mixed mode 
> linux-amd64 compressed oops)
> # Problematic frame:
> # C  [libzip.so+0x124e2]  newEntry+0x62
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core 
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/runners/google-cloud-dataflow-java/hs_err_pid7054.log
> Compiled method (nm)1382  211 n 0   
> java.util.zip.ZipFile::getEntry (native)
>  total in heap  [0x7fc1313495d0,0x7fc131349940] = 880
>  relocation [0x7fc1313496f8,0x7fc131349740] = 72
>  main code  [0x7fc131349740,0x7fc131349940] = 512
> #
> # If you would like to submit a bug report, please visit:
> #   
> [http://bugreport.java.com/bugreport/crash.jsp]
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5207) :beam-runners-apex:compileTestJava dependency issue

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590521#comment-16590521
 ] 

Andrew Pilloud commented on BEAM-5207:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1315

> :beam-runners-apex:compileTestJava dependency issue
> ---
>
> Key: BEAM-5207
> URL: https://issues.apache.org/jira/browse/BEAM-5207
> Project: Beam
>  Issue Type: New Feature
>  Components: build-system
>Reporter: Andrew Pilloud
>Assignee: Luke Cwik
>Priority: Major
>
> Build failed without much in the way of errors, it appears to be some sort of 
> race condition. 
> https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1321
>  
> All input files are considered out-of-date for incremental task 
> ':beam-runners-apex:compileTestJava'



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5122) beam_PostCommit_Java_GradleBuild org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590506#comment-16590506
 ] 

Andrew Pilloud commented on BEAM-5122:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1314/testReport/junit/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/

> beam_PostCommit_Java_GradleBuild 
> org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq
> -
>
> Key: BEAM-5122
> URL: https://issues.apache.org/jira/browse/BEAM-5122
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Anton Kedin
>Priority: Critical
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1216/testReport/junit/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/history/]
> Test flakes with timeout of getting update on pubsub:
> java.lang.AssertionError: Did not receive signal on 
> projects/apache-beam-testing/subscriptions/result-subscription--6677803195159868432
>  in 60s at 
> org.apache.beam.sdk.io.gcp.pubsub.TestPubsubSignal.pollForResultForDuration(TestPubsubSignal.java:269)
>  at 
> org.apache.beam.sdk.io.gcp.pubsub.TestPubsubSignal.waitForSuccess(TestPubsubSignal.java:237)
>  at 
> org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq(PubsubJsonIT.java:206)
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1216/testReport/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/]
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5035) beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava FAILED

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590504#comment-16590504
 ] 

Andrew Pilloud commented on BEAM-5035:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1316/

> beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava 
> FAILED
> 
>
> Key: BEAM-5035
> URL: https://issues.apache.org/jira/browse/BEAM-5035
> Project: Beam
>  Issue Type: Improvement
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Alan Myrvold
>Priority: Critical
>
> Compilation failed for
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1105/]
>  > Task :beam-examples-java:compileTestJava FAILED
>  
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java:22:
>  error: cannot access BigqueryMatcher
>  import org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher;
>  ^
>  bad class file: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.7.0-SNAPSHOT-tests.jar(/org/apache/beam/sdk/io/gcp/testing/BigqueryMatcher.class)
>  unable to access file: java.util.zip.ZipException: invalid stored block 
> lengths
>  Please remove or make sure it appears in the correct subdirectory of the 
> classpath.
>  1 error
>  
> https://github.com/apache/beam/blame/328129bf033bc6be16bc8e09af905f37b7516412/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5035) beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava FAILED

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590497#comment-16590497
 ] 

Andrew Pilloud commented on BEAM-5035:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1319

> beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava 
> FAILED
> 
>
> Key: BEAM-5035
> URL: https://issues.apache.org/jira/browse/BEAM-5035
> Project: Beam
>  Issue Type: Improvement
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Alan Myrvold
>Priority: Critical
>
> Compilation failed for
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1105/]
>  > Task :beam-examples-java:compileTestJava FAILED
>  
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java:22:
>  error: cannot access BigqueryMatcher
>  import org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher;
>  ^
>  bad class file: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.7.0-SNAPSHOT-tests.jar(/org/apache/beam/sdk/io/gcp/testing/BigqueryMatcher.class)
>  unable to access file: java.util.zip.ZipException: invalid stored block 
> lengths
>  Please remove or make sure it appears in the correct subdirectory of the 
> classpath.
>  1 error
>  
> https://github.com/apache/beam/blame/328129bf033bc6be16bc8e09af905f37b7516412/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5207) :beam-runners-apex:compileTestJava dependency issue

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5207:


 Summary: :beam-runners-apex:compileTestJava dependency issue
 Key: BEAM-5207
 URL: https://issues.apache.org/jira/browse/BEAM-5207
 Project: Beam
  Issue Type: New Feature
  Components: build-system
Reporter: Andrew Pilloud
Assignee: Luke Cwik


Build failed without much in the way of errors, it appears to be some sort of 
race condition. 
https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1321

 
All input files are considered out-of-date for incremental task 
':beam-runners-apex:compileTestJava'



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5035) beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava FAILED

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590420#comment-16590420
 ] 

Andrew Pilloud commented on BEAM-5035:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1326

 
*05:09:00* > Task :beam-examples-java:compileTestJava FAILED*05:09:00* 
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java:22:
 error: cannot access BigqueryMatcher*05:09:00* import 
org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher;*05:09:00*   
   ^*05:09:00*   bad class file: 
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.7.0-SNAPSHOT-tests.jar(/org/apache/beam/sdk/io/gcp/testing/BigqueryMatcher.class)*05:09:00*
 unable to access file: java.util.zip.ZipException: invalid stored block 
lengths*05:09:00* Please remove or make sure it appears in the correct 
subdirectory of the classpath.

> beam_PostCommit_Java_GradleBuild/1105 :beam-examples-java:compileTestJava 
> FAILED
> 
>
> Key: BEAM-5035
> URL: https://issues.apache.org/jira/browse/BEAM-5035
> Project: Beam
>  Issue Type: Improvement
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Alan Myrvold
>Priority: Critical
>
> Compilation failed for
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1105/]
>  > Task :beam-examples-java:compileTestJava FAILED
>  
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java:22:
>  error: cannot access BigqueryMatcher
>  import org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher;
>  ^
>  bad class file: 
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_GradleBuild/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.7.0-SNAPSHOT-tests.jar(/org/apache/beam/sdk/io/gcp/testing/BigqueryMatcher.class)
>  unable to access file: java.util.zip.ZipException: invalid stored block 
> lengths
>  Please remove or make sure it appears in the correct subdirectory of the 
> classpath.
>  1 error
>  
> https://github.com/apache/beam/blame/328129bf033bc6be16bc8e09af905f37b7516412/examples/java/src/test/java/org/apache/beam/examples/cookbook/BigQueryTornadoesIT.java
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5122) beam_PostCommit_Java_GradleBuild org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590422#comment-16590422
 ] 

Andrew Pilloud commented on BEAM-5122:
--

https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1324/

> beam_PostCommit_Java_GradleBuild 
> org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq
> -
>
> Key: BEAM-5122
> URL: https://issues.apache.org/jira/browse/BEAM-5122
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Anton Kedin
>Priority: Critical
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1216/testReport/junit/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/history/]
> Test flakes with timeout of getting update on pubsub:
> java.lang.AssertionError: Did not receive signal on 
> projects/apache-beam-testing/subscriptions/result-subscription--6677803195159868432
>  in 60s at 
> org.apache.beam.sdk.io.gcp.pubsub.TestPubsubSignal.pollForResultForDuration(TestPubsubSignal.java:269)
>  at 
> org.apache.beam.sdk.io.gcp.pubsub.TestPubsubSignal.waitForSuccess(TestPubsubSignal.java:237)
>  at 
> org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubJsonIT.testUsesDlq(PubsubJsonIT.java:206)
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1216/testReport/org.apache.beam.sdk.extensions.sql.meta.provider.pubsub/PubsubJsonIT/testUsesDlq/]
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4772) TextIO.read transform does not respect .withEmptyMatchTreatment

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-4772.
--
   Resolution: Fixed
Fix Version/s: 2.7.0

> TextIO.read transform does not respect .withEmptyMatchTreatment
> ---
>
> Key: BEAM-4772
> URL: https://issues.apache.org/jira/browse/BEAM-4772
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Affects Versions: 2.5.0
>Reporter: Samuel Waggoner
>Assignee: Kyle Winkelman
>Priority: Major
> Fix For: 2.7.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> I modified the MinimalWordCount example to reproduce. I expect the read 
> transform to read 0 lines rather than give an exception, since I used 
> EmptyMatchTreatment.ALLOW. I see the same behavior with ALLOW_IF_WILDCARD. 
> The EmptyMatchTreatment value seems to be ignored.
> {code:java}
> public class MinimalWordCount {
>  public static void main(String[] args) {
>PipelineOptions options = PipelineOptionsFactory.create();
>Pipeline p = Pipeline.create(options);
>p.apply(TextIO.read()
>  .from("gs://apache-beam-samples/doesnotexist/*")
>  .withEmptyMatchTreatment(EmptyMatchTreatment.ALLOW))
> .apply(TextIO.write().to("wordcounts"));
>p.run().waitUntilFinish();
>  }
> }
> {code}
> {code:java}
> Exception in thread "main" 
> org.apache.beam.sdk.Pipeline$PipelineExecutionException: 
> java.io.FileNotFoundException: No files matched spec: 
> gs://apache-beam-samples/doesnotexist/*
>  at org.apache.beam.sdk.Pipeline.run(Pipeline.java:317)
>  at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
>  at org.apache.beam.examples.MinimalWordCount.main(MinimalWordCount.java:124)
> Caused by: java.io.FileNotFoundException: No files matched spec: 
> gs://apache-beam-samples/doesnotexist/*
>  at 
> org.apache.beam.sdk.io.FileSystems.maybeAdjustEmptyMatchResult(FileSystems.java:172)
>  at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:158)
>  at 
> org.apache.beam.sdk.io.FileBasedSource.getEstimatedSizeBytes(FileBasedSource.java:222)
>  at 
> org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$InputProvider.getInitialInputs(BoundedReadEvaluatorFactory.java:212)
>  at 
> org.apache.beam.runners.direct.ReadEvaluatorFactory$InputProvider.getInitialInputs(ReadEvaluatorFactory.java:91)
>  at 
> org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:81){code}
> We see this behavior both when using DirectRunner and DataflowRunner 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-4772) TextIO.read transform does not respect .withEmptyMatchTreatment

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud reassigned BEAM-4772:


Assignee: Kyle Winkelman  (was: Kenneth Knowles)

> TextIO.read transform does not respect .withEmptyMatchTreatment
> ---
>
> Key: BEAM-4772
> URL: https://issues.apache.org/jira/browse/BEAM-4772
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Affects Versions: 2.5.0
>Reporter: Samuel Waggoner
>Assignee: Kyle Winkelman
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I modified the MinimalWordCount example to reproduce. I expect the read 
> transform to read 0 lines rather than give an exception, since I used 
> EmptyMatchTreatment.ALLOW. I see the same behavior with ALLOW_IF_WILDCARD. 
> The EmptyMatchTreatment value seems to be ignored.
> {code:java}
> public class MinimalWordCount {
>  public static void main(String[] args) {
>PipelineOptions options = PipelineOptionsFactory.create();
>Pipeline p = Pipeline.create(options);
>p.apply(TextIO.read()
>  .from("gs://apache-beam-samples/doesnotexist/*")
>  .withEmptyMatchTreatment(EmptyMatchTreatment.ALLOW))
> .apply(TextIO.write().to("wordcounts"));
>p.run().waitUntilFinish();
>  }
> }
> {code}
> {code:java}
> Exception in thread "main" 
> org.apache.beam.sdk.Pipeline$PipelineExecutionException: 
> java.io.FileNotFoundException: No files matched spec: 
> gs://apache-beam-samples/doesnotexist/*
>  at org.apache.beam.sdk.Pipeline.run(Pipeline.java:317)
>  at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
>  at org.apache.beam.examples.MinimalWordCount.main(MinimalWordCount.java:124)
> Caused by: java.io.FileNotFoundException: No files matched spec: 
> gs://apache-beam-samples/doesnotexist/*
>  at 
> org.apache.beam.sdk.io.FileSystems.maybeAdjustEmptyMatchResult(FileSystems.java:172)
>  at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:158)
>  at 
> org.apache.beam.sdk.io.FileBasedSource.getEstimatedSizeBytes(FileBasedSource.java:222)
>  at 
> org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$InputProvider.getInitialInputs(BoundedReadEvaluatorFactory.java:212)
>  at 
> org.apache.beam.runners.direct.ReadEvaluatorFactory$InputProvider.getInitialInputs(ReadEvaluatorFactory.java:91)
>  at 
> org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:81){code}
> We see this behavior both when using DirectRunner and DataflowRunner 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-4807) Upgrade calcite to 1.17.0

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-4807:
-
Fix Version/s: (was: Not applicable)
   2.7.0

> Upgrade calcite to 1.17.0
> -
>
> Key: BEAM-4807
> URL: https://issues.apache.org/jira/browse/BEAM-4807
> Project: Beam
>  Issue Type: New Feature
>  Components: dsl-sql
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.7.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> We should upgrade calcite.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (BEAM-4774) Intergrate Nexmark SQL with Perfkit

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud closed BEAM-4774.

   Resolution: Fixed
Fix Version/s: 2.7.0

> Intergrate Nexmark SQL with Perfkit
> ---
>
> Key: BEAM-4774
> URL: https://issues.apache.org/jira/browse/BEAM-4774
> Project: Beam
>  Issue Type: New Feature
>  Components: dsl-sql
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.7.0
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> We should publish to a dashboard like pure JAVA does. See BEAM-4283.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-2848) Validate nexmark with the Google Dataflow runner

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-2848.
--
   Resolution: Duplicate
Fix Version/s: 2.7.0

> Validate nexmark with the Google Dataflow runner
> 
>
> Key: BEAM-2848
> URL: https://issues.apache.org/jira/browse/BEAM-2848
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-dataflow, testing
>Reporter: Ismaël Mejía
>Priority: Minor
>  Labels: dataflow, nexmark
> Fix For: 2.7.0
>
>
> Run all the queries with both bounded/unbounded sources with the dataflow 
> runner and update the execution matrix and create new specific JIRAs as 
> needed for the runner.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud closed BEAM-4761.

   Resolution: Fixed
Fix Version/s: 2.7.0

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.7.0
>
>  Time Spent: 10h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5092) Nexmark 10x performance regression

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570868#comment-16570868
 ] 

Andrew Pilloud commented on BEAM-5092:
--

Copy and paste from `job_PostCommit_Java_Nexmark_Direct.groovy`?

 
{code:java}
 ./gradlew :beam-sdks-java-nexmark:run 
-Pnexmark.runner=":beam-runners-direct-java" 
-Pnexmark.args="–runner=DirectRunner --streaming=false --suite=SMOKE 
--manageResources=false --monitorJobs=true --enforceEncodability=true 
--enforceImmutability=true"
{code}
 

> Nexmark 10x performance regression
> --
>
> Key: BEAM-5092
> URL: https://issues.apache.org/jira/browse/BEAM-5092
> Project: Beam
>  Issue Type: New Feature
>  Components: sdk-java-core
>Reporter: Andrew Pilloud
>Assignee: Reuven Lax
>Priority: Critical
>
> There looks to be a 10x performance hit on the DirectRunner and Flink nexmark 
> jobs. It first showed up in this build:
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Direct/151/changes]
> [https://apache-beam-testing.appspot.com/explore?dashboard=5084698770407424]
> [https://apache-beam-testing.appspot.com/explore?dashboard=5699257587728384]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (BEAM-5092) Nexmark 10x performance regression

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570854#comment-16570854
 ] 

Andrew Pilloud edited comment on BEAM-5092 at 8/6/18 10:16 PM:
---

It looks like Query 12 is broken in Streaming mode, I suspect it is related. 
Hopefully whatever you change to fix the performance will also fix this.

https://apache-beam-testing.appspot.com/explore?dashboard=5099379773931520


was (Author: apilloud):
It looks like Query 12 is broken in Streaming mode, I suspect it is related. 
Hopefully whatever you change to fix the performance will also fix this.

> Nexmark 10x performance regression
> --
>
> Key: BEAM-5092
> URL: https://issues.apache.org/jira/browse/BEAM-5092
> Project: Beam
>  Issue Type: New Feature
>  Components: sdk-java-core
>Reporter: Andrew Pilloud
>Assignee: Reuven Lax
>Priority: Critical
>
> There looks to be a 10x performance hit on the DirectRunner and Flink nexmark 
> jobs. It first showed up in this build:
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Direct/151/changes]
> [https://apache-beam-testing.appspot.com/explore?dashboard=5084698770407424]
> [https://apache-beam-testing.appspot.com/explore?dashboard=5699257587728384]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5092) Nexmark 10x performance regression

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570854#comment-16570854
 ] 

Andrew Pilloud commented on BEAM-5092:
--

It looks like Query 12 is broken in Streaming mode, I suspect it is related. 
Hopefully whatever you change to fix the performance will also fix this.

> Nexmark 10x performance regression
> --
>
> Key: BEAM-5092
> URL: https://issues.apache.org/jira/browse/BEAM-5092
> Project: Beam
>  Issue Type: New Feature
>  Components: sdk-java-core
>Reporter: Andrew Pilloud
>Assignee: Reuven Lax
>Priority: Critical
>
> There looks to be a 10x performance hit on the DirectRunner and Flink nexmark 
> jobs. It first showed up in this build:
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Direct/151/changes]
> [https://apache-beam-testing.appspot.com/explore?dashboard=5084698770407424]
> [https://apache-beam-testing.appspot.com/explore?dashboard=5699257587728384]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-5092) Nexmark 10x performance regression

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud updated BEAM-5092:
-
Description: 
There looks to be a 10x performance hit on the DirectRunner and Flink nexmark 
jobs. It first showed up in this build:

[https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Direct/151/changes]

[https://apache-beam-testing.appspot.com/explore?dashboard=5084698770407424]

[https://apache-beam-testing.appspot.com/explore?dashboard=5699257587728384]

  was:
There looks to be a 10x performance hit on the DirectRunner and Flink nexmark 
jobs. It first showed up in this build:

[https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Direct/151/changes]

[https://apache-beam-testing.appspot.com/explore?dashboard=5084698770407424
https://apache-beam-testing.appspot.com/explore?dashboard=5699257587728384

|https://apache-beam-testing.appspot.com/explore?dashboard=5084698770407424]


> Nexmark 10x performance regression
> --
>
> Key: BEAM-5092
> URL: https://issues.apache.org/jira/browse/BEAM-5092
> Project: Beam
>  Issue Type: New Feature
>  Components: sdk-java-core
>Reporter: Andrew Pilloud
>Assignee: Reuven Lax
>Priority: Critical
>
> There looks to be a 10x performance hit on the DirectRunner and Flink nexmark 
> jobs. It first showed up in this build:
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Direct/151/changes]
> [https://apache-beam-testing.appspot.com/explore?dashboard=5084698770407424]
> [https://apache-beam-testing.appspot.com/explore?dashboard=5699257587728384]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5092) Nexmark 10x performance regression

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5092:


 Summary: Nexmark 10x performance regression
 Key: BEAM-5092
 URL: https://issues.apache.org/jira/browse/BEAM-5092
 Project: Beam
  Issue Type: New Feature
  Components: sdk-java-core
Reporter: Andrew Pilloud
Assignee: Reuven Lax


There looks to be a 10x performance hit on the DirectRunner and Flink nexmark 
jobs. It first showed up in this build:

[https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Direct/151/changes]

[https://apache-beam-testing.appspot.com/explore?dashboard=5084698770407424
https://apache-beam-testing.appspot.com/explore?dashboard=5699257587728384

|https://apache-beam-testing.appspot.com/explore?dashboard=5084698770407424]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4971) Beam Dependency Update Request: jline:jline 3.0.0.M1

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-4971.
--
   Resolution: Invalid
Fix Version/s: Not applicable

> Beam Dependency Update Request: jline:jline 3.0.0.M1
> 
>
> Key: BEAM-4971
> URL: https://issues.apache.org/jira/browse/BEAM-4971
> Project: Beam
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Assignee: Kenneth Knowles
>Priority: Major
> Fix For: Not applicable
>
>
> 2018-07-25 20:30:39.308366
> Please review and upgrade the jline:jline to the latest version 
> 3.0.0.M1 
>  
> cc: 
> 2018-08-06 12:13:19.570559
> Please review and upgrade the jline:jline to the latest version 
> 3.0.0.M1 
>  
> cc: [~kedin], [~apilloud], [~amaliujia], 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-4971) Beam Dependency Update Request: jline:jline 3.0.0.M1

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570479#comment-16570479
 ] 

Andrew Pilloud commented on BEAM-4971:
--

We are pined to the calcite version of jline.

> Beam Dependency Update Request: jline:jline 3.0.0.M1
> 
>
> Key: BEAM-4971
> URL: https://issues.apache.org/jira/browse/BEAM-4971
> Project: Beam
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Priority: Major
>
> 2018-07-25 20:30:39.308366
> Please review and upgrade the jline:jline to the latest version 
> 3.0.0.M1 
>  
> cc: 
> 2018-08-06 12:13:19.570559
> Please review and upgrade the jline:jline to the latest version 
> 3.0.0.M1 
>  
> cc: [~kedin], [~apilloud], [~amaliujia], 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (BEAM-4804) Need dedicated jenkins workers for perf tests

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud closed BEAM-4804.


> Need dedicated jenkins workers for perf tests
> -
>
> Key: BEAM-4804
> URL: https://issues.apache.org/jira/browse/BEAM-4804
> Project: Beam
>  Issue Type: Bug
>  Components: examples-nexmark
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.7.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Our perf tests (such as nexmark) are currently sharing jenkins workers with 
> other builds. This results in huge variations in performance due to resource 
> sharing with another build. We should make the perf tests take the exclusive 
> lock on whatever they are running on.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4804) Need dedicated jenkins workers for perf tests

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-4804.
--
   Resolution: Fixed
Fix Version/s: 2.7.0

> Need dedicated jenkins workers for perf tests
> -
>
> Key: BEAM-4804
> URL: https://issues.apache.org/jira/browse/BEAM-4804
> Project: Beam
>  Issue Type: Bug
>  Components: examples-nexmark
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.7.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Our perf tests (such as nexmark) are currently sharing jenkins workers with 
> other builds. This results in huge variations in performance due to resource 
> sharing with another build. We should make the perf tests take the exclusive 
> lock on whatever they are running on.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-4804) Need dedicated jenkins workers for perf tests

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud reassigned BEAM-4804:


Assignee: Andrew Pilloud  (was: Kenneth Knowles)

> Need dedicated jenkins workers for perf tests
> -
>
> Key: BEAM-4804
> URL: https://issues.apache.org/jira/browse/BEAM-4804
> Project: Beam
>  Issue Type: Bug
>  Components: examples-nexmark
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
>
> Our perf tests (such as nexmark) are currently sharing jenkins workers with 
> other builds. This results in huge variations in performance due to resource 
> sharing with another build. We should make the perf tests take the exclusive 
> lock on whatever they are running on.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5051) Cleanup pubsub subscriptions

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-5051:


 Summary: Cleanup pubsub subscriptions
 Key: BEAM-5051
 URL: https://issues.apache.org/jira/browse/BEAM-5051
 Project: Beam
  Issue Type: New Feature
  Components: io-java-gcp
Reporter: Andrew Pilloud
Assignee: Andrew Pilloud


[https://github.com/apache/beam/pull/346#discussion_r63624693]

We should cleanup pubsub subscriptions when the pipeline is canceled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-3386) Dependency conflict when Calcite is included in a project.

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554519#comment-16554519
 ] 

Andrew Pilloud commented on BEAM-3386:
--

Good to know this still exists. Looks like calcite isn't easily repackageable. 
This issue can be fixed either in our shading (by also rewriting the 
"jdbc:calcite:" string when we shade calcite) or in calcite (by not using the 
driver manager to connect between calcite modules).

> Dependency conflict when Calcite is included in a project.
> --
>
> Key: BEAM-3386
> URL: https://issues.apache.org/jira/browse/BEAM-3386
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Affects Versions: 2.2.0, 2.3.0, 2.4.0, 2.5.0, 2.6.0
>Reporter: Austin Haas
>Priority: Critical
>
> When Calcite (v. 1.13.0) is included in a project that also includes Beam and 
> the Beam SQL extension, then the following error is thrown when trying to run 
> Beam code.
> ClassCastException 
> org.apache.beam.sdk.extensions.sql.impl.planner.BeamRelDataTypeSystem cannot 
> be cast to org.apache.calcite.rel.type.RelDataTypeSystem
> org.apache.calcite.jdbc.CalciteConnectionImpl. 
> (CalciteConnectionImpl.java:120)
> 
> org.apache.calcite.jdbc.CalciteJdbc41Factory$CalciteJdbc41Connection. 
> (CalciteJdbc41Factory.java:114)
> org.apache.calcite.jdbc.CalciteJdbc41Factory.newConnection 
> (CalciteJdbc41Factory.java:59)
> org.apache.calcite.jdbc.CalciteJdbc41Factory.newConnection 
> (CalciteJdbc41Factory.java:44)
> org.apache.calcite.jdbc.CalciteFactory.newConnection 
> (CalciteFactory.java:53)
> org.apache.calcite.avatica.UnregisteredDriver.connect 
> (UnregisteredDriver.java:138)
> java.sql.DriverManager.getConnection (DriverManager.java:664)
> java.sql.DriverManager.getConnection (DriverManager.java:208)
> 
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.tools.Frameworks.withPrepare
>  (Frameworks.java:145)
> 
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.tools.Frameworks.withPlanner
>  (Frameworks.java:106)
> 
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.prepare.PlannerImpl.ready
>  (PlannerImpl.java:140)
> 
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.prepare.PlannerImpl.parse
>  (PlannerImpl.java:170)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-4807) Upgrade calcite to 1.17.0

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-4807:


 Summary: Upgrade calcite to 1.17.0
 Key: BEAM-4807
 URL: https://issues.apache.org/jira/browse/BEAM-4807
 Project: Beam
  Issue Type: New Feature
  Components: dsl-sql
Reporter: Andrew Pilloud
Assignee: Andrew Pilloud


We should upgrade calcite.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-4804) Need dedicated jenkins workers for perf tests

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-4804:


 Summary: Need dedicated jenkins workers for perf tests
 Key: BEAM-4804
 URL: https://issues.apache.org/jira/browse/BEAM-4804
 Project: Beam
  Issue Type: Bug
  Components: examples-nexmark
Reporter: Andrew Pilloud
Assignee: Kenneth Knowles


Our perf tests (such as nexmark) are currently sharing jenkins workers with 
other builds. This results in huge variations in performance due to resource 
sharing with another build. We should make the perf tests take the exclusive 
lock on whatever they are running on.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4562) [SQL] Fix INSERT VALUES in JdbcDriver

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-4562.
--
   Resolution: Fixed
Fix Version/s: 2.6.0

> [SQL] Fix INSERT VALUES in JdbcDriver 
> --
>
> Key: BEAM-4562
> URL: https://issues.apache.org/jira/browse/BEAM-4562
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.6.0
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> Executing INSERT VALUES against JdbcDriver fails. Executing similar 
> statements against BeamSqlEnv works fine. Example:
> {code:java}
> TestTableProvider tableProvider = new TestTableProvider();
> Connection connection = JdbcDriver.connect(tableProvider);
> connection
> .createStatement()
> .executeUpdate("CREATE TABLE person (id BIGINT, name VARCHAR) TYPE 
> 'test'");
> connection.createStatement().executeQuery("INSERT INTO person VALUES(3, 
> 'yyy')");
> {code}
>  Output:
> {code}
> java.sql.SQLException: Error while executing SQL "INSERT INTO person 
> VALUES(3, 'yyy')": Node [rel#9:Subset#1.ENUMERABLE.[]] could not be 
> implemented; planner state:
> Root: rel#9:Subset#1.ENUMERABLE.[]
> Original rel:
> BeamIOSinkRel(subset=[rel#9:Subset#1.ENUMERABLE.[]], table=[[beam, person]], 
> operation=[INSERT], flattened=[false]): rowcount = 1.0, cumulative cost = 
> {1.0 rows, 0.0 cpu, 0.0 io}, id = 6
>   LogicalValues(subset=[rel#5:Subset#0.NONE.[]], tuples=[[{ 3, 'yyy' }]]): 
> rowcount = 1.0, cumulative cost = {1.0 rows, 1.0 cpu, 0.0 io}, id = 0
> Sets:
> Set#0, type: RecordType(BIGINT id, VARCHAR name)
>   rel#5:Subset#0.NONE.[], best=null, importance=0.81
>   rel#0:LogicalValues.NONE.[[0, 1], [1]](type=RecordType(BIGINT 
> id, VARCHAR name),tuples=[{ 3, 'yyy' }]), rowcount=1.0, cumulative cost={inf}
>   rel#14:Subset#0.BEAM_LOGICAL.[], best=null, importance=0.81
>   rel#20:Subset#0.ENUMERABLE.[], best=rel#19, importance=0.405
>   rel#19:EnumerableValues.ENUMERABLE.[[0, 1], 
> [1]](type=RecordType(BIGINT id, VARCHAR name),tuples=[{ 3, 'yyy' }]), 
> rowcount=1.0, cumulative cost={1.0 rows, 1.0 cpu, 0.0 io}
> Set#1, type: RecordType(BIGINT ROWCOUNT)
>   rel#7:Subset#1.BEAM_LOGICAL.[], best=null, importance=0.9
>   
> rel#6:BeamIOSinkRel.BEAM_LOGICAL.[](input=rel#5:Subset#0.NONE.[],table=[beam, 
> person],operation=INSERT,flattened=false), rowcount=1.0, cumulative cost={inf}
>   
> rel#15:BeamIOSinkRel.BEAM_LOGICAL.[](input=rel#14:Subset#0.BEAM_LOGICAL.[],table=[beam,
>  person],operation=INSERT,flattened=false), rowcount=1.0, cumulative 
> cost={inf}
>   rel#9:Subset#1.ENUMERABLE.[], best=null, importance=1.0
>   
> rel#10:AbstractConverter.ENUMERABLE.[](input=rel#7:Subset#1.BEAM_LOGICAL.[],convention=ENUMERABLE,sort=[]),
>  rowcount=1.0, cumulative cost={inf}
>   
> rel#11:BeamEnumerableConverter.ENUMERABLE.[](input=rel#7:Subset#1.BEAM_LOGICAL.[]),
>  rowcount=1.0, cumulative cost={inf}{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4799) Beam SQL JDBC broken

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-4799.
--
   Resolution: Fixed
Fix Version/s: 2.6.0

> Beam SQL JDBC broken
> 
>
> Key: BEAM-4799
> URL: https://issues.apache.org/jira/browse/BEAM-4799
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.6.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Rui found the JdbcIT is not running in precommits and failing:
>  
> {code:java}
> org.apache.beam.sdk.extensions.sql.jdbc.JdbcIT > classLoader_readFile FAILED
> java.sql.SQLException at JdbcIT.java:86
> Caused by: java.lang.IllegalArgumentException at JdbcIT.java:86
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4700) JDBC driver cannot support TIMESTAMP data type

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-4700.
--
   Resolution: Fixed
Fix Version/s: 2.6.0

> JDBC driver cannot support TIMESTAMP data type
> --
>
> Key: BEAM-4700
> URL: https://issues.apache.org/jira/browse/BEAM-4700
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.6.0
>
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> Avatica allows column representation to be customized, so a timestamp can be 
> stored as a variety of types. Joda ReadableInstant is none of these types: 
> https://github.com/apache/calcite-avatica/blob/acb675de97b9b0743c09368820a770e2ceda05f8/core/src/main/java/org/apache/calcite/avatica/util/AbstractCursor.java#L162
> By default, it seems to be configured to store {{TIMESTAMP}} columns as 
> {{long}} values. If you run the SQL shell and select a {{TIMESTAMP}} column, 
> you get:
> {code}
> ava.lang.ClassCastException: org.joda.time.Instant cannot be cast to 
> java.lang.Number
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$NumberAccessor.getNumber(AbstractCursor.java:726)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$TimestampFromNumberAccessor.getString(AbstractCursor.java:1026)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:225)
> at sqlline.Rows$Row.(Rows.java:183)
> {code}
> So, essentially, Beam SQL Shell does not support timestamps.
> We may be able to:
>  - override how the accessor for our existing storage is created
>  - configure what the column representation is (this doesn't really help, 
> since none of the choices are ours)
>  - convert timestamps to longs in BeamEnumerableConverter; not sure how many 
> conversions will be required here



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-4799) Beam SQL JDBC broken

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-4799:


 Summary: Beam SQL JDBC broken
 Key: BEAM-4799
 URL: https://issues.apache.org/jira/browse/BEAM-4799
 Project: Beam
  Issue Type: Bug
  Components: dsl-sql
Reporter: Andrew Pilloud
Assignee: Andrew Pilloud


Rui found the JdbcIT is not running in precommits and failing:

 
{code:java}
org.apache.beam.sdk.extensions.sql.jdbc.JdbcIT > classLoader_readFile FAILED
java.sql.SQLException at JdbcIT.java:86
Caused by: java.lang.IllegalArgumentException at JdbcIT.java:86
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-4761) Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud reassigned BEAM-4761:


Assignee: Andrew Pilloud  (was: Kenneth Knowles)

> Add postCommit scripts and perfkit dashboards for nexmark on Dataflow runner
> 
>
> Key: BEAM-4761
> URL: https://issues.apache.org/jira/browse/BEAM-4761
> Project: Beam
>  Issue Type: Test
>  Components: examples-nexmark
>Reporter: Etienne Chauchot
>Assignee: Andrew Pilloud
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-4774) Intergrate Nexmark SQL with Perfkit

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-4774:


 Summary: Intergrate Nexmark SQL with Perfkit
 Key: BEAM-4774
 URL: https://issues.apache.org/jira/browse/BEAM-4774
 Project: Beam
  Issue Type: New Feature
  Components: dsl-sql
Reporter: Andrew Pilloud
Assignee: Andrew Pilloud


We should publish to a dashboard like pure JAVA does. See BEAM-4283.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-4562) [SQL] Fix INSERT VALUES in JdbcDriver

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud reassigned BEAM-4562:


Assignee: Andrew Pilloud

> [SQL] Fix INSERT VALUES in JdbcDriver 
> --
>
> Key: BEAM-4562
> URL: https://issues.apache.org/jira/browse/BEAM-4562
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Andrew Pilloud
>Priority: Major
>
> Executing INSERT VALUES against JdbcDriver fails. Executing similar 
> statements against BeamSqlEnv works fine. Example:
> {code:java}
> TestTableProvider tableProvider = new TestTableProvider();
> Connection connection = JdbcDriver.connect(tableProvider);
> connection
> .createStatement()
> .executeUpdate("CREATE TABLE person (id BIGINT, name VARCHAR) TYPE 
> 'test'");
> connection.createStatement().executeQuery("INSERT INTO person VALUES(3, 
> 'yyy')");
> {code}
>  Output:
> {code}
> java.sql.SQLException: Error while executing SQL "INSERT INTO person 
> VALUES(3, 'yyy')": Node [rel#9:Subset#1.ENUMERABLE.[]] could not be 
> implemented; planner state:
> Root: rel#9:Subset#1.ENUMERABLE.[]
> Original rel:
> BeamIOSinkRel(subset=[rel#9:Subset#1.ENUMERABLE.[]], table=[[beam, person]], 
> operation=[INSERT], flattened=[false]): rowcount = 1.0, cumulative cost = 
> {1.0 rows, 0.0 cpu, 0.0 io}, id = 6
>   LogicalValues(subset=[rel#5:Subset#0.NONE.[]], tuples=[[{ 3, 'yyy' }]]): 
> rowcount = 1.0, cumulative cost = {1.0 rows, 1.0 cpu, 0.0 io}, id = 0
> Sets:
> Set#0, type: RecordType(BIGINT id, VARCHAR name)
>   rel#5:Subset#0.NONE.[], best=null, importance=0.81
>   rel#0:LogicalValues.NONE.[[0, 1], [1]](type=RecordType(BIGINT 
> id, VARCHAR name),tuples=[{ 3, 'yyy' }]), rowcount=1.0, cumulative cost={inf}
>   rel#14:Subset#0.BEAM_LOGICAL.[], best=null, importance=0.81
>   rel#20:Subset#0.ENUMERABLE.[], best=rel#19, importance=0.405
>   rel#19:EnumerableValues.ENUMERABLE.[[0, 1], 
> [1]](type=RecordType(BIGINT id, VARCHAR name),tuples=[{ 3, 'yyy' }]), 
> rowcount=1.0, cumulative cost={1.0 rows, 1.0 cpu, 0.0 io}
> Set#1, type: RecordType(BIGINT ROWCOUNT)
>   rel#7:Subset#1.BEAM_LOGICAL.[], best=null, importance=0.9
>   
> rel#6:BeamIOSinkRel.BEAM_LOGICAL.[](input=rel#5:Subset#0.NONE.[],table=[beam, 
> person],operation=INSERT,flattened=false), rowcount=1.0, cumulative cost={inf}
>   
> rel#15:BeamIOSinkRel.BEAM_LOGICAL.[](input=rel#14:Subset#0.BEAM_LOGICAL.[],table=[beam,
>  person],operation=INSERT,flattened=false), rowcount=1.0, cumulative 
> cost={inf}
>   rel#9:Subset#1.ENUMERABLE.[], best=null, importance=1.0
>   
> rel#10:AbstractConverter.ENUMERABLE.[](input=rel#7:Subset#1.BEAM_LOGICAL.[],convention=ENUMERABLE,sort=[]),
>  rowcount=1.0, cumulative cost={inf}
>   
> rel#11:BeamEnumerableConverter.ENUMERABLE.[](input=rel#7:Subset#1.BEAM_LOGICAL.[]),
>  rowcount=1.0, cumulative cost={inf}{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4590) Beam SQL JDBC driver should set User-agent PipelineOption

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-4590.
--
   Resolution: Fixed
Fix Version/s: 2.6.0

> Beam SQL JDBC driver should set User-agent PipelineOption
> -
>
> Key: BEAM-4590
> URL: https://issues.apache.org/jira/browse/BEAM-4590
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.6.0
>
>
> It makes sense to clearly call out what is a SQL pipeline, and this is 
> already a feature of the Java SDK at a global level (for all of its 
> interactions with any service).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-3634) [SQL] Refactor BeamRelNodes into PTransforms

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-3634.
--
   Resolution: Fixed
Fix Version/s: 2.6.0

> [SQL] Refactor BeamRelNodes into PTransforms
> 
>
> Key: BEAM-3634
> URL: https://issues.apache.org/jira/browse/BEAM-3634
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.6.0
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> BeamRelNode exposes PCollection buildBeamPipeline() which builds 
> a pipeline when parsing.
> It feels like it should instead implement a 
> PTransform, PCollection> which would 
> receive a prepared PCollection, and apply sub-expressions instead of manually 
> invoking expression evaluation to get the input.
> And maybe consider building it lazily.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-4746) Validate UDF/UDAF types on creation

[Andrew Pilloud (JIRA)]

Andrew Pilloud created BEAM-4746:


 Summary: Validate UDF/UDAF types on creation
 Key: BEAM-4746
 URL: https://issues.apache.org/jira/browse/BEAM-4746
 Project: Beam
  Issue Type: New Feature
  Components: dsl-sql
Reporter: Andrew Pilloud
Assignee: Xu Mingmin


We should validate the primitive types used in UDFs and UDAFs at creation time. 
Similar to the verifyPrimitiveType() call in Row. This will ensure the 
functions don't fail with a cryptic error message.

See: 
https://stackoverflow.com/questions/51239226/not-able-to-call-udf-in-beamsql



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-4704) String operations yield incorrect results when executed through SQL shell

[Andrew Pilloud (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16530127#comment-16530127
 ] 

Andrew Pilloud commented on BEAM-4704:
--

Correct. Queries that do not involve a Beam table will be implemented purely in 
calcite. There is a variant of this bug where queries that don't begin with a 
Beam table are unplannable.

> String operations yield incorrect results when executed through SQL shell
> -
>
> Key: BEAM-4704
> URL: https://issues.apache.org/jira/browse/BEAM-4704
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>
> {{TRIM}} is defined to trim _all_ the characters in the first string from the 
> string-to-be-trimmed. Calcite has an incorrect implementation of this. We use 
> our own fixed implementation. But when executed through the SQL shell, the 
> results do not match what we get from the PTransform path. Here two test 
> cases that pass on {{master}} but are incorrect in the shell:
> {code:sql}
> BeamSQL> select TRIM(LEADING 'eh' FROM 'hehe__hehe');
> ++
> | EXPR$0 |
> ++
> | hehe__hehe |
> ++
> {code}
> {code:sql}
> BeamSQL> select TRIM(TRAILING 'eh' FROM 'hehe__hehe');
> ++
> |   EXPR$0   |
> ++
> | hehe__heh  |
> ++
> {code}
> {code:sql}
> BeamSQL> select TRIM(BOTH 'eh' FROM 'hehe__hehe');
> ++
> |   EXPR$0   |
> ++
> | hehe__heh  |
> ++
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)