[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=323185=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323185
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 04/Oct/19 05:03
Start Date: 04/Oct/19 05:03
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on pull request #9446: 
[BEAM-8111] Enable CloudObjectsTest$DefaultCoders
URL: https://github.com/apache/beam/pull/9446
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323185)
Time Spent: 4h 50m  (was: 4h 40m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=318562=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318562
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 25/Sep/19 20:07
Start Date: 25/Sep/19 20:07
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on pull request #9446: [BEAM-8111] 
Enable CloudObjectsTest$DefaultCoders
URL: https://github.com/apache/beam/pull/9446#discussion_r328316138
 
 

 ##
 File path: 
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaCoder.java
 ##
 @@ -100,4 +99,47 @@ public boolean consistentWithEquals() {
   public String toString() {
 return "SchemaCoder: " + rowCoder.toString();
   }
+
+  @Override
+  public boolean equals(Object o) {
+if (this == o) {
+  return true;
+}
+if (o == null || getClass() != o.getClass()) {
+  return false;
+}
+SchemaCoder that = (SchemaCoder) o;
+return rowCoder.equals(that.rowCoder)
+&& toRowFunction.equals(that.toRowFunction)
+&& fromRowFunction.equals(that.fromRowFunction);
 
 Review comment:
   BTW this is not just for tests. The Flink runner appears to rely on coder 
equality (even though you can argue it shouldn't). 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 318562)
Time Spent: 4h 40m  (was: 4.5h)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=318470=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318470
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 25/Sep/19 17:26
Start Date: 25/Sep/19 17:26
Worklog Time Spent: 10m 
  Work Description: TheNeuralBit commented on issue #9446: [BEAM-8111] 
Enable CloudObjectsTest$DefaultCoders
URL: https://github.com/apache/beam/pull/9446#issuecomment-535126596
 
 
   R: @kennknowles 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 318470)
Time Spent: 4.5h  (was: 4h 20m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 4.5h
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=318398=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318398
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 25/Sep/19 15:14
Start Date: 25/Sep/19 15:14
Worklog Time Spent: 10m 
  Work Description: markflyhigh commented on pull request #9550: Revert 
"[BEAM-8111] Add ValidatesRunner test to AvroSchemaTest"
URL: https://github.com/apache/beam/pull/9550
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 318398)
Time Spent: 4h 20m  (was: 4h 10m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=317803=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317803
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 24/Sep/19 20:05
Start Date: 24/Sep/19 20:05
Worklog Time Spent: 10m 
  Work Description: robertwb commented on issue #9550: Revert "[BEAM-8111] 
Add ValidatesRunner test to AvroSchemaTest"
URL: https://github.com/apache/beam/pull/9550#issuecomment-534725691
 
 
   So should this be closed? 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 317803)
Time Spent: 4h 10m  (was: 4h)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 4h 10m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=316978=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-316978
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 23/Sep/19 21:09
Start Date: 23/Sep/19 21:09
Worklog Time Spent: 10m 
  Work Description: TheNeuralBit commented on pull request #9446: 
[BEAM-8111] Enable CloudObjectsTest$DefaultCoders
URL: https://github.com/apache/beam/pull/9446#discussion_r327330801
 
 

 ##
 File path: 
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaCoder.java
 ##
 @@ -100,4 +99,47 @@ public boolean consistentWithEquals() {
   public String toString() {
 return "SchemaCoder: " + rowCoder.toString();
   }
+
+  @Override
+  public boolean equals(Object o) {
+if (this == o) {
+  return true;
+}
+if (o == null || getClass() != o.getClass()) {
+  return false;
+}
+SchemaCoder that = (SchemaCoder) o;
+return rowCoder.equals(that.rowCoder)
+&& toRowFunction.equals(that.toRowFunction)
+&& fromRowFunction.equals(that.fromRowFunction);
 
 Review comment:
   I have a PR up now (https://github.com/apache/beam/pull/9493) that adds 
`equals` and `hashCode` to the `fromRow` and `toRow` functions created by all 
the `GetterBasedSchemaProvider` sub-classes.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 316978)
Time Spent: 4h  (was: 3h 50m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=314536=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314536
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 18/Sep/19 18:07
Start Date: 18/Sep/19 18:07
Worklog Time Spent: 10m 
  Work Description: TheNeuralBit commented on pull request #9446: 
[BEAM-8111] Enable CloudObjectsTest$DefaultCoders
URL: https://github.com/apache/beam/pull/9446#discussion_r325818608
 
 

 ##
 File path: 
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaCoder.java
 ##
 @@ -100,4 +99,47 @@ public boolean consistentWithEquals() {
   public String toString() {
 return "SchemaCoder: " + rowCoder.toString();
   }
+
+  @Override
+  public boolean equals(Object o) {
+if (this == o) {
+  return true;
+}
+if (o == null || getClass() != o.getClass()) {
+  return false;
+}
+SchemaCoder that = (SchemaCoder) o;
+return rowCoder.equals(that.rowCoder)
+&& toRowFunction.equals(that.toRowFunction)
+&& fromRowFunction.equals(that.fromRowFunction);
 
 Review comment:
   Could we go ahead and merge this as is? I could follow up with more changes 
to the SchemaCoder equals (plumbing through a type descriptor and using that 
for comparison, as well as possibly changing the toRow/fromRow functions 
created by the existing SchemaProviders to make them comparable)
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 314536)
Time Spent: 3h 50m  (was: 3h 40m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=311795=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311795
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 12/Sep/19 22:45
Start Date: 12/Sep/19 22:45
Worklog Time Spent: 10m 
  Work Description: TheNeuralBit commented on pull request #9446: 
[BEAM-8111] Enable CloudObjectsTest$DefaultCoders
URL: https://github.com/apache/beam/pull/9446#discussion_r323979264
 
 

 ##
 File path: 
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaCoder.java
 ##
 @@ -100,4 +99,47 @@ public boolean consistentWithEquals() {
   public String toString() {
 return "SchemaCoder: " + rowCoder.toString();
   }
+
+  @Override
+  public boolean equals(Object o) {
+if (this == o) {
+  return true;
+}
+if (o == null || getClass() != o.getClass()) {
+  return false;
+}
+SchemaCoder that = (SchemaCoder) o;
+return rowCoder.equals(that.rowCoder)
+&& toRowFunction.equals(that.toRowFunction)
+&& fromRowFunction.equals(that.fromRowFunction);
 
 Review comment:
   Another alternative could be to add something like 
`assertEquivalentSchemaCoder` that just checks schema and type, rather than 
continuing down this rabbit hole.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 311795)
Time Spent: 3h 40m  (was: 3.5h)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=311774=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311774
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 12/Sep/19 21:57
Start Date: 12/Sep/19 21:57
Worklog Time Spent: 10m 
  Work Description: TheNeuralBit commented on pull request #9446: 
[BEAM-8111] Enable CloudObjectsTest$DefaultCoders
URL: https://github.com/apache/beam/pull/9446#discussion_r323966059
 
 

 ##
 File path: 
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaCoder.java
 ##
 @@ -100,4 +99,47 @@ public boolean consistentWithEquals() {
   public String toString() {
 return "SchemaCoder: " + rowCoder.toString();
   }
+
+  @Override
+  public boolean equals(Object o) {
+if (this == o) {
+  return true;
+}
+if (o == null || getClass() != o.getClass()) {
+  return false;
+}
+SchemaCoder that = (SchemaCoder) o;
+return rowCoder.equals(that.rowCoder)
+&& toRowFunction.equals(that.toRowFunction)
+&& fromRowFunction.equals(that.fromRowFunction);
 
 Review comment:
   That's true. I was thinking it's not such a big deal to get false negatives 
when lambdas are used, since I really just want the equality check to use in 
tests.
   
   What do you think about updating the various schema providers to create 
Function sub-classes (with equals implemented) instead of using lambdas?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 311774)
Time Spent: 3.5h  (was: 3h 20m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=311718=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311718
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 12/Sep/19 20:55
Start Date: 12/Sep/19 20:55
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on pull request #9446: [BEAM-8111] 
Enable CloudObjectsTest$DefaultCoders
URL: https://github.com/apache/beam/pull/9446#discussion_r323945107
 
 

 ##
 File path: 
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaCoder.java
 ##
 @@ -100,4 +99,47 @@ public boolean consistentWithEquals() {
   public String toString() {
 return "SchemaCoder: " + rowCoder.toString();
   }
+
+  @Override
+  public boolean equals(Object o) {
+if (this == o) {
+  return true;
+}
+if (o == null || getClass() != o.getClass()) {
+  return false;
+}
+SchemaCoder that = (SchemaCoder) o;
+return rowCoder.equals(that.rowCoder)
+&& toRowFunction.equals(that.toRowFunction)
+&& fromRowFunction.equals(that.fromRowFunction);
 
 Review comment:
   Sounds good in theory. In practice these functions are usually lambdas, so 
we might have trouble making this work.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 311718)
Time Spent: 3h 20m  (was: 3h 10m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=311200=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311200
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 12/Sep/19 06:18
Start Date: 12/Sep/19 06:18
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on pull request #9446: 
[BEAM-8111] Enable CloudObjectsTest$DefaultCoders
URL: https://github.com/apache/beam/pull/9446#discussion_r323571191
 
 

 ##
 File path: 
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaCoder.java
 ##
 @@ -100,4 +99,47 @@ public boolean consistentWithEquals() {
   public String toString() {
 return "SchemaCoder: " + rowCoder.toString();
   }
+
+  @Override
+  public boolean equals(Object o) {
+if (this == o) {
+  return true;
+}
+if (o == null || getClass() != o.getClass()) {
+  return false;
+}
+SchemaCoder that = (SchemaCoder) o;
+return rowCoder.equals(that.rowCoder)
+&& toRowFunction.equals(that.toRowFunction)
+&& fromRowFunction.equals(that.fromRowFunction);
 
 Review comment:
   The way I would phrase this is: let the functions own their equals. If they 
say they are equal, they are. If they say they aren't, they aren't. So this 
equals() is relative to that.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 311200)
Time Spent: 3h 10m  (was: 3h)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=310983=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310983
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 11/Sep/19 20:33
Start Date: 11/Sep/19 20:33
Worklog Time Spent: 10m 
  Work Description: TheNeuralBit commented on pull request #9446: 
[BEAM-8111] Enable CloudObjectsTest$DefaultCoders
URL: https://github.com/apache/beam/pull/9446#discussion_r323445919
 
 

 ##
 File path: 
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaCoder.java
 ##
 @@ -100,4 +99,47 @@ public boolean consistentWithEquals() {
   public String toString() {
 return "SchemaCoder: " + rowCoder.toString();
   }
+
+  @Override
+  public boolean equals(Object o) {
+if (this == o) {
+  return true;
+}
+if (o == null || getClass() != o.getClass()) {
+  return false;
+}
+SchemaCoder that = (SchemaCoder) o;
+return rowCoder.equals(that.rowCoder)
+&& toRowFunction.equals(that.toRowFunction)
+&& fromRowFunction.equals(that.fromRowFunction);
 
 Review comment:
   Yeah - I discussed this offline a bit with @kennknowles and he convinced me 
that it was better to have an equals function that might have some false 
negatives (if the toRowFunction and fromRowFunction don't have a good equals), 
rather than one that could have false positives (like if we rely on just 
checking the schema and typeDescriptor, and assume that the toRow/fromRow are 
the same).
   
   I managed to make the CloudObjectsTest work by adding RowIdentity with an 
equals() function here.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 310983)
Time Spent: 3h  (was: 2h 50m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=310964=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310964
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 11/Sep/19 20:04
Start Date: 11/Sep/19 20:04
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on pull request #9446: [BEAM-8111] 
Enable CloudObjectsTest$DefaultCoders
URL: https://github.com/apache/beam/pull/9446#discussion_r323433725
 
 

 ##
 File path: 
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaCoder.java
 ##
 @@ -100,4 +99,47 @@ public boolean consistentWithEquals() {
   public String toString() {
 return "SchemaCoder: " + rowCoder.toString();
   }
+
+  @Override
+  public boolean equals(Object o) {
+if (this == o) {
+  return true;
+}
+if (o == null || getClass() != o.getClass()) {
+  return false;
+}
+SchemaCoder that = (SchemaCoder) o;
+return rowCoder.equals(that.rowCoder)
+&& toRowFunction.equals(that.toRowFunction)
+&& fromRowFunction.equals(that.fromRowFunction);
 
 Review comment:
   Doesn't this just revert to object equality comparison on the to/from 
functions?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 310964)
Time Spent: 2h 50m  (was: 2h 40m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=310962=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310962
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 11/Sep/19 19:59
Start Date: 11/Sep/19 19:59
Worklog Time Spent: 10m 
  Work Description: markflyhigh commented on issue #9550: Revert 
"[BEAM-8111] Add ValidatesRunner test to AvroSchemaTest"
URL: https://github.com/apache/beam/pull/9550#issuecomment-530541628
 
 
   It should be Flink. Thank you Reuven!
   
   Robin already had https://github.com/apache/beam/pull/9541 to skip this test 
and test is done. I think we can continue with 
https://github.com/apache/beam/pull/9541 and ignore this rollback.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 310962)
Time Spent: 2h 40m  (was: 2.5h)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=310946=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310946
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 11/Sep/19 19:15
Start Date: 11/Sep/19 19:15
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #9550: Revert "[BEAM-8111] 
Add ValidatesRunner test to AvroSchemaTest"
URL: https://github.com/apache/beam/pull/9550#issuecomment-530525184
 
 
   Also I suspect that https://github.com/apache/beam/pull/9446 will fix this.
   
   On Wed, Sep 11, 2019 at 12:05 PM Reuven Lax  wrote:
   
   > Also it appears that Flink relies on all the Coders being exactly the
   > same, which is far more restrictive than other runners. Is there another
   > way for Flink to do this - maybe comparing the TypeDescriptor instead of
   > the Coder?
   >
   > On Wed, Sep 11, 2019 at 11:54 AM Reuven Lax  wrote:
   >
   >> FYI this appears to be Flink ValidatesRunner, not Dataflow?
   >>
   >> On Wed, Sep 11, 2019 at 11:30 AM Mark Liu 
   >> wrote:
   >>
   >>> @markflyhigh  requested your review on:
   >>> #9550  Revert "[BEAM-8111]
   >>> Add ValidatesRunner test to AvroSchemaTest".
   >>>
   >>> —
   >>> You are receiving this because your review was requested.
   >>> Reply to this email directly, view it on GitHub
   >>> 
,
   >>> or mute the thread
   >>> 

   >>> .
   >>>
   >>
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 310946)
Time Spent: 2.5h  (was: 2h 20m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=310940=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310940
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 11/Sep/19 19:05
Start Date: 11/Sep/19 19:05
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #9550: Revert "[BEAM-8111] 
Add ValidatesRunner test to AvroSchemaTest"
URL: https://github.com/apache/beam/pull/9550#issuecomment-530521285
 
 
   Also it appears that Flink relies on all the Coders being exactly the same,
   which is far more restrictive than other runners. Is there another way for
   Flink to do this - maybe comparing the TypeDescriptor instead of the Coder?
   
   On Wed, Sep 11, 2019 at 11:54 AM Reuven Lax  wrote:
   
   > FYI this appears to be Flink ValidatesRunner, not Dataflow?
   >
   > On Wed, Sep 11, 2019 at 11:30 AM Mark Liu 
   > wrote:
   >
   >> @markflyhigh  requested your review on:
   >> #9550  Revert "[BEAM-8111] Add
   >> ValidatesRunner test to AvroSchemaTest".
   >>
   >> —
   >> You are receiving this because your review was requested.
   >> Reply to this email directly, view it on GitHub
   >> 
,
   >> or mute the thread
   >> 

   >> .
   >>
   >
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 310940)
Time Spent: 2h 20m  (was: 2h 10m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=310933=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310933
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 11/Sep/19 18:54
Start Date: 11/Sep/19 18:54
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #9550: Revert "[BEAM-8111] 
Add ValidatesRunner test to AvroSchemaTest"
URL: https://github.com/apache/beam/pull/9550#issuecomment-530517191
 
 
   FYI this appears to be Flink ValidatesRunner, not Dataflow?
   
   On Wed, Sep 11, 2019 at 11:30 AM Mark Liu  wrote:
   
   > @markflyhigh  requested your review on:
   > #9550  Revert "[BEAM-8111] Add
   > ValidatesRunner test to AvroSchemaTest".
   >
   > —
   > You are receiving this because your review was requested.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or mute the thread
   > 

   > .
   >
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 310933)
Time Spent: 2h 10m  (was: 2h)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=310918=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310918
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 11/Sep/19 18:30
Start Date: 11/Sep/19 18:30
Worklog Time Spent: 10m 
  Work Description: markflyhigh commented on pull request #9550: Revert 
"[BEAM-8111] Add ValidatesRunner test to AvroSchemaTest"
URL: https://github.com/apache/beam/pull/9550
 
 
   Reverts apache/beam#9454 due to AvroSchemaTest.testAvroPipelineGroupBy 
consistently failed in Dataflow ValidatesRunner test suite.
   
   
[This](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/5178/)
 is the first failure instance. You can go to console output or Gradle scan for 
error details.
   
   +R: @reuvenlax 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 310918)
Time Spent: 2h  (was: 1h 50m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=308238=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-308238
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 07/Sep/19 00:20
Start Date: 07/Sep/19 00:20
Worklog Time Spent: 10m 
  Work Description: TheNeuralBit commented on issue #9446: [BEAM-8111] 
Enable CloudObjectsTest$DefaultCoders
URL: https://github.com/apache/beam/pull/9446#issuecomment-529053161
 
 
   R: @reuvenlax would you mind reviewing this?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 308238)
Time Spent: 1h 50m  (was: 1h 40m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=308195=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-308195
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 06/Sep/19 22:14
Start Date: 06/Sep/19 22:14
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on pull request #9454: [BEAM-8111] 
Add ValidatesRunner test to AvroSchemaTest
URL: https://github.com/apache/beam/pull/9454
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 308195)
Time Spent: 1h 40m  (was: 1.5h)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=307301=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-307301
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 05/Sep/19 16:31
Start Date: 05/Sep/19 16:31
Worklog Time Spent: 10m 
  Work Description: TheNeuralBit commented on issue #9454: [BEAM-8111] Add 
ValidatesRunner test to AvroSchemaTest
URL: https://github.com/apache/beam/pull/9454#issuecomment-528450826
 
 
   Yeah I ran the Dataflow validates runner tests on commit 
https://github.com/apache/beam/pull/9454/commits/fb93067aee5af06025ce0318f8a29d6d5ebf3500
   
   Here's the Jenkins run: 
https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_PR/100/
   
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 307301)
Time Spent: 1.5h  (was: 1h 20m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=307295=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-307295
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 05/Sep/19 16:23
Start Date: 05/Sep/19 16:23
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #9454: [BEAM-8111] Add 
ValidatesRunner test to AvroSchemaTest
URL: https://github.com/apache/beam/pull/9454#issuecomment-528447906
 
 
   did you verify that this test would've caught the regression?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 307295)
Time Spent: 1h 20m  (was: 1h 10m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=306792=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-306792
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 05/Sep/19 00:21
Start Date: 05/Sep/19 00:21
Worklog Time Spent: 10m 
  Work Description: TheNeuralBit commented on issue #9454: [BEAM-8111] Add 
ValidatesRunner test to AvroSchemaTest
URL: https://github.com/apache/beam/pull/9454#issuecomment-528143139
 
 
   R: @reuvenlax 
   CC: @lukecwik 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 306792)
Time Spent: 1h 10m  (was: 1h)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
> Fix For: 2.16.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=304345=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-304345
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 30/Aug/19 15:56
Start Date: 30/Aug/19 15:56
Worklog Time Spent: 10m 
  Work Description: aaltay commented on pull request #9455: [BEAM-8111] 
Update dataflow container version to revert SchemaCoder change
URL: https://github.com/apache/beam/pull/9455
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 304345)
Time Spent: 1h  (was: 50m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=304054=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-304054
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 30/Aug/19 05:37
Start Date: 30/Aug/19 05:37
Worklog Time Spent: 10m 
  Work Description: TheNeuralBit commented on issue #9455: [BEAM-8111] 
Update dataflow container version to revert SchemaCoder change
URL: https://github.com/apache/beam/pull/9455#issuecomment-526463400
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 304054)
Time Spent: 50m  (was: 40m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=304042=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-304042
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 30/Aug/19 04:52
Start Date: 30/Aug/19 04:52
Worklog Time Spent: 10m 
  Work Description: TheNeuralBit commented on pull request #9455: 
[BEAM-8111] Update dataflow container version to revert SchemaCoder change
URL: https://github.com/apache/beam/pull/9455
 
 
   R: @aaltay 
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
 | --- | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/)
 | --- | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)
 | --- | --- | [![Build 

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=303288=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303288
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 28/Aug/19 23:04
Start Date: 28/Aug/19 23:04
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on pull request #9443: [BEAM-8111] 
SchemaCoder broken on DataflowRunner
URL: https://github.com/apache/beam/pull/9443
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 303288)
Time Spent: 0.5h  (was: 20m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.
> This means any pipelines that use SqlTransform or schema transforms will fail 
> on Dataflow in 2.15.0



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=303268=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303268
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 28/Aug/19 22:13
Start Date: 28/Aug/19 22:13
Worklog Time Spent: 10m 
  Work Description: TheNeuralBit commented on issue #9443: [BEAM-8111] 
SchemaCoder broken on DataflowRunner
URL: https://github.com/apache/beam/pull/9443#issuecomment-525942861
 
 
   Run Python PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 303268)
Time Spent: 20m  (was: 10m)

> SchemaCoder broken on DataflowRunner
> 
>
> Key: BEAM-8111
> URL: https://issues.apache.org/jira/browse/BEAM-8111
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core
>Affects Versions: 2.15.0
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Blocker
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c
>  broke SchemaCoder on Dataflow. When translating a schema that uses logical 
> types from a cloud object dataflow encounters a runtime error.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Work logged] (BEAM-8111) SchemaCoder broken on DataflowRunner

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8111?focusedWorklogId=303266=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303266
 ]

ASF GitHub Bot logged work on BEAM-8111:


Author: ASF GitHub Bot
Created on: 28/Aug/19 22:06
Start Date: 28/Aug/19 22:06
Worklog Time Spent: 10m 
  Work Description: TheNeuralBit commented on pull request #9443: 
[BEAM-8111] SchemaCoder broken on DataflowRunner
URL: https://github.com/apache/beam/pull/9443
 
 
   Reverts 
https://github.com/apache/beam/commit/e65c176a9f34e45d408281e1101a2ae54cef0f6c 
and 
https://github.com/apache/beam/commit/559708610f13e660296d1639e3fdc2e9eea3fd9c 
in order to fix SchemaCoder on Dataflow runner.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
 | --- | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/)
 | --- | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | [![Build