[beam] 01/01: Merge pull request #5918: [BEAM-4562] [SQL] Fix INSERT VALUES in JDBC

[mingmxu]

This is an automated email from the ASF dual-hosted git repository.

mingmxu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 39db610a9b2f1aba7381fd424a0ed742e2326b8e
Merge: 109f03c 1275032
Author: XuMingmin 
AuthorDate: Tue Jul 10 22:45:38 2018 -0700

Merge pull request #5918: [BEAM-4562] [SQL] Fix INSERT VALUES in JDBC

 .../sdk/extensions/sql/impl/BeamCalciteTable.java  |  3 ++-
 .../beam/sdk/extensions/sql/impl/JdbcDriver.java   | 22 ++
 .../sdk/extensions/sql/impl/rel/BeamIOSinkRel.java | 14 --
 .../sql/impl/rel/BeamLogicalConvention.java| 13 +
 .../sql/impl/rel/BeamEnumerableConverterTest.java  |  3 ++-
 5 files changed, 39 insertions(+), 16 deletions(-)

[jira] [Work logged] (BEAM-4562) [SQL] Fix INSERT VALUES in JdbcDriver

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4562?focusedWorklogId=121770=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121770
 ]

ASF GitHub Bot logged work on BEAM-4562:


Author: ASF GitHub Bot
Created on: 11/Jul/18 05:45
Start Date: 11/Jul/18 05:45
Worklog Time Spent: 10m 
  Work Description: XuMingmin closed pull request #5918: BEAM-4562: [SQL] 
Fix INSERT VALUES in JDBC
URL: https://github.com/apache/beam/pull/5918
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamCalciteTable.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamCalciteTable.java
index 8007d1eae92..fd2eddf171d 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamCalciteTable.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamCalciteTable.java
@@ -91,6 +91,7 @@ public TableModify toModificationRel(
 updateColumnList,
 sourceExpressionList,
 flattened,
-beamTable);
+beamTable,
+pipelineOptions);
   }
 }
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcDriver.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcDriver.java
index 75a2e901eba..4b66e93d514 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcDriver.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcDriver.java
@@ -21,12 +21,14 @@
 
 import com.google.auto.service.AutoService;
 import com.google.common.annotations.VisibleForTesting;
+import com.google.common.base.Function;
 import java.sql.Connection;
 import java.sql.SQLException;
 import java.util.Map;
 import java.util.Properties;
 import org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
 import org.apache.beam.sdk.extensions.sql.impl.planner.BeamRelDataTypeSystem;
+import org.apache.beam.sdk.extensions.sql.impl.planner.BeamRuleSets;
 import org.apache.beam.sdk.extensions.sql.meta.provider.TableProvider;
 import org.apache.beam.sdk.options.PipelineOptions;
 import org.apache.beam.sdk.util.ReleaseInfo;
@@ -37,7 +39,12 @@
 import org.apache.calcite.jdbc.CalciteConnection;
 import org.apache.calcite.jdbc.CalciteSchema;
 import org.apache.calcite.jdbc.Driver;
+import org.apache.calcite.plan.RelOptPlanner;
+import org.apache.calcite.plan.RelOptRule;
+import org.apache.calcite.rel.rules.CalcRemoveRule;
+import org.apache.calcite.runtime.Hook;
 import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.tools.RuleSet;
 
 /**
  * Calcite JDBC driver with Beam defaults.
@@ -73,6 +80,21 @@
 } finally {
   Thread.currentThread().setContextClassLoader(origLoader);
 }
+// inject beam rules into planner
+Hook.PLANNER.addThread(
+new Function() {
+  @Override
+  public Void apply(RelOptPlanner planner) {
+for (RuleSet ruleSet : BeamRuleSets.getRuleSets()) {
+  for (RelOptRule rule : ruleSet) {
+planner.addRule(rule);
+  }
+}
+planner.removeRule(CalcRemoveRule.INSTANCE);
+return null;
+  }
+});
+// register JDBC driver
 INSTANCE.register();
   }
 
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamIOSinkRel.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamIOSinkRel.java
index 350bccfa02b..65c97c4f88d 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamIOSinkRel.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamIOSinkRel.java
@@ -20,6 +20,7 @@
 import static com.google.common.base.Preconditions.checkArgument;
 
 import java.util.List;
+import java.util.Map;
 import org.apache.beam.sdk.extensions.sql.BeamSqlTable;
 import org.apache.beam.sdk.extensions.sql.impl.rule.BeamIOSinkRule;
 import org.apache.beam.sdk.transforms.PTransform;
@@ -41,6 +42,7 @@
 implements BeamRelNode, RelStructuredTypeFlattener.SelfFlatteningRel {
 
   private final BeamSqlTable sqlTable;
+  private final Map pipelineOptions;
   private boolean isFlattening = false;
 
   public BeamIOSinkRel(
@@ -52,7 +54,8 @@ public BeamIOSinkRel(
   List updateColumnList,
   List sourceExpressionList,
   boolean flattened,
-  BeamSqlTable sqlTable) {
+  BeamSqlTable sqlTable,
+  Map 

[beam] branch master updated (109f03c -> 39db610)

[mingmxu]

This is an automated email from the ASF dual-hosted git repository.

mingmxu pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 109f03c  [BEAM-4481, BEAM-4484] Start vendoring portability 
dependencies to not have dependency conflicts
 add 22896fa  [SQL] Inject JDBC rules through Hook.
 add 1275032  [SQL] Plumb pipelineOptions through IOSinkRel
 new 39db610  Merge pull request #5918: [BEAM-4562] [SQL] Fix INSERT VALUES 
in JDBC

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../sdk/extensions/sql/impl/BeamCalciteTable.java  |  3 ++-
 .../beam/sdk/extensions/sql/impl/JdbcDriver.java   | 22 ++
 .../sdk/extensions/sql/impl/rel/BeamIOSinkRel.java | 14 --
 .../sql/impl/rel/BeamLogicalConvention.java| 13 +
 .../sql/impl/rel/BeamEnumerableConverterTest.java  |  3 ++-
 5 files changed, 39 insertions(+), 16 deletions(-)

[jira] [Work logged] (BEAM-4663) Implement Cost calculations for Cost-Based Optimization (CBO)

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4663?focusedWorklogId=121762=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121762
 ]

ASF GitHub Bot logged work on BEAM-4663:


Author: ASF GitHub Bot
Created on: 11/Jul/18 04:40
Start Date: 11/Jul/18 04:40
Worklog Time Spent: 10m 
  Work Description: vectorijk commented on issue #5825: [WIP] [BEAM-4663] 
CBO cost calculation
URL: https://github.com/apache/beam/pull/5825#issuecomment-404043101
 
 
   spotlessApply and rebased


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121762)
Time Spent: 0.5h  (was: 20m)

> Implement Cost calculations for Cost-Based Optimization (CBO) 
> --
>
> Key: BEAM-4663
> URL: https://issues.apache.org/jira/browse/BEAM-4663
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kai Jiang
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> To support CBO, we should implement methods in each Beam*Rel.java.  
> computeSelfCost(...) as our first step.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4599) Test arithmetic operators at the DSL level

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4599?focusedWorklogId=121760=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121760
 ]

ASF GitHub Bot logged work on BEAM-4599:


Author: ASF GitHub Bot
Created on: 11/Jul/18 04:25
Start Date: 11/Jul/18 04:25
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #5921: [BEAM-4599] [SQL] 
add testArithmeticOperator
URL: https://github.com/apache/beam/pull/5921#issuecomment-404040988
 
 
   run java precommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121760)
Time Spent: 1h 50m  (was: 1h 40m)

> Test arithmetic operators at the DSL level
> --
>
> Key: BEAM-4599
> URL: https://issues.apache.org/jira/browse/BEAM-4599
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4599) Test arithmetic operators at the DSL level

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4599?focusedWorklogId=121761=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121761
 ]

ASF GitHub Bot logged work on BEAM-4599:


Author: ASF GitHub Bot
Created on: 11/Jul/18 04:25
Start Date: 11/Jul/18 04:25
Worklog Time Spent: 10m 
  Work Description: amaliujia removed a comment on issue #5921: [BEAM-4599] 
[SQL] add testArithmeticOperator
URL: https://github.com/apache/beam/pull/5921#issuecomment-404013318
 
 
   run java precommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121761)
Time Spent: 2h  (was: 1h 50m)

> Test arithmetic operators at the DSL level
> --
>
> Key: BEAM-4599
> URL: https://issues.apache.org/jira/browse/BEAM-4599
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4745) SDF tests broken by innocent change due to Dataflow worker dependencies

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4745?focusedWorklogId=121740=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121740
 ]

ASF GitHub Bot logged work on BEAM-4745:


Author: ASF GitHub Bot
Created on: 11/Jul/18 01:47
Start Date: 11/Jul/18 01:47
Worklog Time Spent: 10m 
  Work Description: jkff commented on issue #5925: [BEAM-4745, BEAM-4016] 
Reintroduces Setup and TearDown on SplitRestrictionFn and PairWithRestrictionFn
URL: https://github.com/apache/beam/pull/5925#issuecomment-404018457
 
 
   Run Dataflow ValidatesRunner


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121740)
Time Spent: 40m  (was: 0.5h)

> SDF tests broken by innocent change due to Dataflow worker dependencies
> ---
>
> Key: BEAM-4745
> URL: https://issues.apache.org/jira/browse/BEAM-4745
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow
>Reporter: Eugene Kirpichov
>Assignee: Eugene Kirpichov
>Priority: Critical
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/pull/5894 broke SDF in Dataflow streaming 
> runner, using SDFs fails with the error below.
> The reason is that Dataflow worker has a staged copy of some stuff including 
> runners-core-construction, and it comes before user code in the classpath. So 
> the pipeline includes a serialized SplittableParDo from master, but the 
> worker deserializes it using a stale class file.
> This needs to be fixed on Dataflow side. Filing this JIRA just to track the 
> externally facing issue.
> Meanwhile to stop the bleeding I'm going to revert the change, even though by 
> itself it's a correct change, but it's better to have SDFs not invoke 
> setup/teardown than to have them not work at all.
> CC: [~iemejia]
> java.lang.RuntimeException: 
> com.google.cloud.dataflow.worker.repackaged.com.google.common.util.concurrent.UncheckedExecutionException:
>  java.lang.IllegalArgumentException: unable to deserialize Serialized DoFnInfo
> 
> com.google.cloud.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:192)
> 
> com.google.cloud.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:163)
> 
> com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:63)
> 
> com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:50)
> 
> com.google.cloud.dataflow.worker.graph.Networks.replaceDirectedNetworkNodes(Networks.java:87)
> 
> com.google.cloud.dataflow.worker.IntrinsicMapTaskExecutorFactory.create(IntrinsicMapTaskExecutorFactory.java:123)
> 
> com.google.cloud.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1143)
> 
> com.google.cloud.dataflow.worker.StreamingDataflowWorker.access$1000(StreamingDataflowWorker.java:136)
> 
> com.google.cloud.dataflow.worker.StreamingDataflowWorker$6.run(StreamingDataflowWorker.java:966)
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> Caused by: 
> com.google.cloud.dataflow.worker.repackaged.com.google.common.util.concurrent.UncheckedExecutionException:
>  java.lang.IllegalArgumentException: unable to deserialize Serialized DoFnInfo
> 
> com.google.cloud.dataflow.worker.repackaged.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2214)
> 
> com.google.cloud.dataflow.worker.repackaged.com.google.common.cache.LocalCache.get(LocalCache.java:4053)
> 
> com.google.cloud.dataflow.worker.repackaged.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4899)
> 
> com.google.cloud.dataflow.worker.UserParDoFnFactory.create(UserParDoFnFactory.java:90)
> 
> com.google.cloud.dataflow.worker.DefaultParDoFnFactory.create(DefaultParDoFnFactory.java:74)
> 
> com.google.cloud.dataflow.worker.IntrinsicMapTaskExecutorFactory.createParDoOperation(IntrinsicMapTaskExecutorFactory.java:262)
> 
> com.google.cloud.dataflow.worker.IntrinsicMapTaskExecutorFactory.access$000(IntrinsicMapTaskExecutorFactory.java:84)
> 
> 

[jira] [Work logged] (BEAM-4745) SDF tests broken by innocent change due to Dataflow worker dependencies

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4745?focusedWorklogId=121739=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121739
 ]

ASF GitHub Bot logged work on BEAM-4745:


Author: ASF GitHub Bot
Created on: 11/Jul/18 01:46
Start Date: 11/Jul/18 01:46
Worklog Time Spent: 10m 
  Work Description: jkff opened a new pull request #5925: [BEAM-4745, 
BEAM-4016] Reintroduces Setup and TearDown on SplitRestrictionFn and 
PairWithRestrictionFn
URL: https://github.com/apache/beam/pull/5925
 
 
   This should be safe now, after a Dataflow worker release that is more 
correctly shaded and picks up fresh versions of this code.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121739)
Time Spent: 0.5h  (was: 20m)

> SDF tests broken by innocent change due to Dataflow worker dependencies
> ---
>
> Key: BEAM-4745
> URL: https://issues.apache.org/jira/browse/BEAM-4745
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow
>Reporter: Eugene Kirpichov
>Assignee: Eugene Kirpichov
>Priority: Critical
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> https://github.com/apache/beam/pull/5894 broke SDF in Dataflow streaming 
> runner, using SDFs fails with the error below.
> The reason is that Dataflow worker has a staged copy of some stuff including 
> runners-core-construction, and it comes before user code in the classpath. So 
> the pipeline includes a serialized SplittableParDo from master, but the 
> worker deserializes it using a stale class file.
> This needs to be fixed on Dataflow side. Filing this JIRA just to track the 
> externally facing issue.
> Meanwhile to stop the bleeding I'm going to revert the change, even though by 
> itself it's a correct change, but it's better to have SDFs not invoke 
> setup/teardown than to have them not work at all.
> CC: [~iemejia]
> java.lang.RuntimeException: 
> com.google.cloud.dataflow.worker.repackaged.com.google.common.util.concurrent.UncheckedExecutionException:
>  java.lang.IllegalArgumentException: unable to deserialize Serialized DoFnInfo
> 
> com.google.cloud.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:192)
> 
> com.google.cloud.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:163)
> 
> com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:63)
> 
> com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:50)
> 
> com.google.cloud.dataflow.worker.graph.Networks.replaceDirectedNetworkNodes(Networks.java:87)
> 
> com.google.cloud.dataflow.worker.IntrinsicMapTaskExecutorFactory.create(IntrinsicMapTaskExecutorFactory.java:123)
> 
> com.google.cloud.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1143)
> 
> com.google.cloud.dataflow.worker.StreamingDataflowWorker.access$1000(StreamingDataflowWorker.java:136)
> 
> com.google.cloud.dataflow.worker.StreamingDataflowWorker$6.run(StreamingDataflowWorker.java:966)
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> Caused by: 
> com.google.cloud.dataflow.worker.repackaged.com.google.common.util.concurrent.UncheckedExecutionException:
>  java.lang.IllegalArgumentException: unable to deserialize Serialized DoFnInfo
> 
> com.google.cloud.dataflow.worker.repackaged.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2214)
> 
> com.google.cloud.dataflow.worker.repackaged.com.google.common.cache.LocalCache.get(LocalCache.java:4053)
> 
> com.google.cloud.dataflow.worker.repackaged.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4899)
> 
> com.google.cloud.dataflow.worker.UserParDoFnFactory.create(UserParDoFnFactory.java:90)
> 
> com.google.cloud.dataflow.worker.DefaultParDoFnFactory.create(DefaultParDoFnFactory.java:74)
> 
> com.google.cloud.dataflow.worker.IntrinsicMapTaskExecutorFactory.createParDoOperation(IntrinsicMapTaskExecutorFactory.java:262)
> 
> com.google.cloud.dataflow.worker.IntrinsicMapTaskExecutorFactory.access$000(IntrinsicMapTaskExecutorFactory.java:84)
> 

[jira] [Work logged] (BEAM-4599) Test arithmetic operators at the DSL level

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4599?focusedWorklogId=121731=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121731
 ]

ASF GitHub Bot logged work on BEAM-4599:


Author: ASF GitHub Bot
Created on: 11/Jul/18 01:12
Start Date: 11/Jul/18 01:12
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #5921: [BEAM-4599] [SQL] 
add testArithmeticOperator
URL: https://github.com/apache/beam/pull/5921#issuecomment-404013318
 
 
   run java precommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121731)
Time Spent: 1h 40m  (was: 1.5h)

> Test arithmetic operators at the DSL level
> --
>
> Key: BEAM-4599
> URL: https://issues.apache.org/jira/browse/BEAM-4599
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4649) Failures in beam_PostCommit_Py_ValCont due to exception in read_log_control_messages

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4649?focusedWorklogId=121730=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121730
 ]

ASF GitHub Bot logged work on BEAM-4649:


Author: ASF GitHub Bot
Created on: 11/Jul/18 01:08
Start Date: 11/Jul/18 01:08
Worklog Time Spent: 10m 
  Work Description: aaltay opened a new pull request #5923: [BEAM-4649] 
Wait on connection channel to be ready
URL: https://github.com/apache/beam/pull/5923
 
 
   A fix similar to https://github.com/apache/beam/pull/5780/files
   
   R: @alanmyrvold @robertwb 
   
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121730)
Time Spent: 1h  (was: 50m)

> Failures in beam_PostCommit_Py_ValCont due to exception in 
> read_log_control_messages
> 
>
> Key: BEAM-4649
> URL: https://issues.apache.org/jira/browse/BEAM-4649
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-harness
>Affects Versions: 2.6.0
>Reporter: Alan Myrvold
>Assignee: Alan Myrvold
>Priority: Major
> Fix For: 2.6.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Example
> https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-26_11_04_04-14383422363721841333?project=apache-beam-testing
>  
> All of the failures have the same exception in the docker logs:
> I  2018/06/26 18:05:12 Executing: python -m 
> apache_beam.runners.worker.sdk_worker_main
> I  Exception in thread read_log_control_messages:
> I  Traceback (most recent call last):
> I    File "/usr/local/lib/python2.7/threading.py", line 801, in 
> __bootstrap_inner
> I      self.run()
> I    File 

Build failed in Jenkins: beam_PostCommit_Python_Verify #5536

[Apache Jenkins Server]

See 


Changes:

[robertwb] object() takes no parameters

[amyrvold] [BEAM-3457] Upgrade version of gogradle and add examples and 
containers

[lcwik] [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not

--
[...truncated 1.06 MB...]
test_repr (apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_type_check_invalid_key_type 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_type_check_invalid_value_type 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_type_check_valid_composite_type 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_type_check_valid_simple_type 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_type_checks_not_dict 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_value_type_must_be_valid_composite_param 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_compatibility (apache_beam.typehints.typehints_test.GeneratorHintTestCase) 
... ok
test_generator_argument_hint_invalid_yield_type 
(apache_beam.typehints.typehints_test.GeneratorHintTestCase) ... ok
test_generator_return_hint_invalid_yield_type 
(apache_beam.typehints.typehints_test.GeneratorHintTestCase) ... ok
test_repr (apache_beam.typehints.typehints_test.GeneratorHintTestCase) ... ok
test_compatibility (apache_beam.typehints.typehints_test.IterableHintTestCase) 
... ok
test_getitem_invalid_composite_type_param 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_repr (apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_tuple_compatibility 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_type_check_must_be_iterable 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_type_check_violation_invalid_composite_type 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_type_check_violation_invalid_simple_type 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_type_check_violation_valid_composite_type 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_type_check_violation_valid_simple_type 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_enforce_kv_type_constraint 
(apache_beam.typehints.typehints_test.KVHintTestCase) ... ok
test_getitem_param_must_be_tuple 
(apache_beam.typehints.typehints_test.KVHintTestCase) ... ok
test_getitem_param_must_have_length_2 
(apache_beam.typehints.typehints_test.KVHintTestCase) ... ok
test_getitem_proxy_to_tuple 
(apache_beam.typehints.typehints_test.KVHintTestCase) ... ok
test_enforce_list_type_constraint_invalid_composite_type 
(apache_beam.typehints.typehints_test.ListHintTestCase) ... ok
test_enforce_list_type_constraint_invalid_simple_type 
(apache_beam.typehints.typehints_test.ListHintTestCase) ... ok
test_enforce_list_type_constraint_valid_composite_type 
(apache_beam.typehints.typehints_test.ListHintTestCase) ... ok
test_enforce_list_type_constraint_valid_simple_type 
(apache_beam.typehints.typehints_test.ListHintTestCase) ... ok
test_getitem_invalid_composite_type_param 
(apache_beam.typehints.typehints_test.ListHintTestCase) ... ok
test_list_constraint_compatibility 
(apache_beam.typehints.typehints_test.ListHintTestCase) ... ok
test_list_repr (apache_beam.typehints.typehints_test.ListHintTestCase) ... ok
test_getitem_proxy_to_union 
(apache_beam.typehints.typehints_test.OptionalHintTestCase) ... ok
test_getitem_sequence_not_allowed 
(apache_beam.typehints.typehints_test.OptionalHintTestCase) ... ok
test_any_return_type_hint 
(apache_beam.typehints.typehints_test.ReturnsDecoratorTestCase) ... ok
test_must_be_primitive_type_or_type_constraint 
(apache_beam.typehints.typehints_test.ReturnsDecoratorTestCase) ... ok
test_must_be_single_return_type 
(apache_beam.typehints.typehints_test.ReturnsDecoratorTestCase) ... ok
test_no_kwargs_accepted 
(apache_beam.typehints.typehints_test.ReturnsDecoratorTestCase) ... ok
test_type_check_composite_type 
(apache_beam.typehints.typehints_test.ReturnsDecoratorTestCase) ... ok
test_type_check_simple_type 
(apache_beam.typehints.typehints_test.ReturnsDecoratorTestCase) ... ok
test_type_check_violation 
(apache_beam.typehints.typehints_test.ReturnsDecoratorTestCase) ... ok
test_compatibility (apache_beam.typehints.typehints_test.SetHintTestCase) ... ok
test_getitem_invalid_composite_type_param 
(apache_beam.typehints.typehints_test.SetHintTestCase) ... ok
test_repr (apache_beam.typehints.typehints_test.SetHintTestCase) ... ok
test_type_check_invalid_elem_type 
(apache_beam.typehints.typehints_test.SetHintTestCase) ... ok
test_type_check_must_be_set 
(apache_beam.typehints.typehints_test.SetHintTestCase) ... ok
test_type_check_valid_elem_composite_type 
(apache_beam.typehints.typehints_test.SetHintTestCase) ... ok

[jira] [Work logged] (BEAM-4599) Test arithmetic operators at the DSL level

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4599?focusedWorklogId=121722=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121722
 ]

ASF GitHub Bot logged work on BEAM-4599:


Author: ASF GitHub Bot
Created on: 11/Jul/18 00:34
Start Date: 11/Jul/18 00:34
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #5921: [BEAM-4599] [SQL] 
add testArithmeticOperator
URL: https://github.com/apache/beam/pull/5921#issuecomment-404007828
 
 
   Updated this PR to merge old arithmetic operator integration test with new 
style test.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121722)
Time Spent: 1.5h  (was: 1h 20m)

> Test arithmetic operators at the DSL level
> --
>
> Key: BEAM-4599
> URL: https://issues.apache.org/jira/browse/BEAM-4599
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PreCommit_Java_Cron #90

[Apache Jenkins Server]

See 


Changes:

[robertwb] object() takes no parameters

[amyrvold] [BEAM-3457] Upgrade version of gogradle and add examples and 
containers

[lcwik] [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not

--
[...truncated 16.73 MB...]
Jul 11, 2018 12:26:33 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-11T00:26:25.173Z: Fusing adjacent ParDo, Read, Write, and 
Flatten operations
Jul 11, 2018 12:26:33 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-11T00:26:25.213Z: Unzipping flatten s13 for input 
s12.org.apache.beam.sdk.values.PCollection.:349#1d275f544daf228c
Jul 11, 2018 12:26:33 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-11T00:26:25.259Z: Fusing unzipped copy of 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Add void 
key/AddKeys/Map, through flatten 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections,
 into producer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Jul 11, 2018 12:26:33 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-11T00:26:25.300Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable
 into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow
Jul 11, 2018 12:26:33 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-11T00:26:25.348Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
 into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair
 with random key
Jul 11, 2018 12:26:33 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-11T00:26:25.395Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
 into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Jul 11, 2018 12:26:33 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-11T00:26:25.439Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
 into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Jul 11, 2018 12:26:33 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-11T00:26:25.485Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
 into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Jul 11, 2018 12:26:33 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-11T00:26:25.532Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Drop 
key/Values/Map into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable
Jul 11, 2018 12:26:33 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-11T00:26:25.578Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Gather 
bundles into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Drop 
key/Values/Map
Jul 11, 2018 12:26:33 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-11T00:26:25.618Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair
 with random key into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Gather 
bundles
Jul 11, 2018 12:26:33 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-11T00:26:25.660Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
 into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Jul 11, 2018 12:26:33 AM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-11T00:26:25.705Z: Fusing consumer 

[jira] [Work logged] (BEAM-4599) Test arithmetic operators at the DSL level

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4599?focusedWorklogId=121720=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121720
 ]

ASF GitHub Bot logged work on BEAM-4599:


Author: ASF GitHub Bot
Created on: 11/Jul/18 00:18
Start Date: 11/Jul/18 00:18
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #5921: [BEAM-4599] [SQL] 
add testArithmeticOperator
URL: https://github.com/apache/beam/pull/5921#issuecomment-404005444
 
 
   @vectorijk 
   
   Yep, I forgot about mod. Thanks!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121720)
Time Spent: 1h 20m  (was: 1h 10m)

> Test arithmetic operators at the DSL level
> --
>
> Key: BEAM-4599
> URL: https://issues.apache.org/jira/browse/BEAM-4599
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4599) Test arithmetic operators at the DSL level

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4599?focusedWorklogId=121719=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121719
 ]

ASF GitHub Bot logged work on BEAM-4599:


Author: ASF GitHub Bot
Created on: 11/Jul/18 00:14
Start Date: 11/Jul/18 00:14
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #5921: [BEAM-4599] [SQL] 
add testArithmeticOperator
URL: https://github.com/apache/beam/pull/5921#issuecomment-404004727
 
 
   Another thought is we can start to deprecate old style tests by merging 
those with this operator test.
   
   I can work on that in a separate PR:  
https://issues.apache.org/jira/browse/BEAM-4573


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121719)
Time Spent: 1h 10m  (was: 1h)

> Test arithmetic operators at the DSL level
> --
>
> Key: BEAM-4599
> URL: https://issues.apache.org/jira/browse/BEAM-4599
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-4757) Deprecate old operator integration tests

[Rui Wang (JIRA)]

Rui Wang created BEAM-4757:
--

 Summary: Deprecate old operator integration tests
 Key: BEAM-4757
 URL: https://issues.apache.org/jira/browse/BEAM-4757
 Project: Beam
  Issue Type: Sub-task
  Components: dsl-sql
Reporter: Rui Wang
Assignee: Rui Wang






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4599) Test arithmetic operators at the DSL level

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4599?focusedWorklogId=121716=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121716
 ]

ASF GitHub Bot logged work on BEAM-4599:


Author: ASF GitHub Bot
Created on: 11/Jul/18 00:11
Start Date: 11/Jul/18 00:11
Worklog Time Spent: 10m 
  Work Description: vectorijk commented on issue #5921: [BEAM-4599] [SQL] 
add testArithmeticOperator
URL: https://github.com/apache/beam/pull/5921#issuecomment-404004350
 
 
   Missing test for `Mod`? Otherwise, LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121716)
Time Spent: 1h  (was: 50m)

> Test arithmetic operators at the DSL level
> --
>
> Key: BEAM-4599
> URL: https://issues.apache.org/jira/browse/BEAM-4599
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4599) Test arithmetic operators at the DSL level

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4599?focusedWorklogId=121711=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121711
 ]

ASF GitHub Bot logged work on BEAM-4599:


Author: ASF GitHub Bot
Created on: 10/Jul/18 23:58
Start Date: 10/Jul/18 23:58
Worklog Time Spent: 10m 
  Work Description: amaliujia edited a comment on issue #5921: [BEAM-4599] 
[SQL] add testArithmeticOperator
URL: https://github.com/apache/beam/pull/5921#issuecomment-404002118
 
 
   There are redundant tests in the old style. 
   
   I'd like to keep the old tests now. As we become confident in the new tests' 
converge at operator level, we can remove the old tests in a batch (unless 
there is some testing performance concern). 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121711)
Time Spent: 50m  (was: 40m)

> Test arithmetic operators at the DSL level
> --
>
> Key: BEAM-4599
> URL: https://issues.apache.org/jira/browse/BEAM-4599
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4599) Test arithmetic operators at the DSL level

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4599?focusedWorklogId=121710=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121710
 ]

ASF GitHub Bot logged work on BEAM-4599:


Author: ASF GitHub Bot
Created on: 10/Jul/18 23:57
Start Date: 10/Jul/18 23:57
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #5921: [BEAM-4599] [SQL] 
add testArithmeticOperator
URL: https://github.com/apache/beam/pull/5921#issuecomment-404002118
 
 
   There are redundant tests in the old style. 
   
   I'd like to keep the old tests now. As we become confident in the new tests' 
converge at operator level, we can remove the old tests in a batch.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121710)
Time Spent: 40m  (was: 0.5h)

> Test arithmetic operators at the DSL level
> --
>
> Key: BEAM-4599
> URL: https://issues.apache.org/jira/browse/BEAM-4599
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-4694) Test Auto JIRA

[Beam JIRA Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539386#comment-16539386
 ] 

Beam JIRA Bot commented on BEAM-4694:
-

LALALA

> Test Auto JIRA
> --
>
> Key: BEAM-4694
> URL: https://issues.apache.org/jira/browse/BEAM-4694
> Project: Beam
>  Issue Type: Bug
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: Not applicable
>
>
> Creating a JIRA by using python API. No actions needed. 
>  cc: [~chamikara]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (BEAM-4756) New Jira Bot Test

[Beam JIRA Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Beam JIRA Bot closed BEAM-4756.
---
   Resolution: Won't Fix
Fix Version/s: Not applicable

> New Jira Bot Test
> -
>
> Key: BEAM-4756
> URL: https://issues.apache.org/jira/browse/BEAM-4756
> Project: Beam
>  Issue Type: Bug
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Priority: Major
> Fix For: Not applicable
>
>
> No actions required. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-4756) New Jira Bot Test

[Beam JIRA Bot (JIRA)]

Beam JIRA Bot created BEAM-4756:
---

 Summary: New Jira Bot Test
 Key: BEAM-4756
 URL: https://issues.apache.org/jira/browse/BEAM-4756
 Project: Beam
  Issue Type: Bug
  Components: dependencies
Reporter: Beam JIRA Bot


No actions required. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4599) Test arithmetic operators at the DSL level

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4599?focusedWorklogId=121705=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121705
 ]

ASF GitHub Bot logged work on BEAM-4599:


Author: ASF GitHub Bot
Created on: 10/Jul/18 23:47
Start Date: 10/Jul/18 23:47
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5921: [BEAM-4599] [SQL] 
add testArithmeticOperator
URL: https://github.com/apache/beam/pull/5921#issuecomment-404000474
 
 
   LGTM.
   
   I remember we had a bunch of tests in the old style that checked the same 
thing. Are there any that are now redundant and can be deleted?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121705)
Time Spent: 0.5h  (was: 20m)

> Test arithmetic operators at the DSL level
> --
>
> Key: BEAM-4599
> URL: https://issues.apache.org/jira/browse/BEAM-4599
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4599) Test arithmetic operators at the DSL level

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4599?focusedWorklogId=121704=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121704
 ]

ASF GitHub Bot logged work on BEAM-4599:


Author: ASF GitHub Bot
Created on: 10/Jul/18 23:44
Start Date: 10/Jul/18 23:44
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #5921: [BEAM-4599] [SQL] 
add testArithmeticOperator
URL: https://github.com/apache/beam/pull/5921#issuecomment-40373
 
 
   to: @akedin @apilloud @vectorijk 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121704)
Time Spent: 20m  (was: 10m)

> Test arithmetic operators at the DSL level
> --
>
> Key: BEAM-4599
> URL: https://issues.apache.org/jira/browse/BEAM-4599
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4599) Test arithmetic operators at the DSL level

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4599?focusedWorklogId=121703=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121703
 ]

ASF GitHub Bot logged work on BEAM-4599:


Author: ASF GitHub Bot
Created on: 10/Jul/18 23:43
Start Date: 10/Jul/18 23:43
Worklog Time Spent: 10m 
  Work Description: amaliujia opened a new pull request #5921: [BEAM-4599] 
[SQL] add testArithmeticOperator
URL: https://github.com/apache/beam/pull/5921
 
 
   Test arithmetic operator in SQL.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121703)
Time Spent: 10m
Remaining Estimate: 0h

> Test arithmetic operators at the DSL level
> --
>
> Key: BEAM-4599
> URL: https://issues.apache.org/jira/browse/BEAM-4599
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4751) Finish io futurize stage 2: fix the missing pylint3 check in tox.ini

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4751?focusedWorklogId=121701=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121701
 ]

ASF GitHub Bot logged work on BEAM-4751:


Author: ASF GitHub Bot
Created on: 10/Jul/18 23:42
Start Date: 10/Jul/18 23:42
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on a change in pull request 
#5916: [BEAM-4751] fix missing pylint3 check for io subpackage
URL: https://github.com/apache/beam/pull/5916#discussion_r201523551
 
 

 ##
 File path: sdks/python/apache_beam/io/concat_source_test.py
 ##
 @@ -78,6 +78,9 @@ def __eq__(self, other):
 return (type(self) == type(other)
 and self._start == other._start and self._end == other._end)
 
+  def __hash__(self):
+return hash((type(self), self._start, self._end))
 
 Review comment:
   Can you add `self._split_freq` to `__eq__` and `__hash__`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121701)

> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini
> 
>
> Key: BEAM-4751
> URL: https://issues.apache.org/jira/browse/BEAM-4751
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Matthias Feys
>Assignee: Matthias Feys
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4751) Finish io futurize stage 2: fix the missing pylint3 check in tox.ini

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4751?focusedWorklogId=121702=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121702
 ]

ASF GitHub Bot logged work on BEAM-4751:


Author: ASF GitHub Bot
Created on: 10/Jul/18 23:42
Start Date: 10/Jul/18 23:42
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on a change in pull request 
#5916: [BEAM-4751] fix missing pylint3 check for io subpackage
URL: https://github.com/apache/beam/pull/5916#discussion_r201528700
 
 

 ##
 File path: sdks/python/apache_beam/io/vcfio.py
 ##
 @@ -221,6 +219,9 @@ def __eq__(self, other):
 return ((self.name, self.genotype, self.phaseset, self.info) ==
 (other.name, other.genotype, other.phaseset, other.info))
 
+  def __hash__(self):
+return hash((self.name, self.genotype, self.phaseset, self.info))
 
 Review comment:
   `self.info` is a dict.  Is this a problem?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121702)
Time Spent: 50m  (was: 40m)

> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini
> 
>
> Key: BEAM-4751
> URL: https://issues.apache.org/jira/browse/BEAM-4751
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Matthias Feys
>Assignee: Matthias Feys
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4751) Finish io futurize stage 2: fix the missing pylint3 check in tox.ini

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4751?focusedWorklogId=121700=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121700
 ]

ASF GitHub Bot logged work on BEAM-4751:


Author: ASF GitHub Bot
Created on: 10/Jul/18 23:42
Start Date: 10/Jul/18 23:42
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on a change in pull request 
#5916: [BEAM-4751] fix missing pylint3 check for io subpackage
URL: https://github.com/apache/beam/pull/5916#discussion_r201524390
 
 

 ##
 File path: sdks/python/apache_beam/io/filebasedsink.py
 ##
 @@ -377,6 +373,9 @@ def __eq__(self, other):
 # pylint: disable=unidiomatic-typecheck
 return type(self) == type(other) and self.__dict__ == other.__dict__
 
+  def __hash__(self):
+return hash((type(self), self.__dict__))
 
 Review comment:
   Does `self.__dict__` work here for hashing?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121700)
Time Spent: 40m  (was: 0.5h)

> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini
> 
>
> Key: BEAM-4751
> URL: https://issues.apache.org/jira/browse/BEAM-4751
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Matthias Feys
>Assignee: Matthias Feys
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4751) Finish io futurize stage 2: fix the missing pylint3 check in tox.ini

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4751?focusedWorklogId=121699=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121699
 ]

ASF GitHub Bot logged work on BEAM-4751:


Author: ASF GitHub Bot
Created on: 10/Jul/18 23:42
Start Date: 10/Jul/18 23:42
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on a change in pull request 
#5916: [BEAM-4751] fix missing pylint3 check for io subpackage
URL: https://github.com/apache/beam/pull/5916#discussion_r201528399
 
 

 ##
 File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
 ##
 @@ -49,6 +49,9 @@ def __init__(self, path, mode='', type='FILE'):
   def __eq__(self, other):
 return self.stat == other.stat and self.getvalue() == self.getvalue()
 
+  def __hash__(self):
+return hash((self.stat, self.getvalue()))
 
 Review comment:
   Can we use the dictionary `self.stat`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121699)
Time Spent: 40m  (was: 0.5h)

> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini
> 
>
> Key: BEAM-4751
> URL: https://issues.apache.org/jira/browse/BEAM-4751
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Matthias Feys
>Assignee: Matthias Feys
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-4599) Test arithmetic operators at the DSL level

[Rui Wang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rui Wang reassigned BEAM-4599:
--

Assignee: Rui Wang  (was: Xu Mingmin)

> Test arithmetic operators at the DSL level
> --
>
> Key: BEAM-4599
> URL: https://issues.apache.org/jira/browse/BEAM-4599
> Project: Beam
>  Issue Type: Sub-task
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Rui Wang
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4700) JDBC driver cannot support TIMESTAMP data type

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4700?focusedWorklogId=121695=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121695
 ]

ASF GitHub Bot logged work on BEAM-4700:


Author: ASF GitHub Bot
Created on: 10/Jul/18 23:33
Start Date: 10/Jul/18 23:33
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5919: BEAM-4700: [SQL] 
Default timezone is UTC
URL: https://github.com/apache/beam/pull/5919#issuecomment-403998185
 
 
   @lukecwik You like timezones. We made them not matter in Beam SQL by setting 
the default timezone to UTC instead of local time.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121695)
Time Spent: 2h 20m  (was: 2h 10m)

> JDBC driver cannot support TIMESTAMP data type
> --
>
> Key: BEAM-4700
> URL: https://issues.apache.org/jira/browse/BEAM-4700
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Avatica allows column representation to be customized, so a timestamp can be 
> stored as a variety of types. Joda ReadableInstant is none of these types: 
> https://github.com/apache/calcite-avatica/blob/acb675de97b9b0743c09368820a770e2ceda05f8/core/src/main/java/org/apache/calcite/avatica/util/AbstractCursor.java#L162
> By default, it seems to be configured to store {{TIMESTAMP}} columns as 
> {{long}} values. If you run the SQL shell and select a {{TIMESTAMP}} column, 
> you get:
> {code}
> ava.lang.ClassCastException: org.joda.time.Instant cannot be cast to 
> java.lang.Number
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$NumberAccessor.getNumber(AbstractCursor.java:726)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$TimestampFromNumberAccessor.getString(AbstractCursor.java:1026)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:225)
> at sqlline.Rows$Row.(Rows.java:183)
> {code}
> So, essentially, Beam SQL Shell does not support timestamps.
> We may be able to:
>  - override how the accessor for our existing storage is created
>  - configure what the column representation is (this doesn't really help, 
> since none of the choices are ours)
>  - convert timestamps to longs in BeamEnumerableConverter; not sure how many 
> conversions will be required here



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam-site] 01/03: Fix typos in programming guide

[mergebot-role]

This is an automated email from the ASF dual-hosted git repository.

mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git

commit 9408d1f7b5160c2f6f7fc9bfb6b0a76c75d82089
Author: Yueyang Qiu 
AuthorDate: Tue Jul 10 12:59:03 2018 -0700

Fix typos in programming guide
---
 src/documentation/programming-guide.md | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/documentation/programming-guide.md 
b/src/documentation/programming-guide.md
index 091c130..4b55fcb 100644
--- a/src/documentation/programming-guide.md
+++ b/src/documentation/programming-guide.md
@@ -79,14 +79,14 @@ include:
 
 A typical Beam driver program works as follows:
 
-* Create a `Pipeline` object and set the pipeline execution options, including
+* **Create** a `Pipeline` object and set the pipeline execution options, 
including
   the Pipeline Runner.
 * Create an initial `PCollection` for pipeline data, either using the IOs
   to read data from an external storage system, or using a `Create` transform 
to
   build a `PCollection` from in-memory data.
-* Apply **PTransforms** to each `PCollection`. Transforms can change, filter,
+* **Apply** `PTransforms` to each `PCollection`. Transforms can change, filter,
   group, analyze, or otherwise process the elements in a `PCollection`. A
-  transform creates a new output `PCollection` *without consuming the input
+  transform creates a new output `PCollection` *without modifying the input
   collection*. A typical pipeline applies subsequent transforms to the each new
   output `PCollection` in turn until processing is complete. However, note that
   a pipeline does not have to be a single straight line of transforms applied
@@ -1072,7 +1072,7 @@ If your input `PCollection` uses the default global 
windowing, the default
 behavior is to return a `PCollection` containing one item. That item's value
 comes from the accumulator in the combine function that you specified when
 applying `Combine`. For example, the Beam provided sum combine function returns
-a zero value (the sum of an empty input), while the max combine function 
returns
+a zero value (the sum of an empty input), while the min combine function 
returns
 a maximal or infinite value.
 
 To have `Combine` instead return an empty `PCollection` if the input is empty,

[beam-site] 02/03: Fix typos in mobile gaming example

[mergebot-role]

This is an automated email from the ASF dual-hosted git repository.

mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git

commit 8c1eacbed089f8e6e13db978263a0d57094e3424
Author: Yueyang Qiu 
AuthorDate: Tue Jul 10 13:01:28 2018 -0700

Fix typos in mobile gaming example
---
 src/get-started/mobile-gaming-example.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/get-started/mobile-gaming-example.md 
b/src/get-started/mobile-gaming-example.md
index f6c6c46..4a289b4 100644
--- a/src/get-started/mobile-gaming-example.md
+++ b/src/get-started/mobile-gaming-example.md
@@ -344,7 +344,7 @@ Below, we'll look at these features in more detail.
 
 Let's suppose scoring in our game depends on the speed at which a user can 
"click" on their phone. `GameStats`'s abuse detection analyzes each user's 
score data to detect if a user has an abnormally high "click rate" and thus an 
abnormally high score. This might indicate that the game is being played by a 
bot that operates significantly faster than a human could play.
 
-To determine whether or not a score is "abnormally" high, `GameStats` 
calculates the average of every score in that fixed-time window, and then 
checks each score individual score against the average score multiplied by an 
arbitrary weight factor (in our case, 2.5). Thus, any score more than 2.5 times 
the average is deemed to be the product of spam. The `GameStats` pipeline 
tracks a list of "spam" users and filters those users out of the team score 
calculations for the team leader board.
+To determine whether or not a score is "abnormally" high, `GameStats` 
calculates the average of every score in that fixed-time window, and then 
checks each individual score against the average score multiplied by an 
arbitrary weight factor (in our case, 2.5). Thus, any score more than 2.5 times 
the average is deemed to be the product of spam. The `GameStats` pipeline 
tracks a list of "spam" users and filters those users out of the team score 
calculations for the team leader board.
 
 Since the average depends on the pipeline data, we need to calculate it, and 
then use that calculated data in a subsequent `ParDo` transform that filters 
scores that exceed the weighted value. To do this, we can pass the calculated 
average to as a [side input]({{ site.baseurl 
}}/documentation/programming-guide/#side-inputs) to the filtering `ParDo`.
 

[beam-site] 03/03: This closes #491

[mergebot-role]

This is an automated email from the ASF dual-hosted git repository.

mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git

commit 04664219270e3ee142798d77d5f3850882b39a50
Merge: f254995 8c1eacb
Author: Mergebot 
AuthorDate: Tue Jul 10 23:28:45 2018 +

This closes #491

 src/documentation/programming-guide.md   | 8 
 src/get-started/mobile-gaming-example.md | 2 +-
 2 files changed, 5 insertions(+), 5 deletions(-)

[beam-site] branch mergebot updated (970da98 -> 0466421)

[mergebot-role]

This is an automated email from the ASF dual-hosted git repository.

mergebot-role pushed a change to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git.


from 970da98  This closes #477
 add f254995  Prepare repository for deployment.
 new 9408d1f  Fix typos in programming guide
 new 8c1eacb  Fix typos in mobile gaming example
 new 0466421  This closes #491

The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 content/community/contact-us/index.html|  1 +
 content/community/logos/index.html |  1 +
 .../community/presentation-materials/index.html|  1 +
 .../{contact-us => youtube-channel}/index.html | 87 +-
 src/documentation/programming-guide.md |  8 +-
 src/get-started/mobile-gaming-example.md   |  2 +-
 6 files changed, 44 insertions(+), 56 deletions(-)
 copy content/community/{contact-us => youtube-channel}/index.html (82%)

[jira] [Work logged] (BEAM-4700) JDBC driver cannot support TIMESTAMP data type

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4700?focusedWorklogId=121694=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121694
 ]

ASF GitHub Bot logged work on BEAM-4700:


Author: ASF GitHub Bot
Created on: 10/Jul/18 23:26
Start Date: 10/Jul/18 23:26
Worklog Time Spent: 10m 
  Work Description: vectorijk commented on issue #5919: BEAM-4700: [SQL] 
Default timezone is UTC
URL: https://github.com/apache/beam/pull/5919#issuecomment-403996976
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121694)
Time Spent: 2h 10m  (was: 2h)

> JDBC driver cannot support TIMESTAMP data type
> --
>
> Key: BEAM-4700
> URL: https://issues.apache.org/jira/browse/BEAM-4700
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> Avatica allows column representation to be customized, so a timestamp can be 
> stored as a variety of types. Joda ReadableInstant is none of these types: 
> https://github.com/apache/calcite-avatica/blob/acb675de97b9b0743c09368820a770e2ceda05f8/core/src/main/java/org/apache/calcite/avatica/util/AbstractCursor.java#L162
> By default, it seems to be configured to store {{TIMESTAMP}} columns as 
> {{long}} values. If you run the SQL shell and select a {{TIMESTAMP}} column, 
> you get:
> {code}
> ava.lang.ClassCastException: org.joda.time.Instant cannot be cast to 
> java.lang.Number
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$NumberAccessor.getNumber(AbstractCursor.java:726)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$TimestampFromNumberAccessor.getString(AbstractCursor.java:1026)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:225)
> at sqlline.Rows$Row.(Rows.java:183)
> {code}
> So, essentially, Beam SQL Shell does not support timestamps.
> We may be able to:
>  - override how the accessor for our existing storage is created
>  - configure what the column representation is (this doesn't really help, 
> since none of the choices are ours)
>  - convert timestamps to longs in BeamEnumerableConverter; not sure how many 
> conversions will be required here



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-4755) Cython cast error for Interactive Runner with FnAPI mode direct runner

[Qinye Li (JIRA)]

Qinye Li created BEAM-4755:
--

 Summary: Cython cast error for Interactive Runner with FnAPI mode 
direct runner
 Key: BEAM-4755
 URL: https://issues.apache.org/jira/browse/BEAM-4755
 Project: Beam
  Issue Type: Bug
  Components: sdk-py-harness
Affects Versions: Not applicable
 Environment: Linux
Reporter: Qinye Li
Assignee: Robert Bradshaw
 Fix For: Not applicable


When Interactive Runner uses FnAPI mode DirectRunner as the underlying runner,

`cython.cast(Receiver, self.receivers[output_index])` in `Operation.output` 
throws an error that complains

 

File 
"/usr/local/home/qinyeli/beam/sdks/python/apache_beam/runners/worker/operations.py",
 line 178, in output
 print(cython.cast(Receiver, self.receivers[output_index]))
 File 
"/usr/local/home/qinyeli/.local/lib/python2.7/site-packages/Cython/Shadow.py", 
line 164, in cast
 return type(*args)
TypeError: object() takes no parameters

 

To replicate the error:

Run a batch pipeline with interactive runner and explicitly specify to use 
direct_runner.SwitchingDirectRunner() as the underlying runner.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-4754) Type conversion testing

[Rui Wang (JIRA)]

Rui Wang created BEAM-4754:
--

 Summary: Type conversion testing 
 Key: BEAM-4754
 URL: https://issues.apache.org/jira/browse/BEAM-4754
 Project: Beam
  Issue Type: Improvement
  Components: dsl-sql
Reporter: Rui Wang
Assignee: Rui Wang


We should have a type system testing to test all types of Beam SQL from 
different types (Avro, JSON, etc.)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-4717) Test JSON to Beam types conversion

[Rui Wang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rui Wang updated BEAM-4717:
---
Summary: Test JSON to Beam types conversion  (was: Improve SQL's PubSub 
integration test)

> Test JSON to Beam types conversion
> --
>
> Key: BEAM-4717
> URL: https://issues.apache.org/jira/browse/BEAM-4717
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>
> Should improve PubSub test coverage by testing more data types.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-3950) Dataflow Runner should supply a wheel version of Python SDK if it is available

[Valentyn Tymofieiev (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539330#comment-16539330
 ] 

Valentyn Tymofieiev commented on BEAM-3950:
---

Yes, this is should be fixed. SDK stages both SDK sources and SDK wheels (if 
available). Dataflow runner prefers SDK wheels if they were staged.

> Dataflow Runner should supply a wheel version of Python SDK if it is available
> --
>
> Key: BEAM-3950
> URL: https://issues.apache.org/jira/browse/BEAM-3950
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-dataflow, sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Major
> Fix For: 2.5.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (BEAM-3950) Dataflow Runner should supply a wheel version of Python SDK if it is available

[Valentyn Tymofieiev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Valentyn Tymofieiev closed BEAM-3950.
-
   Resolution: Fixed
Fix Version/s: 2.5.0

> Dataflow Runner should supply a wheel version of Python SDK if it is available
> --
>
> Key: BEAM-3950
> URL: https://issues.apache.org/jira/browse/BEAM-3950
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-dataflow, sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Major
> Fix For: 2.5.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4700) JDBC driver cannot support TIMESTAMP data type

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4700?focusedWorklogId=121673=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121673
 ]

ASF GitHub Bot logged work on BEAM-4700:


Author: ASF GitHub Bot
Created on: 10/Jul/18 22:52
Start Date: 10/Jul/18 22:52
Worklog Time Spent: 10m 
  Work Description: akedin commented on issue #5919: BEAM-4700: [SQL] 
Default timezone is UTC
URL: https://github.com/apache/beam/pull/5919#issuecomment-403990888
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121673)
Time Spent: 2h  (was: 1h 50m)

> JDBC driver cannot support TIMESTAMP data type
> --
>
> Key: BEAM-4700
> URL: https://issues.apache.org/jira/browse/BEAM-4700
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> Avatica allows column representation to be customized, so a timestamp can be 
> stored as a variety of types. Joda ReadableInstant is none of these types: 
> https://github.com/apache/calcite-avatica/blob/acb675de97b9b0743c09368820a770e2ceda05f8/core/src/main/java/org/apache/calcite/avatica/util/AbstractCursor.java#L162
> By default, it seems to be configured to store {{TIMESTAMP}} columns as 
> {{long}} values. If you run the SQL shell and select a {{TIMESTAMP}} column, 
> you get:
> {code}
> ava.lang.ClassCastException: org.joda.time.Instant cannot be cast to 
> java.lang.Number
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$NumberAccessor.getNumber(AbstractCursor.java:726)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$TimestampFromNumberAccessor.getString(AbstractCursor.java:1026)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:225)
> at sqlline.Rows$Row.(Rows.java:183)
> {code}
> So, essentially, Beam SQL Shell does not support timestamps.
> We may be able to:
>  - override how the accessor for our existing storage is created
>  - configure what the column representation is (this doesn't really help, 
> since none of the choices are ours)
>  - convert timestamps to longs in BeamEnumerableConverter; not sure how many 
> conversions will be required here



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-4647) Support String type in Create Table statement

[Rui Wang (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539314#comment-16539314
 ] 

Rui Wang commented on BEAM-4647:


We might not want to support STRING type for the following reasons:
 # Beam SQL only controls CREATE TABLE. It could be inconsistent if only 
supporting STRING in CREATE TABLE.
 # Calcite has VARCHAR type already. STRING type is less needed. 

> Support String type in Create Table statement 
> --
>
> Key: BEAM-4647
> URL: https://issues.apache.org/jira/browse/BEAM-4647
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Fix For: Not applicable
>
>
> Investigate if we can support STRING type in CREATE TABLE statement.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (BEAM-4647) Support String type in Create Table statement

[Rui Wang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rui Wang closed BEAM-4647.
--
   Resolution: Won't Do
Fix Version/s: Not applicable

> Support String type in Create Table statement 
> --
>
> Key: BEAM-4647
> URL: https://issues.apache.org/jira/browse/BEAM-4647
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Fix For: Not applicable
>
>
> Investigate if we can support STRING type in CREATE TABLE statement.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4700) JDBC driver cannot support TIMESTAMP data type

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4700?focusedWorklogId=121666=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121666
 ]

ASF GitHub Bot logged work on BEAM-4700:


Author: ASF GitHub Bot
Created on: 10/Jul/18 22:37
Start Date: 10/Jul/18 22:37
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5919: BEAM-4700: [SQL] 
Default timezone is UTC
URL: https://github.com/apache/beam/pull/5919#issuecomment-403988096
 
 
   run java postcommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121666)
Time Spent: 1h 50m  (was: 1h 40m)

> JDBC driver cannot support TIMESTAMP data type
> --
>
> Key: BEAM-4700
> URL: https://issues.apache.org/jira/browse/BEAM-4700
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Avatica allows column representation to be customized, so a timestamp can be 
> stored as a variety of types. Joda ReadableInstant is none of these types: 
> https://github.com/apache/calcite-avatica/blob/acb675de97b9b0743c09368820a770e2ceda05f8/core/src/main/java/org/apache/calcite/avatica/util/AbstractCursor.java#L162
> By default, it seems to be configured to store {{TIMESTAMP}} columns as 
> {{long}} values. If you run the SQL shell and select a {{TIMESTAMP}} column, 
> you get:
> {code}
> ava.lang.ClassCastException: org.joda.time.Instant cannot be cast to 
> java.lang.Number
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$NumberAccessor.getNumber(AbstractCursor.java:726)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$TimestampFromNumberAccessor.getString(AbstractCursor.java:1026)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:225)
> at sqlline.Rows$Row.(Rows.java:183)
> {code}
> So, essentially, Beam SQL Shell does not support timestamps.
> We may be able to:
>  - override how the accessor for our existing storage is created
>  - configure what the column representation is (this doesn't really help, 
> since none of the choices are ours)
>  - convert timestamps to longs in BeamEnumerableConverter; not sure how many 
> conversions will be required here



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4700) JDBC driver cannot support TIMESTAMP data type

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4700?focusedWorklogId=121665=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121665
 ]

ASF GitHub Bot logged work on BEAM-4700:


Author: ASF GitHub Bot
Created on: 10/Jul/18 22:35
Start Date: 10/Jul/18 22:35
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #5919: BEAM-4700: [SQL] 
Default timezone is UTC
URL: https://github.com/apache/beam/pull/5919#issuecomment-403987646
 
 
   How about trigger the java post commit test as well so it can still pass SQL 
read/write tests?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121665)
Time Spent: 1h 40m  (was: 1.5h)

> JDBC driver cannot support TIMESTAMP data type
> --
>
> Key: BEAM-4700
> URL: https://issues.apache.org/jira/browse/BEAM-4700
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Avatica allows column representation to be customized, so a timestamp can be 
> stored as a variety of types. Joda ReadableInstant is none of these types: 
> https://github.com/apache/calcite-avatica/blob/acb675de97b9b0743c09368820a770e2ceda05f8/core/src/main/java/org/apache/calcite/avatica/util/AbstractCursor.java#L162
> By default, it seems to be configured to store {{TIMESTAMP}} columns as 
> {{long}} values. If you run the SQL shell and select a {{TIMESTAMP}} column, 
> you get:
> {code}
> ava.lang.ClassCastException: org.joda.time.Instant cannot be cast to 
> java.lang.Number
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$NumberAccessor.getNumber(AbstractCursor.java:726)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$TimestampFromNumberAccessor.getString(AbstractCursor.java:1026)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:225)
> at sqlline.Rows$Row.(Rows.java:183)
> {code}
> So, essentially, Beam SQL Shell does not support timestamps.
> We may be able to:
>  - override how the accessor for our existing storage is created
>  - configure what the column representation is (this doesn't really help, 
> since none of the choices are ours)
>  - convert timestamps to longs in BeamEnumerableConverter; not sure how many 
> conversions will be required here



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4700) JDBC driver cannot support TIMESTAMP data type

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4700?focusedWorklogId=121664=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121664
 ]

ASF GitHub Bot logged work on BEAM-4700:


Author: ASF GitHub Bot
Created on: 10/Jul/18 22:34
Start Date: 10/Jul/18 22:34
Worklog Time Spent: 10m 
  Work Description: amaliujia commented on issue #5919: BEAM-4700: [SQL] 
Default timezone is UTC
URL: https://github.com/apache/beam/pull/5919#issuecomment-403987426
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121664)
Time Spent: 1.5h  (was: 1h 20m)

> JDBC driver cannot support TIMESTAMP data type
> --
>
> Key: BEAM-4700
> URL: https://issues.apache.org/jira/browse/BEAM-4700
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Avatica allows column representation to be customized, so a timestamp can be 
> stored as a variety of types. Joda ReadableInstant is none of these types: 
> https://github.com/apache/calcite-avatica/blob/acb675de97b9b0743c09368820a770e2ceda05f8/core/src/main/java/org/apache/calcite/avatica/util/AbstractCursor.java#L162
> By default, it seems to be configured to store {{TIMESTAMP}} columns as 
> {{long}} values. If you run the SQL shell and select a {{TIMESTAMP}} column, 
> you get:
> {code}
> ava.lang.ClassCastException: org.joda.time.Instant cannot be cast to 
> java.lang.Number
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$NumberAccessor.getNumber(AbstractCursor.java:726)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$TimestampFromNumberAccessor.getString(AbstractCursor.java:1026)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:225)
> at sqlline.Rows$Row.(Rows.java:183)
> {code}
> So, essentially, Beam SQL Shell does not support timestamps.
> We may be able to:
>  - override how the accessor for our existing storage is created
>  - configure what the column representation is (this doesn't really help, 
> since none of the choices are ours)
>  - convert timestamps to longs in BeamEnumerableConverter; not sure how many 
> conversions will be required here



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4700) JDBC driver cannot support TIMESTAMP data type

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4700?focusedWorklogId=121662=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121662
 ]

ASF GitHub Bot logged work on BEAM-4700:


Author: ASF GitHub Bot
Created on: 10/Jul/18 22:29
Start Date: 10/Jul/18 22:29
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5919: BEAM-4700: [SQL] 
Default timezone is UTC
URL: https://github.com/apache/beam/pull/5919#issuecomment-403986452
 
 
   R: @akedin
   cc: @amaliujia


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121662)
Time Spent: 1h 20m  (was: 1h 10m)

> JDBC driver cannot support TIMESTAMP data type
> --
>
> Key: BEAM-4700
> URL: https://issues.apache.org/jira/browse/BEAM-4700
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Avatica allows column representation to be customized, so a timestamp can be 
> stored as a variety of types. Joda ReadableInstant is none of these types: 
> https://github.com/apache/calcite-avatica/blob/acb675de97b9b0743c09368820a770e2ceda05f8/core/src/main/java/org/apache/calcite/avatica/util/AbstractCursor.java#L162
> By default, it seems to be configured to store {{TIMESTAMP}} columns as 
> {{long}} values. If you run the SQL shell and select a {{TIMESTAMP}} column, 
> you get:
> {code}
> ava.lang.ClassCastException: org.joda.time.Instant cannot be cast to 
> java.lang.Number
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$NumberAccessor.getNumber(AbstractCursor.java:726)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$TimestampFromNumberAccessor.getString(AbstractCursor.java:1026)
> at 
> org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:225)
> at sqlline.Rows$Row.(Rows.java:183)
> {code}
> So, essentially, Beam SQL Shell does not support timestamps.
> We may be able to:
>  - override how the accessor for our existing storage is created
>  - configure what the column representation is (this doesn't really help, 
> since none of the choices are ours)
>  - convert timestamps to longs in BeamEnumerableConverter; not sure how many 
> conversions will be required here



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4562) [SQL] Fix INSERT VALUES in JdbcDriver

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4562?focusedWorklogId=121661=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121661
 ]

ASF GitHub Bot logged work on BEAM-4562:


Author: ASF GitHub Bot
Created on: 10/Jul/18 22:28
Start Date: 10/Jul/18 22:28
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5918: BEAM-4562: [SQL] Fix 
INSERT VALUES in JDBC
URL: https://github.com/apache/beam/pull/5918#issuecomment-403986118
 
 
   R: @XuMingmin Can you be the committer for this?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121661)
Time Spent: 40m  (was: 0.5h)

> [SQL] Fix INSERT VALUES in JdbcDriver 
> --
>
> Key: BEAM-4562
> URL: https://issues.apache.org/jira/browse/BEAM-4562
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Executing INSERT VALUES against JdbcDriver fails. Executing similar 
> statements against BeamSqlEnv works fine. Example:
> {code:java}
> TestTableProvider tableProvider = new TestTableProvider();
> Connection connection = JdbcDriver.connect(tableProvider);
> connection
> .createStatement()
> .executeUpdate("CREATE TABLE person (id BIGINT, name VARCHAR) TYPE 
> 'test'");
> connection.createStatement().executeQuery("INSERT INTO person VALUES(3, 
> 'yyy')");
> {code}
>  Output:
> {code}
> java.sql.SQLException: Error while executing SQL "INSERT INTO person 
> VALUES(3, 'yyy')": Node [rel#9:Subset#1.ENUMERABLE.[]] could not be 
> implemented; planner state:
> Root: rel#9:Subset#1.ENUMERABLE.[]
> Original rel:
> BeamIOSinkRel(subset=[rel#9:Subset#1.ENUMERABLE.[]], table=[[beam, person]], 
> operation=[INSERT], flattened=[false]): rowcount = 1.0, cumulative cost = 
> {1.0 rows, 0.0 cpu, 0.0 io}, id = 6
>   LogicalValues(subset=[rel#5:Subset#0.NONE.[]], tuples=[[{ 3, 'yyy' }]]): 
> rowcount = 1.0, cumulative cost = {1.0 rows, 1.0 cpu, 0.0 io}, id = 0
> Sets:
> Set#0, type: RecordType(BIGINT id, VARCHAR name)
>   rel#5:Subset#0.NONE.[], best=null, importance=0.81
>   rel#0:LogicalValues.NONE.[[0, 1], [1]](type=RecordType(BIGINT 
> id, VARCHAR name),tuples=[{ 3, 'yyy' }]), rowcount=1.0, cumulative cost={inf}
>   rel#14:Subset#0.BEAM_LOGICAL.[], best=null, importance=0.81
>   rel#20:Subset#0.ENUMERABLE.[], best=rel#19, importance=0.405
>   rel#19:EnumerableValues.ENUMERABLE.[[0, 1], 
> [1]](type=RecordType(BIGINT id, VARCHAR name),tuples=[{ 3, 'yyy' }]), 
> rowcount=1.0, cumulative cost={1.0 rows, 1.0 cpu, 0.0 io}
> Set#1, type: RecordType(BIGINT ROWCOUNT)
>   rel#7:Subset#1.BEAM_LOGICAL.[], best=null, importance=0.9
>   
> rel#6:BeamIOSinkRel.BEAM_LOGICAL.[](input=rel#5:Subset#0.NONE.[],table=[beam, 
> person],operation=INSERT,flattened=false), rowcount=1.0, cumulative cost={inf}
>   
> rel#15:BeamIOSinkRel.BEAM_LOGICAL.[](input=rel#14:Subset#0.BEAM_LOGICAL.[],table=[beam,
>  person],operation=INSERT,flattened=false), rowcount=1.0, cumulative 
> cost={inf}
>   rel#9:Subset#1.ENUMERABLE.[], best=null, importance=1.0
>   
> rel#10:AbstractConverter.ENUMERABLE.[](input=rel#7:Subset#1.BEAM_LOGICAL.[],convention=ENUMERABLE,sort=[]),
>  rowcount=1.0, cumulative cost={inf}
>   
> rel#11:BeamEnumerableConverter.ENUMERABLE.[](input=rel#7:Subset#1.BEAM_LOGICAL.[]),
>  rowcount=1.0, cumulative cost={inf}{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4562) [SQL] Fix INSERT VALUES in JdbcDriver

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4562?focusedWorklogId=121660=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121660
 ]

ASF GitHub Bot logged work on BEAM-4562:


Author: ASF GitHub Bot
Created on: 10/Jul/18 22:26
Start Date: 10/Jul/18 22:26
Worklog Time Spent: 10m 
  Work Description: akedin commented on issue #5918: BEAM-4562: [SQL] Fix 
INSERT VALUES in JDBC
URL: https://github.com/apache/beam/pull/5918#issuecomment-403985745
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121660)
Time Spent: 0.5h  (was: 20m)

> [SQL] Fix INSERT VALUES in JdbcDriver 
> --
>
> Key: BEAM-4562
> URL: https://issues.apache.org/jira/browse/BEAM-4562
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Executing INSERT VALUES against JdbcDriver fails. Executing similar 
> statements against BeamSqlEnv works fine. Example:
> {code:java}
> TestTableProvider tableProvider = new TestTableProvider();
> Connection connection = JdbcDriver.connect(tableProvider);
> connection
> .createStatement()
> .executeUpdate("CREATE TABLE person (id BIGINT, name VARCHAR) TYPE 
> 'test'");
> connection.createStatement().executeQuery("INSERT INTO person VALUES(3, 
> 'yyy')");
> {code}
>  Output:
> {code}
> java.sql.SQLException: Error while executing SQL "INSERT INTO person 
> VALUES(3, 'yyy')": Node [rel#9:Subset#1.ENUMERABLE.[]] could not be 
> implemented; planner state:
> Root: rel#9:Subset#1.ENUMERABLE.[]
> Original rel:
> BeamIOSinkRel(subset=[rel#9:Subset#1.ENUMERABLE.[]], table=[[beam, person]], 
> operation=[INSERT], flattened=[false]): rowcount = 1.0, cumulative cost = 
> {1.0 rows, 0.0 cpu, 0.0 io}, id = 6
>   LogicalValues(subset=[rel#5:Subset#0.NONE.[]], tuples=[[{ 3, 'yyy' }]]): 
> rowcount = 1.0, cumulative cost = {1.0 rows, 1.0 cpu, 0.0 io}, id = 0
> Sets:
> Set#0, type: RecordType(BIGINT id, VARCHAR name)
>   rel#5:Subset#0.NONE.[], best=null, importance=0.81
>   rel#0:LogicalValues.NONE.[[0, 1], [1]](type=RecordType(BIGINT 
> id, VARCHAR name),tuples=[{ 3, 'yyy' }]), rowcount=1.0, cumulative cost={inf}
>   rel#14:Subset#0.BEAM_LOGICAL.[], best=null, importance=0.81
>   rel#20:Subset#0.ENUMERABLE.[], best=rel#19, importance=0.405
>   rel#19:EnumerableValues.ENUMERABLE.[[0, 1], 
> [1]](type=RecordType(BIGINT id, VARCHAR name),tuples=[{ 3, 'yyy' }]), 
> rowcount=1.0, cumulative cost={1.0 rows, 1.0 cpu, 0.0 io}
> Set#1, type: RecordType(BIGINT ROWCOUNT)
>   rel#7:Subset#1.BEAM_LOGICAL.[], best=null, importance=0.9
>   
> rel#6:BeamIOSinkRel.BEAM_LOGICAL.[](input=rel#5:Subset#0.NONE.[],table=[beam, 
> person],operation=INSERT,flattened=false), rowcount=1.0, cumulative cost={inf}
>   
> rel#15:BeamIOSinkRel.BEAM_LOGICAL.[](input=rel#14:Subset#0.BEAM_LOGICAL.[],table=[beam,
>  person],operation=INSERT,flattened=false), rowcount=1.0, cumulative 
> cost={inf}
>   rel#9:Subset#1.ENUMERABLE.[], best=null, importance=1.0
>   
> rel#10:AbstractConverter.ENUMERABLE.[](input=rel#7:Subset#1.BEAM_LOGICAL.[],convention=ENUMERABLE,sort=[]),
>  rowcount=1.0, cumulative cost={inf}
>   
> rel#11:BeamEnumerableConverter.ENUMERABLE.[](input=rel#7:Subset#1.BEAM_LOGICAL.[]),
>  rowcount=1.0, cumulative cost={inf}{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4727) Reduce metrics overhead

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4727?focusedWorklogId=121647=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121647
 ]

ASF GitHub Bot logged work on BEAM-4727:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:59
Start Date: 10/Jul/18 21:59
Worklog Time Spent: 10m 
  Work Description: herohde commented on a change in pull request #5884: 
[BEAM-4727] Re-use metric context throughout bundle
URL: https://github.com/apache/beam/pull/5884#discussion_r201507007
 
 

 ##
 File path: sdks/go/pkg/beam/core/metrics/metrics.go
 ##
 @@ -69,18 +69,58 @@ type ctxKey string
 const bundleKey ctxKey = "beam:bundle"
 const ptransformKey ctxKey = "beam:ptransform"
 
+// context is a caching context since reads & writes are expensive.
+type beamCtx struct {
+   context.Context
+   bundleID, ptransformID string
+}
+
+// Lift the keys value for faster lookups when not available.
+func (ctx *beamCtx) Value(key interface{}) interface{} {
+   switch key {
+   case bundleKey:
+   if ctx.bundleID == "" {
+   if id := ctx.Value(key); id != nil {
+   ctx.bundleID = id.(string)
+   }
+   }
+   return ctx.bundleID
+   case ptransformKey:
+   if ctx.ptransformID == "" {
+   if id := ctx.Value(key); id != nil {
+   ctx.ptransformID = id.(string)
+   }
+   }
+   return ctx.ptransformID
+   }
+   return ctx.Context.Value(key)
+}
+
 // SetBundleID sets the id of the current Bundle.
 func SetBundleID(ctx context.Context, id string) context.Context {
-   return context.WithValue(ctx, bundleKey, id)
+   var ptransformID string
+   if bctx, ok := ctx.(*beamCtx); ok {
+   ptransformID = bctx.ptransformID
+   }
+   return {Context: ctx, bundleID: id, ptransformID: ptransformID}
 
 Review comment:
   We can also use the beamCtx's context as Context here to avoid nesting 
beamCtxs.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121647)
Time Spent: 1h 50m  (was: 1h 40m)

> Reduce metrics overhead
> ---
>
> Key: BEAM-4727
> URL: https://issues.apache.org/jira/browse/BEAM-4727
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-go
>Reporter: Robert Burke
>Assignee: Robert Burke
>Priority: Major
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> There are a few opportunities to avoid metrics overhead.
> First when setting state in the context, we allocate a new one for the stored 
> value, per element. This generates a fair amount of objects for the garbage 
> collector too. If we retain and re-use contexts within a bundle, we would 
> have the opportunity to save on these costs.
> Also, it's possible that we have overhead on the metric updating paths. We 
> can possibly do better than the general sync.Map, and avoid the type 
> assertion cost for extracting values of known types from the maps.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4687) Automatically file JIRA for dependency updates

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4687?focusedWorklogId=121646=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121646
 ]

ASF GitHub Bot logged work on BEAM-4687:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:59
Start Date: 10/Jul/18 21:59
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #5920: DO NOT MERGE, 
[BEAM-4687] test Jenkins JIRA plugin
URL: https://github.com/apache/beam/pull/5920#issuecomment-403979625
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121646)
Time Spent: 1h  (was: 50m)

> Automatically file JIRA for dependency updates
> --
>
> Key: BEAM-4687
> URL: https://issues.apache.org/jira/browse/BEAM-4687
> Project: Beam
>  Issue Type: New Feature
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 1h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4727) Reduce metrics overhead

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4727?focusedWorklogId=121650=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121650
 ]

ASF GitHub Bot logged work on BEAM-4727:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:59
Start Date: 10/Jul/18 21:59
Worklog Time Spent: 10m 
  Work Description: herohde commented on a change in pull request #5884: 
[BEAM-4727] Re-use metric context throughout bundle
URL: https://github.com/apache/beam/pull/5884#discussion_r201506572
 
 

 ##
 File path: sdks/go/pkg/beam/core/metrics/metrics.go
 ##
 @@ -69,18 +69,58 @@ type ctxKey string
 const bundleKey ctxKey = "beam:bundle"
 const ptransformKey ctxKey = "beam:ptransform"
 
+// context is a caching context since reads & writes are expensive.
+type beamCtx struct {
+   context.Context
+   bundleID, ptransformID string
+}
+
+// Lift the keys value for faster lookups when not available.
+func (ctx *beamCtx) Value(key interface{}) interface{} {
+   switch key {
+   case bundleKey:
+   if ctx.bundleID == "" {
+   if id := ctx.Value(key); id != nil {
+   ctx.bundleID = id.(string)
+   }
+   }
+   return ctx.bundleID
+   case ptransformKey:
+   if ctx.ptransformID == "" {
+   if id := ctx.Value(key); id != nil {
+   ctx.ptransformID = id.(string)
+   }
+   }
+   return ctx.ptransformID
+   }
+   return ctx.Context.Value(key)
+}
+
 // SetBundleID sets the id of the current Bundle.
 func SetBundleID(ctx context.Context, id string) context.Context {
-   return context.WithValue(ctx, bundleKey, id)
+   var ptransformID string
+   if bctx, ok := ctx.(*beamCtx); ok {
 
 Review comment:
   The semantics here is subtle and deserves a comment. I would suggest making 
a note that checking for beamCtx here is just an optimization. Always using "" 
as ptransformID would work as well.
   
   Ditto for the other set helper.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121650)
Time Spent: 2h  (was: 1h 50m)

> Reduce metrics overhead
> ---
>
> Key: BEAM-4727
> URL: https://issues.apache.org/jira/browse/BEAM-4727
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-go
>Reporter: Robert Burke
>Assignee: Robert Burke
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> There are a few opportunities to avoid metrics overhead.
> First when setting state in the context, we allocate a new one for the stored 
> value, per element. This generates a fair amount of objects for the garbage 
> collector too. If we retain and re-use contexts within a bundle, we would 
> have the opportunity to save on these costs.
> Also, it's possible that we have overhead on the metric updating paths. We 
> can possibly do better than the general sync.Map, and avoid the type 
> assertion cost for extracting values of known types from the maps.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4727) Reduce metrics overhead

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4727?focusedWorklogId=121648=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121648
 ]

ASF GitHub Bot logged work on BEAM-4727:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:59
Start Date: 10/Jul/18 21:59
Worklog Time Spent: 10m 
  Work Description: herohde commented on a change in pull request #5884: 
[BEAM-4727] Re-use metric context throughout bundle
URL: https://github.com/apache/beam/pull/5884#discussion_r201483579
 
 

 ##
 File path: sdks/go/pkg/beam/core/metrics/metrics.go
 ##
 @@ -69,18 +69,58 @@ type ctxKey string
 const bundleKey ctxKey = "beam:bundle"
 const ptransformKey ctxKey = "beam:ptransform"
 
+// context is a caching context since reads & writes are expensive.
 
 Review comment:
   nit: beamCtx is ... Maybe also elaborate what it caches.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121648)
Time Spent: 1h 50m  (was: 1h 40m)

> Reduce metrics overhead
> ---
>
> Key: BEAM-4727
> URL: https://issues.apache.org/jira/browse/BEAM-4727
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-go
>Reporter: Robert Burke
>Assignee: Robert Burke
>Priority: Major
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> There are a few opportunities to avoid metrics overhead.
> First when setting state in the context, we allocate a new one for the stored 
> value, per element. This generates a fair amount of objects for the garbage 
> collector too. If we retain and re-use contexts within a bundle, we would 
> have the opportunity to save on these costs.
> Also, it's possible that we have overhead on the metric updating paths. We 
> can possibly do better than the general sync.Map, and avoid the type 
> assertion cost for extracting values of known types from the maps.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4687) Automatically file JIRA for dependency updates

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4687?focusedWorklogId=121645=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121645
 ]

ASF GitHub Bot logged work on BEAM-4687:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:59
Start Date: 10/Jul/18 21:59
Worklog Time Spent: 10m 
  Work Description: yifanzou opened a new pull request #5920: DO NOT MERGE, 
[BEAM-4687] test Jenkins JIRA plugin
URL: https://github.com/apache/beam/pull/5920
 
 
   **Please** add a meaningful description for your change here
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121645)
Time Spent: 50m  (was: 40m)

> Automatically file JIRA for dependency updates
> --
>
> Key: BEAM-4687
> URL: https://issues.apache.org/jira/browse/BEAM-4687
> Project: Beam
>  Issue Type: New Feature
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121644=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121644
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:59
Start Date: 10/Jul/18 21:59
Worklog Time Spent: 10m 
  Work Description: superbobry commented on issue #5869: [BEAM-1251] 
Replace NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869#issuecomment-403979524
 
 
   @aaltay thanks for the feedback. I've added a relevant comment on the 
migration proposal doc.
   
   @cclauss done.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121644)
Time Spent: 17h 40m  (was: 17.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 17h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4727) Reduce metrics overhead

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4727?focusedWorklogId=121649=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121649
 ]

ASF GitHub Bot logged work on BEAM-4727:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:59
Start Date: 10/Jul/18 21:59
Worklog Time Spent: 10m 
  Work Description: herohde commented on a change in pull request #5884: 
[BEAM-4727] Re-use metric context throughout bundle
URL: https://github.com/apache/beam/pull/5884#discussion_r201505368
 
 

 ##
 File path: sdks/go/pkg/beam/core/metrics/metrics.go
 ##
 @@ -69,18 +69,58 @@ type ctxKey string
 const bundleKey ctxKey = "beam:bundle"
 const ptransformKey ctxKey = "beam:ptransform"
 
+// context is a caching context since reads & writes are expensive.
+type beamCtx struct {
+   context.Context
+   bundleID, ptransformID string
+}
+
+// Lift the keys value for faster lookups when not available.
+func (ctx *beamCtx) Value(key interface{}) interface{} {
+   switch key {
+   case bundleKey:
+   if ctx.bundleID == "" {
+   if id := ctx.Value(key); id != nil {
+   ctx.bundleID = id.(string)
+   }
+   }
+   return ctx.bundleID
+   case ptransformKey:
+   if ctx.ptransformID == "" {
+   if id := ctx.Value(key); id != nil {
+   ctx.ptransformID = id.(string)
+   }
+   }
+   return ctx.ptransformID
+   }
+   return ctx.Context.Value(key)
+}
+
 // SetBundleID sets the id of the current Bundle.
 func SetBundleID(ctx context.Context, id string) context.Context {
-   return context.WithValue(ctx, bundleKey, id)
+   var ptransformID string
+   if bctx, ok := ctx.(*beamCtx); ok {
+   ptransformID = bctx.ptransformID
+   }
+   return {Context: ctx, bundleID: id, ptransformID: ptransformID}
 }
 
 // SetPTransformID sets the id of the current PTransform.
 func SetPTransformID(ctx context.Context, id string) context.Context {
-   return context.WithValue(ctx, ptransformKey, id)
+   var bundleID string
+   if bctx, ok := ctx.(*beamCtx); ok {
+   bundleID = bctx.bundleID
+   }
+   return {Context: ctx, bundleID: bundleID, ptransformID: id}
 }
 
 func getContextKey(ctx context.Context, n name) key {
key := key{name: n, bundle: "(bundle id unset)", ptransform: 
"(ptransform id unset)"}
+   if bctx, ok := ctx.(*beamCtx); ok {
 
 Review comment:
   This inserted block would be wrong, if we used context.WithValue on a 
beamCtx and went around the on-write caching. Is this block even needed given 
the behavior of beamCtx.Value as a genuine cache?
   
   At any rate, please document the behavior and assumptions of beamCtx. It 
feels subtle. Perhaps also add some test where we mix normal contexts and 
operations with beamCtx and its helpers. IMO it should be a semantic no-op.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121649)
Time Spent: 2h  (was: 1h 50m)

> Reduce metrics overhead
> ---
>
> Key: BEAM-4727
> URL: https://issues.apache.org/jira/browse/BEAM-4727
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-go
>Reporter: Robert Burke
>Assignee: Robert Burke
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> There are a few opportunities to avoid metrics overhead.
> First when setting state in the context, we allocate a new one for the stored 
> value, per element. This generates a fair amount of objects for the garbage 
> collector too. If we retain and re-use contexts within a bundle, we would 
> have the opportunity to save on these costs.
> Also, it's possible that we have overhead on the metric updating paths. We 
> can possibly do better than the general sync.Map, and avoid the type 
> assertion cost for extracting values of known types from the maps.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4700) JDBC driver cannot support TIMESTAMP data type

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4700?focusedWorklogId=121643=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121643
 ]

ASF GitHub Bot logged work on BEAM-4700:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:57
Start Date: 10/Jul/18 21:57
Worklog Time Spent: 10m 
  Work Description: apilloud opened a new pull request #5919: BEAM-4700: 
[SQL] Default timezone is UTC
URL: https://github.com/apache/beam/pull/5919
 
 
   This fixes JDBC timezone conversion issues.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [X] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121643)
Time Spent: 1h 10m  (was: 1h)

> JDBC driver cannot support TIMESTAMP data type
> --
>
> Key: BEAM-4700
> URL: https://issues.apache.org/jira/browse/BEAM-4700
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Avatica allows column representation to be customized, so a timestamp can be 
> stored as a variety of types. Joda 

[jira] [Work logged] (BEAM-4562) [SQL] Fix INSERT VALUES in JdbcDriver

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4562?focusedWorklogId=121637=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121637
 ]

ASF GitHub Bot logged work on BEAM-4562:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:48
Start Date: 10/Jul/18 21:48
Worklog Time Spent: 10m 
  Work Description: apilloud commented on issue #5918: BEAM-4562 BEAM-4700: 
[SQL] Fix INSERT VALUES and timezone issues in JDBC
URL: https://github.com/apache/beam/pull/5918#issuecomment-403977032
 
 
   R: @akedin 
   cc: @amaliujia


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121637)
Time Spent: 20m  (was: 10m)

> [SQL] Fix INSERT VALUES in JdbcDriver 
> --
>
> Key: BEAM-4562
> URL: https://issues.apache.org/jira/browse/BEAM-4562
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Executing INSERT VALUES against JdbcDriver fails. Executing similar 
> statements against BeamSqlEnv works fine. Example:
> {code:java}
> TestTableProvider tableProvider = new TestTableProvider();
> Connection connection = JdbcDriver.connect(tableProvider);
> connection
> .createStatement()
> .executeUpdate("CREATE TABLE person (id BIGINT, name VARCHAR) TYPE 
> 'test'");
> connection.createStatement().executeQuery("INSERT INTO person VALUES(3, 
> 'yyy')");
> {code}
>  Output:
> {code}
> java.sql.SQLException: Error while executing SQL "INSERT INTO person 
> VALUES(3, 'yyy')": Node [rel#9:Subset#1.ENUMERABLE.[]] could not be 
> implemented; planner state:
> Root: rel#9:Subset#1.ENUMERABLE.[]
> Original rel:
> BeamIOSinkRel(subset=[rel#9:Subset#1.ENUMERABLE.[]], table=[[beam, person]], 
> operation=[INSERT], flattened=[false]): rowcount = 1.0, cumulative cost = 
> {1.0 rows, 0.0 cpu, 0.0 io}, id = 6
>   LogicalValues(subset=[rel#5:Subset#0.NONE.[]], tuples=[[{ 3, 'yyy' }]]): 
> rowcount = 1.0, cumulative cost = {1.0 rows, 1.0 cpu, 0.0 io}, id = 0
> Sets:
> Set#0, type: RecordType(BIGINT id, VARCHAR name)
>   rel#5:Subset#0.NONE.[], best=null, importance=0.81
>   rel#0:LogicalValues.NONE.[[0, 1], [1]](type=RecordType(BIGINT 
> id, VARCHAR name),tuples=[{ 3, 'yyy' }]), rowcount=1.0, cumulative cost={inf}
>   rel#14:Subset#0.BEAM_LOGICAL.[], best=null, importance=0.81
>   rel#20:Subset#0.ENUMERABLE.[], best=rel#19, importance=0.405
>   rel#19:EnumerableValues.ENUMERABLE.[[0, 1], 
> [1]](type=RecordType(BIGINT id, VARCHAR name),tuples=[{ 3, 'yyy' }]), 
> rowcount=1.0, cumulative cost={1.0 rows, 1.0 cpu, 0.0 io}
> Set#1, type: RecordType(BIGINT ROWCOUNT)
>   rel#7:Subset#1.BEAM_LOGICAL.[], best=null, importance=0.9
>   
> rel#6:BeamIOSinkRel.BEAM_LOGICAL.[](input=rel#5:Subset#0.NONE.[],table=[beam, 
> person],operation=INSERT,flattened=false), rowcount=1.0, cumulative cost={inf}
>   
> rel#15:BeamIOSinkRel.BEAM_LOGICAL.[](input=rel#14:Subset#0.BEAM_LOGICAL.[],table=[beam,
>  person],operation=INSERT,flattened=false), rowcount=1.0, cumulative 
> cost={inf}
>   rel#9:Subset#1.ENUMERABLE.[], best=null, importance=1.0
>   
> rel#10:AbstractConverter.ENUMERABLE.[](input=rel#7:Subset#1.BEAM_LOGICAL.[],convention=ENUMERABLE,sort=[]),
>  rowcount=1.0, cumulative cost={inf}
>   
> rel#11:BeamEnumerableConverter.ENUMERABLE.[](input=rel#7:Subset#1.BEAM_LOGICAL.[]),
>  rowcount=1.0, cumulative cost={inf}{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4562) [SQL] Fix INSERT VALUES in JdbcDriver

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4562?focusedWorklogId=121634=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121634
 ]

ASF GitHub Bot logged work on BEAM-4562:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:45
Start Date: 10/Jul/18 21:45
Worklog Time Spent: 10m 
  Work Description: apilloud opened a new pull request #5918: BEAM-4562 
BEAM-4700: [SQL] Fix INSERT VALUES and timezone issues
URL: https://github.com/apache/beam/pull/5918
 
 
   This fixes the critical SQL issues:
* BEAM-4562: INSERT values not working
* BEAM-4700: Timezone offset being applied to timestamp without timezone
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [X] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121634)
Time Spent: 10m
Remaining Estimate: 0h

> [SQL] Fix INSERT VALUES in JdbcDriver 
> --
>
> Key: BEAM-4562
> URL: https://issues.apache.org/jira/browse/BEAM-4562
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Andrew Pilloud
>Priority: Major
>  Time Spent: 10m

[jira] [Assigned] (BEAM-4562) [SQL] Fix INSERT VALUES in JdbcDriver

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud reassigned BEAM-4562:


Assignee: Andrew Pilloud

> [SQL] Fix INSERT VALUES in JdbcDriver 
> --
>
> Key: BEAM-4562
> URL: https://issues.apache.org/jira/browse/BEAM-4562
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Andrew Pilloud
>Priority: Major
>
> Executing INSERT VALUES against JdbcDriver fails. Executing similar 
> statements against BeamSqlEnv works fine. Example:
> {code:java}
> TestTableProvider tableProvider = new TestTableProvider();
> Connection connection = JdbcDriver.connect(tableProvider);
> connection
> .createStatement()
> .executeUpdate("CREATE TABLE person (id BIGINT, name VARCHAR) TYPE 
> 'test'");
> connection.createStatement().executeQuery("INSERT INTO person VALUES(3, 
> 'yyy')");
> {code}
>  Output:
> {code}
> java.sql.SQLException: Error while executing SQL "INSERT INTO person 
> VALUES(3, 'yyy')": Node [rel#9:Subset#1.ENUMERABLE.[]] could not be 
> implemented; planner state:
> Root: rel#9:Subset#1.ENUMERABLE.[]
> Original rel:
> BeamIOSinkRel(subset=[rel#9:Subset#1.ENUMERABLE.[]], table=[[beam, person]], 
> operation=[INSERT], flattened=[false]): rowcount = 1.0, cumulative cost = 
> {1.0 rows, 0.0 cpu, 0.0 io}, id = 6
>   LogicalValues(subset=[rel#5:Subset#0.NONE.[]], tuples=[[{ 3, 'yyy' }]]): 
> rowcount = 1.0, cumulative cost = {1.0 rows, 1.0 cpu, 0.0 io}, id = 0
> Sets:
> Set#0, type: RecordType(BIGINT id, VARCHAR name)
>   rel#5:Subset#0.NONE.[], best=null, importance=0.81
>   rel#0:LogicalValues.NONE.[[0, 1], [1]](type=RecordType(BIGINT 
> id, VARCHAR name),tuples=[{ 3, 'yyy' }]), rowcount=1.0, cumulative cost={inf}
>   rel#14:Subset#0.BEAM_LOGICAL.[], best=null, importance=0.81
>   rel#20:Subset#0.ENUMERABLE.[], best=rel#19, importance=0.405
>   rel#19:EnumerableValues.ENUMERABLE.[[0, 1], 
> [1]](type=RecordType(BIGINT id, VARCHAR name),tuples=[{ 3, 'yyy' }]), 
> rowcount=1.0, cumulative cost={1.0 rows, 1.0 cpu, 0.0 io}
> Set#1, type: RecordType(BIGINT ROWCOUNT)
>   rel#7:Subset#1.BEAM_LOGICAL.[], best=null, importance=0.9
>   
> rel#6:BeamIOSinkRel.BEAM_LOGICAL.[](input=rel#5:Subset#0.NONE.[],table=[beam, 
> person],operation=INSERT,flattened=false), rowcount=1.0, cumulative cost={inf}
>   
> rel#15:BeamIOSinkRel.BEAM_LOGICAL.[](input=rel#14:Subset#0.BEAM_LOGICAL.[],table=[beam,
>  person],operation=INSERT,flattened=false), rowcount=1.0, cumulative 
> cost={inf}
>   rel#9:Subset#1.ENUMERABLE.[], best=null, importance=1.0
>   
> rel#10:AbstractConverter.ENUMERABLE.[](input=rel#7:Subset#1.BEAM_LOGICAL.[],convention=ENUMERABLE,sort=[]),
>  rowcount=1.0, cumulative cost={inf}
>   
> rel#11:BeamEnumerableConverter.ENUMERABLE.[](input=rel#7:Subset#1.BEAM_LOGICAL.[]),
>  rowcount=1.0, cumulative cost={inf}{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-4687) Automatically file JIRA for dependency updates

[yifan zou (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539257#comment-16539257
 ] 

yifan zou commented on BEAM-4687:
-

design proposal: 
https://docs.google.com/document/d/1XXTMnofizSQZSorZy4NFlxKfx5f17PGRmws-DngoTms/edit#

> Automatically file JIRA for dependency updates
> --
>
> Key: BEAM-4687
> URL: https://issues.apache.org/jira/browse/BEAM-4687
> Project: Beam
>  Issue Type: New Feature
>  Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4432) Performance tests need a way to generate Synthetic data

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4432?focusedWorklogId=121626=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121626
 ]

ASF GitHub Bot logged work on BEAM-4432:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:27
Start Date: 10/Jul/18 21:27
Worklog Time Spent: 10m 
  Work Description: pabloem commented on a change in pull request #5519: 
[BEAM-4432] Adding Sources to produce Synthetic output for Batch pipelines
URL: https://github.com/apache/beam/pull/5519#discussion_r201445730
 
 

 ##
 File path: sdks/java/io/synthetic/pom.xml
 ##
 @@ -0,0 +1,84 @@
+
 
 Review comment:
   Done.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121626)
Time Spent: 8h 10m  (was: 8h)

> Performance tests need a way to generate Synthetic data
> ---
>
> Key: BEAM-4432
> URL: https://issues.apache.org/jira/browse/BEAM-4432
> Project: Beam
>  Issue Type: Improvement
>  Components: testing
>Reporter: Pablo Estrada
>Assignee: Pablo Estrada
>Priority: Minor
>  Time Spent: 8h 10m
>  Remaining Estimate: 0h
>
> GenerateSequence fal.lls short in this regard, as we may want to generate 
> data in custom distributions, or with specific repeatability requirements / 
> and hardcoded delays for autoscaling.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=121623=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121623
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:24
Start Date: 10/Jul/18 21:24
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4943: [BEAM-3906] Automate 
Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-403970713
 
 
   @aaltay The blocker bug was resolved and we used this PR in 2.5 RC 
verification. Please review it and feel free to put comments.
   
   Overview of this PR:
   In the 2.4.0 release, we have added Wheel files for Python SDK. Changes in 
this pull request aim to automate the Quickstarts and MobileGaming examples to 
make an easy release validation process.
   
   General workflow:
   download & validate sha512 and gpg keys
   setup virtualenv & install python sdk (tar)
   run wordcount and streaming wordcount on core runners (direct, dataflow)
   run userscore, leader_board and hourly_team_score on core runnres
   repeat from the step 2 to validate wheel versions.
   
   Major changes:
   change directory hierarchy of python automation shell scripts, 
release/src/main/groovy -> release/src/main/python-release
   Functionalize each steps of quickstart and mobile-gaming validation in 
purpose of reusing code for wheel validations.
   Add validations for wheel files.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121623)
Time Spent: 26h 10m  (was: 26h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 26h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4484) Shading model-pipeline / model-fn-execution / model-job-management produces corrupted classes

[Luke Cwik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luke Cwik resolved BEAM-4484.
-
   Resolution: Fixed
Fix Version/s: 2.6.0

> Shading model-pipeline / model-fn-execution / model-job-management produces 
> corrupted classes
> -
>
> Key: BEAM-4484
> URL: https://issues.apache.org/jira/browse/BEAM-4484
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.6.0
>
>
> During investigation for test failures on 
> [https://github.com/apache/beam/pull/5445,] I discovered that the the post 
> shaded beam-model-pipeline proto filedescriptors were corrupted. It turns out 
> that during the shading process, an over eager string replacement inside a 
> class is corrupting an internal field, in this case it modifies the RunnerApi 
> file descriptor storing
> {code:java}
> org.apache.beam.model.pipeline.v1.AccumulationMode.Enum{code}
> and changing it to
> {code:java}
> org.apache.beam.repackaged.beam_runners_direct_java.model.pipeline.v1.AccumulationMode.Enum{code}
> This problem exists because the proto package name and the java package name 
> collide and use org.apache.beam.model.pipeline.v1.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java

[Luke Cwik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luke Cwik resolved BEAM-4481.
-
   Resolution: Fixed
Fix Version/s: 2.6.0

> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.6.0
>
>  Time Spent: 7h
>  Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4126) Delete maven build files

[Luke Cwik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luke Cwik resolved BEAM-4126.
-
   Resolution: Fixed
Fix Version/s: Not applicable

> Delete maven build files
> 
>
> Key: BEAM-4126
> URL: https://issues.apache.org/jira/browse/BEAM-4126
> Project: Beam
>  Issue Type: Sub-task
>  Components: build-system
>Reporter: Scott Wegner
>Assignee: Luke Cwik
>Priority: Minor
> Fix For: Not applicable
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Once we are fully-migrated to Gradle, we should proactively remove Maven 
> build files (pom.xml etc) so that it's clear they no longer need to be 
> maintained.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4622) Many Beam SQL expressions never have their validation called

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4622?focusedWorklogId=121620=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121620
 ]

ASF GitHub Bot logged work on BEAM-4622:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:18
Start Date: 10/Jul/18 21:18
Worklog Time Spent: 10m 
  Work Description: vectorijk commented on a change in pull request #5912: 
[BEAM-4622] Makes required to call Beam SQL expressions validation
URL: https://github.com/apache/beam/pull/5912#discussion_r201498240
 
 

 ##
 File path: 
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/operator/reinterpret/BeamSqlReinterpretExpression.java
 ##
 @@ -47,6 +48,13 @@ public BeamSqlReinterpretExpression(List 
operands, SqlTypeNam
 
   @Override
   public boolean accept() {
+// Interval types will be already converted into BIGINT after evaluation.
+SqlTypeFamily opTypeFamily = opType(0).getFamily();
+if (opTypeFamily.equals(SqlTypeFamily.INTERVAL_DAY_TIME)
 
 Review comment:
   I think it's good to add a unit test for it


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121620)
Time Spent: 0.5h  (was: 20m)

> Many Beam SQL expressions never have their validation called
> 
>
> Key: BEAM-4622
> URL: https://issues.apache.org/jira/browse/BEAM-4622
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Alexey Romanenko
>Priority: Major
>  Labels: easyfix, newbie, starter
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> In {{BeamSqlFnExecutor}} there is a pattern where first the returned 
> expression is assigned to a variable {{ret}} and then after a giant switch 
> statement the validation is invoked. But there are many code paths that just 
> call {{return}} and skip validation. This should be refactored so it is 
> impossible to short-circuit on accident like this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam] 01/01: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have dependency conflicts

[lcwik]

This is an automated email from the ASF dual-hosted git repository.

lcwik pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 109f03c45af5fa287d938d696c95ef50dad2ab91
Merge: 50a4baa f9f9e87
Author: Lukasz Cwik 
AuthorDate: Tue Jul 10 14:09:04 2018 -0700

[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have 
dependency conflicts

 .../org/apache/beam/gradle/BeamModulePlugin.groovy | 138 -
 examples/java/build.gradle |   2 +-
 model/fn-execution/build.gradle|  13 +-
 model/job-management/build.gradle  |  13 +-
 model/pipeline/build.gradle|   7 +-
 runners/core-construction-java/build.gradle|   6 +-
 .../core/construction/ArtifactServiceStager.java   |   6 +-
 .../beam/runners/core/construction/BeamUrns.java   |   2 +-
 .../core/construction/CoderTranslation.java|   2 +-
 .../core/construction/CombineTranslation.java  |   2 +-
 .../CreatePCollectionViewTranslation.java  |   2 +-
 .../core/construction/DisplayDataTranslation.java  |   4 +-
 .../runners/core/construction/Environments.java|   2 +-
 .../construction/PCollectionViewTranslation.java   |   2 +-
 .../core/construction/ParDoTranslation.java|   4 +-
 .../construction/PipelineOptionsTranslation.java   |   4 +-
 .../runners/core/construction/ReadTranslation.java |   4 +-
 .../core/construction/TestStreamTranslation.java   |   2 +-
 .../core/construction/WindowIntoTranslation.java   |   2 +-
 .../construction/WindowingStrategyTranslation.java |   8 +-
 .../core/construction/WriteFilesTranslation.java   |   2 +-
 .../graph/GreedyPCollectionFusers.java |   2 +-
 .../core/construction/graph/QueryablePipeline.java |   2 +-
 .../construction/ArtifactServiceStagerTest.java|   8 +-
 .../InMemoryArtifactStagerService.java |   2 +-
 .../PipelineOptionsTranslationTest.java|   6 +-
 .../construction/WindowIntoTranslationTest.java|   2 +-
 .../construction/graph/ProtoOverridesTest.java |   2 +-
 runners/core-java/build.gradle |   2 -
 runners/direct-java/build.gradle   |  19 ++-
 .../runners/direct/portable/ReferenceRunner.java   |   2 +-
 .../LocalFileSystemArtifactRetrievalService.java   |   6 +-
 .../LocalFileSystemArtifactStagerService.java  |   8 +-
 .../runners/direct/portable/job/PreparingJob.java  |   2 +-
 .../portable/job/ReferenceRunnerJobService.java|   6 +-
 ...ocalFileSystemArtifactRetrievalServiceTest.java |   4 +-
 .../LocalFileSystemArtifactStagerServiceTest.java  |  14 +--
 .../UnsupportedArtifactRetrievalServiceTest.java   |   2 +-
 .../job/ReferenceRunnerJobServiceTest.java |   4 +-
 .../FlinkBatchPortablePipelineTranslator.java  |   2 +-
 .../apache/beam/runners/flink/FlinkJobInvoker.java |   2 +-
 .../FlinkStreamingPortablePipelineTranslator.java  |   2 +-
 .../streaming/ExecutableStageDoFnOperatorTest.java |   2 +-
 .../FlinkExecutableStageFunctionTest.java  |   2 +-
 .../dataflow/DataflowPipelineTranslator.java   |   2 +-
 runners/java-fn-execution/build.gradle |   3 -
 .../apache/beam/runners/fnexecution/FnService.java |   2 +-
 .../GrpcContextHeaderAccessorProvider.java |  16 +--
 .../beam/runners/fnexecution/GrpcFnServer.java |   2 +-
 .../fnexecution/InProcessServerFactory.java|   8 +-
 .../beam/runners/fnexecution/ServerFactory.java|   8 +-
 .../BeamFileSystemArtifactRetrievalService.java|  10 +-
 .../BeamFileSystemArtifactStagingService.java  |  10 +-
 .../fnexecution/control/FnApiControlClient.java|   6 +-
 .../control/FnApiControlClientPoolService.java |   2 +-
 .../control/ProcessBundleDescriptors.java  |   2 +-
 .../runners/fnexecution/data/GrpcDataService.java  |   2 +-
 .../jobsubmission/InMemoryJobService.java  |  10 +-
 .../fnexecution/jobsubmission/JobInvoker.java  |   2 +-
 .../fnexecution/jobsubmission/JobPreparation.java  |   2 +-
 .../fnexecution/logging/GrpcLoggingService.java|   2 +-
 .../runners/fnexecution/provisioning/JobInfo.java  |   2 +-
 .../provisioning/StaticGrpcProvisionService.java   |   2 +-
 .../splittabledofn/SDFFeederViaStateAndTimers.java |   2 +-
 .../fnexecution/state/GrpcStateService.java|   2 +-
 .../fnexecution/state/StateRequestHandlers.java|   2 +-
 .../GrpcContextHeaderAccessorProviderTest.java |  20 +--
 .../runners/fnexecution/ServerFactoryTest.java |   8 +-
 .../BeamFileSystemArtifactServicesTest.java|   8 +-
 .../control/FnApiControlClientPoolServiceTest.java |   4 +-
 .../control/FnApiControlClientTest.java|   2 +-
 .../fnexecution/data/GrpcDataServiceTest.java  |   8 +-
 .../jobsubmission/InMemoryJobServiceTest.java  |   4 +-
 .../logging/GrpcLoggingServiceTest.java|   6 +-
 .../StaticGrpcProvisionServiceTest.java|  

[beam] branch master updated (50a4baa -> 109f03c)

[lcwik]

This is an automated email from the ASF dual-hosted git repository.

lcwik pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 50a4baa  [BEAM-3457] Upgrade version of gogradle and add examples and 
containers to :goPrecommit
 add f9f9e87  [BEAM-4481, BEAM-4484] Start vendoring portability 
dependencies to not have dependency conflicts
 new 109f03c  [BEAM-4481, BEAM-4484] Start vendoring portability 
dependencies to not have dependency conflicts

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../org/apache/beam/gradle/BeamModulePlugin.groovy | 138 -
 examples/java/build.gradle |   2 +-
 model/fn-execution/build.gradle|  13 +-
 model/job-management/build.gradle  |  13 +-
 model/pipeline/build.gradle|   7 +-
 runners/core-construction-java/build.gradle|   6 +-
 .../core/construction/ArtifactServiceStager.java   |   6 +-
 .../beam/runners/core/construction/BeamUrns.java   |   2 +-
 .../core/construction/CoderTranslation.java|   2 +-
 .../core/construction/CombineTranslation.java  |   2 +-
 .../CreatePCollectionViewTranslation.java  |   2 +-
 .../core/construction/DisplayDataTranslation.java  |   4 +-
 .../runners/core/construction/Environments.java|   2 +-
 .../construction/PCollectionViewTranslation.java   |   2 +-
 .../core/construction/ParDoTranslation.java|   4 +-
 .../construction/PipelineOptionsTranslation.java   |   4 +-
 .../runners/core/construction/ReadTranslation.java |   4 +-
 .../core/construction/TestStreamTranslation.java   |   2 +-
 .../core/construction/WindowIntoTranslation.java   |   2 +-
 .../construction/WindowingStrategyTranslation.java |   8 +-
 .../core/construction/WriteFilesTranslation.java   |   2 +-
 .../graph/GreedyPCollectionFusers.java |   2 +-
 .../core/construction/graph/QueryablePipeline.java |   2 +-
 .../construction/ArtifactServiceStagerTest.java|   8 +-
 .../InMemoryArtifactStagerService.java |   2 +-
 .../PipelineOptionsTranslationTest.java|   6 +-
 .../construction/WindowIntoTranslationTest.java|   2 +-
 .../construction/graph/ProtoOverridesTest.java |   2 +-
 runners/core-java/build.gradle |   2 -
 runners/direct-java/build.gradle   |  19 ++-
 .../runners/direct/portable/ReferenceRunner.java   |   2 +-
 .../LocalFileSystemArtifactRetrievalService.java   |   6 +-
 .../LocalFileSystemArtifactStagerService.java  |   8 +-
 .../runners/direct/portable/job/PreparingJob.java  |   2 +-
 .../portable/job/ReferenceRunnerJobService.java|   6 +-
 ...ocalFileSystemArtifactRetrievalServiceTest.java |   4 +-
 .../LocalFileSystemArtifactStagerServiceTest.java  |  14 +--
 .../UnsupportedArtifactRetrievalServiceTest.java   |   2 +-
 .../job/ReferenceRunnerJobServiceTest.java |   4 +-
 .../FlinkBatchPortablePipelineTranslator.java  |   2 +-
 .../apache/beam/runners/flink/FlinkJobInvoker.java |   2 +-
 .../FlinkStreamingPortablePipelineTranslator.java  |   2 +-
 .../streaming/ExecutableStageDoFnOperatorTest.java |   2 +-
 .../FlinkExecutableStageFunctionTest.java  |   2 +-
 .../dataflow/DataflowPipelineTranslator.java   |   2 +-
 runners/java-fn-execution/build.gradle |   3 -
 .../apache/beam/runners/fnexecution/FnService.java |   2 +-
 .../GrpcContextHeaderAccessorProvider.java |  16 +--
 .../beam/runners/fnexecution/GrpcFnServer.java |   2 +-
 .../fnexecution/InProcessServerFactory.java|   8 +-
 .../beam/runners/fnexecution/ServerFactory.java|   8 +-
 .../BeamFileSystemArtifactRetrievalService.java|  10 +-
 .../BeamFileSystemArtifactStagingService.java  |  10 +-
 .../fnexecution/control/FnApiControlClient.java|   6 +-
 .../control/FnApiControlClientPoolService.java |   2 +-
 .../control/ProcessBundleDescriptors.java  |   2 +-
 .../runners/fnexecution/data/GrpcDataService.java  |   2 +-
 .../jobsubmission/InMemoryJobService.java  |  10 +-
 .../fnexecution/jobsubmission/JobInvoker.java  |   2 +-
 .../fnexecution/jobsubmission/JobPreparation.java  |   2 +-
 .../fnexecution/logging/GrpcLoggingService.java|   2 +-
 .../runners/fnexecution/provisioning/JobInfo.java  |   2 +-
 .../provisioning/StaticGrpcProvisionService.java   |   2 +-
 .../splittabledofn/SDFFeederViaStateAndTimers.java |   2 +-
 .../fnexecution/state/GrpcStateService.java|   2 +-
 .../fnexecution/state/StateRequestHandlers.java|   2 +-
 .../GrpcContextHeaderAccessorProviderTest.java |  20 +--
 .../runners/fnexecution/ServerFactoryTest.java |   8 +-
 .../BeamFileSystemArtifactServicesTest.java|   8 +-
 

[jira] [Work logged] (BEAM-4006) Futurize and fix python 2 compatibility for transforms subpackage

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4006?focusedWorklogId=121566=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121566
 ]

ASF GitHub Bot logged work on BEAM-4006:


Author: ASF GitHub Bot
Created on: 10/Jul/18 20:29
Start Date: 10/Jul/18 20:29
Worklog Time Spent: 10m 
  Work Description: Fematich commented on a change in pull request #5729: 
[BEAM-4006] Futurize transforms subpackage
URL: https://github.com/apache/beam/pull/5729#discussion_r201483568
 
 

 ##
 File path: sdks/python/apache_beam/transforms/util.py
 ##
 @@ -19,11 +19,17 @@
 """
 
 from __future__ import absolute_import
+from __future__ import division
 
 Review comment:
   treshold variable is now an int and Python PostCommit succeeds


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121566)
Time Spent: 5h 50m  (was: 5h 40m)

> Futurize and fix python 2 compatibility for transforms subpackage
> -
>
> Key: BEAM-4006
> URL: https://issues.apache.org/jira/browse/BEAM-4006
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Robbe
>Assignee: Matthias Feys
>Priority: Major
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (BEAM-4724) Maven Go build broken

[Henning Rohde (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539186#comment-16539186
 ] 

Henning Rohde edited comment on BEAM-4724 at 7/10/18 8:25 PM:
--

Poms have now been deleted, so there is no longer any Maven build.


was (Author: herohde):
Poms have now been deleted.

> Maven Go build broken
> -
>
> Key: BEAM-4724
> URL: https://issues.apache.org/jira/browse/BEAM-4724
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Reporter: Henning Rohde
>Assignee: Henning Rohde
>Priority: Major
> Fix For: 2.6.0
>
>
> Reported on stack overflow:
> https://stackoverflow.com/questions/51137744/build-failure-of-apache-beam-2-6-0-snapshot
> Repros on mac:
> $ mvn clean verify
> [...]
> [ERROR] Failed to execute goal 
> com.igormaznitsa:mvn-golang-wrapper:2.1.7:build (go-build-linux-amd64) on 
> project beam-sdks-go: Can't find generated target file : 
> /Users/herohde/go/src/github.com/apache/beam/sdks/go/target/linux_amd64/beamctl
>  -> [Help 1]
> [ERROR] 
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR] 
> [ERROR] For more information about the errors and possible solutions, please 
> read the following articles:
> [ERROR] [Help 1] 
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [ERROR] 
> [ERROR] After correcting the problems, you can resume the build with the 
> command
> [ERROR]   mvn  -rf :beam-sdks-go



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-4724) Maven Go build broken

[Henning Rohde (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539186#comment-16539186
 ] 

Henning Rohde commented on BEAM-4724:
-

Poms have now been deleted.

> Maven Go build broken
> -
>
> Key: BEAM-4724
> URL: https://issues.apache.org/jira/browse/BEAM-4724
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Reporter: Henning Rohde
>Assignee: Henning Rohde
>Priority: Major
> Fix For: 2.6.0
>
>
> Reported on stack overflow:
> https://stackoverflow.com/questions/51137744/build-failure-of-apache-beam-2-6-0-snapshot
> Repros on mac:
> $ mvn clean verify
> [...]
> [ERROR] Failed to execute goal 
> com.igormaznitsa:mvn-golang-wrapper:2.1.7:build (go-build-linux-amd64) on 
> project beam-sdks-go: Can't find generated target file : 
> /Users/herohde/go/src/github.com/apache/beam/sdks/go/target/linux_amd64/beamctl
>  -> [Help 1]
> [ERROR] 
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR] 
> [ERROR] For more information about the errors and possible solutions, please 
> read the following articles:
> [ERROR] [Help 1] 
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [ERROR] 
> [ERROR] After correcting the problems, you can resume the build with the 
> command
> [ERROR]   mvn  -rf :beam-sdks-go



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-4724) Maven Go build broken

[Henning Rohde (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Henning Rohde resolved BEAM-4724.
-
   Resolution: Won't Fix
Fix Version/s: 2.6.0

> Maven Go build broken
> -
>
> Key: BEAM-4724
> URL: https://issues.apache.org/jira/browse/BEAM-4724
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Reporter: Henning Rohde
>Assignee: Henning Rohde
>Priority: Major
> Fix For: 2.6.0
>
>
> Reported on stack overflow:
> https://stackoverflow.com/questions/51137744/build-failure-of-apache-beam-2-6-0-snapshot
> Repros on mac:
> $ mvn clean verify
> [...]
> [ERROR] Failed to execute goal 
> com.igormaznitsa:mvn-golang-wrapper:2.1.7:build (go-build-linux-amd64) on 
> project beam-sdks-go: Can't find generated target file : 
> /Users/herohde/go/src/github.com/apache/beam/sdks/go/target/linux_amd64/beamctl
>  -> [Help 1]
> [ERROR] 
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR] 
> [ERROR] For more information about the errors and possible solutions, please 
> read the following articles:
> [ERROR] [Help 1] 
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [ERROR] 
> [ERROR] After correcting the problems, you can resume the build with the 
> command
> [ERROR]   mvn  -rf :beam-sdks-go



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4751) Finish io futurize stage 2: fix the missing pylint3 check in tox.ini

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4751?focusedWorklogId=121560=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121560
 ]

ASF GitHub Bot logged work on BEAM-4751:


Author: ASF GitHub Bot
Created on: 10/Jul/18 20:16
Start Date: 10/Jul/18 20:16
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5916: [BEAM-4751] 
fix missing pylint3 check for io subpackage
URL: https://github.com/apache/beam/pull/5916#issuecomment-403951741
 
 
   Run Python PostCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121560)
Time Spent: 20m  (was: 10m)

> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini
> 
>
> Key: BEAM-4751
> URL: https://issues.apache.org/jira/browse/BEAM-4751
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Matthias Feys
>Assignee: Matthias Feys
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4751) Finish io futurize stage 2: fix the missing pylint3 check in tox.ini

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4751?focusedWorklogId=121561=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121561
 ]

ASF GitHub Bot logged work on BEAM-4751:


Author: ASF GitHub Bot
Created on: 10/Jul/18 20:16
Start Date: 10/Jul/18 20:16
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5916: [BEAM-4751] 
fix missing pylint3 check for io subpackage
URL: https://github.com/apache/beam/pull/5916#issuecomment-403951830
 
 
   Run Python Dataflow ValidatesRunner


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121561)
Time Spent: 0.5h  (was: 20m)

> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini
> 
>
> Key: BEAM-4751
> URL: https://issues.apache.org/jira/browse/BEAM-4751
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Matthias Feys
>Assignee: Matthias Feys
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3457) Migrate Go Jenkins PreCommit to Gradle

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3457?focusedWorklogId=121559=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121559
 ]

ASF GitHub Bot logged work on BEAM-3457:


Author: ASF GitHub Bot
Created on: 10/Jul/18 20:08
Start Date: 10/Jul/18 20:08
Worklog Time Spent: 10m 
  Work Description: lukecwik closed pull request #5917: [BEAM-3457] Upgrade 
version of gogradle and add examples and containers to :goPrecommit
URL: https://github.com/apache/beam/pull/5917
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/build.gradle b/build.gradle
index 36aee89dccf..af2c411ed18 100644
--- a/build.gradle
+++ b/build.gradle
@@ -56,7 +56,7 @@ buildscript {
 classpath "gradle.plugin.org.nosphere.apache:creadur-rat-gradle:0.3.1" 
 // Enable Apache license enforcement
 classpath "com.commercehub.gradle.plugin:gradle-avro-plugin:0.11.0"
 // Enable Avro code generation
 classpath "com.diffplug.spotless:spotless-plugin-gradle:3.6.0" 
 // Enable a code formatting plugin
-classpath "gradle.plugin.com.github.blindpirate:gogradle:0.9"  
 // Enable Go code compilation
+classpath "gradle.plugin.com.github.blindpirate:gogradle:0.10" 
 // Enable Go code compilation
 classpath "gradle.plugin.com.palantir.gradle.docker:gradle-docker:0.13.0"  
 // Enable building Docker containers
 classpath "cz.malohlava:visteg:1.0.3"  
 // Enable generating Gradle task dependencies as 
".dot" files
 classpath "com.github.jengelman.gradle.plugins:shadow:2.0.4"   
 // Enable shading Java dependencies
@@ -174,16 +174,15 @@ task goPreCommit() {
   dependsOn ":rat"
   dependsOn ":beam-sdks-go:test"
 
-  // TODO: reenable when https://github.com/gogradle/gogradle/issues/225 is 
fixed
-  // dependsOn ":beam-sdks-go-examples:build"
-  // dependsOn ":beam-sdks-go-test:build"
+  dependsOn ":beam-sdks-go-examples:build"
+  dependsOn ":beam-sdks-go-test:build"
 
   // Ensure all container Go boot code builds as well.
-  // dependsOn ":beam-sdks-java-container:build"
-  // dependsOn ":beam-sdks-python-container:build"
-  // dependsOn ":beam-sdks-go-container:build"
-  // dependsOn ":beam-runners-gcp-gcemd:build"
-  // dependsOn ":beam-runners-gcp-gcsproxy:build"
+  dependsOn ":beam-sdks-java-container:build"
+  dependsOn ":beam-sdks-python-container:build"
+  dependsOn ":beam-sdks-go-container:build"
+  dependsOn ":beam-runners-gcp-gcemd:build"
+  dependsOn ":beam-runners-gcp-gcsproxy:build"
 }
 
 task goPostCommit() {


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121559)
Time Spent: 5h 50m  (was: 5h 40m)

> Migrate Go Jenkins PreCommit to Gradle
> --
>
> Key: BEAM-3457
> URL: https://issues.apache.org/jira/browse/BEAM-3457
> Project: Beam
>  Issue Type: Sub-task
>  Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Henning Rohde
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>
> The code is here: 
> https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_beam_PreCommit_Go_MavenInstall.groovy



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam] branch master updated (e65015f -> 50a4baa)

[lcwik]

This is an automated email from the ASF dual-hosted git repository.

lcwik pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from e65015f  Merge pull request #5891 from robertwb/cleanup
 add b824531  [BEAM-3457] Upgrade version of gogradle and add examples and 
containers to :goPrecommit
 add 50a4baa  [BEAM-3457] Upgrade version of gogradle and add examples and 
containers to :goPrecommit

No new revisions were added by this update.

Summary of changes:
 build.gradle | 17 -
 1 file changed, 8 insertions(+), 9 deletions(-)

[beam] branch master updated (4c81d4e -> e65015f)

[ccy]

This is an automated email from the ASF dual-hosted git repository.

ccy pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 4c81d4e  Merge pull request #5843 from cclauss/Define-cmp-in-Python3
 add 684330f  object() takes no parameters
 new e65015f  Merge pull request #5891 from robertwb/cleanup

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 sdks/python/apache_beam/io/gcp/gcsio.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

[beam] 01/01: Merge pull request #5891 from robertwb/cleanup

[ccy]

This is an automated email from the ASF dual-hosted git repository.

ccy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit e65015fba11d55c8b13cd31dcd253c66d44f58f3
Merge: 4c81d4e 684330f
Author: Charles Chen 
AuthorDate: Tue Jul 10 13:06:41 2018 -0700

Merge pull request #5891 from robertwb/cleanup

object() takes no parameters

 sdks/python/apache_beam/io/gcp/gcsio.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

[jira] [Work logged] (BEAM-3457) Migrate Go Jenkins PreCommit to Gradle

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3457?focusedWorklogId=121555=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121555
 ]

ASF GitHub Bot logged work on BEAM-3457:


Author: ASF GitHub Bot
Created on: 10/Jul/18 19:35
Start Date: 10/Jul/18 19:35
Worklog Time Spent: 10m 
  Work Description: alanmyrvold commented on issue #5917: [BEAM-3457] 
Upgrade version of gogradle and add examples and containers to :goPrecommit
URL: https://github.com/apache/beam/pull/5917#issuecomment-403940724
 
 
   FYI: @herohde @lostluck 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121555)
Time Spent: 5h 40m  (was: 5.5h)

> Migrate Go Jenkins PreCommit to Gradle
> --
>
> Key: BEAM-3457
> URL: https://issues.apache.org/jira/browse/BEAM-3457
> Project: Beam
>  Issue Type: Sub-task
>  Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Henning Rohde
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 5h 40m
>  Remaining Estimate: 0h
>
> The code is here: 
> https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_beam_PreCommit_Go_MavenInstall.groovy



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Jenkins build is back to normal : beam_PostCommit_Java_GradleBuild #1044

[Apache Jenkins Server]

See 

[jira] [Work logged] (BEAM-3457) Migrate Go Jenkins PreCommit to Gradle

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3457?focusedWorklogId=121552=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121552
 ]

ASF GitHub Bot logged work on BEAM-3457:


Author: ASF GitHub Bot
Created on: 10/Jul/18 19:31
Start Date: 10/Jul/18 19:31
Worklog Time Spent: 10m 
  Work Description: alanmyrvold commented on issue #5917: [BEAM-3457] 
Upgrade version of gogradle and add examples and containers to :goPrecommit
URL: https://github.com/apache/beam/pull/5917#issuecomment-403939586
 
 
   +R: @jkff


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121552)
Time Spent: 5.5h  (was: 5h 20m)

> Migrate Go Jenkins PreCommit to Gradle
> --
>
> Key: BEAM-3457
> URL: https://issues.apache.org/jira/browse/BEAM-3457
> Project: Beam
>  Issue Type: Sub-task
>  Components: build-system, testing
>Reporter: Luke Cwik
>Assignee: Henning Rohde
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 5.5h
>  Remaining Estimate: 0h
>
> The code is here: 
> https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_beam_PreCommit_Go_MavenInstall.groovy



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (BEAM-4752) Import error in apache_beam.internal.pickler: "'module' object has no attribute 'dill'"

[Barry Hart (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Barry Hart closed BEAM-4752.

   Resolution: Invalid
Fix Version/s: 2.4.0

> Import error in apache_beam.internal.pickler: "'module' object has no 
> attribute 'dill'"
> ---
>
> Key: BEAM-4752
> URL: https://issues.apache.org/jira/browse/BEAM-4752
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Affects Versions: 2.4.0
> Environment: CentOS Linux release 7.4.1708
> Python 2.7.13
>Reporter: Barry Hart
>Assignee: Ahmet Altay
>Priority: Major
> Fix For: 2.4.0
>
>
> I'm seeing the following error (stack trace below). I looked at the module 
> structure of the {{dill}} library, and it does not have a {{dill}} submodule 
> (although it *does* have a {{_dill}} submodule). I think the correct way to 
> reference {{Pickler}} is simply {{dill.Pickler.}}
> {noformat}
> Traceback (most recent call last):
>   File "script/beam_run_model.py", line 29, in 
> import apache_beam as beam
>   File 
> "/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/__init__.py",
>  line 84, in 
> import apache_beam.internal.pickler
>   File 
> "/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/internal/pickler.py",
>  line 107, in 
> dill.dill.Pickler.dispatch[type])
> AttributeError: 'module' object has no attribute 'dill'{noformat}
> Oddly, I have successfully used Beam 2.4.0 in the past with this version of 
> Dill.  ¯_(ツ)_/¯



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-4752) Import error in apache_beam.internal.pickler: "'module' object has no attribute 'dill'"

[Barry Hart (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539141#comment-16539141
 ] 

Barry Hart commented on BEAM-4752:
--

Apparently something odd happened when I was installing local Python libraries. 
I suspect I got a newer version of {{dill}}, as newer versions have 
{{_dill.py}} but not {{dill.py}}.

Closing...

> Import error in apache_beam.internal.pickler: "'module' object has no 
> attribute 'dill'"
> ---
>
> Key: BEAM-4752
> URL: https://issues.apache.org/jira/browse/BEAM-4752
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Affects Versions: 2.4.0
> Environment: CentOS Linux release 7.4.1708
> Python 2.7.13
>Reporter: Barry Hart
>Assignee: Ahmet Altay
>Priority: Major
>
> I'm seeing the following error (stack trace below). I looked at the module 
> structure of the {{dill}} library, and it does not have a {{dill}} submodule 
> (although it *does* have a {{_dill}} submodule). I think the correct way to 
> reference {{Pickler}} is simply {{dill.Pickler.}}
> {noformat}
> Traceback (most recent call last):
>   File "script/beam_run_model.py", line 29, in 
> import apache_beam as beam
>   File 
> "/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/__init__.py",
>  line 84, in 
> import apache_beam.internal.pickler
>   File 
> "/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/internal/pickler.py",
>  line 107, in 
> dill.dill.Pickler.dispatch[type])
> AttributeError: 'module' object has no attribute 'dill'{noformat}
> Oddly, I have successfully used Beam 2.4.0 in the past with this version of 
> Dill.  ¯_(ツ)_/¯



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3457) Migrate Go Jenkins PreCommit to Gradle

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3457?focusedWorklogId=121548=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121548
 ]

ASF GitHub Bot logged work on BEAM-3457:


Author: ASF GitHub Bot
Created on: 10/Jul/18 19:26
Start Date: 10/Jul/18 19:26
Worklog Time Spent: 10m 
  Work Description: alanmyrvold opened a new pull request #5917: 
[BEAM-3457] Upgrade version of gogradle and add examples and containers to 
:goPrecommit
URL: https://github.com/apache/beam/pull/5917
 
 
   Upgrade version of gogradle and add examples and containers to :goPrecommit
   
   The new version of gogradle includes fix for 
https://github.com/gogradle/gogradle/issues/225 OverlappingFileLockException 
resolving dependencies in DefaultGlobalCacheManager.java in 
https://github.com/gogradle/gogradle/pull/227
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121548)
Time Spent: 5h 20m  (was: 5h 10m)

> Migrate Go Jenkins PreCommit to Gradle
> --
>
> Key: BEAM-3457
> URL: https://issues.apache.org/jira/browse/BEAM-3457
> Project: Beam
>  Issue Type: Sub-task
>   

[jira] [Resolved] (BEAM-4590) Beam SQL JDBC driver should set User-agent PipelineOption

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-4590.
--
   Resolution: Fixed
Fix Version/s: 2.6.0

> Beam SQL JDBC driver should set User-agent PipelineOption
> -
>
> Key: BEAM-4590
> URL: https://issues.apache.org/jira/browse/BEAM-4590
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Kenneth Knowles
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.6.0
>
>
> It makes sense to clearly call out what is a SQL pipeline, and this is 
> already a feature of the Java SDK at a global level (for all of its 
> interactions with any service).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-3634) [SQL] Refactor BeamRelNodes into PTransforms

[Andrew Pilloud (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Pilloud resolved BEAM-3634.
--
   Resolution: Fixed
Fix Version/s: 2.6.0

> [SQL] Refactor BeamRelNodes into PTransforms
> 
>
> Key: BEAM-3634
> URL: https://issues.apache.org/jira/browse/BEAM-3634
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: 2.6.0
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> BeamRelNode exposes PCollection buildBeamPipeline() which builds 
> a pipeline when parsing.
> It feels like it should instead implement a 
> PTransform, PCollection> which would 
> receive a prepared PCollection, and apply sub-expressions instead of manually 
> invoking expression evaluation to get the input.
> And maybe consider building it lazily.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4751) Finish io futurize stage 2: fix the missing pylint3 check in tox.ini

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4751?focusedWorklogId=121544=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121544
 ]

ASF GitHub Bot logged work on BEAM-4751:


Author: ASF GitHub Bot
Created on: 10/Jul/18 19:08
Start Date: 10/Jul/18 19:08
Worklog Time Spent: 10m 
  Work Description: Fematich opened a new pull request #5916: [BEAM-4751] 
fix missing pylint3 check for io subpackage
URL: https://github.com/apache/beam/pull/5916
 
 
   This pull request finishes the preparation of the io subpackage for Python 3 
support. The work for io was started in #5715, however the py27-lint3 test was 
missing in tox.ini, resulting in some open issues and missing checks for new 
PRs.
   
   The PR is part of a series in which all subpackages will be updated using 
the same approach.
   This approach has been documented 
[here](https://docs.google.com/document/d/1xDG0MWVlDKDPu_IW9gtMvxi2S9I0GB0VDTkPhjXT0nE/edit)
 and the first pull request in the series (Futurize coders subpackage) 
demonstrating this approach can be found at #5053.
   
   R: @aaltay @tvalentyn @RobbeSneyders @charlesccychen @superbobry
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121544)
Time Spent: 10m
Remaining Estimate: 0h

> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini
> 
>
> Key: BEAM-4751
> URL: https://issues.apache.org/jira/browse/BEAM-4751
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Matthias Feys
>Assignee: Matthias Feys
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Finish io futurize stage 2: fix 

Build failed in Jenkins: beam_PreCommit_Java_Cron #89

[Apache Jenkins Server]

See 


Changes:

[Pablo] Improving signing of all published artifacts and adding test publication

[Pablo] Removing extra publication

[cclauss] [BEAM-3761] Define cmp() in Python 3

[cclauss] [BEAM-3959] Add Python 3 undefined names to flake8

--
[...truncated 17.12 MB...]
Jul 10, 2018 7:04:07 PM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-10T19:04:04.858Z: Unzipping flatten s13 for input 
s12.org.apache.beam.sdk.values.PCollection.:349#1d275f544daf228c
Jul 10, 2018 7:04:07 PM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-10T19:04:04.883Z: Fusing unzipped copy of 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Add void 
key/AddKeys/Map, through flatten 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections,
 into producer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Jul 10, 2018 7:04:07 PM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-10T19:04:04.914Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable
 into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow
Jul 10, 2018 7:04:07 PM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-10T19:04:04.927Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
 into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair
 with random key
Jul 10, 2018 7:04:07 PM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-10T19:04:04.957Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
 into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Jul 10, 2018 7:04:07 PM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-10T19:04:04.985Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
 into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Jul 10, 2018 7:04:07 PM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-10T19:04:05.009Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
 into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Jul 10, 2018 7:04:07 PM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-10T19:04:05.038Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Drop 
key/Values/Map into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable
Jul 10, 2018 7:04:07 PM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-10T19:04:05.070Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Gather 
bundles into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Drop 
key/Values/Map
Jul 10, 2018 7:04:07 PM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-10T19:04:05.104Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair
 with random key into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Gather 
bundles
Jul 10, 2018 7:04:07 PM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-10T19:04:05.132Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
 into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Jul 10, 2018 7:04:07 PM 
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-07-10T19:04:05.162Z: Fusing consumer 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify
 into 
WriteOneFilePerWindow/TextIO.Write/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign
Jul 10, 2018 7:04:07 PM 

[jira] [Created] (BEAM-4753) Reuse validateShadedJarDoesntLeakNonOrgApacheBeamClasses across all subprojects which perform shading

[Luke Cwik (JIRA)]

Luke Cwik created BEAM-4753:
---

 Summary: Reuse validateShadedJarDoesntLeakNonOrgApacheBeamClasses 
across all subprojects which perform shading
 Key: BEAM-4753
 URL: https://issues.apache.org/jira/browse/BEAM-4753
 Project: Beam
  Issue Type: Bug
  Components: build-system
Reporter: Luke Cwik


This task prevents artifacts from incorrectly packaging dependencies without 
relocating them simplifying any code reviews which perform shading.

 

Reuse this task in all modules which perform shading.

 

Note that it may be preferable to start vendoring guava (BEAM-3608) before 
doing this task since guava is the most common shaded dependency.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3761) Fix Python 3 cmp function

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3761?focusedWorklogId=121537=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121537
 ]

ASF GitHub Bot logged work on BEAM-3761:


Author: ASF GitHub Bot
Created on: 10/Jul/18 18:45
Start Date: 10/Jul/18 18:45
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #5843: [BEAM-3761] Define 
cmp() in Python 3
URL: https://github.com/apache/beam/pull/5843#issuecomment-403926630
 
 
   Thanks all!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121537)
Time Spent: 14h 20m  (was: 14h 10m)

> Fix Python 3 cmp function
> -
>
> Key: BEAM-3761
> URL: https://issues.apache.org/jira/browse/BEAM-3761
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: holdenk
>Priority: Major
>  Time Spent: 14h 20m
>  Remaining Estimate: 0h
>
> Various functions don't exist in Python 3 that did in python 2. This Jira is 
> to fix the use of cmp (which often will involve rewriting __cmp__ as well).
>  
> Note: there are existing PRs for basestring and unicode ( 
> [https://github.com/apache/beam/pull/4697|https://github.com/apache/beam/pull/4697,]
>  , [https://github.com/apache/beam/pull/4730] )
>  
> Note once all of the missing names/functions are fixed we can enable F821 in 
> falke8 python 3.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-2810) Consider a faster Avro library in Python

[Barry Hart (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539079#comment-16539079
 ] 

Barry Hart commented on BEAM-2810:
--

I am a fairly frequent contributor to the fastavro library. I'm happy to try 
and help if it needs some tweaks.

FWIW, the library is pretty mature and works well for our project. It has 
several small changes from time to time, but generally nothing major. Probably 
the last big changes were late 2017, when we did some Cython work to make reads 
about 30% faster and writes about 2x faster. 

> Consider a faster Avro library in Python
> 
>
> Key: BEAM-2810
> URL: https://issues.apache.org/jira/browse/BEAM-2810
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Eugene Kirpichov
>Assignee: Ryan Williams
>Priority: Major
>  Time Spent: 5h 40m
>  Remaining Estimate: 0h
>
> https://stackoverflow.com/questions/45870789/bottleneck-on-data-source
> Seems like this job is reading Avro files (exported by BigQuery) at about 2 
> MB/s.
> We use the standard Python "avro" library which is apparently known to be 
> very slow (10x+ slower than Java) 
> http://apache-avro.679487.n3.nabble.com/Avro-decode-very-slow-in-Python-td4034422.html,
>  and there are alternatives e.g. https://pypi.python.org/pypi/fastavro/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3761) Fix Python 3 cmp function

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3761?focusedWorklogId=121535=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121535
 ]

ASF GitHub Bot logged work on BEAM-3761:


Author: ASF GitHub Bot
Created on: 10/Jul/18 18:39
Start Date: 10/Jul/18 18:39
Worklog Time Spent: 10m 
  Work Description: charlesccychen closed pull request #5843: [BEAM-3761] 
Define cmp() in Python 3
URL: https://github.com/apache/beam/pull/5843
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/apache_beam/transforms/window.py 
b/sdks/python/apache_beam/transforms/window.py
index a67e2cf1b50..5bc047b48c7 100644
--- a/sdks/python/apache_beam/transforms/window.py
+++ b/sdks/python/apache_beam/transforms/window.py
@@ -53,6 +53,7 @@
 
 from google.protobuf import duration_pb2
 from google.protobuf import timestamp_pb2
+from past.builtins import cmp
 
 from apache_beam.coders import coders
 from apache_beam.portability import common_urns
diff --git a/sdks/python/scripts/run_mini_py3lint.sh 
b/sdks/python/scripts/run_mini_py3lint.sh
index 63e71771d73..0729c7ba2cf 100755
--- a/sdks/python/scripts/run_mini_py3lint.sh
+++ b/sdks/python/scripts/run_mini_py3lint.sh
@@ -48,5 +48,4 @@ if test $# -gt 0; then
 fi
 
 echo "Running flake8 for module $MODULE:"
-# TODO(BEAM-3959): Add F821 (undefined names) as soon as that test passes
-flake8 $MODULE --count --select=E9,F822,F823 --show-source --statistics
+flake8 $MODULE --count --select=E9,F821,F822,F823 --show-source --statistics


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121535)
Time Spent: 14h 10m  (was: 14h)

> Fix Python 3 cmp function
> -
>
> Key: BEAM-3761
> URL: https://issues.apache.org/jira/browse/BEAM-3761
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: holdenk
>Priority: Major
>  Time Spent: 14h 10m
>  Remaining Estimate: 0h
>
> Various functions don't exist in Python 3 that did in python 2. This Jira is 
> to fix the use of cmp (which often will involve rewriting __cmp__ as well).
>  
> Note: there are existing PRs for basestring and unicode ( 
> [https://github.com/apache/beam/pull/4697|https://github.com/apache/beam/pull/4697,]
>  , [https://github.com/apache/beam/pull/4730] )
>  
> Note once all of the missing names/functions are fixed we can enable F821 in 
> falke8 python 3.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-4752) Import error in apache_beam.internal.pickler: "'module' object has no attribute 'dill'"

[Barry Hart (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539075#comment-16539075
 ] 

Barry Hart commented on BEAM-4752:
--

I can work around the problem by doing the following before {{import 
apache_beam}}.

 
{code:java}
import dill
if not hasattr(dill, 'dill'):
dill.dill = dill._dill{code}

> Import error in apache_beam.internal.pickler: "'module' object has no 
> attribute 'dill'"
> ---
>
> Key: BEAM-4752
> URL: https://issues.apache.org/jira/browse/BEAM-4752
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Affects Versions: 2.4.0
> Environment: CentOS Linux release 7.4.1708
> Python 2.7.13
>Reporter: Barry Hart
>Assignee: Ahmet Altay
>Priority: Major
>
> I'm seeing the following error (stack trace below). I looked at the module 
> structure of the {{dill}} library, and it does not have a {{dill}} submodule 
> (although it *does* have a {{_dill}} submodule). I think the correct way to 
> reference {{Pickler}} is simply {{dill.Pickler.}}
> {noformat}
> Traceback (most recent call last):
>   File "script/beam_run_model.py", line 29, in 
> import apache_beam as beam
>   File 
> "/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/__init__.py",
>  line 84, in 
> import apache_beam.internal.pickler
>   File 
> "/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/internal/pickler.py",
>  line 107, in 
> dill.dill.Pickler.dispatch[type])
> AttributeError: 'module' object has no attribute 'dill'{noformat}
> Oddly, I have successfully used Beam 2.4.0 in the past with this version of 
> Dill.  ¯_(ツ)_/¯



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3761) Fix Python 3 cmp function

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3761?focusedWorklogId=121534=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121534
 ]

ASF GitHub Bot logged work on BEAM-3761:


Author: ASF GitHub Bot
Created on: 10/Jul/18 18:38
Start Date: 10/Jul/18 18:38
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5843: [BEAM-3761] 
Define cmp() in Python 3
URL: https://github.com/apache/beam/pull/5843#issuecomment-403924594
 
 
   This looks fine, and the ValidatesRunner tests pass, so I'm going to merge 
even though Jenkins seems to buggy right now for the PostSubmit run.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121534)
Time Spent: 14h  (was: 13h 50m)

> Fix Python 3 cmp function
> -
>
> Key: BEAM-3761
> URL: https://issues.apache.org/jira/browse/BEAM-3761
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: holdenk
>Priority: Major
>  Time Spent: 14h
>  Remaining Estimate: 0h
>
> Various functions don't exist in Python 3 that did in python 2. This Jira is 
> to fix the use of cmp (which often will involve rewriting __cmp__ as well).
>  
> Note: there are existing PRs for basestring and unicode ( 
> [https://github.com/apache/beam/pull/4697|https://github.com/apache/beam/pull/4697,]
>  , [https://github.com/apache/beam/pull/4730] )
>  
> Note once all of the missing names/functions are fixed we can enable F821 in 
> falke8 python 3.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-4752) Import error in apache_beam.internal.pickler: "'module' object has no attribute 'dill'"

[Barry Hart (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Barry Hart updated BEAM-4752:
-
Description: 
I'm seeing the following error (stack trace below). I looked at the module 
structure of the {{dill}} library, and it does not have a {{dill}} submodule 
(although it *does* have a {{_dill}} submodule). I think the correct way to 
reference {{Pickler}} is simply {{dill.Pickler.}}
{noformat}
Traceback (most recent call last):
  File "script/beam_run_model.py", line 29, in 
import apache_beam as beam
  File 
"/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/__init__.py",
 line 84, in 
import apache_beam.internal.pickler
  File 
"/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/internal/pickler.py",
 line 107, in 
dill.dill.Pickler.dispatch[type])
AttributeError: 'module' object has no attribute 'dill'{noformat}
Oddly, I have successfully used Beam 2.4.0 in the past with this version of 
Dill.  ¯_(ツ)_/¯

  was:
I'm seeing the following error. The {{dill}} library has no {{dill}} submodule. 
I think the correct way to reference {{Pickler}} is simply {{dill.Pickler.}}
{noformat}
Traceback (most recent call last):
  File "script/beam_run_model.py", line 29, in 
import apache_beam as beam
  File 
"/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/__init__.py",
 line 84, in 
import apache_beam.internal.pickler
  File 
"/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/internal/pickler.py",
 line 107, in 
dill.dill.Pickler.dispatch[type])
AttributeError: 'module' object has no attribute 'dill'{noformat}
Oddly, I have successfully used Beam 2.4.0 in the past with this version of 
Dill.  ¯\_(ツ)_/¯


> Import error in apache_beam.internal.pickler: "'module' object has no 
> attribute 'dill'"
> ---
>
> Key: BEAM-4752
> URL: https://issues.apache.org/jira/browse/BEAM-4752
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Affects Versions: 2.4.0
> Environment: CentOS Linux release 7.4.1708
> Python 2.7.13
>Reporter: Barry Hart
>Assignee: Ahmet Altay
>Priority: Major
>
> I'm seeing the following error (stack trace below). I looked at the module 
> structure of the {{dill}} library, and it does not have a {{dill}} submodule 
> (although it *does* have a {{_dill}} submodule). I think the correct way to 
> reference {{Pickler}} is simply {{dill.Pickler.}}
> {noformat}
> Traceback (most recent call last):
>   File "script/beam_run_model.py", line 29, in 
> import apache_beam as beam
>   File 
> "/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/__init__.py",
>  line 84, in 
> import apache_beam.internal.pickler
>   File 
> "/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/internal/pickler.py",
>  line 107, in 
> dill.dill.Pickler.dispatch[type])
> AttributeError: 'module' object has no attribute 'dill'{noformat}
> Oddly, I have successfully used Beam 2.4.0 in the past with this version of 
> Dill.  ¯_(ツ)_/¯



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-4752) Import error in apache_beam.internal.pickler: "'module' object has no attribute 'dill'"

[Barry Hart (JIRA)]

Barry Hart created BEAM-4752:


 Summary: Import error in apache_beam.internal.pickler: "'module' 
object has no attribute 'dill'"
 Key: BEAM-4752
 URL: https://issues.apache.org/jira/browse/BEAM-4752
 Project: Beam
  Issue Type: Bug
  Components: sdk-py-core
Affects Versions: 2.4.0
 Environment: CentOS Linux release 7.4.1708
Python 2.7.13
Reporter: Barry Hart
Assignee: Ahmet Altay


I'm seeing the following error. The {{dill}} library has no {{dill}} submodule. 
I think the correct way to reference {{Pickler}} is simply {{dill.Pickler.}}
{noformat}
Traceback (most recent call last):
  File "script/beam_run_model.py", line 29, in 
import apache_beam as beam
  File 
"/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/__init__.py",
 line 84, in 
import apache_beam.internal.pickler
  File 
"/usr/local/pyenv/versions/2.7.13/lib/python2.7/site-packages/apache_beam/internal/pickler.py",
 line 107, in 
dill.dill.Pickler.dispatch[type])
AttributeError: 'module' object has no attribute 'dill'{noformat}
Oddly, I have successfully used Beam 2.4.0 in the past with this version of 
Dill.  ¯\_(ツ)_/¯



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-4751) Finish io futurize stage 2: fix the missing pylint3 check in tox.ini

[Matthias Feys (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Feys updated BEAM-4751:

Issue Type: Sub-task  (was: Task)
Parent: BEAM-1251

> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini
> 
>
> Key: BEAM-4751
> URL: https://issues.apache.org/jira/browse/BEAM-4751
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Matthias Feys
>Assignee: Matthias Feys
>Priority: Major
>
> Finish io futurize stage 2: fix the missing pylint3 check in tox.ini



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-4751) Finish io futurize stage 2: fix the missing pylint3 check in tox.ini

[Matthias Feys (JIRA)]

Matthias Feys created BEAM-4751:
---

 Summary: Finish io futurize stage 2: fix the missing pylint3 check 
in tox.ini
 Key: BEAM-4751
 URL: https://issues.apache.org/jira/browse/BEAM-4751
 Project: Beam
  Issue Type: Task
  Components: sdk-py-core
Reporter: Matthias Feys
Assignee: Matthias Feys


Finish io futurize stage 2: fix the missing pylint3 check in tox.ini



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)