date:20180314

See

Jenkins build became unstable: beam_PostCommit_Java_ValidatesRunner_Apex #3695

See

Jenkins build is back to stable : beam_PostCommit_Java_ValidatesRunner_Flink #5237

See

[jira] [Work logged] (BEAM-3785) [SQL] Add support for arrays


 [ 
https://issues.apache.org/jira/browse/BEAM-3785?focusedWorklogId=80714=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80714
 ]

ASF GitHub Bot logged work on BEAM-3785:


Author: ASF GitHub Bot
Created on: 15/Mar/18 04:36
Start Date: 15/Mar/18 04:36
Worklog Time Spent: 10m 
  Work Description: akedin commented on issue #4863: [BEAM-3785] Support 
dot operator
URL: https://github.com/apache/beam/pull/4863#issuecomment-373258278
 
 
   rebased


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80714)
Time Spent: 3h 20m  (was: 3h 10m)

> [SQL] Add support for arrays
> 
>
> Key: BEAM-3785
> URL: https://issues.apache.org/jira/browse/BEAM-3785
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Anton Kedin
>Priority: Major
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> Support fields of Array type



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-2281) call SqlFunctions in operator implementation


 [ 
https://issues.apache.org/jira/browse/BEAM-2281?focusedWorklogId=80713=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80713
 ]

ASF GitHub Bot logged work on BEAM-2281:


Author: ASF GitHub Bot
Created on: 15/Mar/18 04:32
Start Date: 15/Mar/18 04:32
Worklog Time Spent: 10m 
  Work Description: XuMingmin commented on issue #4865: [BEAM-2281][Sql] 
Use SqlFunctions.toBigDecimal not toString
URL: https://github.com/apache/beam/pull/4865#issuecomment-373257869
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80713)
Time Spent: 1.5h  (was: 1h 20m)

> call SqlFunctions in operator implementation
> 
>
> Key: BEAM-2281
> URL: https://issues.apache.org/jira/browse/BEAM-2281
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Xu Mingmin
>Priority: Major
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Calcite has a collections of functions in 
> {{org.apache.calcite.runtime.SqlFunctions}}. It sounds a good source to 
> leverage when adding operators as {{BeamSqlExpression}}. 
> [~xumingming] [~app-tarush], any comments?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3785) [SQL] Add support for arrays


 [ 
https://issues.apache.org/jira/browse/BEAM-3785?focusedWorklogId=80706=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80706
 ]

ASF GitHub Bot logged work on BEAM-3785:


Author: ASF GitHub Bot
Created on: 15/Mar/18 04:20
Start Date: 15/Mar/18 04:20
Worklog Time Spent: 10m 
  Work Description: XuMingmin closed pull request #4857: [BEAM-3785][SQL] 
Add support for arrays of rows
URL: https://github.com/apache/beam/pull/4857
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/RowSqlType.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/RowSqlType.java
index 77eda6a84de..fc16c848db2 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/RowSqlType.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/RowSqlType.java
@@ -102,9 +102,18 @@ public Builder withTimestampField(String fieldName) {
   return withField(fieldName, SqlTypeCoders.TIMESTAMP);
 }
 
+/**
+ * Adds an ARRAY field with elements of {@code elementCoder}.
+ */
 public Builder withArrayField(String fieldName, SqlTypeCoder elementCoder) 
{
   return withField(fieldName, SqlTypeCoders.arrayOf(elementCoder));
+}
 
+/**
+ * Adds an ARRAY field with elements of {@code rowType}.
+ */
+public Builder withArrayField(String fieldName, RowType rowType) {
+  return withField(fieldName, SqlTypeCoders.arrayOf(rowType));
 }
 
 public Builder withRowField(String fieldName, RowType rowType) {
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoder.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoder.java
index 5b6e104a3fd..c2a6a3ac081 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoder.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoder.java
@@ -65,10 +65,6 @@ public int hashCode() {
 return this.getClass().hashCode();
   }
 
-  public static boolean isArray(SqlTypeCoder sqlTypeCoder) {
-return sqlTypeCoder instanceof SqlArrayCoder;
-  }
-
   static class SqlTinyIntCoder extends SqlTypeCoder {
 
 @Override
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoders.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoders.java
index 9eea2df1a99..d9a132ca623 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoders.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoders.java
@@ -58,10 +58,6 @@
   public static final SqlTypeCoder DATE = new SqlDateCoder();
   public static final SqlTypeCoder TIMESTAMP = new SqlTimestampCoder();
 
-  public static SqlTypeCoder arrayOf(SqlTypeCoder elementCoder) {
-return SqlArrayCoder.of(elementCoder);
-  }
-
   public static final Set NUMERIC_TYPES =
   ImmutableSet.of(
   SqlTypeCoders.TINYINT,
@@ -72,6 +68,18 @@ public static SqlTypeCoder arrayOf(SqlTypeCoder 
elementCoder) {
   SqlTypeCoders.DOUBLE,
   SqlTypeCoders.DECIMAL);
 
+  public static SqlTypeCoder arrayOf(SqlTypeCoder elementCoder) {
+return SqlArrayCoder.of(elementCoder);
+  }
+
+  public static SqlTypeCoder arrayOf(RowType rowType) {
+return SqlArrayCoder.of(rowOf(rowType));
+  }
+
+  public static boolean isArray(SqlTypeCoder sqlTypeCoder) {
+return sqlTypeCoder instanceof SqlArrayCoder;
+  }
+
   public static boolean isRow(SqlTypeCoder sqlTypeCoder) {
 return sqlTypeCoder instanceof SqlRowCoder;
   }
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamSqlEnv.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamSqlEnv.java
index e81b9278842..7bd04f2b1f5 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamSqlEnv.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamSqlEnv.java
@@ -169,8 +169,7 @@ public BeamCalciteTable(RowType beamRowType) {
 
 @Override
 public RelDataType getRowType(RelDataTypeFactory typeFactory) {
-  return CalciteUtils.toCalciteRowType(this.beamRowType)
-  .apply(BeamQueryPlanner.TYPE_FACTORY);
+  return CalciteUtils.toCalciteRowType(this.beamRowType, 
BeamQueryPlanner.TYPE_FACTORY);
 }
 
 @Override
diff --git

[beam] branch master updated: [SQL] Add support for arrays of rows (#4857)

2018-03-14 Thread mingmxu

This is an automated email from the ASF dual-hosted git repository.

mingmxu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 7d96ee6  [SQL] Add support for arrays of rows (#4857)
7d96ee6 is described below

commit 7d96ee6250ea93b4822c32ace84f2f5ea21e2da9
Author: Anton Kedin <33067037+ake...@users.noreply.github.com>
AuthorDate: Wed Mar 14 21:20:02 2018 -0700

[SQL] Add support for arrays of rows (#4857)

Support array fields containing rows
---
 .../apache/beam/sdk/extensions/sql/RowSqlType.java |   9 ++
 .../beam/sdk/extensions/sql/SqlTypeCoder.java  |   4 -
 .../beam/sdk/extensions/sql/SqlTypeCoders.java |  16 ++-
 .../beam/sdk/extensions/sql/impl/BeamSqlEnv.java   |   3 +-
 .../sql/impl/interpreter/BeamSqlFnExecutor.java|   8 +-
 .../interpreter/operator/BeamSqlPrimitive.java |   5 +-
 .../operator/row/BeamSqlFieldAccessExpression.java |  27 -
 .../extensions/sql/impl/utils/CalciteUtils.java|  51 -
 .../beam/sdk/extensions/sql/BeamSqlArrayTest.java  | 121 +
 .../row/BeamSqlFieldAccessExpressionTest.java  |  91 
 10 files changed, 290 insertions(+), 45 deletions(-)

diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/RowSqlType.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/RowSqlType.java
index 77eda6a..fc16c84 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/RowSqlType.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/RowSqlType.java
@@ -102,9 +102,18 @@ public class RowSqlType {
   return withField(fieldName, SqlTypeCoders.TIMESTAMP);
 }
 
+/**
+ * Adds an ARRAY field with elements of {@code elementCoder}.
+ */
 public Builder withArrayField(String fieldName, SqlTypeCoder elementCoder) 
{
   return withField(fieldName, SqlTypeCoders.arrayOf(elementCoder));
+}
 
+/**
+ * Adds an ARRAY field with elements of {@code rowType}.
+ */
+public Builder withArrayField(String fieldName, RowType rowType) {
+  return withField(fieldName, SqlTypeCoders.arrayOf(rowType));
 }
 
 public Builder withRowField(String fieldName, RowType rowType) {
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoder.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoder.java
index 5b6e104..c2a6a3a 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoder.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoder.java
@@ -65,10 +65,6 @@ public abstract class SqlTypeCoder extends 
CustomCoder {
 return this.getClass().hashCode();
   }
 
-  public static boolean isArray(SqlTypeCoder sqlTypeCoder) {
-return sqlTypeCoder instanceof SqlArrayCoder;
-  }
-
   static class SqlTinyIntCoder extends SqlTypeCoder {
 
 @Override
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoders.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoders.java
index 9eea2df..d9a132c 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoders.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTypeCoders.java
@@ -58,10 +58,6 @@ public class SqlTypeCoders {
   public static final SqlTypeCoder DATE = new SqlDateCoder();
   public static final SqlTypeCoder TIMESTAMP = new SqlTimestampCoder();
 
-  public static SqlTypeCoder arrayOf(SqlTypeCoder elementCoder) {
-return SqlArrayCoder.of(elementCoder);
-  }
-
   public static final Set NUMERIC_TYPES =
   ImmutableSet.of(
   SqlTypeCoders.TINYINT,
@@ -72,6 +68,18 @@ public class SqlTypeCoders {
   SqlTypeCoders.DOUBLE,
   SqlTypeCoders.DECIMAL);
 
+  public static SqlTypeCoder arrayOf(SqlTypeCoder elementCoder) {
+return SqlArrayCoder.of(elementCoder);
+  }
+
+  public static SqlTypeCoder arrayOf(RowType rowType) {
+return SqlArrayCoder.of(rowOf(rowType));
+  }
+
+  public static boolean isArray(SqlTypeCoder sqlTypeCoder) {
+return sqlTypeCoder instanceof SqlArrayCoder;
+  }
+
   public static boolean isRow(SqlTypeCoder sqlTypeCoder) {
 return sqlTypeCoder instanceof SqlRowCoder;
   }
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamSqlEnv.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamSqlEnv.java
index e81b927..7bd04f2 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamSqlEnv.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamSqlEnv.java
@@ -169,8

[jira] [Created] (BEAM-3858) Data from JdbcIO.read() cannot pass to next transform on ApexRunner

2018-03-14 Thread huangjianhuang (JIRA)

huangjianhuang created BEAM-3858:


 Summary: Data from JdbcIO.read() cannot pass to next transform on 
ApexRunner
 Key: BEAM-3858
 URL: https://issues.apache.org/jira/browse/BEAM-3858
 Project: Beam
  Issue Type: Bug
  Components: io-java-jdbc, runner-apex
Affects Versions: 2.3.0
 Environment: ubuntu16.04
Reporter: huangjianhuang
Assignee: Jean-Baptiste Onofré


{code:java}

public static void testJDBCRead(Pipeline pipeline) {
System.out.println("in testJDBCRead()");
pipeline.apply(JdbcIO.read()
.withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
"com.mysql.jdbc.Driver", 
"jdbc:mysql://localhost:3307/libra")
.withUsername("root")
.withPassword("123456"))
.withQuery("SELECT * FROM o_flow_account_login limit 3")
.withCoder(StringUtf8Coder.of())
.withRowMapper(new JdbcIO.RowMapper() {
public String mapRow(ResultSet resultSet) throws Exception {
System.out.println("maprow");
return "tmp";
}
})

)

.apply(ParDo.of(new DoFn() {
@ProcessElement
public void process(ProcessContext context) {
System.out.println("??");
context.output(" ");
}
}));
}
{code}
On DirectRunner or FlinkRunner, screen shows:
{code:java}
maprow
maprow
maprow
??
??
??
{code}
however on ApexRunner, screen only shows:
{code:java}
maprow
maprow
maprow
{code}
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Dataflow #5153

See 


Changes:

[tgroh] Add an InProcess SdkHarness Rule

[tgroh] Close Outstanding Control Clients on Service close

[tgroh] Don't try to close a cancelled Multiplexer

[tgroh] fixup! Close Outstanding Control Clients on Service close

[tgroh] fixup! Don't try to close a cancelled Multiplexer

[tgroh] fixup! Add an InProcess SdkHarness Rule

[tgroh] fixup! Don't try to close a cancelled Multiplexer

[ehudm] Use Python 3 compatible string.

--
[...truncated 24.87 MB...]
[INFO] 2018-03-15T02:02:39.582Z: (b6198bbdcd056431): Fusing consumer 
PAssert$377/GroupGlobally/GroupDummyAndContents/Reify into 
PAssert$377/GroupGlobally/WindowIntoDummy/Window.Assign
[INFO] 2018-03-15T02:02:39.616Z: (b6198bbdcd056c57): Fusing consumer 
PAssert$377/GroupGlobally/WindowIntoDummy/Window.Assign into 
PAssert$377/GroupGlobally/Create.Values/Read(CreateSource)
[INFO] 2018-03-15T02:02:40.122Z: (15244c67deeb0900): Executing operation 
PAssert$377/GroupGlobally/GatherAllOutputs/GroupByKey/Create
[INFO] 2018-03-15T02:02:40.190Z: (15f7ab0cbc7b6f10): Starting 1 workers in 
us-central1-f...
[INFO] 2018-03-15T02:02:40.326Z: (5c5f5f711bf28ed4): Executing operation 
Create.Values/Read(CreateSource)+KvSwap/KvSwap/Map+PAssert$377/GroupGlobally/Window.Into()/Window.Assign+PAssert$377/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$377/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$377/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$377/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$377/GroupGlobally/GatherAllOutputs/GroupByKey/Write
[INFO] 2018-03-15T02:02:48.524Z: (89bb1449654ebd73): Cleaning up.
[INFO] 2018-03-15T02:02:48.608Z: (89bb1449654eb311): Stopping worker pool...
[INFO] 2018-03-15T02:02:48.509Z: Autoscaling: Raised the number of workers to 0 
based on the rate of progress in the currently running step(s).
[INFO] 2018-03-15T02:02:48.909Z: (59a51e12d1f4ecd4): Executing operation 
PAssert$370/GroupGlobally/GroupDummyAndContents/Close
[INFO] 2018-03-15T02:02:48.987Z: (59a51e12d1f4e9f5): Executing operation 
PAssert$370/GroupGlobally/GroupDummyAndContents/Read+PAssert$370/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$370/GroupGlobally/Values/Values/Map+PAssert$370/GroupGlobally/ParDo(Concat)+PAssert$370/GetPane/Map+PAssert$370/RunChecks+PAssert$370/VerifyAssertions/ParDo(DefaultConclude)
[INFO] 2018-03-15T02:02:50.731Z: (1024ff05ad78a0ce): Workers have started 
successfully.
[INFO] 2018-03-15T02:02:50.154Z: (7698105668e5849e): Workers have started 
successfully.
[INFO] 2018-03-15T02:02:51.517Z: (ddb7cdb99ad8cb97): Autoscaling: Resized 
worker pool from 1 to 0.
[INFO] 2018-03-15T02:02:51.540Z: (ddb7cdb99ad8c3bd): Autoscaling: Would further 
reduce the number of workers but reached the minimum number allowed for the job.
[INFO] 2018-03-15T02:02:43.075Z: (cd90d2b09e9933fb): Autoscaling is enabled for 
job 2018-03-14_19_02_43-14812570830438480351. The number of workers will be 
between 1 and 100.
[INFO] 2018-03-15T02:02:43.096Z: (cd90d2b09e993a82): Autoscaling was 
automatically enabled for job 2018-03-14_19_02_43-14812570830438480351.
[INFO] 2018-03-15T02:02:45.664Z: (b74543d965bf982a): Checking required Cloud 
APIs are enabled.
[INFO] 2018-03-15T02:02:45.752Z: (b74543d965bf97d3): Checking permissions 
granted to controller Service Account.
[INFO] 2018-03-15T02:02:49.866Z: (b74543d965bf9933): Expanding CoGroupByKey 
operations into optimizable parts.
[INFO] 2018-03-15T02:02:49.893Z: (b74543d965bf98dc): Expanding GroupByKey 
operations into optimizable parts.
[INFO] 2018-03-15T02:02:49.920Z: (b74543d965bf967a): Lifting 
ValueCombiningMappingFns into MergeBucketsMappingFns
[INFO] 2018-03-15T02:02:49.985Z: (b74543d965bf9a90): Fusing adjacent ParDo, 
Read, Write, and Flatten operations
[INFO] 2018-03-15T02:02:50.011Z: (b74543d965bf982e): Elided trivial flatten 
[INFO] 2018-03-15T02:02:50.031Z: (b74543d965bf95cc): Elided trivial flatten 
[INFO] 2018-03-15T02:02:50.070Z: (b74543d965bf936a): Elided trivial flatten 
[INFO] 2018-03-15T02:02:50.092Z: (b74543d965bf9108): Unzipping flatten s15 for 
input s10.org.apache.beam.sdk.values.PCollection.:329#89dcb817761b7ddd
[INFO] 2018-03-15T02:02:50.118Z: (b74543d965bf9ea6): Fusing unzipped copy of 
PAssert$378/GroupGlobally/GroupDummyAndContents/Reify, through flatten 
PAssert$378/GroupGlobally/FlattenDummyAndContents, into producer 
PAssert$378/GroupGlobally/KeyForDummy/AddKeys/Map
[INFO] 2018-03-15T02:02:50.141Z: (b74543d965bf9c44): Unzipping flatten s15-u22 
for input s17-reify-value0-c20
[INFO] 2018-03-15T02:02:50.163Z: (b74543d965bf99e2): Fusing unzipped copy of 
PAssert$378/GroupGlobally/GroupDummyAndContents/Write, through flatten s15-u22, 
into producer PAssert$378/GroupGlobally/GroupDummyAndContents/Reify
[INFO] 2018-03-15T02:02:50.183Z: (b74543d965bf9780): Fusing

Build failed in Jenkins: beam_PostCommit_Java_MavenInstall #6206

See 


Changes:

[tgroh] Add an InProcess SdkHarness Rule

[tgroh] Close Outstanding Control Clients on Service close

[tgroh] Don't try to close a cancelled Multiplexer

[tgroh] fixup! Close Outstanding Control Clients on Service close

[tgroh] fixup! Don't try to close a cancelled Multiplexer

[tgroh] fixup! Add an InProcess SdkHarness Rule

[tgroh] fixup! Don't try to close a cancelled Multiplexer

--
[...truncated 1.44 MB...]
2018-03-15T01:04:25.736 [INFO] Excluding 
org.apache.avro:avro-mapred:jar:hadoop2:1.7.7 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding org.apache.avro:avro-ipc:jar:1.7.7 
from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
org.apache.avro:avro-ipc:jar:tests:1.7.7 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding com.twitter:chill_2.11:jar:0.8.0 from 
the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
com.esotericsoftware:kryo-shaded:jar:3.0.3 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding com.esotericsoftware:minlog:jar:1.3.0 
from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding com.twitter:chill-java:jar:0.8.0 from 
the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
org.apache.xbean:xbean-asm5-shaded:jar:4.4 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
org.apache.hadoop:hadoop-client:jar:2.7.3 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
org.apache.hadoop:hadoop-common:jar:2.7.3 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding xmlenc:xmlenc:jar:0.52 from the shaded 
jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
commons-httpclient:commons-httpclient:jar:3.1 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding javax.servlet.jsp:jsp-api:jar:2.1 from 
the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
commons-configuration:commons-configuration:jar:1.6 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
commons-digester:commons-digester:jar:1.8 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
commons-beanutils:commons-beanutils-core:jar:1.8.0 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
org.apache.hadoop:hadoop-auth:jar:2.7.3 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15 from the 
shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
org.apache.directory.api:api-asn1-api:jar:1.0.0-M20 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
org.apache.directory.api:api-util:jar:1.0.0-M20 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
org.apache.curator:curator-client:jar:2.7.1 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
org.mortbay.jetty:jetty-util:jar:6.1.26 from the shaded jar.
2018-03-15T01:04:25.736 [INFO] Excluding 
org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.3 from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding 
org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.3 from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding 
org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.3 from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding 
org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.3 from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding 
org.apache.hadoop:hadoop-yarn-api:jar:2.7.3 from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding 
org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3 from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding 
org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.3 from the shaded 
jar.
2018-03-15T01:04:25.737 [INFO] Excluding 
org.apache.hadoop:hadoop-annotations:jar:2.7.3 from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding 
org.apache.spark:spark-launcher_2.11:jar:2.2.1 from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding 
org.apache.spark:spark-network-common_2.11:jar:2.2.1 from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding 
org.fusesource.leveldbjni:leveldbjni-all:jar:1.8 from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding 
org.apache.spark:spark-network-shuffle_2.11:jar:2.2.1 from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding 
org.apache.spark:spark-unsafe_2.11:jar:2.2.1 from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding net.java.dev.jets3t:jets3t:jar:0.9.3 
from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding javax.activation:activation:jar:1.1.1 
from the shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding mx4j:mx4j:jar:3.0.2 from the shaded 
jar.
2018-03-15T01:04:25.737 [INFO] Excluding javax.mail:mail:jar:1.4.7 from the 
shaded jar.
2018-03-15T01:04:25.737 [INFO] Excluding 
org.bouncycastle:bcprov-jdk15on:jar:1.51 from the shaded jar.

Jenkins build became unstable: beam_PostCommit_Java_ValidatesRunner_Flink #5236

See

Build failed in Jenkins: beam_PerformanceTests_Spark #1467

See 


Changes:

[tgroh] Add an InProcess SdkHarness Rule

[tgroh] Close Outstanding Control Clients on Service close

[tgroh] Don't try to close a cancelled Multiplexer

[tgroh] fixup! Close Outstanding Control Clients on Service close

[tgroh] fixup! Don't try to close a cancelled Multiplexer

[tgroh] fixup! Add an InProcess SdkHarness Rule

[tgroh] fixup! Don't try to close a cancelled Multiplexer

[ehudm] Use Python 3 compatible string.

--
[...truncated 87.98 KB...]
'apache-beam-testing:bqjob_r15cd19075a27eaa1_01622705e51f_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT

STDERR: 
/usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73:
 UserWarning: You have requested explicit scopes to be used with a GCE service 
account.
Using this argument will have no effect on the actual scopes for tokens
requested. These scopes are set at VM instance creation time and
can't be overridden in the request.

  warnings.warn(_SCOPES_WARNING)

2018-03-15 00:18:42,572 237e779b MainThread INFO Retrying exception running 
IssueRetryableCommand: Command returned a non-zero exit code.

2018-03-15 00:19:05,198 237e779b MainThread INFO Running: bq load 
--autodetect --source_format=NEWLINE_DELIMITED_JSON 
beam_performance.pkb_results 

2018-03-15 00:19:07,474 237e779b MainThread INFO Ran: {bq load --autodetect 
--source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results 

  ReturnCode:1
STDOUT: Upload complete.
Waiting on bqjob_r4f7e2f81fe5a2ab0_016227064671_1 ... (0s) Current status: 
RUNNING 
 Waiting on bqjob_r4f7e2f81fe5a2ab0_016227064671_1 ... (0s) 
Current status: DONE   
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4f7e2f81fe5a2ab0_016227064671_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT

STDERR: 
/usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73:
 UserWarning: You have requested explicit scopes to be used with a GCE service 
account.
Using this argument will have no effect on the actual scopes for tokens
requested. These scopes are set at VM instance creation time and
can't be overridden in the request.

  warnings.warn(_SCOPES_WARNING)

2018-03-15 00:19:07,474 237e779b MainThread INFO Retrying exception running 
IssueRetryableCommand: Command returned a non-zero exit code.

2018-03-15 00:19:35,076 237e779b MainThread INFO Running: bq load 
--autodetect --source_format=NEWLINE_DELIMITED_JSON 
beam_performance.pkb_results 

2018-03-15 00:19:39,486 237e779b MainThread INFO Ran: {bq load --autodetect 
--source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results 

  ReturnCode:1
STDOUT: Upload complete.
Waiting on bqjob_r2a24a67e7c536611_01622706bf03_1 ... (0s) Current status: 
RUNNING 
 Waiting on bqjob_r2a24a67e7c536611_01622706bf03_1 ... (1s) 
Current status: RUNNING 
 Waiting on 
bqjob_r2a24a67e7c536611_01622706bf03_1 ... (1s) Current status: DONE   
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r2a24a67e7c536611_01622706bf03_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT

STDERR: 
/usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73:
 UserWarning: You have requested explicit scopes to be used with a GCE service 
account.
Using this argument will have no effect on the actual scopes for tokens
requested. These scopes are set at VM instance creation time and
can't be overridden in the request.

  warnings.warn(_SCOPES_WARNING)

2018-03-15 00:19:39,486 237e779b MainThread INFO Retrying exception running 
IssueRetryableCommand: Command returned a non-zero exit code.

2018-03-15 00:19:56,739 237e779b MainThread INFO Running: bq load 
--autodetect --source_format=NEWLINE_DELIMITED_JSON 
beam_performance.pkb_results 

2018-03-15 00:19:58,897 237e779b MainThread INFO Ran: {bq load --autodetect 
--source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results

Build failed in Jenkins: beam_PerformanceTests_JDBC #330

See 


Changes:

[tgroh] Add an InProcess SdkHarness Rule

[tgroh] Close Outstanding Control Clients on Service close

[tgroh] Don't try to close a cancelled Multiplexer

[tgroh] fixup! Close Outstanding Control Clients on Service close

[tgroh] fixup! Don't try to close a cancelled Multiplexer

[tgroh] fixup! Add an InProcess SdkHarness Rule

[tgroh] fixup! Don't try to close a cancelled Multiplexer

[ehudm] Use Python 3 compatible string.

--
[...truncated 47.39 KB...]
[INFO] Replacing 

 with 

[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing 

 with 

[INFO] Dependency-reduced POM written at: 

[INFO] 
[INFO] --- maven-failsafe-plugin:2.20.1:integration-test (default) @ 
beam-sdks-java-io-jdbc ---
[INFO] Failsafe report directory: 

[INFO] parallel='all', perCoreThreadCount=true, threadCount=4, 
useUnlimitedThreads=false, threadCountSuites=0, threadCountClasses=0, 
threadCountMethods=0, parallelOptimized=true
[INFO] 
[INFO] ---
[INFO]  T E S T S
[INFO] ---
[INFO] Running org.apache.beam.sdk.io.jdbc.JdbcIOIT
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 
1,052.688 s <<< FAILURE! - in org.apache.beam.sdk.io.jdbc.JdbcIOIT
[ERROR] testWriteThenRead(org.apache.beam.sdk.io.jdbc.JdbcIOIT)  Time elapsed: 
1,052.688 s  <<< ERROR!
java.lang.RuntimeException: 
(c26c5e77916f3930): java.lang.RuntimeException: 
org.apache.beam.sdk.util.UserCodeException: org.postgresql.util.PSQLException: 
The connection attempt failed.
at 
com.google.cloud.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:190)
at 
com.google.cloud.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:161)
at 
com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:63)
at 
com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:50)
at 
com.google.cloud.dataflow.worker.graph.Networks.replaceDirectedNetworkNodes(Networks.java:87)
at 
com.google.cloud.dataflow.worker.IntrinsicMapTaskExecutorFactory.create(IntrinsicMapTaskExecutorFactory.java:121)
at 
com.google.cloud.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:328)
at 
com.google.cloud.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:284)
at 
com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:134)
at 
com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:114)
at 
com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:101)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.beam.sdk.util.UserCodeException: 
org.postgresql.util.PSQLException: The connection attempt failed.
at 
org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at 
org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn$DoFnInvoker.invokeSetup(Unknown 
Source)
at 
com.google.cloud.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.deserializeCopy(DoFnInstanceManagers.java:63)
at 
com.google.cloud.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.peek(DoFnInstanceManagers.java:45)
at 
com.google.cloud.dataflow.worker.UserParDoFnFactory.create(UserParDoFnFactory.java:94)
at 
com.google.cloud.dataflow.worker.DefaultParDoFnFactory.create(DefaultParDoFnFactory.java:74)
at

Jenkins build is back to normal : beam_PerformanceTests_HadoopInputFormat #19

See

Build failed in Jenkins: beam_PerformanceTests_Python #1023

See 


Changes:

[tgroh] Add an InProcess SdkHarness Rule

[tgroh] Close Outstanding Control Clients on Service close

[tgroh] Don't try to close a cancelled Multiplexer

[tgroh] fixup! Close Outstanding Control Clients on Service close

[tgroh] fixup! Don't try to close a cancelled Multiplexer

[tgroh] fixup! Add an InProcess SdkHarness Rule

[tgroh] fixup! Don't try to close a cancelled Multiplexer

[ehudm] Use Python 3 compatible string.

--
[...truncated 1.25 KB...]
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins6177379409296578440.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins25413355220260393.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins3443169984416437619.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins4769829087826744124.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from 
https://pypi.python.org/packages/ad/dc/fcced9ec3f2561c0cbe8eb6527eef7cf4f4919a2b3a07891a36e846635af/setuptools-38.5.2-py2.py3-none-any.whl#md5=abd3307cdce6fb543b5a4d0e3e98bdb6
Downloading/unpacking pip from 
https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
  Found existing installation: setuptools 2.2
Uninstalling setuptools:
  Successfully uninstalled setuptools
  Found existing installation: pip 1.5.4
Uninstalling pip:
  Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins832150981919416220.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins7332908729815608270.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in 
/usr/local/lib/python2.7/dist-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages 
(from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
  Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in 
/home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r 
PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in 
/usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r 
PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in 
/usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r 
PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in 
/home/jenkins/.local/lib/python2.7/site-packages (from

Build failed in Jenkins: beam_PerformanceTests_Compressed_TextIOIT #252

See 


Changes:

[tgroh] Add an InProcess SdkHarness Rule

[tgroh] Close Outstanding Control Clients on Service close

[tgroh] Don't try to close a cancelled Multiplexer

[tgroh] fixup! Close Outstanding Control Clients on Service close

[tgroh] fixup! Don't try to close a cancelled Multiplexer

[tgroh] fixup! Add an InProcess SdkHarness Rule

[tgroh] fixup! Don't try to close a cancelled Multiplexer

[ehudm] Use Python 3 compatible string.

--
[...truncated 37.07 KB...]
[INFO] Excluding 
org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.5.0-SNAPSHOT from 
the shaded jar.
[INFO] Excluding 
org.apache.beam:beam-sdks-java-extensions-protobuf:jar:2.5.0-SNAPSHOT from the 
shaded jar.
[INFO] Excluding io.grpc:grpc-core:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.errorprone:error_prone_annotations:jar:2.0.15 from 
the shaded jar.
[INFO] Excluding io.grpc:grpc-context:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.instrumentation:instrumentation-api:jar:0.3.0 from 
the shaded jar.
[INFO] Excluding 
com.google.apis:google-api-services-bigquery:jar:v2-rev374-1.22.0 from the 
shaded jar.
[INFO] Excluding com.google.api:gax-grpc:jar:0.20.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-protobuf:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.api:api-common:jar:1.0.0-rc2 from the shaded jar.
[INFO] Excluding com.google.auto.value:auto-value:jar:1.5.3 from the shaded jar.
[INFO] Excluding com.google.api:gax:jar:1.3.1 from the shaded jar.
[INFO] Excluding org.threeten:threetenbp:jar:1.3.3 from the shaded jar.
[INFO] Excluding com.google.cloud:google-cloud-core-grpc:jar:1.2.0 from the 
shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java-util:jar:3.2.0 from the 
shaded jar.
[INFO] Excluding com.google.code.gson:gson:jar:2.7 from the shaded jar.
[INFO] Excluding com.google.apis:google-api-services-pubsub:jar:v1-rev10-1.22.0 
from the shaded jar.
[INFO] Excluding com.google.api.grpc:grpc-google-cloud-pubsub-v1:jar:0.1.18 
from the shaded jar.
[INFO] Excluding com.google.api.grpc:proto-google-cloud-pubsub-v1:jar:0.1.18 
from the shaded jar.
[INFO] Excluding com.google.api.grpc:proto-google-iam-v1:jar:0.1.18 from the 
shaded jar.
[INFO] Excluding com.google.cloud.datastore:datastore-v1-proto-client:jar:1.4.0 
from the shaded jar.
[INFO] Excluding com.google.http-client:google-http-client-protobuf:jar:1.22.0 
from the shaded jar.
[INFO] Excluding com.google.http-client:google-http-client-jackson:jar:1.22.0 
from the shaded jar.
[INFO] Excluding com.google.cloud.datastore:datastore-v1-protos:jar:1.3.0 from 
the shaded jar.
[INFO] Excluding com.google.api.grpc:grpc-google-common-protos:jar:0.1.9 from 
the shaded jar.
[INFO] Excluding io.grpc:grpc-auth:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-netty:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http2:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler-proxy:jar:4.1.8.Final from the shaded 
jar.
[INFO] Excluding io.netty:netty-codec-socks:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-buffer:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-common:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-transport:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-resolver:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.grpc:grpc-stub:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-all:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-okhttp:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.squareup.okhttp:okhttp:jar:2.5.0 from the shaded jar.
[INFO] Excluding com.squareup.okio:okio:jar:1.6.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-protobuf-lite:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-protobuf-nano:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.protobuf.nano:protobuf-javanano:jar:3.0.0-alpha-5 
from the shaded jar.
[INFO] Excluding com.google.cloud:google-cloud-core:jar:1.0.2 from the shaded 
jar.
[INFO] Excluding org.json:json:jar:20160810 from the shaded jar.
[INFO] Excluding com.google.cloud:google-cloud-spanner:jar:0.20.0b-beta from 
the shaded jar.
[INFO] Excluding com.google.api.grpc:proto-google-cloud-spanner-v1:jar:0.1.11b 
from the shaded jar.
[INFO] Excluding 
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:jar:0.1.11 
from the shaded jar.
[INFO] Excluding com.google.api.grpc:grpc-google-cloud-spanner-v1:jar:0.1.11b 
from the shaded jar.
[INFO] Excluding

[jira] [Created] (BEAM-3857) Dynamic output support for python

2018-03-14 Thread Eila Arich-Landkof (JIRA)

Eila Arich-Landkof created BEAM-3857:


 Summary: Dynamic output support for python
 Key: BEAM-3857
 URL: https://issues.apache.org/jira/browse/BEAM-3857
 Project: Beam
  Issue Type: New Feature
  Components: io-ideas, sdk-py-core
Reporter: Eila Arich-Landkof
Assignee: Eugene Kirpichov


If possible, I would like to have a dynamic output support for python as 
available in Java

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-3826) Complete Go SDK documentation

2018-03-14 Thread Henning Rohde (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Henning Rohde updated BEAM-3826:

Description: 
We need to add a Go QuickStart and development guides on the website. The godoc 
should also be self-contained and published similarly to java and python api 
docs.

https://godoc.org/github.com/apache/beam/sdks/go/pkg/beam

  was:We need to add a Go QuickStart and development guides on the website. The 
godoc should also be self-contained and published similarly to java and python 
api docs.


> Complete Go SDK documentation
> -
>
> Key: BEAM-3826
> URL: https://issues.apache.org/jira/browse/BEAM-3826
> Project: Beam
>  Issue Type: Task
>  Components: sdk-go
>Reporter: Henning Rohde
>Assignee: Henning Rohde
>Priority: Major
>
> We need to add a Go QuickStart and development guides on the website. The 
> godoc should also be self-contained and published similarly to java and 
> python api docs.
> https://godoc.org/github.com/apache/beam/sdks/go/pkg/beam



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-3532) Add Distinct cookbook example to Go SDK

2018-03-14 Thread Henning Rohde (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-3532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Henning Rohde reassigned BEAM-3532:
---

Assignee: Willy Lulciuc  (was: Henning Rohde)

> Add Distinct cookbook example to Go SDK
> ---
>
> Key: BEAM-3532
> URL: https://issues.apache.org/jira/browse/BEAM-3532
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-go
>Reporter: Henning Rohde
>Assignee: Willy Lulciuc
>Priority: Minor
>
> Add the Go equivalent of
> [https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/cookbook/DistinctExample.java]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-3532) Add Distinct cookbook example to Go SDK

2018-03-14 Thread Henning Rohde (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-3532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Henning Rohde reassigned BEAM-3532:
---

Assignee: Henning Rohde

> Add Distinct cookbook example to Go SDK
> ---
>
> Key: BEAM-3532
> URL: https://issues.apache.org/jira/browse/BEAM-3532
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-go
>Reporter: Henning Rohde
>Assignee: Henning Rohde
>Priority: Minor
>
> Add the Go equivalent of
> [https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/cookbook/DistinctExample.java]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3565) Add utilities for producing a collection of PTransforms that can execute in a single SDK Harness


 [ 
https://issues.apache.org/jira/browse/BEAM-3565?focusedWorklogId=80630=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80630
 ]

ASF GitHub Bot logged work on BEAM-3565:


Author: ASF GitHub Bot
Created on: 14/Mar/18 23:28
Start Date: 14/Mar/18 23:28
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #4777: [BEAM-3565] Add 
FusedPipeline#toPipeline
URL: https://github.com/apache/beam/pull/4777#issuecomment-373208733
 
 
   Please address all the comments.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80630)
Time Spent: 12h 50m  (was: 12h 40m)

> Add utilities for producing a collection of PTransforms that can execute in a 
> single SDK Harness
> 
>
> Key: BEAM-3565
> URL: https://issues.apache.org/jira/browse/BEAM-3565
> Project: Beam
>  Issue Type: Bug
>  Components: runner-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>Priority: Major
>  Labels: portability
> Fix For: 2.4.0
>
>  Time Spent: 12h 50m
>  Remaining Estimate: 0h
>
> An SDK Harness executes some ("fused") collection of PTransforms. The java 
> runner libraries should provide some way to take a Pipeline that executes in 
> both a runner and an environment and construct a collection of transforms 
> which can execute within a single environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-3856) Add prototype support for Go SDK streaming

2018-03-14 Thread Willy Lulciuc (JIRA)

Willy Lulciuc created BEAM-3856:
---

 Summary: Add prototype support for Go SDK streaming
 Key: BEAM-3856
 URL: https://issues.apache.org/jira/browse/BEAM-3856
 Project: Beam
  Issue Type: Improvement
  Components: sdk-go
Reporter: Willy Lulciuc
Assignee: Henning Rohde






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-3854) Add Go SDK support for PubSubIO

2018-03-14 Thread Willy Lulciuc (JIRA)

Willy Lulciuc created BEAM-3854:
---

 Summary: Add Go SDK support for PubSubIO
 Key: BEAM-3854
 URL: https://issues.apache.org/jira/browse/BEAM-3854
 Project: Beam
  Issue Type: Improvement
  Components: sdk-go
Reporter: Willy Lulciuc
Assignee: Henning Rohde


Both read and write on Dataflow runner.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-3855) Add Go SDK support for protobuf coder

2018-03-14 Thread Willy Lulciuc (JIRA)

Willy Lulciuc created BEAM-3855:
---

 Summary: Add Go SDK support for protobuf coder
 Key: BEAM-3855
 URL: https://issues.apache.org/jira/browse/BEAM-3855
 Project: Beam
  Issue Type: Improvement
  Components: sdk-go
Reporter: Willy Lulciuc
Assignee: Henning Rohde






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-2817) Bigquery queries should allow options to run in batch mode or not


 [ 
https://issues.apache.org/jira/browse/BEAM-2817?focusedWorklogId=80624=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80624
 ]

ASF GitHub Bot logged work on BEAM-2817:


Author: ASF GitHub Bot
Created on: 14/Mar/18 22:52
Start Date: 14/Mar/18 22:52
Worklog Time Spent: 10m 
  Work Description: justintumale commented on a change in pull request 
#4606: [BEAM-2817] BigQuery queries are allowed to run in either BATCH or IN…
URL: https://github.com/apache/beam/pull/4606#discussion_r174634008
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQuerySource.java
 ##
 @@ -74,12 +77,19 @@ private BigQueryQuerySource(
   Boolean useLegacySql,
   BigQueryServices bqServices,
   Coder coder,
-  SerializableFunction parseFn) {
+  SerializableFunction parseFn,
+  Priority priority) {
 super(stepUuid, bqServices, coder, parseFn);
 this.query = checkNotNull(query, "query");
 this.flattenResults = checkNotNull(flattenResults, "flattenResults");
 this.useLegacySql = checkNotNull(useLegacySql, "useLegacySql");
 this.dryRunJobStats = new AtomicReference<>();
+if (priority != BigQueryIO.TypedRead.Priority.BATCH
 
 Review comment:
   Good find. This should be && as this sets the Priority to default. This was 
done in the case that an incorrect priority is passed in that is neither batch 
or interactive. Please let me know the next steps as I see that you will fix 
this part of the PR. I can fix the unit tests if needed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80624)
Time Spent: 2h 50m  (was: 2h 40m)

> Bigquery queries should allow options to run in batch mode or not
> -
>
> Key: BEAM-2817
> URL: https://issues.apache.org/jira/browse/BEAM-2817
> Project: Beam
>  Issue Type: Bug
>  Components: io-java-gcp
>Affects Versions: 2.0.0
>Reporter: Lara Schmidt
>Assignee: Justin Tumale
>Priority: Major
>  Labels: newbie, starter
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> When bigquery read does a query it sets the mode to batch. A batch query can 
> be very slow to schedule as it batches it with other queries. However it 
> doesn't use batch quota which is better for some cases. However, in some 
> cases a fast query is better (especially in timed tests). It would be a good 
> idea to have a configuration to the bigquery source to set this per-read.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-2817) Bigquery queries should allow options to run in batch mode or not


 [ 
https://issues.apache.org/jira/browse/BEAM-2817?focusedWorklogId=80622=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80622
 ]

ASF GitHub Bot logged work on BEAM-2817:


Author: ASF GitHub Bot
Created on: 14/Mar/18 22:50
Start Date: 14/Mar/18 22:50
Worklog Time Spent: 10m 
  Work Description: justintumale commented on a change in pull request 
#4606: [BEAM-2817] BigQuery queries are allowed to run in either BATCH or IN…
URL: https://github.com/apache/beam/pull/4606#discussion_r174634008
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQuerySource.java
 ##
 @@ -74,12 +77,19 @@ private BigQueryQuerySource(
   Boolean useLegacySql,
   BigQueryServices bqServices,
   Coder coder,
-  SerializableFunction parseFn) {
+  SerializableFunction parseFn,
+  Priority priority) {
 super(stepUuid, bqServices, coder, parseFn);
 this.query = checkNotNull(query, "query");
 this.flattenResults = checkNotNull(flattenResults, "flattenResults");
 this.useLegacySql = checkNotNull(useLegacySql, "useLegacySql");
 this.dryRunJobStats = new AtomicReference<>();
+if (priority != BigQueryIO.TypedRead.Priority.BATCH
 
 Review comment:
   Good find. This should be && as this sets the Priority to default. This was 
done in the case that an incorrect priority is passed in.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80622)
Time Spent: 2h 40m  (was: 2.5h)

> Bigquery queries should allow options to run in batch mode or not
> -
>
> Key: BEAM-2817
> URL: https://issues.apache.org/jira/browse/BEAM-2817
> Project: Beam
>  Issue Type: Bug
>  Components: io-java-gcp
>Affects Versions: 2.0.0
>Reporter: Lara Schmidt
>Assignee: Justin Tumale
>Priority: Major
>  Labels: newbie, starter
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> When bigquery read does a query it sets the mode to batch. A batch query can 
> be very slow to schedule as it batches it with other queries. However it 
> doesn't use batch quota which is better for some cases. However, in some 
> cases a fast query is better (especially in timed tests). It would be a good 
> idea to have a configuration to the bigquery source to set this per-read.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-2817) Bigquery queries should allow options to run in batch mode or not


 [ 
https://issues.apache.org/jira/browse/BEAM-2817?focusedWorklogId=80621=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80621
 ]

ASF GitHub Bot logged work on BEAM-2817:


Author: ASF GitHub Bot
Created on: 14/Mar/18 22:47
Start Date: 14/Mar/18 22:47
Worklog Time Spent: 10m 
  Work Description: jkff commented on a change in pull request #4606: 
[BEAM-2817] BigQuery queries are allowed to run in either BATCH or IN…
URL: https://github.com/apache/beam/pull/4606#discussion_r174633409
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQuerySource.java
 ##
 @@ -74,12 +77,19 @@ private BigQueryQuerySource(
   Boolean useLegacySql,
   BigQueryServices bqServices,
   Coder coder,
-  SerializableFunction parseFn) {
+  SerializableFunction parseFn,
+  Priority priority) {
 super(stepUuid, bqServices, coder, parseFn);
 this.query = checkNotNull(query, "query");
 this.flattenResults = checkNotNull(flattenResults, "flattenResults");
 this.useLegacySql = checkNotNull(useLegacySql, "useLegacySql");
 this.dryRunJobStats = new AtomicReference<>();
+if (priority != BigQueryIO.TypedRead.Priority.BATCH
 
 Review comment:
   I'm fixing this in my PR. Probably not too big of a deal, just this feature 
won't really be usable in the release, but will be in the next one. No 
regression.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80621)
Time Spent: 2.5h  (was: 2h 20m)

> Bigquery queries should allow options to run in batch mode or not
> -
>
> Key: BEAM-2817
> URL: https://issues.apache.org/jira/browse/BEAM-2817
> Project: Beam
>  Issue Type: Bug
>  Components: io-java-gcp
>Affects Versions: 2.0.0
>Reporter: Lara Schmidt
>Assignee: Justin Tumale
>Priority: Major
>  Labels: newbie, starter
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> When bigquery read does a query it sets the mode to batch. A batch query can 
> be very slow to schedule as it batches it with other queries. However it 
> doesn't use batch quota which is better for some cases. However, in some 
> cases a fast query is better (especially in timed tests). It would be a good 
> idea to have a configuration to the bigquery source to set this per-read.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80618=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80618
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 22:46
Start Date: 14/Mar/18 22:46
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners
URL: https://github.com/apache/beam/pull/4853#issuecomment-373200164
 
 
   +R: @aaltay Could you please review it when you have time? Thanks~


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80618)
Time Spent: 12h  (was: 11h 50m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 12h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80617=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80617
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 22:45
Start Date: 14/Mar/18 22:45
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners
URL: https://github.com/apache/beam/pull/4853#issuecomment-373200164
 
 
   @aaltay Could you please review it when you have time? Thanks~


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80617)
Time Spent: 11h 50m  (was: 11h 40m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 11h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80616=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80616
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 22:41
Start Date: 14/Mar/18 22:41
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-373199296
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80616)
Time Spent: 11h 40m  (was: 11.5h)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 11h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-3853) 2.4.0 Python mobile gaming leader_board example is not working


[ 
https://issues.apache.org/jira/browse/BEAM-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399538#comment-16399538
 ] 

yifan zou commented on BEAM-3853:
-

Hi [~altay], I added the steps I did for running that example. In most of 
cases, the pipeline runs without any results nor console info. In a few cases, 
it throws errors: 

"error": {
 "errors": [
 {
 "domain": "global",
 "reason": "invalid",
 "message": "Invalid value for: STRING is not a valid value",
 "debugInfo": "..."
 }
 ],
 "code": 400,
 "message": "Invalid value for: STRING is not a valid value"
 }
}
> [while running 'WriteTeamScoreSums/WriteToBigQuery']

> 2.4.0 Python mobile gaming leader_board example is not working
> --
>
> Key: BEAM-3853
> URL: https://issues.apache.org/jira/browse/BEAM-3853
> Project: Beam
>  Issue Type: Bug
>  Components: examples-python
>Reporter: yifan zou
>Assignee: Ahmet Altay
>Priority: Major
> Fix For: 2.5.0
>
>
> The leader_board example does not work on DirectRunner. No result are 
> generated.
> *Run Injector to simulate data:*
>   mvn compile exec:java \
>    -Dexec.mainClass=org.apache.beam.examples.complete.game.injector.Injector \
>    Dexec.args="  none" 
> *Setup virtualenv and install python sdk from 
> [https://dist.apache.org/repos/dist/dev/beam/2.4.0/]*
>   virtualenv temp_virtualenv
>    . temp_virtualenv/bin/activate && pip install apache-beam-2.4.0-python.zip
> *Run leader_board*:
>   python -m apache_beam.examples.complete.game.leader_board \
>   --project=\
>   --topic projects//topics/\
>   --dataset= \
>   --runner=DirectRunner \
>   --table=
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-3853) 2.4.0 Python mobile gaming leader_board example is not working


 [ 
https://issues.apache.org/jira/browse/BEAM-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yifan zou updated BEAM-3853:

Description: 
The leader_board example does not work on DirectRunner. No result are generated.

*Run Injector to simulate data:*

  mvn compile exec:java \
   -Dexec.mainClass=org.apache.beam.examples.complete.game.injector.Injector \
   Dexec.args="  none" 

*Setup virtualenv and install python sdk from 
[https://dist.apache.org/repos/dist/dev/beam/2.4.0/]*

  virtualenv temp_virtualenv
   . temp_virtualenv/bin/activate && pip install apache-beam-2.4.0-python.zip

*Run leader_board*:

  python -m apache_beam.examples.complete.game.leader_board \
  --project=\
  --topic projects//topics/\
  --dataset= \
  --runner=DirectRunner \
  --table=

 

 

  was:
The leader_board example does not work on DirectRunner. No result are 
generated. 

*Run Injector to simulate data:*

  mvn compile exec:java \
  -Dexec.mainClass=org.apache.beam.examples.complete.game.injector.Injector \
  Dexec.args="  none" 

*Setup virtualenv and install python sdk from 
[https://dist.apache.org/repos/dist/dev/beam/2.4.0/]*

  virtualenv temp_virtualenv
  . temp_virtualenv/bin/activate && pip install apache-beam-2.4.0-python.zip

Run leader_board:

 

 

 


> 2.4.0 Python mobile gaming leader_board example is not working
> --
>
> Key: BEAM-3853
> URL: https://issues.apache.org/jira/browse/BEAM-3853
> Project: Beam
>  Issue Type: Bug
>  Components: examples-python
>Reporter: yifan zou
>Assignee: Ahmet Altay
>Priority: Major
> Fix For: 2.5.0
>
>
> The leader_board example does not work on DirectRunner. No result are 
> generated.
> *Run Injector to simulate data:*
>   mvn compile exec:java \
>    -Dexec.mainClass=org.apache.beam.examples.complete.game.injector.Injector \
>    Dexec.args="  none" 
> *Setup virtualenv and install python sdk from 
> [https://dist.apache.org/repos/dist/dev/beam/2.4.0/]*
>   virtualenv temp_virtualenv
>    . temp_virtualenv/bin/activate && pip install apache-beam-2.4.0-python.zip
> *Run leader_board*:
>   python -m apache_beam.examples.complete.game.leader_board \
>   --project=\
>   --topic projects//topics/\
>   --dataset= \
>   --runner=DirectRunner \
>   --table=
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-3820) SolrIO: Allow changing batchSize for writes


[ 
https://issues.apache.org/jira/browse/BEAM-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399530#comment-16399530
 ] 

Eugene Kirpichov commented on BEAM-3820:


The lack of control is one of the biggest reasons why we are strongly against 
tunable parameters - unless the user has full control over the environment 
(which they don't - they don't control the runner or the implementation of the 
PTransform), they can not assume that setting the parameter to a particular 
value will reliably achieve a particular effect. See discussion on 
https://github.com/apache/beam/pull/4461 for more elaboration - I'm happy to 
add this argument to 
https://beam.apache.org/contribute/ptransform-style-guide/#what-parameters-to-expose
 which already recommends against tunable parameters.

Could you elaborate about the tuning of batch sizes that you needed to do to 
get your job to work, on top of the retry strategy (which I agree needs to be 
implemented)? Perhaps we can make SolrIO do it automatically. If it turns out 
that there's no way to automatically set a value that will work reliably for 
all users, then that falls under "unless it’s impossible to automatically 
supply or compute a good-enough value" and then we can add it. But we should 
not do so lightly.

> SolrIO: Allow changing batchSize for writes
> ---
>
> Key: BEAM-3820
> URL: https://issues.apache.org/jira/browse/BEAM-3820
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-solr
>Affects Versions: 2.2.0, 2.3.0
>Reporter: Tim Robertson
>Assignee: Ismaël Mejía
>Priority: Trivial
>
> The SolrIO hard codes the batchSize for writes at 1000.  It would be a good 
> addition to allow the user to set the batchSize explicitly (similar to the 
> ElasticsearchIO)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-3853) 2.4.0 Python mobile gaming leader_board example is not working


 [ 
https://issues.apache.org/jira/browse/BEAM-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yifan zou updated BEAM-3853:

Description: 
The leader_board example does not work on DirectRunner. No result are 
generated. 

*Run Injector to simulate data:*

  mvn compile exec:java \
  -Dexec.mainClass=org.apache.beam.examples.complete.game.injector.Injector \
  Dexec.args="  none" 

*Setup virtualenv and install python sdk from 
[https://dist.apache.org/repos/dist/dev/beam/2.4.0/]*

  virtualenv temp_virtualenv
  . temp_virtualenv/bin/activate && pip install apache-beam-2.4.0-python.zip

Run leader_board:

 

 

 

  was:
The leader_board example does not work on DirectRunner. No result are 
generated. 

Run Injector to simulate data:

mvn compile exec:java \
-Dexec.mainClass=org.apache.beam.examples.complete.game.injector.Injector \
Dexec.args="  none" 

Setup virtualenv and install python sdk from 
[https://dist.apache.org/repos/dist/dev/beam/2.4.0/]

 

 


> 2.4.0 Python mobile gaming leader_board example is not working
> --
>
> Key: BEAM-3853
> URL: https://issues.apache.org/jira/browse/BEAM-3853
> Project: Beam
>  Issue Type: Bug
>  Components: examples-python
>Reporter: yifan zou
>Assignee: Ahmet Altay
>Priority: Major
> Fix For: 2.5.0
>
>
> The leader_board example does not work on DirectRunner. No result are 
> generated. 
> *Run Injector to simulate data:*
>   mvn compile exec:java \
>   -Dexec.mainClass=org.apache.beam.examples.complete.game.injector.Injector \
>   Dexec.args="  none" 
> *Setup virtualenv and install python sdk from 
> [https://dist.apache.org/repos/dist/dev/beam/2.4.0/]*
>   virtualenv temp_virtualenv
>   . temp_virtualenv/bin/activate && pip install apache-beam-2.4.0-python.zip
> Run leader_board:
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-3853) 2.4.0 Python mobile gaming leader_board example is not working


 [ 
https://issues.apache.org/jira/browse/BEAM-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yifan zou updated BEAM-3853:

Description: 
The leader_board example does not work on DirectRunner. No result are 
generated. 

Run Injector to simulate data:

mvn compile exec:java \
-Dexec.mainClass=org.apache.beam.examples.complete.game.injector.Injector \
Dexec.args="  none" 

Setup virtualenv and install python sdk from 
[https://dist.apache.org/repos/dist/dev/beam/2.4.0/]

 

 

  was:
The leader_board example does not work on DirectRunner. No result are 
generated. 

Run Injector to simulate data:

 


> 2.4.0 Python mobile gaming leader_board example is not working
> --
>
> Key: BEAM-3853
> URL: https://issues.apache.org/jira/browse/BEAM-3853
> Project: Beam
>  Issue Type: Bug
>  Components: examples-python
>Reporter: yifan zou
>Assignee: Ahmet Altay
>Priority: Major
> Fix For: 2.5.0
>
>
> The leader_board example does not work on DirectRunner. No result are 
> generated. 
> Run Injector to simulate data:
> mvn compile exec:java \
> -Dexec.mainClass=org.apache.beam.examples.complete.game.injector.Injector \
> Dexec.args="  none" 
> Setup virtualenv and install python sdk from 
> [https://dist.apache.org/repos/dist/dev/beam/2.4.0/]
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-3853) 2.4.0 Python mobile gaming leader_board example is not working


 [ 
https://issues.apache.org/jira/browse/BEAM-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yifan zou updated BEAM-3853:

Description: 
The leader_board example does not work on DirectRunner. No result are 
generated. 

Run Injector to simulate data:

 

  was:The leader_board example does not work on DirectRunner. No result are 
generated. 


> 2.4.0 Python mobile gaming leader_board example is not working
> --
>
> Key: BEAM-3853
> URL: https://issues.apache.org/jira/browse/BEAM-3853
> Project: Beam
>  Issue Type: Bug
>  Components: examples-python
>Reporter: yifan zou
>Assignee: Ahmet Altay
>Priority: Major
> Fix For: 2.5.0
>
>
> The leader_board example does not work on DirectRunner. No result are 
> generated. 
> Run Injector to simulate data:
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-3849) SolrIO: Expose connection timeout tuning for writes


[ 
https://issues.apache.org/jira/browse/BEAM-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399512#comment-16399512
 ] 

Eugene Kirpichov commented on BEAM-3849:


It sounds like we need a retrying wrapper over the Solr client - I definitely 
agree with that. Assuming we develop such a wrapper: is there any critical 
reason not to set the socket/connection timeouts to e.g. 10 seconds?

> SolrIO: Expose connection timeout tuning for writes
> ---
>
> Key: BEAM-3849
> URL: https://issues.apache.org/jira/browse/BEAM-3849
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-solr
>Affects Versions: 2.2.0, 2.3.0
>Reporter: Tim Robertson
>Assignee: Ismaël Mejía
>Priority: Minor
>
> A useful addition would be the ability to tune the socket and conncetion 
> timeouts for the underlying solrj client.
> Currently the createHttpClient() uses defaults only.
> This relates to BEAM-3820 and BEAM-3848 which together will help improve 
> stability of jobs doing large (billions of docs) loading of SOLR.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3500) JdbcIO: Improve connection management


 [ 
https://issues.apache.org/jira/browse/BEAM-3500?focusedWorklogId=80607=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80607
 ]

ASF GitHub Bot logged work on BEAM-3500:


Author: ASF GitHub Bot
Created on: 14/Mar/18 22:05
Start Date: 14/Mar/18 22:05
Worklog Time Spent: 10m 
  Work Description: jkff commented on a change in pull request #4461: 
[BEAM-3500] "Attach" JDBC connection to the bundle and add DataSourceFactory 
allowing full control of the way the DataSource is created
URL: https://github.com/apache/beam/pull/4461#discussion_r174624347
 
 

 ##
 File path: 
sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java
 ##
 @@ -509,7 +530,9 @@ public void populateDisplayData(DisplayData.Builder 
builder) {
   builder.add(DisplayData.item("query", getQuery()));
   builder.add(DisplayData.item("rowMapper", 
getRowMapper().getClass().getName()));
   builder.add(DisplayData.item("coder", getCoder().getClass().getName()));
-  getDataSourceConfiguration().populateDisplayData(builder);
+  if (getDataSourceConfiguration() != null) {
 
 Review comment:
   Ditto


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80607)
Time Spent: 4h 50m  (was: 4h 40m)

> JdbcIO: Improve connection management
> -
>
> Key: BEAM-3500
> URL: https://issues.apache.org/jira/browse/BEAM-3500
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-jdbc
>Affects Versions: 2.2.0
>Reporter: Pawel Bartoszek
>Assignee: Jean-Baptiste Onofré
>Priority: Major
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> JdbcIO write DoFn acquires connection in {{@Setup}} and release it in 
> {{@Teardown}} methods, which means that connection might stay opened for days 
> in streaming job case. Keeping single connection open for so long might be 
> very risky as it's exposed to database, network etc issues.
> *Taking connection from the pool when it is actually needed*
> I suggest that connection would be taken from the connection pool in 
> {{executeBatch}} method and released when the batch is flushed. This will 
> allow the pool to take care of any returned unhealthy connections etc.
> *Make JdbcIO accept data source factory*
>  It would be nice if JdbcIO accepted DataSourceFactory rather than DataSource 
> itself. I am saying that because sink checks if DataSource implements 
> `Serializable` interface, which make it impossible to pass 
> BasicDataSource(used internally by sink) as it doesn’t implement this 
> interface. Something like:
> {code:java}
> interface DataSourceFactory extends Serializable{
>  DataSource create();
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3500) JdbcIO: Improve connection management


 [ 
https://issues.apache.org/jira/browse/BEAM-3500?focusedWorklogId=80609=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80609
 ]

ASF GitHub Bot logged work on BEAM-3500:


Author: ASF GitHub Bot
Created on: 14/Mar/18 22:05
Start Date: 14/Mar/18 22:05
Worklog Time Spent: 10m 
  Work Description: jkff commented on a change in pull request #4461: 
[BEAM-3500] "Attach" JDBC connection to the bundle and add DataSourceFactory 
allowing full control of the way the DataSource is created
URL: https://github.com/apache/beam/pull/4461#discussion_r174624327
 
 

 ##
 File path: 
sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java
 ##
 @@ -423,7 +444,9 @@ public void populateDisplayData(DisplayData.Builder 
builder) {
   builder.add(DisplayData.item("query", getQuery()));
   builder.add(DisplayData.item("rowMapper", 
getRowMapper().getClass().getName()));
   builder.add(DisplayData.item("coder", getCoder().getClass().getName()));
-  getDataSourceConfiguration().populateDisplayData(builder);
+  if (getDataSourceConfiguration() != null) {
 
 Review comment:
   It can't be null, as per the check at line 422


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80609)
Time Spent: 5h 10m  (was: 5h)

> JdbcIO: Improve connection management
> -
>
> Key: BEAM-3500
> URL: https://issues.apache.org/jira/browse/BEAM-3500
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-jdbc
>Affects Versions: 2.2.0
>Reporter: Pawel Bartoszek
>Assignee: Jean-Baptiste Onofré
>Priority: Major
>  Time Spent: 5h 10m
>  Remaining Estimate: 0h
>
> JdbcIO write DoFn acquires connection in {{@Setup}} and release it in 
> {{@Teardown}} methods, which means that connection might stay opened for days 
> in streaming job case. Keeping single connection open for so long might be 
> very risky as it's exposed to database, network etc issues.
> *Taking connection from the pool when it is actually needed*
> I suggest that connection would be taken from the connection pool in 
> {{executeBatch}} method and released when the batch is flushed. This will 
> allow the pool to take care of any returned unhealthy connections etc.
> *Make JdbcIO accept data source factory*
>  It would be nice if JdbcIO accepted DataSourceFactory rather than DataSource 
> itself. I am saying that because sink checks if DataSource implements 
> `Serializable` interface, which make it impossible to pass 
> BasicDataSource(used internally by sink) as it doesn’t implement this 
> interface. Something like:
> {code:java}
> interface DataSourceFactory extends Serializable{
>  DataSource create();
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3500) JdbcIO: Improve connection management


 [ 
https://issues.apache.org/jira/browse/BEAM-3500?focusedWorklogId=80608=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80608
 ]

ASF GitHub Bot logged work on BEAM-3500:


Author: ASF GitHub Bot
Created on: 14/Mar/18 22:05
Start Date: 14/Mar/18 22:05
Worklog Time Spent: 10m 
  Work Description: jkff commented on a change in pull request #4461: 
[BEAM-3500] "Attach" JDBC connection to the bundle and add DataSourceFactory 
allowing full control of the way the DataSource is created
URL: https://github.com/apache/beam/pull/4461#discussion_r174624597
 
 

 ##
 File path: 
sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java
 ##
 @@ -327,8 +332,25 @@ DataSource buildDatasource() throws Exception{
 if (getConnectionProperties() != null && 
getConnectionProperties().get() != null) {
   
basicDataSource.setConnectionProperties(getConnectionProperties().get());
 }
-return basicDataSource;
+current = basicDataSource;
   }
+
+  // wrapping the datasource as a pooling datasource
+  DataSourceConnectionFactory connectionFactory = new 
DataSourceConnectionFactory(current);
+  PoolableConnectionFactory poolableConnectionFactory =
+  new PoolableConnectionFactory(connectionFactory, null);
+  GenericObjectPoolConfig poolConfig = new GenericObjectPoolConfig();
+  poolConfig.setMaxTotal(1);
+  poolConfig.setMinIdle(0);
+  poolConfig.setMinEvictableIdleTimeMillis(1);
 
 Review comment:
   Thanks, this looks reasonable, though is there any way to manually verify 
that this is working as intended in practice?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80608)
Time Spent: 5h  (was: 4h 50m)

> JdbcIO: Improve connection management
> -
>
> Key: BEAM-3500
> URL: https://issues.apache.org/jira/browse/BEAM-3500
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-jdbc
>Affects Versions: 2.2.0
>Reporter: Pawel Bartoszek
>Assignee: Jean-Baptiste Onofré
>Priority: Major
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> JdbcIO write DoFn acquires connection in {{@Setup}} and release it in 
> {{@Teardown}} methods, which means that connection might stay opened for days 
> in streaming job case. Keeping single connection open for so long might be 
> very risky as it's exposed to database, network etc issues.
> *Taking connection from the pool when it is actually needed*
> I suggest that connection would be taken from the connection pool in 
> {{executeBatch}} method and released when the batch is flushed. This will 
> allow the pool to take care of any returned unhealthy connections etc.
> *Make JdbcIO accept data source factory*
>  It would be nice if JdbcIO accepted DataSourceFactory rather than DataSource 
> itself. I am saying that because sink checks if DataSource implements 
> `Serializable` interface, which make it impossible to pass 
> BasicDataSource(used internally by sink) as it doesn’t implement this 
> interface. Something like:
> {code:java}
> interface DataSourceFactory extends Serializable{
>  DataSource create();
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-2817) Bigquery queries should allow options to run in batch mode or not


 [ 
https://issues.apache.org/jira/browse/BEAM-2817?focusedWorklogId=80605=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80605
 ]

ASF GitHub Bot logged work on BEAM-2817:


Author: ASF GitHub Bot
Created on: 14/Mar/18 22:04
Start Date: 14/Mar/18 22:04
Worklog Time Spent: 10m 
  Work Description: chamikaramj commented on a change in pull request 
#4606: [BEAM-2817] BigQuery queries are allowed to run in either BATCH or IN…
URL: https://github.com/apache/beam/pull/4606#discussion_r174624324
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQuerySource.java
 ##
 @@ -74,12 +77,19 @@ private BigQueryQuerySource(
   Boolean useLegacySql,
   BigQueryServices bqServices,
   Coder coder,
-  SerializableFunction parseFn) {
+  SerializableFunction parseFn,
+  Priority priority) {
 super(stepUuid, bqServices, coder, parseFn);
 this.query = checkNotNull(query, "query");
 this.flattenResults = checkNotNull(flattenResults, "flattenResults");
 this.useLegacySql = checkNotNull(useLegacySql, "useLegacySql");
 this.dryRunJobStats = new AtomicReference<>();
+if (priority != BigQueryIO.TypedRead.Priority.BATCH
 
 Review comment:
   Is there a need to rollback before the release ? Looks like this just keeps 
the previous default, right ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80605)
Time Spent: 2h 20m  (was: 2h 10m)

> Bigquery queries should allow options to run in batch mode or not
> -
>
> Key: BEAM-2817
> URL: https://issues.apache.org/jira/browse/BEAM-2817
> Project: Beam
>  Issue Type: Bug
>  Components: io-java-gcp
>Affects Versions: 2.0.0
>Reporter: Lara Schmidt
>Assignee: Justin Tumale
>Priority: Major
>  Labels: newbie, starter
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> When bigquery read does a query it sets the mode to batch. A batch query can 
> be very slow to schedule as it batches it with other queries. However it 
> doesn't use batch quota which is better for some cases. However, in some 
> cases a fast query is better (especially in timed tests). It would be a good 
> idea to have a configuration to the bigquery source to set this per-read.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-3853) 2.4.0 Python mobile gaming leader_board example is not working

2018-03-14 Thread Ahmet Altay (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399502#comment-16399502
 ] 

Ahmet Altay commented on BEAM-3853:
---

Thank you [~yifanzou], Could you add context related to what did you try, and 
how it failed?

> 2.4.0 Python mobile gaming leader_board example is not working
> --
>
> Key: BEAM-3853
> URL: https://issues.apache.org/jira/browse/BEAM-3853
> Project: Beam
>  Issue Type: Bug
>  Components: examples-python
>Reporter: yifan zou
>Assignee: Ahmet Altay
>Priority: Major
> Fix For: 2.5.0
>
>
> The leader_board example does not work on DirectRunner. No result are 
> generated. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-3853) 2.4.0 Python mobile gaming leader_board example is not working

yifan zou created BEAM-3853:
---

 Summary: 2.4.0 Python mobile gaming leader_board example is not 
working
 Key: BEAM-3853
 URL: https://issues.apache.org/jira/browse/BEAM-3853
 Project: Beam
  Issue Type: Bug
  Components: examples-python
Reporter: yifan zou
Assignee: Ahmet Altay
 Fix For: 2.5.0


The leader_board example does not work on DirectRunner. No result are 
generated. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80581=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80581
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372571599
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80581)
Time Spent: 8h 40m  (was: 8.5h)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 8h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80574=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80574
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372544755
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80574)
Time Spent: 7.5h  (was: 7h 20m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 7.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80578=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80578
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372556821
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80578)
Time Spent: 8h 10m  (was: 8h)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 8h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80594=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80594
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-373103589
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80594)
Time Spent: 10h 50m  (was: 10h 40m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 10h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80585=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80585
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372769983
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80585)
Time Spent: 9h 20m  (was: 9h 10m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 9h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80582=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80582
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372748104
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80582)
Time Spent: 8h 50m  (was: 8h 40m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 8h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80583=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80583
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372750919
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80583)
Time Spent: 9h  (was: 8h 50m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 9h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80593=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80593
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372882599
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80593)
Time Spent: 10h 40m  (was: 10.5h)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 10h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80572=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80572
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372543232
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80572)
Time Spent: 7h 10m  (was: 7h)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 7h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80580=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80580
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372556616
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80580)
Time Spent: 8.5h  (was: 8h 20m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 8.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80589=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80589
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372837621
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80589)
Time Spent: 10h  (was: 9h 50m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 10h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80571=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80571
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372523145
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80571)
Time Spent: 7h  (was: 6h 50m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 7h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80598=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80598
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-373169118
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80598)
Time Spent: 11.5h  (was: 11h 20m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 11.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80596=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80596
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-373121779
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80596)
Time Spent: 11h 10m  (was: 11h)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 11h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80595=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80595
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-373110940
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80595)
Time Spent: 11h  (was: 10h 50m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 11h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80597=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80597
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-373131286
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80597)
Time Spent: 11h 20m  (was: 11h 10m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 11h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80579=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80579
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372569042
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80579)
Time Spent: 8h 20m  (was: 8h 10m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 8h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80592=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80592
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372871497
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80592)
Time Spent: 10.5h  (was: 10h 20m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 10.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80575=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80575
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372551729
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80575)
Time Spent: 7h 40m  (was: 7.5h)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 7h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80584=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80584
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372753418
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80584)
Time Spent: 9h 10m  (was: 9h)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 9h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80590=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80590
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372838000
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80590)
Time Spent: 10h 10m  (was: 10h)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 10h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80577=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80577
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372553716
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80577)
Time Spent: 8h  (was: 7h 50m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 8h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80591=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80591
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372839589
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80591)
Time Spent: 10h 20m  (was: 10h 10m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 10h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80576=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80576
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372552829
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80576)
Time Spent: 7h 50m  (was: 7h 40m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 7h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80586=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80586
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372783721
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80586)
Time Spent: 9.5h  (was: 9h 20m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 9.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80588=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80588
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372804300
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80588)
Time Spent: 9h 50m  (was: 9h 40m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 9h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80573=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80573
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372543999
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80573)
Time Spent: 7h 20m  (was: 7h 10m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 7h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80587=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80587
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372802414
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80587)
Time Spent: 9h 40m  (was: 9.5h)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 9h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80570=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80570
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372518423
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80570)
Time Spent: 6h 50m  (was: 6h 40m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80566=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80566
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:31
Start Date: 14/Mar/18 21:31
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372498162
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80566)
Time Spent: 6h 10m  (was: 6h)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80567=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80567
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:31
Start Date: 14/Mar/18 21:31
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372500364
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80567)
Time Spent: 6h 20m  (was: 6h 10m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 6h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80568=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80568
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:31
Start Date: 14/Mar/18 21:31
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372511269
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80568)
Time Spent: 6.5h  (was: 6h 20m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80569=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80569
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:32
Start Date: 14/Mar/18 21:32
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-372517417
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80569)
Time Spent: 6h 40m  (was: 6.5h)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 6h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-3849) SolrIO: Expose connection timeout tuning for writes

2018-03-14 Thread Tim Robertson (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399446#comment-16399446
 ] 

Tim Robertson commented on BEAM-3849:
-

Unless I'm mistaken currently defaults are infinite. 

I observed this when running on a Beam/Spark/YARN cluster talking to a SOLR 5.4 
cloud (hdfs) cluster.  In my case the result was Spark task failures due to 
inactivity resulting in redoing the whole partition of the Spark RDD - the root 
cause in this case was in the SOLR server not responding but it seem to me like 
a useful addition to be able to explicitly control failure scenarios rather 
than relying on the retrying of the execution engine. Timing out and then 
explicitly being able to control retry behaviour for the batch (BEAM-3848) 
seemed like sensible additions which I patched in my version.

> SolrIO: Expose connection timeout tuning for writes
> ---
>
> Key: BEAM-3849
> URL: https://issues.apache.org/jira/browse/BEAM-3849
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-solr
>Affects Versions: 2.2.0, 2.3.0
>Reporter: Tim Robertson
>Assignee: Ismaël Mejía
>Priority: Minor
>
> A useful addition would be the ability to tune the socket and conncetion 
> timeouts for the underlying solrj client.
> Currently the createHttpClient() uses defaults only.
> This relates to BEAM-3820 and BEAM-3848 which together will help improve 
> stability of jobs doing large (billions of docs) loading of SOLR.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-2817) Bigquery queries should allow options to run in batch mode or not


 [ 
https://issues.apache.org/jira/browse/BEAM-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Kirpichov updated BEAM-2817:
---
Fix Version/s: (was: 2.4.0)

> Bigquery queries should allow options to run in batch mode or not
> -
>
> Key: BEAM-2817
> URL: https://issues.apache.org/jira/browse/BEAM-2817
> Project: Beam
>  Issue Type: Bug
>  Components: io-java-gcp
>Affects Versions: 2.0.0
>Reporter: Lara Schmidt
>Assignee: Justin Tumale
>Priority: Major
>  Labels: newbie, starter
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> When bigquery read does a query it sets the mode to batch. A batch query can 
> be very slow to schedule as it batches it with other queries. However it 
> doesn't use batch quota which is better for some cases. However, in some 
> cases a fast query is better (especially in timed tests). It would be a good 
> idea to have a configuration to the bigquery source to set this per-read.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Reopened] (BEAM-2817) Bigquery queries should allow options to run in batch mode or not


 [ 
https://issues.apache.org/jira/browse/BEAM-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Kirpichov reopened BEAM-2817:


Unfortunately the PR is not correct. I left some comments on 
https://github.com/apache/beam/pull/4606 - please take a look?

> Bigquery queries should allow options to run in batch mode or not
> -
>
> Key: BEAM-2817
> URL: https://issues.apache.org/jira/browse/BEAM-2817
> Project: Beam
>  Issue Type: Bug
>  Components: io-java-gcp
>Affects Versions: 2.0.0
>Reporter: Lara Schmidt
>Assignee: Justin Tumale
>Priority: Major
>  Labels: newbie, starter
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> When bigquery read does a query it sets the mode to batch. A batch query can 
> be very slow to schedule as it batches it with other queries. However it 
> doesn't use batch quota which is better for some cases. However, in some 
> cases a fast query is better (especially in timed tests). It would be a good 
> idea to have a configuration to the bigquery source to set this per-read.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-2817) Bigquery queries should allow options to run in batch mode or not


 [ 
https://issues.apache.org/jira/browse/BEAM-2817?focusedWorklogId=80562=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80562
 ]

ASF GitHub Bot logged work on BEAM-2817:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:13
Start Date: 14/Mar/18 21:13
Worklog Time Spent: 10m 
  Work Description: jkff commented on a change in pull request #4606: 
[BEAM-2817] BigQuery queries are allowed to run in either BATCH or IN…
URL: https://github.com/apache/beam/pull/4606#discussion_r174611761
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIOReadTest.java
 ##
 @@ -376,6 +377,124 @@ public void processElement(ProcessContext c) throws 
Exception {
 p.run();
   }
 
+  @Test
+  public void testReadFromTableInteractive()
+  throws IOException, InterruptedException {
+Table sometable = new Table();
+sometable.setSchema(
+new TableSchema()
+.setFields(
+ImmutableList.of(
+new TableFieldSchema().setName("name").setType("STRING"),
+new 
TableFieldSchema().setName("number").setType("INTEGER";
+sometable.setTableReference(
+new TableReference()
+.setProjectId("non-executing-project")
+.setDatasetId("somedataset")
+.setTableId("sometable"));
+sometable.setNumBytes(1024L * 1024L);
+FakeDatasetService fakeDatasetService = new FakeDatasetService();
+fakeDatasetService.createDataset("non-executing-project", "somedataset", 
"", "", null);
+fakeDatasetService.createTable(sometable);
+
+List records = Lists.newArrayList(
+new TableRow().set("name", "a").set("number", 1L),
+new TableRow().set("name", "b").set("number", 2L),
+new TableRow().set("name", "c").set("number", 3L));
+fakeDatasetService.insertAll(sometable.getTableReference(), records, null);
+
+FakeBigQueryServices fakeBqServices = new FakeBigQueryServices()
+.withJobService(new FakeJobService())
+.withDatasetService(fakeDatasetService);
+
+PTransform readTransform;
+BigQueryIO.TypedRead read =
+BigQueryIO.readTableRows()
+.from("non-executing-project:somedataset.sometable")
+.withTestServices(fakeBqServices)
+.withoutValidation()
+.usingInteractivePriority();
+readTransform = read;
+
+PCollection> output =
+p.apply(readTransform)
+.apply(
+ParDo.of(
+new DoFn>() {
+  @ProcessElement
+  public void processElement(ProcessContext c) throws 
Exception {
+c.output(
+KV.of(
+(String) c.element().get("name"),
+Long.valueOf((String) 
c.element().get("number";
+  }
+}));
+
+PAssert.that(output)
+.containsInAnyOrder(ImmutableList.of(KV.of("a", 1L), KV.of("b", 2L), 
KV.of("c", 3L)));
+assertEquals(read.getPriority(), 
BigQueryIO.TypedRead.Priority.INTERACTIVE);
 
 Review comment:
   Unfortunately this doesn't actually do anything, and the tests are vacuous 
(pass without verifying what they are supposed to verify).
   
   `read.getPriority()` simply returns what you set on 
`.usingInteractivePriority()` - you need to be verifying the priority that was 
used on the actual BigQuery load job.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80562)
Time Spent: 2h  (was: 1h 50m)

> Bigquery queries should allow options to run in batch mode or not
> -
>
> Key: BEAM-2817
> URL: https://issues.apache.org/jira/browse/BEAM-2817
> Project: Beam
>  Issue Type: Bug
>  Components: io-java-gcp
>Affects Versions: 2.0.0
>Reporter: Lara Schmidt
>Assignee: Justin Tumale
>Priority: Major
>  Labels: newbie, starter
> Fix For: 2.4.0
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> When bigquery read does a query it sets the mode to batch. A batch query can 
> be very slow to schedule as it batches it with other queries. However it 
> doesn't use batch quota which is better for some cases. However, in some 
> cases a fast query is better (especially in timed tests). It

[jira] [Work logged] (BEAM-2817) Bigquery queries should allow options to run in batch mode or not


 [ 
https://issues.apache.org/jira/browse/BEAM-2817?focusedWorklogId=80563=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80563
 ]

ASF GitHub Bot logged work on BEAM-2817:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:13
Start Date: 14/Mar/18 21:13
Worklog Time Spent: 10m 
  Work Description: jkff commented on a change in pull request #4606: 
[BEAM-2817] BigQuery queries are allowed to run in either BATCH or IN…
URL: https://github.com/apache/beam/pull/4606#discussion_r174611949
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQuerySource.java
 ##
 @@ -74,12 +77,19 @@ private BigQueryQuerySource(
   Boolean useLegacySql,
   BigQueryServices bqServices,
   Coder coder,
-  SerializableFunction parseFn) {
+  SerializableFunction parseFn,
+  Priority priority) {
 super(stepUuid, bqServices, coder, parseFn);
 this.query = checkNotNull(query, "query");
 this.flattenResults = checkNotNull(flattenResults, "flattenResults");
 this.useLegacySql = checkNotNull(useLegacySql, "useLegacySql");
 this.dryRunJobStats = new AtomicReference<>();
+if (priority != BigQueryIO.TypedRead.Priority.BATCH
 
 Review comment:
   This condition is always true - "priority != a || priority != b" is true by 
definition, because it can't equal a and b at the same time. I suppose you 
probably meant "&&", but then again I'm not sure why this condition is needed 
at all - why not just "this.priority = priority"?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80563)
Time Spent: 2h 10m  (was: 2h)

> Bigquery queries should allow options to run in batch mode or not
> -
>
> Key: BEAM-2817
> URL: https://issues.apache.org/jira/browse/BEAM-2817
> Project: Beam
>  Issue Type: Bug
>  Components: io-java-gcp
>Affects Versions: 2.0.0
>Reporter: Lara Schmidt
>Assignee: Justin Tumale
>Priority: Major
>  Labels: newbie, starter
> Fix For: 2.4.0
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> When bigquery read does a query it sets the mode to batch. A batch query can 
> be very slow to schedule as it batches it with other queries. However it 
> doesn't use batch quota which is better for some cases. However, in some 
> cases a fast query is better (especially in timed tests). It would be a good 
> idea to have a configuration to the bigquery source to set this per-read.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-3820) SolrIO: Allow changing batchSize for writes

2018-03-14 Thread Tim Robertson (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399419#comment-16399419
 ] 

Tim Robertson commented on BEAM-3820:
-

That is a laudable goal [~jkff] but the reality is Beam application developers 
will often be deploying on environments they don't fully control.  Without 
running diagnostics I don't see how Beam could self tune for every occassion.  

For my case I'm running batch (i.e. bounded sources) ETLs using Beam on Spark 
running in a YARN cluster on pre-production hardware (i.e. not top end by 
current standards nor carefully tuned).  Neither SolrIO nor ElasticSearchIO 
worked out the box, and resulted in a lot of task failures and significant 
retries before complete failure.  I was able to control some aspects of client 
throttling to get jobs to completion, but ended up tuning http timeouts, client 
retry behaviour (to stop full partition retries) and batch sizes in a patched 
SolrIO.

Given the IO connectors are all about making it easy to interface I'd urge 
sensible defaults be set on IO projects but let people turn the knobs they need 
to for various environments.

If it is a project decision not to support this closing as "won't fix" would be 
a good resolution so folk know to just write project specific adapters 
(BEAM-3849, BEAM-3848, BEAM-3026 would also fall under this category). 

> SolrIO: Allow changing batchSize for writes
> ---
>
> Key: BEAM-3820
> URL: https://issues.apache.org/jira/browse/BEAM-3820
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-solr
>Affects Versions: 2.2.0, 2.3.0
>Reporter: Tim Robertson
>Assignee: Ismaël Mejía
>Priority: Trivial
>
> The SolrIO hard codes the batchSize for writes at 1000.  It would be a good 
> addition to allow the user to set the batchSize explicitly (similar to the 
> ElasticsearchIO)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80559=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80559
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 21:04
Start Date: 14/Mar/18 21:04
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-373174140
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80559)
Time Spent: 6h  (was: 5h 50m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 6h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3840) Get Python Mobile-Gaming Running on Core Runners


 [ 
https://issues.apache.org/jira/browse/BEAM-3840?focusedWorklogId=80551=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80551
 ]

ASF GitHub Bot logged work on BEAM-3840:


Author: ASF GitHub Bot
Created on: 14/Mar/18 20:47
Start Date: 14/Mar/18 20:47
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4853: [BEAM-3840] Get 
python mobile-gaming automating on core runners -- testing
URL: https://github.com/apache/beam/pull/4853#issuecomment-373169118
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80551)
Time Spent: 5h 50m  (was: 5h 40m)

> Get Python Mobile-Gaming Running on Core Runners
> 
>
> Key: BEAM-3840
> URL: https://issues.apache.org/jira/browse/BEAM-3840
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python
>Affects Versions: 2.5.0
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3852) Migrate existing users to new channel ASF slack channel


 [ 
https://issues.apache.org/jira/browse/BEAM-3852?focusedWorklogId=80546=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80546
 ]

ASF GitHub Bot logged work on BEAM-3852:


Author: ASF GitHub Bot
Created on: 14/Mar/18 20:42
Start Date: 14/Mar/18 20:42
Worklog Time Spent: 10m 
  Work Description: asfgit commented on issue #401: [BEAM-3852] Update 
slack channel link and slack invite for new channel.
URL: https://github.com/apache/beam-site/pull/401#issuecomment-373167730
 
 
   Error: MergeBot encountered an unexpected error while processing this PR: 
502 Server Error: Proxy Error for url: 
https://builds.apache.org/job/beam_PreCommit_Website_Merge/api/python?tree=builds%5Bnumber%2Curl%5D.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80546)
Time Spent: 1h 20m  (was: 1h 10m)

> Migrate existing users to new channel ASF slack channel
> ---
>
> Key: BEAM-3852
> URL: https://issues.apache.org/jira/browse/BEAM-3852
> Project: Beam
>  Issue Type: New Feature
>  Components: website
>Reporter: Luke Cwik
>Assignee: Romain Manni-Bucau
>Priority: Minor
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Channel is https://the-asf.slack.com/messages/C9H0YNP3P/
> Created short link for #beam channel directly: 
> [https://s.apache.org/beam-slack-channel]
> Created short link for self-enrollment: https://s.apache.org/slack-invite



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3852) Migrate existing users to new channel ASF slack channel


 [ 
https://issues.apache.org/jira/browse/BEAM-3852?focusedWorklogId=80545=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80545
 ]

ASF GitHub Bot logged work on BEAM-3852:


Author: ASF GitHub Bot
Created on: 14/Mar/18 20:42
Start Date: 14/Mar/18 20:42
Worklog Time Spent: 10m 
  Work Description: asfgit commented on issue #401: [BEAM-3852] Update 
slack channel link and slack invite for new channel.
URL: https://github.com/apache/beam-site/pull/401#issuecomment-373167726
 
 
   MergeBot: Merge: PR 401 in state error with description Unexpected Error: 
502 Server Error: Proxy Error for url: 
https://builds.apache.org/job/beam_PreCommit_Website_Merge/api/python?tree=builds%5Bnumber%2Curl%5D..
 URL: http://mergebot-vm2.apache.org:8080/beam-website/401.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80545)
Time Spent: 1h 10m  (was: 1h)

> Migrate existing users to new channel ASF slack channel
> ---
>
> Key: BEAM-3852
> URL: https://issues.apache.org/jira/browse/BEAM-3852
> Project: Beam
>  Issue Type: New Feature
>  Components: website
>Reporter: Luke Cwik
>Assignee: Romain Manni-Bucau
>Priority: Minor
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Channel is https://the-asf.slack.com/messages/C9H0YNP3P/
> Created short link for #beam channel directly: 
> [https://s.apache.org/beam-slack-channel]
> Created short link for self-enrollment: https://s.apache.org/slack-invite



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam] branch master updated (dd675df -> e7f3d8b)

This is an automated email from the ASF dual-hosted git repository.

lcwik pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.

from dd675df Merge pull request #4807 from udim/pubsub-protobuf
add 070109d Add an InProcess SdkHarness Rule
add 4cba6ac Close Outstanding Control Clients on Service close
add 18e0b4d Don't try to close a cancelled Multiplexer
add 8ac47d9 fixup! Close Outstanding Control Clients on Service close
add 452471a fixup! Don't try to close a cancelled Multiplexer
add 0bda8d9 fixup! Add an InProcess SdkHarness Rule
add eaf0e64 fixup! Don't try to close a cancelled Multiplexer
new e7f3d8b [BEAM-3326] Add an InProcess SdkHarness Rule

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.

Summary of changes:
.../control/FnApiControlClientPoolService.java | 19 +++-
.../runners/fnexecution/data/GrpcDataService.java | 31 +++---
.../runners/fnexecution/InProcessSdkHarness.java | 113 +
.../control/FnApiControlClientPoolServiceTest.java | 58 ++-
.../fnexecution/control/SdkHarnessClientTest.java | 55 ++
5 files changed, 211 insertions(+), 65 deletions(-)
create mode 100644
runners/java-fn-execution/src/test/java/org/apache/beam/runners/fnexecution/InProcessSdkHarness.java

--
To stop receiving notification emails like this one, please contact
lc...@apache.org.

[jira] [Work logged] (BEAM-3326) Execute a Stage via the portability framework in the ReferenceRunner


 [ 
https://issues.apache.org/jira/browse/BEAM-3326?focusedWorklogId=80533=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80533
 ]

ASF GitHub Bot logged work on BEAM-3326:


Author: ASF GitHub Bot
Created on: 14/Mar/18 20:30
Start Date: 14/Mar/18 20:30
Worklog Time Spent: 10m 
  Work Description: lukecwik closed pull request #4825: [BEAM-3326] Add an 
InProcess SdkHarness Rule
URL: https://github.com/apache/beam/pull/4825
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/FnApiControlClientPoolService.java
 
b/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/FnApiControlClientPoolService.java
index 9d443427dc1..5a82a32a0b4 100644
--- 
a/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/FnApiControlClientPoolService.java
+++ 
b/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/FnApiControlClientPoolService.java
@@ -18,7 +18,10 @@
 package org.apache.beam.runners.fnexecution.control;
 
 import io.grpc.stub.StreamObserver;
+import java.util.Collection;
 import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.CopyOnWriteArrayList;
+import java.util.concurrent.atomic.AtomicBoolean;
 import org.apache.beam.model.fnexecution.v1.BeamFnApi;
 import org.apache.beam.model.fnexecution.v1.BeamFnControlGrpc;
 import org.apache.beam.runners.fnexecution.FnService;
@@ -31,6 +34,8 @@
   private static final Logger LOGGER = 
LoggerFactory.getLogger(FnApiControlClientPoolService.class);
 
   private final BlockingQueue clientPool;
+  private final Collection vendedClients = new 
CopyOnWriteArrayList<>();
+  private AtomicBoolean closed = new AtomicBoolean();
 
   private FnApiControlClientPoolService(BlockingQueue 
clientPool) {
 this.clientPool = clientPool;
@@ -61,6 +66,12 @@ public static FnApiControlClientPoolService 
offeringClientsToPool(
 LOGGER.info("Beam Fn Control client connected.");
 FnApiControlClient newClient = 
FnApiControlClient.forRequestObserver(requestObserver);
 try {
+  // Add the client to the pool of vended clients before making it 
available - we should close
+  // the client when we close even if no one has picked it up yet. This 
can occur after the
+  // service is closed, in which case the client will be discarded when 
the service is
+  // discarded, which should be performed by a call to #shutdownNow. The 
remote caller must be
+  // able to handle an unexpectedly terminated connection.
+  vendedClients.add(newClient);
   clientPool.put(newClient);
 } catch (InterruptedException e) {
   Thread.currentThread().interrupt();
@@ -70,7 +81,11 @@ public static FnApiControlClientPoolService 
offeringClientsToPool(
   }
 
   @Override
-  public void close() throws Exception {
-// The clients in the pool are owned by the consumer, which is responsible 
for closing them
+  public void close() {
+if (!closed.getAndSet(true)) {
+  for (FnApiControlClient vended : vendedClients) {
+vended.close();
+  }
+}
   }
 }
diff --git 
a/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/data/GrpcDataService.java
 
b/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/data/GrpcDataService.java
index 00b2a9afb92..a13d3d7d38c 100644
--- 
a/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/data/GrpcDataService.java
+++ 
b/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/data/GrpcDataService.java
@@ -53,8 +53,7 @@
 implements FnService, FnDataService {
   private static final Logger LOG = 
LoggerFactory.getLogger(GrpcDataService.class);
 
-  public static GrpcDataService create(
-  ExecutorService executor) {
+  public static GrpcDataService create(ExecutorService executor) {
 return new GrpcDataService(executor);
   }
 
@@ -62,6 +61,9 @@ public static GrpcDataService create(
   /**
* A collection of multiplexers which are not used to send data. A handle to 
these multiplexers is
* maintained in order to perform an orderly shutdown.
+   *
+   * TODO: (BEAM-3811) Replace with some cancellable collection, to ensure 
that new clients of a
+   * closed {@link GrpcDataService} are closed with that {@link 
GrpcDataService}.
*/
   private final Queue additionalMultiplexers;
 
@@ -105,7 +107,9 @@ public void close() throws Exception {
 // Shutdown remaining clients
   }
 }
-connectedClient.get().close();
+if (!connectedClient.isCancelled()) {
+

[beam] 01/01: [BEAM-3326] Add an InProcess SdkHarness Rule

This is an automated email from the ASF dual-hosted git repository.

lcwik pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit e7f3d8b21c38ceba6e1f89e853c7d79eeefed364
Merge: dd675df eaf0e64
Author: Lukasz Cwik 
AuthorDate: Wed Mar 14 13:30:40 2018 -0700

[BEAM-3326] Add an InProcess SdkHarness Rule

 .../control/FnApiControlClientPoolService.java |  19 +++-
 .../runners/fnexecution/data/GrpcDataService.java  |  31 +++---
 .../runners/fnexecution/InProcessSdkHarness.java   | 113 +
 .../control/FnApiControlClientPoolServiceTest.java |  58 ++-
 .../fnexecution/control/SdkHarnessClientTest.java  |  55 ++
 5 files changed, 211 insertions(+), 65 deletions(-)

-- 
To stop receiving notification emails like this one, please contact
lc...@apache.org.

[beam-site] 01/01: [BEAM-3852] Update slack channel link and slack invite for new channel.

This is an automated email from the ASF dual-hosted git repository.

lcwik pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam-site.git

commit 04bf78e059bbad23d154a1deb1e8c2f852638020
Merge: c0ea71c ad3307a
Author: Luke Cwik 
AuthorDate: Wed Mar 14 13:22:13 2018 -0700

[BEAM-3852] Update slack channel link and slack invite for new channel.

This closes #401

 src/get-started/support.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

-- 
To stop receiving notification emails like this one, please contact
lc...@apache.org.

[jira] [Work logged] (BEAM-3852) Migrate existing users to new channel ASF slack channel


 [ 
https://issues.apache.org/jira/browse/BEAM-3852?focusedWorklogId=80531=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80531
 ]

ASF GitHub Bot logged work on BEAM-3852:


Author: ASF GitHub Bot
Created on: 14/Mar/18 20:22
Start Date: 14/Mar/18 20:22
Worklog Time Spent: 10m 
  Work Description: asfgit closed pull request #401: [BEAM-3852] Update 
slack channel link and slack invite for new channel.
URL: https://github.com/apache/beam-site/pull/401
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/src/get-started/support.md b/src/get-started/support.md
index 5fd7fee58..3b17d7487 100644
--- a/src/get-started/support.md
+++ b/src/get-started/support.md
@@ -71,5 +71,5 @@ You can find answers to many questions about [Beam on Stack 
Overflow](http://sta
 
 ## Beam Slack Channel
 
-You can participate in discussions and ask questions on the [Beam Slack 
Channel](https://apachebeam.slack.com/).
-[Join](https://join.slack.com/t/apachebeam/shared_invite/enQtMzI4ODYzODY3MTY5LTIxOTJmMmFkMGVkMThhYmIwOWRkMTFiOGI3NDdlYzNmMmE2ZTA4N2JiMjc5ZDNmYTgxZGY5OTNlMDljMzM5NDU)
 the discussion now.
+You can participate in discussions and ask questions on the [Beam Slack 
Channel](https://s.apache.org/beam-slack-channel).
+[Join](https://s.apache.org/slack-invite) the discussion now.


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80531)
Time Spent: 1h  (was: 50m)

> Migrate existing users to new channel ASF slack channel
> ---
>
> Key: BEAM-3852
> URL: https://issues.apache.org/jira/browse/BEAM-3852
> Project: Beam
>  Issue Type: New Feature
>  Components: website
>Reporter: Luke Cwik
>Assignee: Romain Manni-Bucau
>Priority: Minor
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Channel is https://the-asf.slack.com/messages/C9H0YNP3P/
> Created short link for #beam channel directly: 
> [https://s.apache.org/beam-slack-channel]
> Created short link for self-enrollment: https://s.apache.org/slack-invite



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-3852) Migrate existing users to new channel ASF slack channel

2018-03-14 Thread Luke Cwik (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399243#comment-16399243
 ] 

Luke Cwik commented on BEAM-3852:
-

Merged in https://github.com/apache/beam-site/pull/401

> Migrate existing users to new channel ASF slack channel
> ---
>
> Key: BEAM-3852
> URL: https://issues.apache.org/jira/browse/BEAM-3852
> Project: Beam
>  Issue Type: New Feature
>  Components: website
>Reporter: Luke Cwik
>Assignee: Romain Manni-Bucau
>Priority: Minor
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Channel is https://the-asf.slack.com/messages/C9H0YNP3P/
> Created short link for #beam channel directly: 
> [https://s.apache.org/beam-slack-channel]
> Created short link for self-enrollment: https://s.apache.org/slack-invite



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam-site] branch asf-site updated (c0ea71c -> 04bf78e)

This is an automated email from the ASF dual-hosted git repository.

lcwik pushed a change to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam-site.git.


from c0ea71c  [BEAM-3846] Add a link to be able to self join the Apache 
Beam slack channel.
 add ad3307a  [BEAM-3852] Update slack channel link and slack invite for 
new channel.
 new 04bf78e  [BEAM-3852] Update slack channel link and slack invite for 
new channel.

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 src/get-started/support.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

-- 
To stop receiving notification emails like this one, please contact
lc...@apache.org.

Build failed in Jenkins: beam_PostCommit_Java_MavenInstall #6204

See 


--
[...truncated 1.44 MB...]
2018-03-14T20:12:39.550 [INFO] Excluding 
org.apache.avro:avro-mapred:jar:hadoop2:1.7.7 from the shaded jar.
2018-03-14T20:12:39.550 [INFO] Excluding org.apache.avro:avro-ipc:jar:1.7.7 
from the shaded jar.
2018-03-14T20:12:39.550 [INFO] Excluding 
org.apache.avro:avro-ipc:jar:tests:1.7.7 from the shaded jar.
2018-03-14T20:12:39.550 [INFO] Excluding com.twitter:chill_2.11:jar:0.8.0 from 
the shaded jar.
2018-03-14T20:12:39.550 [INFO] Excluding 
com.esotericsoftware:kryo-shaded:jar:3.0.3 from the shaded jar.
2018-03-14T20:12:39.550 [INFO] Excluding com.esotericsoftware:minlog:jar:1.3.0 
from the shaded jar.
2018-03-14T20:12:39.550 [INFO] Excluding com.twitter:chill-java:jar:0.8.0 from 
the shaded jar.
2018-03-14T20:12:39.551 [INFO] Excluding 
org.apache.xbean:xbean-asm5-shaded:jar:4.4 from the shaded jar.
2018-03-14T20:12:39.551 [INFO] Excluding 
org.apache.hadoop:hadoop-client:jar:2.7.3 from the shaded jar.
2018-03-14T20:12:39.551 [INFO] Excluding 
org.apache.hadoop:hadoop-common:jar:2.7.3 from the shaded jar.
2018-03-14T20:12:39.551 [INFO] Excluding xmlenc:xmlenc:jar:0.52 from the shaded 
jar.
2018-03-14T20:12:39.551 [INFO] Excluding 
commons-httpclient:commons-httpclient:jar:3.1 from the shaded jar.
2018-03-14T20:12:39.551 [INFO] Excluding javax.servlet.jsp:jsp-api:jar:2.1 from 
the shaded jar.
2018-03-14T20:12:39.551 [INFO] Excluding 
commons-configuration:commons-configuration:jar:1.6 from the shaded jar.
2018-03-14T20:12:39.552 [INFO] Excluding 
commons-digester:commons-digester:jar:1.8 from the shaded jar.
2018-03-14T20:12:39.552 [INFO] Excluding 
commons-beanutils:commons-beanutils-core:jar:1.8.0 from the shaded jar.
2018-03-14T20:12:39.552 [INFO] Excluding 
org.apache.hadoop:hadoop-auth:jar:2.7.3 from the shaded jar.
2018-03-14T20:12:39.552 [INFO] Excluding 
org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15 from the 
shaded jar.
2018-03-14T20:12:39.552 [INFO] Excluding 
org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15 from the shaded jar.
2018-03-14T20:12:39.552 [INFO] Excluding 
org.apache.directory.api:api-asn1-api:jar:1.0.0-M20 from the shaded jar.
2018-03-14T20:12:39.552 [INFO] Excluding 
org.apache.directory.api:api-util:jar:1.0.0-M20 from the shaded jar.
2018-03-14T20:12:39.553 [INFO] Excluding 
org.apache.curator:curator-client:jar:2.7.1 from the shaded jar.
2018-03-14T20:12:39.553 [INFO] Excluding 
org.mortbay.jetty:jetty-util:jar:6.1.26 from the shaded jar.
2018-03-14T20:12:39.553 [INFO] Excluding 
org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.3 from the shaded jar.
2018-03-14T20:12:39.553 [INFO] Excluding 
org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.3 from the shaded jar.
2018-03-14T20:12:39.553 [INFO] Excluding 
org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.3 from the shaded jar.
2018-03-14T20:12:39.553 [INFO] Excluding 
org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.3 from the shaded jar.
2018-03-14T20:12:39.553 [INFO] Excluding 
org.apache.hadoop:hadoop-yarn-api:jar:2.7.3 from the shaded jar.
2018-03-14T20:12:39.554 [INFO] Excluding 
org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3 from the shaded jar.
2018-03-14T20:12:39.554 [INFO] Excluding 
org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.3 from the shaded 
jar.
2018-03-14T20:12:39.554 [INFO] Excluding 
org.apache.hadoop:hadoop-annotations:jar:2.7.3 from the shaded jar.
2018-03-14T20:12:39.554 [INFO] Excluding 
org.apache.spark:spark-launcher_2.11:jar:2.2.1 from the shaded jar.
2018-03-14T20:12:39.554 [INFO] Excluding 
org.apache.spark:spark-network-common_2.11:jar:2.2.1 from the shaded jar.
2018-03-14T20:12:39.554 [INFO] Excluding 
org.fusesource.leveldbjni:leveldbjni-all:jar:1.8 from the shaded jar.
2018-03-14T20:12:39.554 [INFO] Excluding 
org.apache.spark:spark-network-shuffle_2.11:jar:2.2.1 from the shaded jar.
2018-03-14T20:12:39.554 [INFO] Excluding 
org.apache.spark:spark-unsafe_2.11:jar:2.2.1 from the shaded jar.
2018-03-14T20:12:39.555 [INFO] Excluding net.java.dev.jets3t:jets3t:jar:0.9.3 
from the shaded jar.
2018-03-14T20:12:39.555 [INFO] Excluding javax.activation:activation:jar:1.1.1 
from the shaded jar.
2018-03-14T20:12:39.555 [INFO] Excluding mx4j:mx4j:jar:3.0.2 from the shaded 
jar.
2018-03-14T20:12:39.555 [INFO] Excluding javax.mail:mail:jar:1.4.7 from the 
shaded jar.
2018-03-14T20:12:39.555 [INFO] Excluding 
org.bouncycastle:bcprov-jdk15on:jar:1.51 from the shaded jar.
2018-03-14T20:12:39.555 [INFO] Excluding 
com.jamesmurty.utils:java-xmlbuilder:jar:1.0 from the shaded jar.
2018-03-14T20:12:39.555 [INFO] Excluding net.iharder:base64:jar:2.3.8 from the 
shaded jar.
2018-03-14T20:12:39.556 [INFO] Excluding 
org.apache.curator:curator-recipes:jar:2.6.0 from the shaded jar.
2018-03-14T20:12:39.556 [INFO] Excluding 
org.apache.curator:curator-framework:jar:2.6.0 from the

[beam-site] 01/01: This closes #401

2018-03-14 Thread mergebot-role

This is an automated email from the ASF dual-hosted git repository.

mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git

commit c7523f5007ea4ee392a5193dc971dc53098b1610
Merge: c0ea71c ad3307a
Author: Mergebot 
AuthorDate: Wed Mar 14 13:13:09 2018 -0700

This closes #401

 src/get-started/support.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

-- 
To stop receiving notification emails like this one, please contact
mergebot-r...@apache.org.

[beam-site] branch mergebot updated (adf002b -> c7523f5)

2018-03-14 Thread mergebot-role

This is an automated email from the ASF dual-hosted git repository.

mergebot-role pushed a change to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git.


 discard adf002b  This closes #401
 new c7523f5  This closes #401

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (adf002b)
\
 N -- N -- N   refs/heads/mergebot (c7523f5)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:

-- 
To stop receiving notification emails like this one, please contact
mergebot-r...@apache.org.

[jira] [Work logged] (BEAM-3852) Migrate existing users to new channel ASF slack channel


 [ 
https://issues.apache.org/jira/browse/BEAM-3852?focusedWorklogId=80526=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80526
 ]

ASF GitHub Bot logged work on BEAM-3852:


Author: ASF GitHub Bot
Created on: 14/Mar/18 20:12
Start Date: 14/Mar/18 20:12
Worklog Time Spent: 10m 
  Work Description: asfgit commented on issue #401: [BEAM-3852] Update 
slack channel link and slack invite for new channel.
URL: https://github.com/apache/beam-site/pull/401#issuecomment-373158795
 
 
   Error: MergeBot encountered an unexpected error while processing this PR: 
502 Server Error: Proxy Error for url: 
https://builds.apache.org/job/beam_PreCommit_Website_Merge/api/python?tree=builds%5Bnumber%2Curl%5D.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80526)
Time Spent: 50m  (was: 40m)

> Migrate existing users to new channel ASF slack channel
> ---
>
> Key: BEAM-3852
> URL: https://issues.apache.org/jira/browse/BEAM-3852
> Project: Beam
>  Issue Type: New Feature
>  Components: website
>Reporter: Luke Cwik
>Assignee: Romain Manni-Bucau
>Priority: Minor
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Channel is https://the-asf.slack.com/messages/C9H0YNP3P/
> Created short link for #beam channel directly: 
> [https://s.apache.org/beam-slack-channel]
> Created short link for self-enrollment: https://s.apache.org/slack-invite



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3852) Migrate existing users to new channel ASF slack channel


 [ 
https://issues.apache.org/jira/browse/BEAM-3852?focusedWorklogId=80525=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80525
 ]

ASF GitHub Bot logged work on BEAM-3852:


Author: ASF GitHub Bot
Created on: 14/Mar/18 20:12
Start Date: 14/Mar/18 20:12
Worklog Time Spent: 10m 
  Work Description: asfgit commented on issue #401: [BEAM-3852] Update 
slack channel link and slack invite for new channel.
URL: https://github.com/apache/beam-site/pull/401#issuecomment-373158788
 
 
   MergeBot: Merge: PR 401 in state error with description Unexpected Error: 
502 Server Error: Proxy Error for url: 
https://builds.apache.org/job/beam_PreCommit_Website_Merge/api/python?tree=builds%5Bnumber%2Curl%5D..
 URL: http://mergebot-vm2.apache.org:8080/beam-website/401.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80525)
Time Spent: 40m  (was: 0.5h)

> Migrate existing users to new channel ASF slack channel
> ---
>
> Key: BEAM-3852
> URL: https://issues.apache.org/jira/browse/BEAM-3852
> Project: Beam
>  Issue Type: New Feature
>  Components: website
>Reporter: Luke Cwik
>Assignee: Romain Manni-Bucau
>Priority: Minor
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Channel is https://the-asf.slack.com/messages/C9H0YNP3P/
> Created short link for #beam channel directly: 
> [https://s.apache.org/beam-slack-channel]
> Created short link for self-enrollment: https://s.apache.org/slack-invite



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3851) Support element timestamps while publishing to Kafka.


 [ 
https://issues.apache.org/jira/browse/BEAM-3851?focusedWorklogId=80511=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80511
 ]

ASF GitHub Bot logged work on BEAM-3851:


Author: ASF GitHub Bot
Created on: 14/Mar/18 20:00
Start Date: 14/Mar/18 20:00
Worklog Time Spent: 10m 
  Work Description: rangadi commented on issue #4868: [BEAM-3851] Option to 
preserve element timestamp while publishing to Kafka.
URL: https://github.com/apache/beam/pull/4868#issuecomment-373155519
 
 
   R: @XuMingmin 
   cc: @aljoscha, @chamikaramj 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80511)
Time Spent: 20m  (was: 10m)

> Support element timestamps while publishing to Kafka.
> -
>
> Key: BEAM-3851
> URL: https://issues.apache.org/jira/browse/BEAM-3851
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-kafka
>Affects Versions: 2.3.0
>Reporter: Raghu Angadi
>Assignee: Raghu Angadi
>Priority: Major
> Fix For: 2.4.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> KafkaIO sink should support using input element timestamp for the message 
> published to Kafka. Otherwise there is no way for user to influence the 
> timestamp of the messages in Kafka sink.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3516) SpannerWriteGroupFn does not respect mutation limits