[jira] [Work logged] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?focusedWorklogId=139534=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139534 ] ASF GitHub Bot logged work on BEAM-5187: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-5254) Add Samza Runner translator registrar and refactor config generation

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5254?focusedWorklogId=139516=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139516 ] ASF GitHub Bot logged work on BEAM-5254: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-5247) Remove slf4j-simple binding from dependencies

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5247?focusedWorklogId=139513=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139513 ] ASF GitHub Bot logged work on BEAM-5247: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-5254) Add Samza Runner translator registrar and refactor config generation

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5254?focusedWorklogId=139512=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139512 ] ASF GitHub Bot logged work on BEAM-5254: Author: ASF GitHub Bot

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596979#comment-16596979 ] Ryan Williams commented on BEAM-5036: - fwiw, on the "write directly to final files" question, there is

[jira] [Work logged] (BEAM-4461) Create a library of useful transforms that use schemas

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4461?focusedWorklogId=139469=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139469 ] ASF GitHub Bot logged work on BEAM-4461: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-4461) Create a library of useful transforms that use schemas

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4461?focusedWorklogId=139470=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139470 ] ASF GitHub Bot logged work on BEAM-4461: Author: ASF GitHub Bot

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Reuven Lax (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596950#comment-16596950 ] Reuven Lax commented on BEAM-5036: -- Actually I believe GCS _does_ support efficient object rename. Check

[jira] [Work logged] (BEAM-5262) JobState support for Reference Runner

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5262?focusedWorklogId=139461=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139461 ] ASF GitHub Bot logged work on BEAM-5262: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-5262) JobState support for Reference Runner

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5262?focusedWorklogId=139460=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139460 ] ASF GitHub Bot logged work on BEAM-5262: Author: ASF GitHub Bot

[jira] [Created] (BEAM-5262) JobState support for Reference Runner

2018-08-29 Thread Ankur Goenka (JIRA)
Ankur Goenka created BEAM-5262: -- Summary: JobState support for Reference Runner Key: BEAM-5262 URL: https://issues.apache.org/jira/browse/BEAM-5262 Project: Beam Issue Type: Bug

[jira] [Work logged] (BEAM-4495) Create website pre-commits for apache/beam repository

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4495?focusedWorklogId=139446=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139446 ] ASF GitHub Bot logged work on BEAM-4495: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-4495) Create website pre-commits for apache/beam repository

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4495?focusedWorklogId=139444=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139444 ] ASF GitHub Bot logged work on BEAM-4495: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-4495) Create website pre-commits for apache/beam repository

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4495?focusedWorklogId=139443=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139443 ] ASF GitHub Bot logged work on BEAM-4495: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-4495) Create website pre-commits for apache/beam repository

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4495?focusedWorklogId=139445=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139445 ] ASF GitHub Bot logged work on BEAM-4495: Author: ASF GitHub Bot

[jira] [Comment Edited] (BEAM-5260) Investigate if nested row constructor works

2018-08-29 Thread Rui Wang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596898#comment-16596898 ] Rui Wang edited comment on BEAM-5260 at 8/29/18 10:12 PM: -- Furthermore, we expect

[jira] [Commented] (BEAM-5260) Investigate if nested row constructor works

2018-08-29 Thread Rui Wang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596898#comment-16596898 ] Rui Wang commented on BEAM-5260: Furthermore, we expect  {code:java} SELECT ROW(row_col.$0,

[jira] [Work logged] (BEAM-5261) Sql array indexing should be 1-based

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5261?focusedWorklogId=139436=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139436 ] ASF GitHub Bot logged work on BEAM-5261: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-5261) Sql array indexing should be 1-based

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5261?focusedWorklogId=139435=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139435 ] ASF GitHub Bot logged work on BEAM-5261: Author: ASF GitHub Bot

[beam] 01/01: Merge pull request #6300 from apilloud/index

2018-08-29 Thread anton
This is an automated email from the ASF dual-hosted git repository. anton pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 6497b0b24888178441f05d79ab101dc40760388b Merge: 693de50 dceffa1 Author: Anton Kedin

[jira] [Work logged] (BEAM-5261) Sql array indexing should be 1-based

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5261?focusedWorklogId=139433=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139433 ] ASF GitHub Bot logged work on BEAM-5261: Author: ASF GitHub Bot

[beam] branch master updated (693de50 -> 6497b0b)

2018-08-29 Thread anton
This is an automated email from the ASF dual-hosted git repository. anton pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 693de50 Moving to 2.8.0-SNAPSHOT on master branch. add dceffa1 [BEAM-5261] Sql array indexing is 1 based

[jira] [Work logged] (BEAM-5261) Sql array indexing should be 1-based

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5261?focusedWorklogId=139432=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139432 ] ASF GitHub Bot logged work on BEAM-5261: Author: ASF GitHub Bot

[jira] [Updated] (BEAM-5261) Sql array indexing should be 1-based

2018-08-29 Thread Andrew Pilloud (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Pilloud updated BEAM-5261: - Fix Version/s: (was: 2.7.0) > Sql array indexing should be 1-based >

[jira] [Updated] (BEAM-5261) Sql array indexing should be 1-based

2018-08-29 Thread Andrew Pilloud (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Pilloud updated BEAM-5261: - Fix Version/s: 2.7.0 > Sql array indexing should be 1-based >

[jira] [Updated] (BEAM-5261) Sql array indexing should be 1-based

2018-08-29 Thread Andrew Pilloud (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Pilloud updated BEAM-5261: - Affects Version/s: 2.7.0 2.5.0 2.6.0 > Sql array

[jira] [Commented] (BEAM-5261) Sql array indexing should be 1-based

2018-08-29 Thread Andrew Pilloud (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596876#comment-16596876 ] Andrew Pilloud commented on BEAM-5261: --

[jira] [Assigned] (BEAM-5261) Sql array indexing should be 1-based

2018-08-29 Thread Andrew Pilloud (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Pilloud reassigned BEAM-5261: Assignee: Andrew Pilloud (was: Xu Mingmin) > Sql array indexing should be 1-based >

[jira] [Created] (BEAM-5261) Sql array indexing should be 1-based

2018-08-29 Thread Andrew Pilloud (JIRA)
Andrew Pilloud created BEAM-5261: Summary: Sql array indexing should be 1-based Key: BEAM-5261 URL: https://issues.apache.org/jira/browse/BEAM-5261 Project: Beam Issue Type: Bug

[jira] [Work logged] (BEAM-1909) BigQuery read transform fails for DirectRunner when querying non-US regions

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1909?focusedWorklogId=139422=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139422 ] ASF GitHub Bot logged work on BEAM-1909: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-1909) BigQuery read transform fails for DirectRunner when querying non-US regions

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1909?focusedWorklogId=139423=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139423 ] ASF GitHub Bot logged work on BEAM-1909: Author: ASF GitHub Bot

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Tim Robertson (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596831#comment-16596831 ] Tim Robertson commented on BEAM-5036: - [~reuvenlax], I believe those scenarios are covered as you

[jira] [Work logged] (BEAM-4495) Create website pre-commits for apache/beam repository

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4495?focusedWorklogId=139412=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139412 ] ASF GitHub Bot logged work on BEAM-4495: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-4495) Create website pre-commits for apache/beam repository

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4495?focusedWorklogId=139407=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139407 ] ASF GitHub Bot logged work on BEAM-4495: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-4495) Create website pre-commits for apache/beam repository

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4495?focusedWorklogId=139409=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139409 ] ASF GitHub Bot logged work on BEAM-4495: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-4495) Create website pre-commits for apache/beam repository

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4495?focusedWorklogId=139406=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139406 ] ASF GitHub Bot logged work on BEAM-4495: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-4495) Create website pre-commits for apache/beam repository

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4495?focusedWorklogId=139410=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139410 ] ASF GitHub Bot logged work on BEAM-4495: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-4495) Create website pre-commits for apache/beam repository

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4495?focusedWorklogId=139411=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139411 ] ASF GitHub Bot logged work on BEAM-4495: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-4495) Create website pre-commits for apache/beam repository

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4495?focusedWorklogId=139408=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139408 ] ASF GitHub Bot logged work on BEAM-4495: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-4461) Create a library of useful transforms that use schemas

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4461?focusedWorklogId=139389=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139389 ] ASF GitHub Bot logged work on BEAM-4461: Author: ASF GitHub Bot

[jira] [Closed] (BEAM-5027) Schemas do not work on Dataflow runner of FnApi Runner

2018-08-29 Thread Reuven Lax (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax closed BEAM-5027. Resolution: Fixed Fix Version/s: 2.7.0 > Schemas do not work on Dataflow runner of FnApi Runner >

[beam] 01/01: Create release branch for version 2.7.0.

2018-08-29 Thread ccy
This is an automated email from the ASF dual-hosted git repository. ccy pushed a commit to branch release-2.7.0 in repository https://gitbox.apache.org/repos/asf/beam.git commit bd9dc3fe79702ead2948eea8c13955c1e50e464b Author: Charles Chen AuthorDate: Wed Aug 29 12:15:49 2018 -0700 Create

[beam] branch release-2.7.0 created (now bd9dc3f)

2018-08-29 Thread ccy
This is an automated email from the ASF dual-hosted git repository. ccy pushed a change to branch release-2.7.0 in repository https://gitbox.apache.org/repos/asf/beam.git. at bd9dc3f Create release branch for version 2.7.0. This branch includes the following new commits: new

[beam] branch master updated: Moving to 2.8.0-SNAPSHOT on master branch.

2018-08-29 Thread ccy
This is an automated email from the ASF dual-hosted git repository. ccy pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 693de50 Moving to 2.8.0-SNAPSHOT on master branch.

[jira] [Work logged] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?focusedWorklogId=139367=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139367 ] ASF GitHub Bot logged work on BEAM-5187: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?focusedWorklogId=139361=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139361 ] ASF GitHub Bot logged work on BEAM-5187: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?focusedWorklogId=139359=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139359 ] ASF GitHub Bot logged work on BEAM-5187: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?focusedWorklogId=139358=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139358 ] ASF GitHub Bot logged work on BEAM-5187: Author: ASF GitHub Bot

[jira] [Updated] (BEAM-5258) Investigate if we can disable Row type flattening in Calcite

2018-08-29 Thread Rui Wang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5258: --- Summary: Investigate if we can disable Row type flattening in Calcite (was: Investigate if we can disable

[jira] [Updated] (BEAM-5260) Investigate if nested row constructor works

2018-08-29 Thread Rui Wang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5260: --- Description: Check if    {code:java} SELECT ROW(row.string, ROW(row.row.int, row.row.long)){code} {{works.}}

[jira] [Updated] (BEAM-5259) Test nested select query with nested Row

2018-08-29 Thread Rui Wang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5259: --- Description: add tests to something like     {code:java} SELECT i.row.string from ( select row from table

[jira] [Updated] (BEAM-5260) Investigate if nested row constructor works

2018-08-29 Thread Rui Wang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5260: --- Description: Check if    `SELECT ROW(row.string, ROW(row.row.int, row.row.long))` works. was: Check if 

[jira] [Updated] (BEAM-5260) Investigate if nested row constructor works

2018-08-29 Thread Rui Wang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5260: --- Description: Check if    ```SELECT ROW(row.string, ROW(row.row.int, row.row.long))``` works. was: Check

[jira] [Created] (BEAM-5260) Investigate if nested row constructor works

2018-08-29 Thread Rui Wang (JIRA)
Rui Wang created BEAM-5260: -- Summary: Investigate if nested row constructor works Key: BEAM-5260 URL: https://issues.apache.org/jira/browse/BEAM-5260 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-5259) Test nested select query with nested Row

2018-08-29 Thread Rui Wang (JIRA)
Rui Wang created BEAM-5259: -- Summary: Test nested select query with nested Row Key: BEAM-5259 URL: https://issues.apache.org/jira/browse/BEAM-5259 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-5258) Investigate if we can disable Row type flattening

2018-08-29 Thread Rui Wang (JIRA)
Rui Wang created BEAM-5258: -- Summary: Investigate if we can disable Row type flattening Key: BEAM-5258 URL: https://issues.apache.org/jira/browse/BEAM-5258 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-5257) Java TextIO should accept an optional Encoding parameter

2018-08-29 Thread Peter Farkas (JIRA)
Peter Farkas created BEAM-5257: -- Summary: Java TextIO should accept an optional Encoding parameter Key: BEAM-5257 URL: https://issues.apache.org/jira/browse/BEAM-5257 Project: Beam Issue Type:

[jira] [Work logged] (BEAM-4495) Create website pre-commits for apache/beam repository

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4495?focusedWorklogId=139308=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139308 ] ASF GitHub Bot logged work on BEAM-4495: Author: ASF GitHub Bot

[jira] [Assigned] (BEAM-5191) Add support for writing to BigQuery clustered tables

2018-08-29 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-5191: --- Assignee: Wout Scheepers (was: Chamikara Jayalath) > Add support for writing to BigQuery clustered

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Neville Li (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596565#comment-16596565 ] Neville Li commented on BEAM-5036: -- Yeah that's why I figured. So there's no way to reduce this overhead

[jira] [Comment Edited] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Neville Li (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596565#comment-16596565 ] Neville Li edited comment on BEAM-5036 at 8/29/18 4:23 PM: --- Yeah that's what I

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Reuven Lax (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596548#comment-16596548 ] Reuven Lax commented on BEAM-5036: -- [~sinisa_lyh] I don't think there is any semantically-correct way to

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Reuven Lax (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596545#comment-16596545 ] Reuven Lax commented on BEAM-5036: -- As to ignoring errors: the one thing we need to make sure is that the

[jira] [Work logged] (BEAM-5255) Fix over-aggressive division futurization.

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5255?focusedWorklogId=139301=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139301 ] ASF GitHub Bot logged work on BEAM-5255: Author: ASF GitHub Bot

[beam] branch master updated (053b1d8 -> e278e57)

2018-08-29 Thread robertwb
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 053b1d8 Merge pull request #6279 from echauchot/BEAM-5172-flaky-es2-tests add 4e5c642 [BEAM-5255] Fix

[beam] 01/01: Merge pull request #6293 [BEAM-5255] Fix over-aggressive division futurization in benchmarks.

2018-08-29 Thread robertwb
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit e278e57077d8ab6459a9833eccd2ef20dc2faeae Merge: 053b1d8 4e5c642 Author: Robert Bradshaw AuthorDate: Wed Aug 29 18:05:09

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Jozef Vilcek (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596542#comment-16596542 ] Jozef Vilcek commented on BEAM-5036: Aha, I see. I did not know it might use to work differently

[jira] [Work logged] (BEAM-5255) Fix over-aggressive division futurization.

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5255?focusedWorklogId=139300=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139300 ] ASF GitHub Bot logged work on BEAM-5255: Author: ASF GitHub Bot

[jira] [Commented] (BEAM-5255) Fix over-aggressive division futurization.

2018-08-29 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596524#comment-16596524 ] Valentyn Tymofieiev commented on BEAM-5255: --- We caught a several incorrect division conversion

[jira] [Assigned] (BEAM-5255) Fix over-aggressive division futurization.

2018-08-29 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev reassigned BEAM-5255: - Assignee: Valentyn Tymofieiev (was: Ahmet Altay) > Fix over-aggressive division

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Neville Li (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596513#comment-16596513 ] Neville Li commented on BEAM-5036: -- {{copy+delete}} is still expensive on GCS, especially when running

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Jozef Vilcek (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596491#comment-16596491 ] Jozef Vilcek commented on BEAM-5036: As far as I understand this, there can be only gain, no

[jira] [Work logged] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?focusedWorklogId=139288=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139288 ] ASF GitHub Bot logged work on BEAM-5187: Author: ASF GitHub Bot

[jira] [Closed] (BEAM-5256) Update Bigtable dependency

2018-08-29 Thread Kevin Si (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Si closed BEAM-5256. -- Resolution: Won't Fix > Update Bigtable dependency > -- > > Key:

[jira] [Commented] (BEAM-5256) Update Bigtable dependency

2018-08-29 Thread Kevin Si (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596427#comment-16596427 ] Kevin Si commented on BEAM-5256: Second thought, we may want to use a different version of Bigtable

[jira] [Work logged] (BEAM-5124) Write Euphoria in Beam documentation

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5124?focusedWorklogId=139285=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139285 ] ASF GitHub Bot logged work on BEAM-5124: Author: ASF GitHub Bot

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Neville Li (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596410#comment-16596410 ] Neville Li commented on BEAM-5036: -- Yeah that's my main concern. We use GCS almost exclusively so all our

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Tim Robertson (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596407#comment-16596407 ] Tim Robertson commented on BEAM-5036: - Yes [~sinisa_lyh] it does. I have observed big performance

[jira] [Work logged] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?focusedWorklogId=139276=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139276 ] ASF GitHub Bot logged work on BEAM-5187: Author: ASF GitHub Bot

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Neville Li (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596392#comment-16596392 ] Neville Li commented on BEAM-5036: -- If I understand this correctly, this issue affects all file based

[jira] [Created] (BEAM-5256) Update Bigtable dependency

2018-08-29 Thread Kevin Si (JIRA)
Kevin Si created BEAM-5256: -- Summary: Update Bigtable dependency Key: BEAM-5256 URL: https://issues.apache.org/jira/browse/BEAM-5256 Project: Beam Issue Type: Improvement Components:

[jira] [Work logged] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?focusedWorklogId=139270=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139270 ] ASF GitHub Bot logged work on BEAM-5187: Author: ASF GitHub Bot

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Tim Robertson (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596353#comment-16596353 ] Tim Robertson commented on BEAM-5036: - Thanks [~echauchot] Gcs and S3 have no notion of a rename they

[jira] [Comment Edited] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Tim Robertson (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596353#comment-16596353 ] Tim Robertson edited comment on BEAM-5036 at 8/29/18 1:51 PM: -- Thanks

[jira] [Commented] (BEAM-3926) Support MetricsPusher in Dataflow Runner

2018-08-29 Thread Etienne Chauchot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596346#comment-16596346 ] Etienne Chauchot commented on BEAM-3926: Hi [~foegler] do you need any help on that ? > Support

[jira] [Comment Edited] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Etienne Chauchot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596336#comment-16596336 ] Etienne Chauchot edited comment on BEAM-5036 at 8/29/18 1:40 PM: - I have

[jira] [Work logged] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?focusedWorklogId=139257=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139257 ] ASF GitHub Bot logged work on BEAM-5187: Author: ASF GitHub Bot

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Etienne Chauchot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596336#comment-16596336 ] Etienne Chauchot commented on BEAM-5036: I have mixed emotions regarding this: - silent "fail"

[jira] [Work logged] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?focusedWorklogId=139256=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139256 ] ASF GitHub Bot logged work on BEAM-5187: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?focusedWorklogId=139244=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139244 ] ASF GitHub Bot logged work on BEAM-5187: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?focusedWorklogId=139242=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139242 ] ASF GitHub Bot logged work on BEAM-5187: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?focusedWorklogId=139233=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139233 ] ASF GitHub Bot logged work on BEAM-5187: Author: ASF GitHub Bot

[jira] [Comment Edited] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Tim Robertson (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596233#comment-16596233 ] Tim Robertson edited comment on BEAM-5036 at 8/29/18 12:20 PM: --- The changes

[jira] [Resolved] (BEAM-5172) org.apache.beam.sdk.io.elasticsearch/ElasticsearchIOTest is flaky

2018-08-29 Thread Etienne Chauchot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Etienne Chauchot resolved BEAM-5172. Resolution: Fixed Fix Version/s: 2.7.0 >

[jira] [Work logged] (BEAM-5172) org.apache.beam.sdk.io.elasticsearch/ElasticsearchIOTest is flaky

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5172?focusedWorklogId=139224=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139224 ] ASF GitHub Bot logged work on BEAM-5172: Author: ASF GitHub Bot

[jira] [Comment Edited] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Tim Robertson (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596233#comment-16596233 ] Tim Robertson edited comment on BEAM-5036 at 8/29/18 12:03 PM: --- The changes

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Tim Robertson (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596233#comment-16596233 ] Tim Robertson commented on BEAM-5036: - The changes (yet to be merged) to rename() in BEAM-4861 now

[jira] [Work logged] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?focusedWorklogId=139217=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139217 ] ASF GitHub Bot logged work on BEAM-5036: Author: ASF GitHub Bot

[jira] [Work logged] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?focusedWorklogId=139218=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139218 ] ASF GitHub Bot logged work on BEAM-5036: Author: ASF GitHub Bot

[jira] [Commented] (BEAM-5036) Optimize FileBasedSink's WriteOperation.moveToOutput()

2018-08-29 Thread Tim Robertson (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596213#comment-16596213 ] Tim Robertson commented on BEAM-5036: - I think the cross FS check is actually already in place here

  1   2   >