[jira] [Closed] (BEAM-2991) BigQueryIO.read().fromQuery() should set a TTL on the temporary tables with query results

[ https://issues.apache.org/jira/browse/BEAM-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov closed BEAM-2991. -- Resolution: Fixed > BigQueryIO.read().fromQuery() should set a TTL on the temporary tables with

[jira] [Commented] (BEAM-2991) BigQueryIO.read().fromQuery() should set a TTL on the temporary tables with query results

[ https://issues.apache.org/jira/browse/BEAM-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181982#comment-16181982 ] ASF GitHub Bot commented on BEAM-2991: -- Github user asfgit closed the pull request at:

[GitHub] beam pull request #3883: [BEAM-2991] Sets a TTL on BigQueryIO.read().fromQue...

Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3883 ---

[1/2] beam git commit: Sets a TTL on BigQueryIO.read().fromQuery() temp dataset

Repository: beam Updated Branches: refs/heads/master b494c4404 -> 41239d808 Sets a TTL on BigQueryIO.read().fromQuery() temp dataset Also fixes a bug where we start the query job twice - once to extract the files, once to get schema. Luckily it doesn't actually run twice, because inserting

[2/2] beam git commit: This closes #3883: [BEAM-2991] Sets a TTL on BigQueryIO.read().fromQuery() temp dataset

This closes #3883: [BEAM-2991] Sets a TTL on BigQueryIO.read().fromQuery() temp dataset Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/41239d80 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/41239d80 Diff:

[jira] [Created] (BEAM-2992) Remove codepaths for reading unsplit BigQuery sources

Eugene Kirpichov created BEAM-2992: -- Summary: Remove codepaths for reading unsplit BigQuery sources Key: BEAM-2992 URL: https://issues.apache.org/jira/browse/BEAM-2992 Project: Beam Issue

[jira] [Created] (BEAM-2991) BigQueryIO.read().fromQuery() should set a TTL on the temporary tables with query results

Eugene Kirpichov created BEAM-2991: -- Summary: BigQueryIO.read().fromQuery() should set a TTL on the temporary tables with query results Key: BEAM-2991 URL: https://issues.apache.org/jira/browse/BEAM-2991

Build failed in Jenkins: beam_PostCommit_Python_Verify #3223

See -- [...truncated 528.33 KB...] ^ x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC

[jira] [Assigned] (BEAM-2857) Create FileIO in Python

[ https://issues.apache.org/jira/browse/BEAM-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-2857: --- Assignee: Jing Chen > Create FileIO in Python > --- > > Key:

Build failed in Jenkins: beam_PostCommit_Python_Verify #3222

See -- [...truncated 952.57 KB...] File "/usr/lib/python2.7/pickle.py", line 1227, in load_build d[intern(k)] = v File

Jenkins build is back to stable : beam_PostCommit_Java_MavenInstall #4889

See

[jira] [Commented] (BEAM-2857) Create FileIO in Python

[ https://issues.apache.org/jira/browse/BEAM-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181843#comment-16181843 ] Jing Chen commented on BEAM-2857: - Ok, I just cannot see “assign to me” button, then I will create a pull

[GitHub] beam pull request #3905: Separating Python test into four units

Github user pabloem closed the pull request at: https://github.com/apache/beam/pull/3905 ---

[jira] [Commented] (BEAM-2724) MSEC counters should support Structured Names in Dataflow

[ https://issues.apache.org/jira/browse/BEAM-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181834#comment-16181834 ] ASF GitHub Bot commented on BEAM-2724: -- Github user pabloem closed the pull request at:

[jira] [Commented] (BEAM-2724) MSEC counters should support Structured Names in Dataflow

[ https://issues.apache.org/jira/browse/BEAM-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181835#comment-16181835 ] ASF GitHub Bot commented on BEAM-2724: -- GitHub user pabloem reopened a pull request:

[GitHub] beam pull request #3786: [BEAM-2724] Preparing statesampler to work with str...

GitHub user pabloem reopened a pull request: https://github.com/apache/beam/pull/3786 [BEAM-2724] Preparing statesampler to work with structured names You can merge this pull request into a Git repository by running: $ git pull https://github.com/pabloem/incubator-beam

[GitHub] beam pull request #3786: [BEAM-2724] Preparing statesampler to work with str...

Github user pabloem closed the pull request at: https://github.com/apache/beam/pull/3786 ---

[GitHub] beam pull request #3908: Add an ArtifactServiceStager

GitHub user tgroh opened a pull request: https://github.com/apache/beam/pull/3908 Add an ArtifactServiceStager Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

[jira] [Commented] (BEAM-2857) Create FileIO in Python

[ https://issues.apache.org/jira/browse/BEAM-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181788#comment-16181788 ] Eugene Kirpichov commented on BEAM-2857: I don't think you need to be formally added to anything in

[jira] [Commented] (BEAM-2857) Create FileIO in Python

[ https://issues.apache.org/jira/browse/BEAM-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181780#comment-16181780 ] Jing Chen commented on BEAM-2857: - i want to take a stab at this ticket, can you please add me to

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3935

See

[jira] [Commented] (BEAM-2837) Writing To Spanner From Google Cloud DataFlow - Failure

[ https://issues.apache.org/jira/browse/BEAM-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181772#comment-16181772 ] ASF GitHub Bot commented on BEAM-2837: -- GitHub user mairbek opened a pull request:

[GitHub] beam pull request #3907: [BEAM-2837] Allows to set a Cloud Spanner host.

GitHub user mairbek opened a pull request: https://github.com/apache/beam/pull/3907 [BEAM-2837] Allows to set a Cloud Spanner host. Allows to set a Cloud Spanner host. https://batch-spanner.googleapis.com/ is set as a default host name. You can merge this pull request into a

[GitHub] beam pull request #3906: Better error for reused pcollections.

GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/3906 Better error for reused pcollections. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

[GitHub] beam pull request #3905: Separating Python test into four units

GitHub user pabloem opened a pull request: https://github.com/apache/beam/pull/3905 Separating Python test into four units Expecting to see the difference.. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pabloem/incubator-beam

[jira] [Commented] (BEAM-2724) MSEC counters should support Structured Names in Dataflow

[ https://issues.apache.org/jira/browse/BEAM-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181748#comment-16181748 ] ASF GitHub Bot commented on BEAM-2724: -- Github user pabloem closed the pull request at:

[jira] [Commented] (BEAM-2724) MSEC counters should support Structured Names in Dataflow

[ https://issues.apache.org/jira/browse/BEAM-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181749#comment-16181749 ] ASF GitHub Bot commented on BEAM-2724: -- GitHub user pabloem reopened a pull request:

[GitHub] beam pull request #3786: [BEAM-2724] Preparing statesampler to work with str...

Github user pabloem closed the pull request at: https://github.com/apache/beam/pull/3786 ---

[GitHub] beam pull request #3786: [BEAM-2724] Preparing statesampler to work with str...

GitHub user pabloem reopened a pull request: https://github.com/apache/beam/pull/3786 [BEAM-2724] Preparing statesampler to work with structured names You can merge this pull request into a Git repository by running: $ git pull https://github.com/pabloem/incubator-beam

Jenkins build is unstable: beam_PostCommit_Java_MavenInstall #4888

See

[jira] [Commented] (BEAM-2959) Fix proto enums to not use 0 for a valid value

[ https://issues.apache.org/jira/browse/BEAM-2959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181693#comment-16181693 ] ASF GitHub Bot commented on BEAM-2959: -- GitHub user lukecwik opened a pull request:

[GitHub] beam pull request #3904: [BEAM-2959] Fix proto enums to use "YYY_UNSPECIFIED...

GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3904 [BEAM-2959] Fix proto enums to use "YYY_UNSPECIFIED" as the first declared enum. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there

[jira] [Commented] (BEAM-2530) Make Beam compatible with Java 9

[ https://issues.apache.org/jira/browse/BEAM-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181658#comment-16181658 ] ASF GitHub Bot commented on BEAM-2530: -- GitHub user iemejia opened a pull request:

[GitHub] beam pull request #3903: [BEAM-2530] Fix javadoc generation on Java 9 and up...

GitHub user iemejia opened a pull request: https://github.com/apache/beam/pull/3903 [BEAM-2530] Fix javadoc generation on Java 9 and update maven plugins to support java 9 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3934

See

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3933

See

[1/2] beam git commit: Closes #3829

Repository: beam Updated Branches: refs/heads/master ba9440f33 -> b494c4404 Closes #3829 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/b494c440 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/b494c440 Diff:

[GitHub] beam pull request #3829: Use de(stributed)tox rather than plain old tox for ...

Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3829 ---

[2/2] beam git commit: Use de(stributed)tox rather than plain old tox for Python testing.

Use de(stributed)tox rather than plain old tox for Python testing. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/9052a16a Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/9052a16a Diff:

[GitHub] beam pull request #3828: Add some timing information to testing scripts.

Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3828 ---

[1/2] beam git commit: Closes #3828

Repository: beam Updated Branches: refs/heads/master 732ae4297 -> ba9440f33 Closes #3828 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/ba9440f3 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/ba9440f3 Diff:

[2/2] beam git commit: Add some timing information to testing scripts.

Add some timing information to testing scripts. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/97f7d0bf Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/97f7d0bf Diff:

[GitHub] beam pull request #3902: Use pip from currently executing Python.

GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/3902 Use pip from currently executing Python. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

[jira] [Assigned] (BEAM-2958) Expose a top level user agent PipelineOption which can be communicated to external services

[ https://issues.apache.org/jira/browse/BEAM-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Oliveira reassigned BEAM-2958: - Assignee: Daniel Oliveira > Expose a top level user agent PipelineOption which can be

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3932

See

[GitHub] beam pull request #3901: Catch subprocess error in proto generation.

Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3901 ---

[2/2] beam git commit: Closes #3901

Closes #3901 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/732ae429 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/732ae429 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/732ae429 Branch: refs/heads/master

[1/2] beam git commit: Catch subprocess error in proto generation.

Repository: beam Updated Branches: refs/heads/master 2aa3b5c69 -> 732ae4297 Catch subprocess error in proto generation. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/6dcdea74 Tree:

Jenkins build is back to normal : beam_PostCommit_Java_ValidatesRunner_Spark #3160

See

[GitHub] beam pull request #3901: Catch subprocess error in proto generation.

GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/3901 Catch subprocess error in proto generation. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3931

See

[jira] [Commented] (BEAM-2718) Add bundle retry logic to the DirectRunner

[ https://issues.apache.org/jira/browse/BEAM-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181449#comment-16181449 ] ASF GitHub Bot commented on BEAM-2718: -- Github user asfgit closed the pull request at:

[1/2] beam git commit: Add test to fix partial writouts after a bundle retry

Repository: beam Updated Branches: refs/heads/master b2138b0d7 -> 2aa3b5c69 Add test to fix partial writouts after a bundle retry Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/ce6a18c1 Tree:

[GitHub] beam pull request #3833: [BEAM-2718] Add test to fix partial writeouts after...

Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3833 ---

[2/2] beam git commit: This closes #3833

This closes #3833 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/2aa3b5c6 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/2aa3b5c6 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/2aa3b5c6 Branch:

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3930

See

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Spark #3159

See -- [...truncated 265.54 KB...] 2017-09-26T18:05:35.526 [INFO] Downloaded:

[GitHub] beam pull request #3900: Created Java snippets file

GitHub user davidcavazos opened a pull request: https://github.com/apache/beam/pull/3900 Created Java snippets file R: @tgroh for Java code R: @melap for snippet tags Starting out with just the CoGroupByKey snippet to make the PR small. Eventually we want to migrate

[jira] [Created] (BEAM-2990) support data type MAP/ARRAY

Xu Mingmin created BEAM-2990: Summary: support data type MAP/ARRAY Key: BEAM-2990 URL: https://issues.apache.org/jira/browse/BEAM-2990 Project: Beam Issue Type: New Feature

[GitHub] beam pull request #3899: Move Runners Core Fn Api classes into a package

GitHub user tgroh opened a pull request: https://github.com/apache/beam/pull/3899 Move Runners Core Fn Api classes into a package Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3929

See

[jira] [Created] (BEAM-2989) Improve coder inference for WithKeys.of(null)

Luke Cwik created BEAM-2989: --- Summary: Improve coder inference for WithKeys.of(null) Key: BEAM-2989 URL: https://issues.apache.org/jira/browse/BEAM-2989 Project: Beam Issue Type: Improvement

[jira] [Closed] (BEAM-2858) temp file garbage collection in BigQuery sink should be in a separate DoFn

[ https://issues.apache.org/jira/browse/BEAM-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax closed BEAM-2858. Resolution: Fixed > temp file garbage collection in BigQuery sink should be in a separate DoFn >

[jira] [Commented] (BEAM-2858) temp file garbage collection in BigQuery sink should be in a separate DoFn

[ https://issues.apache.org/jira/browse/BEAM-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180957#comment-16180957 ] ASF GitHub Bot commented on BEAM-2858: -- Github user asfgit closed the pull request at:

[1/2] beam git commit: Move file deletion into subsequent ParDo.

Repository: beam Updated Branches: refs/heads/master 352f106f9 -> b2138b0d7 Move file deletion into subsequent ParDo. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/6d9b8f06 Tree:

[GitHub] beam pull request #3863: [BEAM-2858] Ensure that file deletion happens consi...

Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3863 ---

[2/2] beam git commit: This closes #3863

This closes #3863 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/b2138b0d Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/b2138b0d Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/b2138b0d Branch:

Build failed in Jenkins: beam_PostCommit_Python_Verify #3217

See -- [...truncated 44.47 KB...] Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.2.0.dev0) Using cached mock-2.0.0-py2.py3-none-any.whl Collecting

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3928

See

Build failed in Jenkins: beam_PostCommit_Python_Verify #3216

See -- [...truncated 44.47 KB...] Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.2.0.dev0) Using cached mock-2.0.0-py2.py3-none-any.whl Collecting

[jira] [Commented] (BEAM-2724) MSEC counters should support Structured Names in Dataflow

[ https://issues.apache.org/jira/browse/BEAM-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180328#comment-16180328 ] ASF GitHub Bot commented on BEAM-2724: -- Github user pabloem closed the pull request at:

[jira] [Commented] (BEAM-2724) MSEC counters should support Structured Names in Dataflow

[ https://issues.apache.org/jira/browse/BEAM-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180329#comment-16180329 ] ASF GitHub Bot commented on BEAM-2724: -- GitHub user pabloem reopened a pull request:

[GitHub] beam pull request #3786: [BEAM-2724] Preparing statesampler to work with str...

GitHub user pabloem reopened a pull request: https://github.com/apache/beam/pull/3786 [BEAM-2724] Preparing statesampler to work with structured names You can merge this pull request into a Git repository by running: $ git pull https://github.com/pabloem/incubator-beam

[GitHub] beam pull request #3786: [BEAM-2724] Preparing statesampler to work with str...

Github user pabloem closed the pull request at: https://github.com/apache/beam/pull/3786 ---

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3927

See