[jira] [Updated] (BEAM-2683) Update GCSFileSystem to support glob patterns of the form {x,y,z}

[ https://issues.apache.org/jira/browse/BEAM-2683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-2683: - Component/s: (was: sdk-java-core) sdk-java-extensions > Update

[jira] [Created] (BEAM-2683) Update GCSFileSystem to support glob patterns of the form {x,y,z}

Chamikara Jayalath created BEAM-2683: Summary: Update GCSFileSystem to support glob patterns of the form {x,y,z} Key: BEAM-2683 URL: https://issues.apache.org/jira/browse/BEAM-2683 Project: Beam

[jira] [Commented] (BEAM-2482) CodedValueMutationDetector should use the coders structural value

[ https://issues.apache.org/jira/browse/BEAM-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101174#comment-16101174 ] ASF GitHub Bot commented on BEAM-2482: -- Github user evindj closed the pull request at:

[jira] [Commented] (BEAM-2556) Client-side throttling for Datastore connector

[ https://issues.apache.org/jira/browse/BEAM-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101173#comment-16101173 ] ASF GitHub Bot commented on BEAM-2556: -- GitHub user cph6 opened a pull request:

[GitHub] beam pull request #3406: [BEAM-2482] CodedValueMutationDetector should use t...

Github user evindj closed the pull request at: https://github.com/apache/beam/pull/3406 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] beam pull request #3644: [BEAM-2556] Add client-side throttling.

GitHub user cph6 opened a pull request: https://github.com/apache/beam/pull/3644 [BEAM-2556] Add client-side throttling. The approach used is as described in https://landing.google.com/sre/book/chapters/handling-overload.html#client-side-throttling-a7sYUg . By backing off

[GitHub] beam pull request #3643: [Beam-2482] - CodedValueMutationDetector should use...

GitHub user evindj opened a pull request: https://github.com/apache/beam/pull/3643 [Beam-2482] - CodedValueMutationDetector should use the coders structural value Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is

[jira] [Resolved] (BEAM-2648) beam_PerformanceTests_Python failing since 2017-07-17

[ https://issues.apache.org/jira/browse/BEAM-2648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmet Altay resolved BEAM-2648. --- Resolution: Fixed Fix Version/s: 2.2.0 > beam_PerformanceTests_Python failing since 2017-07-17

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Dataflow #3650

See

[GitHub] beam pull request #3642: Sets desired bundle size on AvroIO.readAll

GitHub user jkff opened a pull request: https://github.com/apache/beam/pull/3642 Sets desired bundle size on AvroIO.readAll This was overlooked cause unit tests set it explicitly to a small value in order to provide more coverage. R: @mairbek You can merge this pull

[jira] [Created] (BEAM-2682) Merge AvroIOTest and AvroIOTransformTest

Eugene Kirpichov created BEAM-2682: -- Summary: Merge AvroIOTest and AvroIOTransformTest Key: BEAM-2682 URL: https://issues.apache.org/jira/browse/BEAM-2682 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-2681) TransformHierarchy.Node getInputs() TupleTags doesn't match previous ParDo.MultiOutput's getOutputs().

Pei He created BEAM-2681: Summary: TransformHierarchy.Node getInputs() TupleTags doesn't match previous ParDo.MultiOutput's getOutputs(). Key: BEAM-2681 URL: https://issues.apache.org/jira/browse/BEAM-2681

[jira] [Commented] (BEAM-79) Gearpump runner

[ https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101071#comment-16101071 ] ASF GitHub Bot commented on BEAM-79: Github user manuzhang closed the pull request at:

[GitHub] beam pull request #3611: [BEAM-79] merge gearpump-runner into master

Github user manuzhang closed the pull request at: https://github.com/apache/beam/pull/3611 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (BEAM-2680) Improve scalability of the Watch transform

Eugene Kirpichov created BEAM-2680: -- Summary: Improve scalability of the Watch transform Key: BEAM-2680 URL: https://issues.apache.org/jira/browse/BEAM-2680 Project: Beam Issue Type: Bug

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Dataflow #3649

See

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2705

See

[jira] [Commented] (BEAM-2648) beam_PerformanceTests_Python failing since 2017-07-17

[ https://issues.apache.org/jira/browse/BEAM-2648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101035#comment-16101035 ] Mark Liu commented on BEAM-2648: PR is merged. Recent Jenkins run passed.

[GitHub] beam pull request #3594: Do not submit: scratch work

Github user charlesccychen closed the pull request at: https://github.com/apache/beam/pull/3594 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Closed] (BEAM-2656) Introduce AvroIO.readAll()

[ https://issues.apache.org/jira/browse/BEAM-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov closed BEAM-2656. -- Resolution: Fixed Fix Version/s: 2.2.0 > Introduce AvroIO.readAll() >

[jira] [Commented] (BEAM-2656) Introduce AvroIO.readAll()

[ https://issues.apache.org/jira/browse/BEAM-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101013#comment-16101013 ] ASF GitHub Bot commented on BEAM-2656: -- Github user asfgit closed the pull request at:

[GitHub] beam pull request #3615: [BEAM-2656] Introduces AvroIO.readAll()

Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3615 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[2/4] beam git commit: Extracts common logic from TextIO.ReadAll into a utility transform

Extracts common logic from TextIO.ReadAll into a utility transform Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/eaf0b363 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/eaf0b363 Diff:

[4/4] beam git commit: This closes #3615: [BEAM-2656] Introduces AvroIO.readAll()

This closes #3615: [BEAM-2656] Introduces AvroIO.readAll() Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/d919394c Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/d919394c Diff:

[3/4] beam git commit: Adds ValueProvider support to AvroIO.Read

Adds ValueProvider support to AvroIO.Read Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/e80c83b2 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/e80c83b2 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/e80c83b2

[1/4] beam git commit: Introduces AvroIO.readAll() and readAllGenericRecords()

Repository: beam Updated Branches: refs/heads/master 71196ec9c -> d919394c7 Introduces AvroIO.readAll() and readAllGenericRecords() Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/ee1bcbae Tree:

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2704

See

Jenkins build is back to normal : beam_PerformanceTests_Python #142

See

[GitHub] beam pull request #3641: Add logging when Staging takes a long time

GitHub user tgroh opened a pull request: https://github.com/apache/beam/pull/3641 Add logging when Staging takes a long time Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

[GitHub] beam pull request #3640: Translate combining operations through the Runner A...

GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/3640 Translate combining operations through the Runner API. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

[jira] [Commented] (BEAM-2566) Java SDK harness should not depend on any runner

[ https://issues.apache.org/jira/browse/BEAM-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100948#comment-16100948 ] Kenneth Knowles commented on BEAM-2566: --- [~tgroh] if I recall, I think you even did this once

[jira] [Assigned] (BEAM-2566) Java SDK harness should not depend on any runner

[ https://issues.apache.org/jira/browse/BEAM-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles reassigned BEAM-2566: - Assignee: Thomas Groh (was: Luke Cwik) > Java SDK harness should not depend on any

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Dataflow #3648

See

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2703

See

[jira] [Commented] (BEAM-1234) Consider a hint ParDo.withHighFanout()

[ https://issues.apache.org/jira/browse/BEAM-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100703#comment-16100703 ] ASF GitHub Bot commented on BEAM-1234: -- Github user asfgit closed the pull request at:

[GitHub] beam pull request #3580: Let IsBounded take True value.

Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3580 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[2/2] beam git commit: Let IsBounded take True value.

Let IsBounded take True value. This is useful for languages like Python that may use this in a conditional statement (or allow assignment from True->1/False->0). Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/2e8ed5a9 Tree:

[1/2] beam git commit: Closes #3580

Repository: beam Updated Branches: refs/heads/master 73da9cc40 -> 71196ec9c Closes #3580 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/71196ec9 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/71196ec9 Diff:

[jira] [Commented] (BEAM-2640) Introduce Create.ofProvider(ValueProvider)

[ https://issues.apache.org/jira/browse/BEAM-2640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100547#comment-16100547 ] ASF GitHub Bot commented on BEAM-2640: -- GitHub user jkff opened a pull request:

[GitHub] beam pull request #3639: [BEAM-2640, BEAM-2641] Introduces TextIO.read().wit...

GitHub user jkff opened a pull request: https://github.com/apache/beam/pull/3639 [BEAM-2640, BEAM-2641] Introduces TextIO.read().withHintMatchesManyFiles() Reincarnation of https://github.com/apache/beam/pull/3598 which got closed by accident In that case it expands to

[jira] [Commented] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100536#comment-16100536 ] Chamikara Jayalath commented on BEAM-2490: -- I suspect this was due to

[jira] [Commented] (BEAM-2679) SQL DSL API Review

[ https://issues.apache.org/jira/browse/BEAM-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100529#comment-16100529 ] Tyler Akidau commented on BEAM-2679: Yes, taken. > SQL DSL API Review > -- > >

[jira] [Assigned] (BEAM-2679) SQL DSL API Review

[ https://issues.apache.org/jira/browse/BEAM-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Akidau reassigned BEAM-2679: -- Assignee: Tyler Akidau > SQL DSL API Review > -- > > Key:

[jira] [Commented] (BEAM-2640) Introduce Create.ofProvider(ValueProvider)

[ https://issues.apache.org/jira/browse/BEAM-2640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100482#comment-16100482 ] ASF GitHub Bot commented on BEAM-2640: -- Github user asfgit closed the pull request at:

[GitHub] beam pull request #3598: [BEAM-2640, BEAM-2641] Introduces TextIO.read().wit...

Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3598 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] beam pull request #3599: ReduceFnRunner: test when watermark leapfrogs EOW a...

Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3599 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[2/2] beam git commit: This closes #3598: ReduceFnRunner: test when watermark leapfrogs EOW and GC

This closes #3598: ReduceFnRunner: test when watermark leapfrogs EOW and GC Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/73da9cc4 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/73da9cc4 Diff:

[1/2] beam git commit: ReduceFnRunner: test when watermark leapfrogs EOW and GC

Repository: beam Updated Branches: refs/heads/master 01408c864 -> 73da9cc40 ReduceFnRunner: test when watermark leapfrogs EOW and GC This is known to fail in older versions; forward porting regression test. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit:

[jira] [Commented] (BEAM-2679) SQL DSL API Review

[ https://issues.apache.org/jira/browse/BEAM-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100474#comment-16100474 ] Xu Mingmin commented on BEAM-2679: -- [~takidau] could you take the task to drive an API review? > SQL DSL

[jira] [Created] (BEAM-2679) SQL DSL API Review

Xu Mingmin created BEAM-2679: Summary: SQL DSL API Review Key: BEAM-2679 URL: https://issues.apache.org/jira/browse/BEAM-2679 Project: Beam Issue Type: Task Components: dsl-sql

[jira] [Commented] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100426#comment-16100426 ] Ahmet Altay commented on BEAM-2490: --- What are the next steps on this issue? Can we close it? >

[jira] [Comment Edited] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062089#comment-16062089 ] Ahmet Altay edited comment on BEAM-2490 at 7/25/17 5:38 PM: gzip issue could be

[jira] [Commented] (BEAM-1347) Basic Java harness capable of understanding process bundle tasks and sending data over the Fn Api

[ https://issues.apache.org/jira/browse/BEAM-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100330#comment-16100330 ] ASF GitHub Bot commented on BEAM-1347: -- GitHub user lukecwik opened a pull request:

[GitHub] beam pull request #3638: [BEAM-1347] Add utility to be able to model inbound...

GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3638 [BEAM-1347] Add utility to be able to model inbound reading as a single input stream Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure

[jira] [Commented] (BEAM-2676) move BeamSqlRow and BeamSqlRowType to sdk/java/core

[ https://issues.apache.org/jira/browse/BEAM-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099989#comment-16099989 ] Luke Cwik commented on BEAM-2676: - I would suggest moving this to *org.apache.beam.sdk.values* instead of

[jira] [Assigned] (BEAM-2670) ParDoTest.testPipelineOptionsParameter* validatesRunner tests fail on Spark runner

[ https://issues.apache.org/jira/browse/BEAM-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-2670: Assignee: Stas Levin (was: Amit Sela) > ParDoTest.testPipelineOptionsParameter* validatesRunner

[jira] [Assigned] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

[ https://issues.apache.org/jira/browse/BEAM-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kobi Salant reassigned BEAM-2669: - Assignee: Kobi Salant (was: Amit Sela) > Kryo serialization exception when DStreams containing

[jira] [Resolved] (BEAM-1773) Consider allowing Source#validate() to throw exception

[ https://issues.apache.org/jira/browse/BEAM-1773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved BEAM-1773. -- Resolution: Later Fix Version/s: Not applicable > Consider allowing Source#validate() to throw exception

[jira] [Updated] (BEAM-1962) Connection should be closed in case start() throws exception

[ https://issues.apache.org/jira/browse/BEAM-1962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated BEAM-1962: - Description: In JmsIO#start() : {code} try { Connection connection; if (spec.getUsername()

[jira] [Updated] (BEAM-2335) Document various maven commands for running tests

[ https://issues.apache.org/jira/browse/BEAM-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated BEAM-2335: - Description: In this discussion thread, various maven commands for running / not running selected tests were

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Dataflow #3647

See

Jenkins build is back to stable : beam_PostCommit_Java_MavenInstall #4450

See

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2702

See

Build failed in Jenkins: beam_PerformanceTests_Python #141

See -- Started by timer [EnvInject] - Loading node environment variables. Building remotely on beam7 (beam) in workspace

Jenkins build is still unstable: beam_PostCommit_Java_MavenInstall #4449

See

[2/2] beam git commit: This closes #3627

This closes #3627 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/04899e41 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/04899e41 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/04899e41 Branch:

[1/2] beam git commit: [BEAM-2636] Make sure we only override the correct class

Repository: beam Updated Branches: refs/heads/release-2.1.0 c7c777d8b -> 04899e414 [BEAM-2636] Make sure we only override the correct class Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/ab21078b Tree:

[GitHub] beam pull request #3627: [BEAM-2636] Make sure we only override the correct ...

Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3627 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (BEAM-2636) user_score on DataflowRunner is broken

[ https://issues.apache.org/jira/browse/BEAM-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099560#comment-16099560 ] ASF GitHub Bot commented on BEAM-2636: -- Github user asfgit closed the pull request at:

[jira] [Commented] (BEAM-2575) ApexRunner doesn't emit watermarks for additional outputs

[ https://issues.apache.org/jira/browse/BEAM-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099556#comment-16099556 ] ASF GitHub Bot commented on BEAM-2575: -- Github user asfgit closed the pull request at:

[jira] [Updated] (BEAM-2636) user_score on DataflowRunner is broken

[ https://issues.apache.org/jira/browse/BEAM-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Baptiste Onofré updated BEAM-2636: --- Fix Version/s: 2.1.0 > user_score on DataflowRunner is broken >

[GitHub] beam pull request #3629: Cherry-pick pull #3529 into Release 2.1.0

Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3629 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[2/2] beam git commit: This closes #3629

This closes #3629 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/c7c777d8 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/c7c777d8 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/c7c777d8 Branch:

[1/2] beam git commit: BEAM-2575 ApexRunner doesn't emit watermarks for additional outputs

Repository: beam Updated Branches: refs/heads/release-2.1.0 1cf560b4b -> c7c777d8b BEAM-2575 ApexRunner doesn't emit watermarks for additional outputs Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/96ff2206 Tree:

[jira] [Created] (BEAM-2678) add a validation in BeamRowType.create

Xu Mingmin created BEAM-2678: Summary: add a validation in BeamRowType.create Key: BEAM-2678 URL: https://issues.apache.org/jira/browse/BEAM-2678 Project: Beam Issue Type: Improvement

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Dataflow #3646

See

Jenkins build became unstable: beam_PostCommit_Java_MavenInstall #4448

See