[jira] [Commented] (BEAM-2574) test unsupported/invalid cases in DSL

2017-07-11 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083436#comment-16083436 ] ASF GitHub Bot commented on BEAM-2574: -- Github user XuMingmin closed the pull request at:

[GitHub] beam pull request #3530: [BEAM-2574] test unsupported/invalid cases in DSL

2017-07-11 Thread XuMingmin
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3530 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[2/2] beam git commit: [BEAM-2574] This closes #3530

2017-07-11 Thread takidau
[BEAM-2574] This closes #3530 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/b8fa0add Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/b8fa0add Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/b8fa0add Branch:

[1/2] beam git commit: Test unsupported/invalid cases in DSL tests.

2017-07-11 Thread takidau
Repository: beam Updated Branches: refs/heads/DSL_SQL ca2bc723d -> b8fa0addc Test unsupported/invalid cases in DSL tests. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/794f1901 Tree:

Jenkins build is back to normal : beam_PostCommit_Python_Verify #2711

2017-07-11 Thread Apache Jenkins Server
See

[jira] [Created] (BEAM-2601) FileBasedSink produces incorrect shards when writing to multiple destinations

2017-07-11 Thread Reuven Lax (JIRA)
Reuven Lax created BEAM-2601: Summary: FileBasedSink produces incorrect shards when writing to multiple destinations Key: BEAM-2601 URL: https://issues.apache.org/jira/browse/BEAM-2601 Project: Beam

[jira] [Created] (BEAM-2602) Fully support dropwizard metrics in beam and runners

2017-07-11 Thread Cody (JIRA)
Cody created BEAM-2602: -- Summary: Fully support dropwizard metrics in beam and runners Key: BEAM-2602 URL: https://issues.apache.org/jira/browse/BEAM-2602 Project: Beam Issue Type: Improvement

[jira] [Commented] (BEAM-2600) Artifact for Python SDK harness that can be referenced in pipeline definition

2017-07-11 Thread Ahmet Altay (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083343#comment-16083343 ] Ahmet Altay commented on BEAM-2600: --- cc: [~tvalentyn] We might consider generalizing the dataflow runner

[jira] [Updated] (BEAM-2600) Artifact for Python SDK harness that can be referenced in pipeline definition

2017-07-11 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-2600: -- Description: In order to build a pipeline that invokes a Python UDF, we need to be able to

[jira] [Commented] (BEAM-2601) FileBasedSink produces incorrect shards when writing to multiple destinations

2017-07-11 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083336#comment-16083336 ] ASF GitHub Bot commented on BEAM-2601: -- GitHub user reuvenlax opened a pull request:

[GitHub] beam pull request #3546: [BEAM-2601] Fix broken per-destination finalization...

2017-07-11 Thread reuvenlax
GitHub user reuvenlax opened a pull request: https://github.com/apache/beam/pull/3546 [BEAM-2601] Fix broken per-destination finalization. We now finalize each destination separately. Since temporary-file cleanup happens (for the non-windowed case) by deleting the entire temp

[jira] [Created] (BEAM-2598) SparkRunner Fn API based ParDo operator

2017-07-11 Thread Kenneth Knowles (JIRA)
Kenneth Knowles created BEAM-2598: - Summary: SparkRunner Fn API based ParDo operator Key: BEAM-2598 URL: https://issues.apache.org/jira/browse/BEAM-2598 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-2597) FlinkRunner Fn API based ParDo operator

2017-07-11 Thread Kenneth Knowles (JIRA)
Kenneth Knowles created BEAM-2597: - Summary: FlinkRunner Fn API based ParDo operator Key: BEAM-2597 URL: https://issues.apache.org/jira/browse/BEAM-2597 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-2600) Artifact for Python SDK harness that can be referenced in pipeline definition

2017-07-11 Thread Kenneth Knowles (JIRA)
Kenneth Knowles created BEAM-2600: - Summary: Artifact for Python SDK harness that can be referenced in pipeline definition Key: BEAM-2600 URL: https://issues.apache.org/jira/browse/BEAM-2600 Project:

[jira] [Created] (BEAM-2599) ApexRunner Fn API based ParDo operator

2017-07-11 Thread Kenneth Knowles (JIRA)
Kenneth Knowles created BEAM-2599: - Summary: ApexRunner Fn API based ParDo operator Key: BEAM-2599 URL: https://issues.apache.org/jira/browse/BEAM-2599 Project: Beam Issue Type: New Feature

[jira] [Assigned] (BEAM-2595) WriteToBigQuery does not work with nested json schema

2017-07-11 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-2595: - Assignee: Ahmet Altay (was: Thomas Groh) Component/s: (was: runner-dataflow)

[jira] [Commented] (BEAM-2595) WriteToBigQuery does not work with nested json schema

2017-07-11 Thread Ahmet Altay (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083300#comment-16083300 ] Ahmet Altay commented on BEAM-2595: --- [~andrea.pierleoni] Thank you for reporting this. Could you share

[jira] [Assigned] (BEAM-2595) WriteToBigQuery does not work with nested json schema

2017-07-11 Thread Ahmet Altay (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmet Altay reassigned BEAM-2595: - Assignee: Sourabh Bajaj (was: Ahmet Altay) > WriteToBigQuery does not work with nested json

[jira] [Updated] (BEAM-2595) WriteToBigQuery does not work with nested json schema

2017-07-11 Thread Ahmet Altay (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmet Altay updated BEAM-2595: -- Fix Version/s: 2.1.0 > WriteToBigQuery does not work with nested json schema >

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2612

2017-07-11 Thread Apache Jenkins Server
See

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3378

2017-07-11 Thread Apache Jenkins Server
See

Jenkins build is back to normal : beam_PostCommit_Java_MavenInstall #4355

2017-07-11 Thread Apache Jenkins Server
See

[jira] [Commented] (BEAM-2572) Implement an S3 filesystem for Python SDK

2017-07-11 Thread Dmitry Demeshchuk (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083268#comment-16083268 ] Dmitry Demeshchuk commented on BEAM-2572: - Nevermind my last comment, credentials would have to be

Build failed in Jenkins: beam_PostCommit_Python_Verify #2710

2017-07-11 Thread Apache Jenkins Server
See Changes: [robertwb] Cleanup and fix ptransform_fn decorator. -- [...truncated 39.75 KB...] ERROR: invocation failed (exit code 1), logfile:

[GitHub] beam pull request #3544: Cleanup and fix ptransform_fn decorator.

2017-07-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3544 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[2/2] beam git commit: Closes #3544

2017-07-11 Thread robertwb
Closes #3544 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/91c7d3d1 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/91c7d3d1 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/91c7d3d1 Branch: refs/heads/master

[1/2] beam git commit: Cleanup and fix ptransform_fn decorator.

2017-07-11 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 84682109b -> 91c7d3d1f Cleanup and fix ptransform_fn decorator. Previously CallablePTransform was being used both as the factory and the transform itself, which could result in state getting carried between pipelines. Project:

[jira] [Commented] (BEAM-2596) Break up Jenkins PreCommit into individual steps.

2017-07-11 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083255#comment-16083255 ] ASF GitHub Bot commented on BEAM-2596: -- GitHub user jasonkuster opened a pull request:

[GitHub] beam pull request #3545: [BEAM-2596] Pipeline job for Jenkins PreCommit

2017-07-11 Thread jasonkuster
GitHub user jasonkuster opened a pull request: https://github.com/apache/beam/pull/3545 [BEAM-2596] Pipeline job for Jenkins PreCommit Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is

[jira] [Created] (BEAM-2596) Break up Jenkins PreCommit into individual steps.

2017-07-11 Thread Jason Kuster (JIRA)
Jason Kuster created BEAM-2596: -- Summary: Break up Jenkins PreCommit into individual steps. Key: BEAM-2596 URL: https://issues.apache.org/jira/browse/BEAM-2596 Project: Beam Issue Type: New

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2611

2017-07-11 Thread Apache Jenkins Server
See

[jira] [Commented] (BEAM-995) Apache Pig DSL

2017-07-11 Thread James Xu (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083235#comment-16083235 ] James Xu commented on BEAM-995: --- I think the former is better, the execution engine for beam program is

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3377

2017-07-11 Thread Apache Jenkins Server
See

[jira] [Updated] (BEAM-2595) WriteToBigQuery does not work with nested json schema

2017-07-11 Thread Andrea Pierleoni (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrea Pierleoni updated BEAM-2595: --- Description: I am trying to use the new `WriteToBigQuery` PTransform added to

[jira] [Created] (BEAM-2595) WriteToBigQuery does not work with nested json schema

2017-07-11 Thread Andrea Pierleoni (JIRA)
Andrea Pierleoni created BEAM-2595: -- Summary: WriteToBigQuery does not work with nested json schema Key: BEAM-2595 URL: https://issues.apache.org/jira/browse/BEAM-2595 Project: Beam Issue

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2610

2017-07-11 Thread Apache Jenkins Server
See

[jira] [Commented] (BEAM-2511) TextIO should support reading a PCollection of filenames

2017-07-11 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083168#comment-16083168 ] ASF GitHub Bot commented on BEAM-2511: -- Github user asfgit closed the pull request at:

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3376

2017-07-11 Thread Apache Jenkins Server
See

[jira] [Closed] (BEAM-2511) TextIO should support reading a PCollection of filenames

2017-07-11 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov closed BEAM-2511. -- Resolution: Fixed Fix Version/s: 2.2.0 > TextIO should support reading a PCollection of

Build failed in Jenkins: beam_PostCommit_Java_MavenInstall #4354

2017-07-11 Thread Apache Jenkins Server
See Changes: [kirpichov] Adds TextIO.readAll(), implemented rather naively -- [...truncated 350.35 KB...] 2017-07-11T23:29:22.756 [INFO] Downloading:

[GitHub] beam pull request #3443: [BEAM-2511] Implements TextIO.ReadAll

2017-07-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3443 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[2/2] beam git commit: This closes #3443: [BEAM-2511] Implements TextIO.ReadAll

2017-07-11 Thread jkff
This closes #3443: [BEAM-2511] Implements TextIO.ReadAll Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/84682109 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/84682109 Diff:

[1/2] beam git commit: Adds TextIO.readAll(), implemented rather naively

2017-07-11 Thread jkff
Repository: beam Updated Branches: refs/heads/master 011e2796d -> 84682109b Adds TextIO.readAll(), implemented rather naively Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/cd216f79 Tree:

[jira] [Commented] (BEAM-2430) Java FnApiDoFnRunner to share across runners

2017-07-11 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083138#comment-16083138 ] Kenneth Knowles commented on BEAM-2430: --- Ah, OK. From within the Java SDK harness, the DoFnRunner

[jira] [Updated] (BEAM-2591) Python shim for submitting to FlinkRunner

2017-07-11 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-2591: -- Component/s: runner-flink > Python shim for submitting to FlinkRunner >

[jira] [Comment Edited] (BEAM-2572) Implement an S3 filesystem for Python SDK

2017-07-11 Thread Dmitry Demeshchuk (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083108#comment-16083108 ] Dmitry Demeshchuk edited comment on BEAM-2572 at 7/11/17 10:39 PM: --- For

[jira] [Commented] (BEAM-2572) Implement an S3 filesystem for Python SDK

2017-07-11 Thread Dmitry Demeshchuk (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083101#comment-16083101 ] Dmitry Demeshchuk commented on BEAM-2572: - [~robertwb]: yes, by "hidden" I meant not exposing some

[jira] [Commented] (BEAM-2572) Implement an S3 filesystem for Python SDK

2017-07-11 Thread Robert Bradshaw (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083088#comment-16083088 ] Robert Bradshaw commented on BEAM-2572: --- It should be noted that there was recently a huge effort to

[jira] [Commented] (BEAM-2572) Implement an S3 filesystem for Python SDK

2017-07-11 Thread Dmitry Demeshchuk (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083113#comment-16083113 ] Dmitry Demeshchuk commented on BEAM-2572: - Or, alternatively, we could state that filesystem

[jira] [Commented] (BEAM-2572) Implement an S3 filesystem for Python SDK

2017-07-11 Thread Dmitry Demeshchuk (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083108#comment-16083108 ] Dmitry Demeshchuk commented on BEAM-2572: - For example, in my particular case, it would be

[jira] [Reopened] (BEAM-2430) Java FnApiDoFnRunner to share across runners

2017-07-11 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reopened BEAM-2430: - Assignee: Kenneth Knowles (was: Luke Cwik) PR/3432 was about writing a DoFnRunner for executing DoFns

[jira] [Updated] (BEAM-2594) Python shim for submitting to Java DirectRunner

2017-07-11 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-2594: -- Component/s: (was: runner-spark) runner-direct > Python shim for submitting

[jira] [Assigned] (BEAM-2430) Java FnApiDoFnRunner to share across runners

2017-07-11 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles reassigned BEAM-2430: - Assignee: Luke Cwik > Java FnApiDoFnRunner to share across runners >

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2609

2017-07-11 Thread Apache Jenkins Server
See

[jira] [Commented] (BEAM-2566) Java SDK harness should not depend on any runner

2017-07-11 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083080#comment-16083080 ] Kenneth Knowles commented on BEAM-2566: --- [~robertwb] any chance you are a good candidate to do

[jira] [Created] (BEAM-2593) Python shim for submitting to SparkRunner

2017-07-11 Thread Kenneth Knowles (JIRA)
Kenneth Knowles created BEAM-2593: - Summary: Python shim for submitting to SparkRunner Key: BEAM-2593 URL: https://issues.apache.org/jira/browse/BEAM-2593 Project: Beam Issue Type: New

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3375

2017-07-11 Thread Apache Jenkins Server
See

[jira] [Created] (BEAM-2589) ApexRunner shim for Job API

2017-07-11 Thread Kenneth Knowles (JIRA)
Kenneth Knowles created BEAM-2589: - Summary: ApexRunner shim for Job API Key: BEAM-2589 URL: https://issues.apache.org/jira/browse/BEAM-2589 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-2588) FlinkRunner shim for serving Job API

2017-07-11 Thread Kenneth Knowles (JIRA)
Kenneth Knowles created BEAM-2588: - Summary: FlinkRunner shim for serving Job API Key: BEAM-2588 URL: https://issues.apache.org/jira/browse/BEAM-2588 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-2590) SparkRunner shim for Job API

2017-07-11 Thread Kenneth Knowles (JIRA)
Kenneth Knowles created BEAM-2590: - Summary: SparkRunner shim for Job API Key: BEAM-2590 URL: https://issues.apache.org/jira/browse/BEAM-2590 Project: Beam Issue Type: New Feature

[GitHub] beam pull request #3544: Cleanup and fix ptransform_fn decorator.

2017-07-11 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/3544 Cleanup and fix ptransform_fn decorator. Previously CallablePTransform was being used both as the factory and the transform itself, which could result in state getting carried between

[jira] [Created] (BEAM-2594) Python shim for submitting to Java DirectRunner

2017-07-11 Thread Kenneth Knowles (JIRA)
Kenneth Knowles created BEAM-2594: - Summary: Python shim for submitting to Java DirectRunner Key: BEAM-2594 URL: https://issues.apache.org/jira/browse/BEAM-2594 Project: Beam Issue Type: New

[jira] [Created] (BEAM-2592) Python shim for submitting to ApexRunner

2017-07-11 Thread Kenneth Knowles (JIRA)
Kenneth Knowles created BEAM-2592: - Summary: Python shim for submitting to ApexRunner Key: BEAM-2592 URL: https://issues.apache.org/jira/browse/BEAM-2592 Project: Beam Issue Type: New

[jira] [Created] (BEAM-2591) Python shim for submitting to FlinkRunner

2017-07-11 Thread Kenneth Knowles (JIRA)
Kenneth Knowles created BEAM-2591: - Summary: Python shim for submitting to FlinkRunner Key: BEAM-2591 URL: https://issues.apache.org/jira/browse/BEAM-2591 Project: Beam Issue Type: New

[GitHub] beam pull request #3542: Remove dead (and wrong) viewFromProto overload

2017-07-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3542 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[1/2] beam git commit: Remove dead (and wrong) viewFromProto overload

2017-07-11 Thread kenn
Repository: beam Updated Branches: refs/heads/master a22f1a05a -> 011e2796d Remove dead (and wrong) viewFromProto overload Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/aaffe15e Tree:

[2/2] beam git commit: This closes #3542: Remove dead (and wrong) viewFromProto overload

2017-07-11 Thread kenn
This closes #3542: Remove dead (and wrong) viewFromProto overload Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/011e2796 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/011e2796 Diff:

[jira] [Resolved] (BEAM-2430) Java FnApiDoFnRunner to share across runners

2017-07-11 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles resolved BEAM-2430. --- Resolution: Implemented Fix Version/s: 2.1.0 > Java FnApiDoFnRunner to share across

[jira] [Updated] (BEAM-2371) Make Java DirectRunner demonstrate language-agnostic Runner API translation wrappers

2017-07-11 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-2371: -- Labels: beam-python-everywhere (was: ) > Make Java DirectRunner demonstrate language-agnostic

[jira] [Updated] (BEAM-2431) Model Runner interactions in RPC layer for Runner API

2017-07-11 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-2431: -- Labels: beam-python-everywhere (was: ) > Model Runner interactions in RPC layer for Runner API

[jira] [Updated] (BEAM-2430) Java FnApiDoFnRunner to share across runners

2017-07-11 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-2430: -- Labels: beam-python-everywhere (was: ) > Java FnApiDoFnRunner to share across runners >

[jira] [Comment Edited] (BEAM-2571) Flink ValidatesRunner failing CombineTest.testSlidingWindowsCombineWithContext

2017-07-11 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082463#comment-16082463 ] Kenneth Knowles edited comment on BEAM-2571 at 7/11/17 9:23 PM: OK, I was

[jira] [Comment Edited] (BEAM-2572) Implement an S3 filesystem for Python SDK

2017-07-11 Thread Sourabh Bajaj (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082887#comment-16082887 ] Sourabh Bajaj edited comment on BEAM-2572 at 7/11/17 8:16 PM: -- I think overall

[jira] [Commented] (BEAM-2573) Better filesystem discovery mechanism in Python SDK

2017-07-11 Thread Ahmet Altay (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082872#comment-16082872 ] Ahmet Altay commented on BEAM-2573: --- Thank you [~demeshchuk]. I agree with documenting this feature.

[jira] [Commented] (BEAM-2573) Better filesystem discovery mechanism in Python SDK

2017-07-11 Thread Dmitry Demeshchuk (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082850#comment-16082850 ] Dmitry Demeshchuk commented on BEAM-2573: - Finally, after some fighting dependency issues, made it

[jira] [Created] (BEAM-2587) Build fails due to python sdk

2017-07-11 Thread Ahmet Altay (JIRA)
Ahmet Altay created BEAM-2587: - Summary: Build fails due to python sdk Key: BEAM-2587 URL: https://issues.apache.org/jira/browse/BEAM-2587 Project: Beam Issue Type: Bug Components:

Jenkins build is back to stable : beam_PostCommit_Java_MavenInstall #4352

2017-07-11 Thread Apache Jenkins Server
See

[jira] [Updated] (BEAM-2585) add math function RAND_INTEGER and RAND

2017-07-11 Thread Xu Mingmin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Mingmin updated BEAM-2585: - Issue Type: Sub-task (was: Task) Parent: BEAM-2160 > add math function RAND_INTEGER and RAND >

[jira] [Assigned] (BEAM-2562) Add integration test for logical operators

2017-07-11 Thread Xu Mingmin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Mingmin reassigned BEAM-2562: Assignee: Xu Mingmin > Add integration test for logical operators >

[jira] [Commented] (BEAM-2586) Accommodate custom delimiters in TextIO

2017-07-11 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082716#comment-16082716 ] ASF GitHub Bot commented on BEAM-2586: -- GitHub user ChristophHebert opened a pull request:

[jira] [Created] (BEAM-2585) add math function RAND_INTEGER and RAND

2017-07-11 Thread Xu Mingmin (JIRA)
Xu Mingmin created BEAM-2585: Summary: add math function RAND_INTEGER and RAND Key: BEAM-2585 URL: https://issues.apache.org/jira/browse/BEAM-2585 Project: Beam Issue Type: Task

[GitHub] beam pull request #3543: [BEAM-2586] Accommodate custom delimiters in TextIO

2017-07-11 Thread ChristophHebert
GitHub user ChristophHebert opened a pull request: https://github.com/apache/beam/pull/3543 [BEAM-2586] Accommodate custom delimiters in TextIO You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChristophHebert/beam modifiedTextIO

[jira] [Created] (BEAM-2586) Accommodate custom delimiters in TextIO

2017-07-11 Thread Christopher Hebert (JIRA)
Christopher Hebert created BEAM-2586: Summary: Accommodate custom delimiters in TextIO Key: BEAM-2586 URL: https://issues.apache.org/jira/browse/BEAM-2586 Project: Beam Issue Type: New

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2608

2017-07-11 Thread Apache Jenkins Server
See

[jira] [Assigned] (BEAM-2563) Add integration test for math operators

2017-07-11 Thread Xu Mingmin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Mingmin reassigned BEAM-2563: Assignee: Xu Mingmin > Add integration test for math operators >

Jenkins build became unstable: beam_PostCommit_Java_MavenInstall #4351

2017-07-11 Thread Apache Jenkins Server
See

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3374

2017-07-11 Thread Apache Jenkins Server
See

[jira] [Resolved] (BEAM-115) Beam Runner API

2017-07-11 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles resolved BEAM-115. -- Resolution: Resolved Fix Version/s: Not applicable I think this umbrella ticket is no

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2607

2017-07-11 Thread Apache Jenkins Server
See

[GitHub] beam pull request #3542: Remove dead (and wrong) viewFromProto overload

2017-07-11 Thread kennknowles
GitHub user kennknowles opened a pull request: https://github.com/apache/beam/pull/3542 Remove dead (and wrong) viewFromProto overload Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is

[jira] [Created] (BEAM-2584) Create a JCache based IO

2017-07-11 Thread JIRA
Ismaël Mejía created BEAM-2584: -- Summary: Create a JCache based IO Key: BEAM-2584 URL: https://issues.apache.org/jira/browse/BEAM-2584 Project: Beam Issue Type: New Feature

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3373

2017-07-11 Thread Apache Jenkins Server
See

[jira] [Commented] (BEAM-2558) Add a CombineFnTester

2017-07-11 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082503#comment-16082503 ] Thomas Groh commented on BEAM-2558: --- The linked file is also only in the Testing JAR, so user's can't use

[jira] [Assigned] (BEAM-2584) Create a JCache based IO

2017-07-11 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía reassigned BEAM-2584: -- Assignee: Ismaël Mejía (was: Davor Bonaci) > Create a JCache based IO >

[jira] [Commented] (BEAM-2556) Client-side throttling for Datastore connector

2017-07-11 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082489#comment-16082489 ] ASF GitHub Bot commented on BEAM-2556: -- Github user asfgit closed the pull request at:

[jira] [Commented] (BEAM-2583) Exceptions in expand() corrupt the TransformHierarchy

2017-07-11 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082486#comment-16082486 ] Kenneth Knowles commented on BEAM-2583: --- Is it a bug to catch an exception and continue to use the

[GitHub] beam pull request #3507: [BEAM-2556] Add client-side throttling.

2017-07-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3507 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[1/2] beam git commit: Add client-side throttling.

2017-07-11 Thread lcwik
Repository: beam Updated Branches: refs/heads/master 138641f14 -> a22f1a05a Add client-side throttling. The approach used is as described in https://landing.google.com/sre/book/chapters/handling-overload.html#client-side-throttling-a7sYUg . By backing off individual workers in response to

[2/2] beam git commit: [BEAM-2556] Add client-side throttling.

2017-07-11 Thread lcwik
[BEAM-2556] Add client-side throttling. This closes #3507 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/a22f1a05 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/a22f1a05 Diff:

[jira] [Commented] (BEAM-2583) Exceptions in expand() corrupt the TransformHierarchy

2017-07-11 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082464#comment-16082464 ] Thomas Groh commented on BEAM-2583: --- Specifically, this only happens if, during a call to {{expand}}, the

  1   2   >