[jira] [Updated] (BEAM-214) Parquet IO

[ https://issues.apache.org/jira/browse/BEAM-214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davor Bonaci updated BEAM-214: -- Assignee: (was: James Malone) > Parquet IO > -- > > Key: BEAM-214 >

[jira] [Updated] (BEAM-214) Parquet IO

[ https://issues.apache.org/jira/browse/BEAM-214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davor Bonaci updated BEAM-214: -- Component/s: (was: sdk-ideas) sdk-java-extensions > Parquet IO > -- > >

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow » Apache Beam :: Runners :: Google Cloud Dataflow #158

See

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #158

See

[GitHub] incubator-beam pull request: [BEAM-53] Pubsub Apiary client

GitHub user mshields822 opened a pull request: https://github.com/apache/incubator-beam/pull/213 [BEAM-53] Pubsub Apiary client Continuing from https://github.com/apache/incubator-beam/pull/120 - Move PubsubClient and friends to sdk.util. - Add PubsubApiaryClient since

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow » Apache Beam :: Runners :: Google Cloud Dataflow #157

See

[2/3] incubator-beam git commit: Remove unused generic arguments in ReduceFnRunnerTest.

Remove unused generic arguments in ReduceFnRunnerTest. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/fa458091 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/fa458091 Diff:

[GitHub] incubator-beam pull request: [BEAM-210] Test that empty final pane...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/211 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[1/3] incubator-beam git commit: Add test for empty ON_TIME and no empty final pane

Repository: incubator-beam Updated Branches: refs/heads/master 0952f4433 -> f1aa490b9 Add test for empty ON_TIME and no empty final pane Add a test that we get an empty `ON_TIME` pane, and don't get the empty final pane when using accumulation mode with the only if non-empty

[jira] [Created] (BEAM-214) Parquet IO

Neville Li created BEAM-214: --- Summary: Parquet IO Key: BEAM-214 URL: https://issues.apache.org/jira/browse/BEAM-214 Project: Beam Issue Type: Improvement Components: sdk-ideas

[jira] [Closed] (BEAM-212) SparkPipelineRunner should support the Read Primitive

[ https://issues.apache.org/jira/browse/BEAM-212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh closed BEAM-212. Resolution: Fixed > SparkPipelineRunner should support the Read Primitive >

[jira] [Commented] (BEAM-210) Be consistent with emitting final empty panes

[ https://issues.apache.org/jira/browse/BEAM-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248630#comment-15248630 ] Kenneth Knowles commented on BEAM-210: -- I don't really agree that adding this knob is what's most

[jira] [Created] (BEAM-213) Fix README to use refactored package names

Amit Sela created BEAM-213: -- Summary: Fix README to use refactored package names Key: BEAM-213 URL: https://issues.apache.org/jira/browse/BEAM-213 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-212) SparkPipelineRunner should support the Read Primitive

Thomas Groh created BEAM-212: Summary: SparkPipelineRunner should support the Read Primitive Key: BEAM-212 URL: https://issues.apache.org/jira/browse/BEAM-212 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-210) Be consistent with emitting final empty panes

[ https://issues.apache.org/jira/browse/BEAM-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248532#comment-15248532 ] Ben Chambers commented on BEAM-210: --- Most important, I think, is to get rid of the implicit "if you have a

[jira] [Closed] (BEAM-189) The Spark runner uses valueInEmptyWindow which causes values to be dropped

[ https://issues.apache.org/jira/browse/BEAM-189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Sela closed BEAM-189. -- Resolution: Fixed PR-179 > The Spark runner uses valueInEmptyWindow which causes values to be dropped >

[jira] [Commented] (BEAM-210) Be consistent with emitting final empty panes

[ https://issues.apache.org/jira/browse/BEAM-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248520#comment-15248520 ] Kenneth Knowles commented on BEAM-210: -- Today the [ON_TIME pane is hardcoded to be

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow » Apache Beam :: Runners :: Google Cloud Dataflow #156

See

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #156

See

[2/2] incubator-beam git commit: This closes #206

This closes #206 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/0952f443 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/0952f443 Diff:

[GitHub] incubator-beam pull request: [BEAM-22] Add WindowedValue#explodeWi...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/206 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[1/2] incubator-beam git commit: Add WindowedValue#explodeWindows

Repository: incubator-beam Updated Branches: refs/heads/master 135cb733f -> 0952f4433 Add WindowedValue#explodeWindows This takes an existing WindowedValue and returns a Collection of WindowedValues, each of which is in exactly one window. Use the explode implementation on DoFnRunnerBase

[jira] [Commented] (BEAM-210) Be consistent with emitting final empty panes

[ https://issues.apache.org/jira/browse/BEAM-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248490#comment-15248490 ] Ben Chambers commented on BEAM-210: --- No. This is in the weird spot. When we say "empty pane" in terms of

[jira] [Commented] (BEAM-210) Be consistent with emitting final empty panes

[ https://issues.apache.org/jira/browse/BEAM-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248488#comment-15248488 ] ASF GitHub Bot commented on BEAM-210: - GitHub user bjchambers opened a pull request:

[GitHub] incubator-beam pull request: [BEAM-210] Test that empty final pane...

GitHub user bjchambers opened a pull request: https://github.com/apache/incubator-beam/pull/211 [BEAM-210] Test that empty final panes are not produced. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #155

See

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow » Apache Beam :: Runners :: Google Cloud Dataflow #155

See

[jira] [Commented] (BEAM-210) Be consistent with emitting final empty panes

[ https://issues.apache.org/jira/browse/BEAM-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248479#comment-15248479 ] Kenneth Knowles commented on BEAM-210: -- Sounds like expected behavior for

[jira] [Commented] (BEAM-189) The Spark runner uses valueInEmptyWindow which causes values to be dropped

[ https://issues.apache.org/jira/browse/BEAM-189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248476#comment-15248476 ] ASF GitHub Bot commented on BEAM-189: - Github user asfgit closed the pull request at:

[GitHub] incubator-beam pull request: [BEAM-189] The Spark runner uses valu...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/179 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[1/5] incubator-beam git commit: Materialize PCollection/RDD as windowed values with the appropriate windows.

Repository: incubator-beam Updated Branches: refs/heads/master f20bf8afd -> 135cb733f Materialize PCollection/RDD as windowed values with the appropriate windows. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[3/5] incubator-beam git commit: Replace valueInEmptyWindows with valueInGlobalWindow

Replace valueInEmptyWindows with valueInGlobalWindow Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5fab1c5f Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5fab1c5f Diff:

[2/5] incubator-beam git commit: Replace valueInEmptyWindows with valueInGlobalWindow in Spark Function, and add per-value (non-RDD) windowing functions

Replace valueInEmptyWindows with valueInGlobalWindow in Spark Function, and add per-value (non-RDD) windowing functions Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d852c5b9 Tree:

[jira] [Created] (BEAM-211) When using Create.of use it's #withCoder method instead of the created PCollection's #setCoder

Amit Sela created BEAM-211: -- Summary: When using Create.of use it's #withCoder method instead of the created PCollection's #setCoder Key: BEAM-211 URL: https://issues.apache.org/jira/browse/BEAM-211

[jira] [Updated] (BEAM-211) When using Create.of use it's #withCoder method instead of the created PCollection's #setCoder

[ https://issues.apache.org/jira/browse/BEAM-211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Sela updated BEAM-211: --- Component/s: runner-spark > When using Create.of use it's #withCoder method instead of the created >

[jira] [Created] (BEAM-210) Be consistent with emitting final empty panes

Mark Shields created BEAM-210: - Summary: Be consistent with emitting final empty panes Key: BEAM-210 URL: https://issues.apache.org/jira/browse/BEAM-210 Project: Beam Issue Type: Bug

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #154

See

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow » Apache Beam :: Runners :: Google Cloud Dataflow #153

See

[2/2] incubator-beam git commit: Closes #208

Closes #208 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f20bf8af Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/f20bf8af Diff:

[jira] [Commented] (BEAM-208) Flaky test: org.apache.beam.runners.flink.streaming.GroupByNullKeyTest.testJob

[ https://issues.apache.org/jira/browse/BEAM-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248132#comment-15248132 ] Thomas Groh commented on BEAM-208: -- This flakiness occurs due to the combination of using

[jira] [Commented] (BEAM-208) Flaky test: org.apache.beam.runners.flink.streaming.GroupByNullKeyTest.testJob

[ https://issues.apache.org/jira/browse/BEAM-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248133#comment-15248133 ] ASF GitHub Bot commented on BEAM-208: - GitHub user tgroh opened a pull request:

[jira] [Commented] (BEAM-209) PCollection-to-batched-gcs-files PTransform

[ https://issues.apache.org/jira/browse/BEAM-209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248125#comment-15248125 ] Daniel Halperin commented on BEAM-209: -- * Link to Nexmark code you're referring? * Can you say more

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #152

See

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow » Apache Beam :: Runners :: Google Cloud Dataflow #152

See

[jira] [Commented] (BEAM-22) DirectPipelineRunner: support for unbounded collections

[ https://issues.apache.org/jira/browse/BEAM-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248040#comment-15248040 ] ASF GitHub Bot commented on BEAM-22: Github user asfgit closed the pull request at:

[GitHub] incubator-beam pull request: [BEAM-22] Remove InProcessBundle

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/203 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[2/2] incubator-beam git commit: This closes #203

This closes #203 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/442435ed Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/442435ed Diff:

[1/2] incubator-beam git commit: Remove InProcessBundle

Repository: incubator-beam Updated Branches: refs/heads/master 861562239 -> 442435ed0 Remove InProcessBundle InProcessBundle is an implementation detail of InProcessBundleFactory, which it should be contained within. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo

[jira] [Created] (BEAM-209) PCollection-to-batched-gcs-files PTransform

Mark Shields created BEAM-209: - Summary: PCollection-to-batched-gcs-files PTransform Key: BEAM-209 URL: https://issues.apache.org/jira/browse/BEAM-209 Project: Beam Issue Type: New Feature

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #151

See

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow » Apache Beam :: Runners :: Google Cloud Dataflow #151

See

[GitHub] incubator-beam pull request: Add CounterNameAndMetadata to support...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/102 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[1/2] incubator-beam git commit: Add CounterNameAndMetadata to support structured counter name

Repository: incubator-beam Updated Branches: refs/heads/master 70e6a1310 -> 861562239 Add CounterNameAndMetadata to support structured counter name Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: This closes #102

This closes #102 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/86156223 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/86156223 Diff:

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #150

See

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow » Apache Beam :: Runners :: Google Cloud Dataflow #150

See

[jira] [Commented] (BEAM-207) Flink test flake in ReadSourceStreamingITCase

[ https://issues.apache.org/jira/browse/BEAM-207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247359#comment-15247359 ] ASF GitHub Bot commented on BEAM-207: - GitHub user mxm opened a pull request:

[GitHub] incubator-beam pull request: [BEAM-207] Flink test flake in ReadSo...

GitHub user mxm opened a pull request: https://github.com/apache/incubator-beam/pull/209 [BEAM-207] Flink test flake in ReadSourceStreamingITCase The `configure(..)` life cycle method is only called on the master but not on the worker nodes. This may lead to an incorrect

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow » Apache Beam :: Runners :: Google Cloud Dataflow #149

See

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #149

See

[jira] [Commented] (BEAM-196) Pipeline options must be available Context in DoFn.startBundle

[ https://issues.apache.org/jira/browse/BEAM-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247333#comment-15247333 ] ASF GitHub Bot commented on BEAM-196: - Github user asfgit closed the pull request at:

[GitHub] incubator-beam pull request: [BEAM-196] Pipeline options must be a...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/200 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Resolved] (BEAM-196) Pipeline options must be available Context in DoFn.startBundle

[ https://issues.apache.org/jira/browse/BEAM-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-196. - Resolution: Fixed Fixed in 43b5ec743718e63c2d9d9532e3ca55bc87370290. > Pipeline options

[2/3] incubator-beam git commit: [BEAM-196] abstraction for PipelineOptions serialization

[BEAM-196] abstraction for PipelineOptions serialization Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/81577b31 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/81577b31 Diff:

[3/3] incubator-beam git commit: This closes #200

This closes #200 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/70e6a131 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/70e6a131 Diff:

[GitHub] incubator-beam pull request: FileBasedSink: remove unused code

GitHub user dhalperi opened a pull request: https://github.com/apache/incubator-beam/pull/208 FileBasedSink: remove unused code R: @lukecwik You can merge this pull request into a Git repository by running: $ git pull https://github.com/dhalperi/incubator-beam rename-parallel

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #148

See

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow » Apache Beam :: Runners :: Google Cloud Dataflow #148

See