[jira] [Created] (BEAM-667) Include code snippets from real examples

Hadar Hod created BEAM-667: -- Summary: Include code snippets from real examples Key: BEAM-667 URL: https://issues.apache.org/jira/browse/BEAM-667 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-666) Add accurate "How to Run" instructions for each of the WC examples

Hadar Hod created BEAM-666: -- Summary: Add accurate "How to Run" instructions for each of the WC examples Key: BEAM-666 URL: https://issues.apache.org/jira/browse/BEAM-666 Project: Beam Issue Type:

[jira] [Created] (BEAM-665) Copy prose (translate html to md, remove Dataflow references, etc.)

Hadar Hod created BEAM-665: -- Summary: Copy prose (translate html to md, remove Dataflow references, etc.) Key: BEAM-665 URL: https://issues.apache.org/jira/browse/BEAM-665 Project: Beam Issue

[jira] [Created] (BEAM-664) Port Dataflow SDK WordCount walkthrough to Beam site

Hadar Hod created BEAM-664: -- Summary: Port Dataflow SDK WordCount walkthrough to Beam site Key: BEAM-664 URL: https://issues.apache.org/jira/browse/BEAM-664 Project: Beam Issue Type: Task

[GitHub] incubator-beam pull request #985: Support BigQuery DATE type

GitHub user peihe opened a pull request: https://github.com/apache/incubator-beam/pull/985 Support BigQuery DATE type You can merge this pull request into a Git repository by running: $ git pull https://github.com/peihe/incubator-beam bq-new-types Alternatively you can

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1185

See

[GitHub] incubator-beam pull request #984: Remove DataflowMatchers

GitHub user tgroh opened a pull request: https://github.com/apache/incubator-beam/pull/984 Remove DataflowMatchers Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like:

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1184

See

[jira] [Updated] (BEAM-663) AvroSource should update progress within blocks

[ https://issues.apache.org/jira/browse/BEAM-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-663: - Issue Type: Improvement (was: Bug) > AvroSource should update progress within blocks >

[jira] [Updated] (BEAM-663) AvroSource should update progress within blocks

[ https://issues.apache.org/jira/browse/BEAM-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-663: - Component/s: sdk-py > AvroSource should update progress within blocks >

[jira] [Commented] (BEAM-618) Python SDKs writes non RFC compliant JSON files for BQ Export

[ https://issues.apache.org/jira/browse/BEAM-618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15511685#comment-15511685 ] ASF GitHub Bot commented on BEAM-618: - Github user ajamato closed the pull request at:

[GitHub] incubator-beam pull request #947: [BEAM-618] Disallow NAN, INF and -INF inva...

Github user ajamato closed the pull request at: https://github.com/apache/incubator-beam/pull/947 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Created] (BEAM-663) AvroSource should update progress within blocks

Chamikara Jayalath created BEAM-663: --- Summary: AvroSource should update progress within blocks Key: BEAM-663 URL: https://issues.apache.org/jira/browse/BEAM-663 Project: Beam Issue Type:

[1/2] incubator-beam git commit: Closes #977

Repository: incubator-beam Updated Branches: refs/heads/master 31a35e5c9 -> f62d04e22 Closes #977 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f62d04e2 Tree:

[GitHub] incubator-beam pull request #977: Support BigQuery BYTES type

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/977 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[2/2] incubator-beam git commit: Support BigQuery BYTES type

Support BigQuery BYTES type Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c7e0010b Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c7e0010b Diff:

[2/2] incubator-beam git commit: Closes #947

Closes #947 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/345fc698 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/345fc698 Diff:

[1/2] incubator-beam git commit: Set allow_nan=False on bigquery JSON encoding

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 701aff074 -> 345fc6985 Set allow_nan=False on bigquery JSON encoding Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e173765a Tree:

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1183

See

[GitHub] incubator-beam pull request #983: [BEAM-657] Support Read.Bounded primitive

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/983 [BEAM-657] Support Read.Bounded primitive Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is

[jira] [Created] (BEAM-662) SlidingWindows should support sub-second periods

Daniel Mills created BEAM-662: - Summary: SlidingWindows should support sub-second periods Key: BEAM-662 URL: https://issues.apache.org/jira/browse/BEAM-662 Project: Beam Issue Type: Bug

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1182

See

[jira] [Reopened] (BEAM-661) CalendarWindows#isCompatibleWith should use equals instead of ==

[ https://issues.apache.org/jira/browse/BEAM-661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reopened BEAM-661: Assignee: (was: Davor Bonaci) > CalendarWindows#isCompatibleWith should use equals instead of == >

[jira] [Resolved] (BEAM-660) CalendarWindows compares DateTimes with ==

[ https://issues.apache.org/jira/browse/BEAM-660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-660. Resolution: Duplicate Fix Version/s: Not applicable Duplicate of

[jira] [Closed] (BEAM-661) CalendarWindows#isCompatibleWith should use equals instead of ==

[ https://issues.apache.org/jira/browse/BEAM-661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Chambers closed BEAM-661. - Resolution: Duplicate Fix Version/s: Not applicable > CalendarWindows#isCompatibleWith should use

[jira] [Created] (BEAM-661) CalendarWindows#isCompatibleWith should use equals instead of ==

Ben Chambers created BEAM-661: - Summary: CalendarWindows#isCompatibleWith should use equals instead of == Key: BEAM-661 URL: https://issues.apache.org/jira/browse/BEAM-661 Project: Beam Issue

[jira] [Created] (BEAM-660) CalendarWindows compares DateTimes with ==

Daniel Mills created BEAM-660: - Summary: CalendarWindows compares DateTimes with == Key: BEAM-660 URL: https://issues.apache.org/jira/browse/BEAM-660 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-613) SimpleStreamingWordCountTest tests only a single batch

[ https://issues.apache.org/jira/browse/BEAM-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510728#comment-15510728 ] ASF GitHub Bot commented on BEAM-613: - GitHub user staslev opened a pull request:

[GitHub] incubator-beam pull request #982: [BEAM-613] Revised SimpleStreamingWordCoun...

GitHub user staslev opened a pull request: https://github.com/apache/incubator-beam/pull/982 [BEAM-613] Revised SimpleStreamingWordCountTest to test multiple batches Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ]

[2/3] incubator-beam git commit: TextIOTest: rewrite for code reuse and actually more test coverage

TextIOTest: rewrite for code reuse and actually more test coverage We had missed a few empty file cases, etc. Suggestions for improvement still requested. Sad I had to delete the TemporaryFolder @Rule but could not see how to get one-time file creation with it. Project:

[1/3] incubator-beam git commit: Closes #980

Repository: incubator-beam Updated Branches: refs/heads/master 1ceb12aeb -> 31a35e5c9 Closes #980 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/31a35e5c Tree:

[3/3] incubator-beam git commit: TextIO/CompressedSource: split AUTO mode files into bundles

TextIO/CompressedSource: split AUTO mode files into bundles Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/370d5924 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/370d5924 Diff:

[jira] [Updated] (BEAM-610) Enable spark's checkpointing mechanism for driver-failure recovery in streaming

[ https://issues.apache.org/jira/browse/BEAM-610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Sela updated BEAM-610: --- Fix Version/s: 0.3.0-incubating > Enable spark's checkpointing mechanism for driver-failure recovery in >

[jira] [Resolved] (BEAM-610) Enable spark's checkpointing mechanism for driver-failure recovery in streaming

[ https://issues.apache.org/jira/browse/BEAM-610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Sela resolved BEAM-610. Resolution: Fixed > Enable spark's checkpointing mechanism for driver-failure recovery in > streaming >

[GitHub] incubator-beam pull request #909: [BEAM-610] Enable spark's checkpointing me...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/909 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[2/4] incubator-beam git commit: [BEAM-610] Enable spark's checkpointing mechanism for driver-failure recovery in streaming.

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/0feb6499/runners/spark/src/main/java/org/apache/beam/runners/spark/translation/TransformTranslator.java -- diff --git

[1/4] incubator-beam git commit: [BEAM-610] Enable spark's checkpointing mechanism for driver-failure recovery in streaming.

Repository: incubator-beam Updated Branches: refs/heads/master 5c23f4954 -> 1ceb12aeb http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/0feb6499/runners/spark/src/test/java/org/apache/beam/runners/spark/ClearAggregatorsRule.java

[4/4] incubator-beam git commit: This closes #909

This closes #909 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/1ceb12ae Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/1ceb12ae Diff:

[3/4] incubator-beam git commit: [BEAM-610] Enable spark's checkpointing mechanism for driver-failure recovery in streaming.

[BEAM-610] Enable spark's checkpointing mechanism for driver-failure recovery in streaming. Refactor translation mechanism to support checkpointing of DStream. Support basic functionality with GroupByKey and ParDo. Added support for grouping operations. Added checkpointDir option, using it

[jira] [Updated] (BEAM-656) Add support for side inputs/outputs to Java 8 lambdas in MapElements

[ https://issues.apache.org/jira/browse/BEAM-656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-656: - Description: Currently there's no way to use side inputs or outputs within Java 8 lambdas and

[jira] [Commented] (BEAM-645) Running Wordcount in Spark Checks Locally and Outputs in HDFS

[ https://issues.apache.org/jira/browse/BEAM-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510280#comment-15510280 ] Jesse Anderson commented on BEAM-645: - They may well be. We can close for now. Just need to test this

[jira] [Comment Edited] (BEAM-645) Running Wordcount in Spark Checks Locally and Outputs in HDFS

[ https://issues.apache.org/jira/browse/BEAM-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509955#comment-15509955 ] Amit Sela edited comment on BEAM-645 at 9/21/16 3:20 PM: - [~eljefe6aa] the SDK

[jira] [Commented] (BEAM-645) Running Wordcount in Spark Checks Locally and Outputs in HDFS

[ https://issues.apache.org/jira/browse/BEAM-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510270#comment-15510270 ] Jesse Anderson commented on BEAM-645: - I don't have a Flink cluster to test this on. > Running

[jira] [Commented] (BEAM-658) Support Read.Unbounded primitive

[ https://issues.apache.org/jira/browse/BEAM-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510215#comment-15510215 ] Jean-Baptiste Onofré commented on BEAM-658: --- +1 IMHO, it would be great to implement it on master

[jira] [Commented] (BEAM-645) Running Wordcount in Spark Checks Locally and Outputs in HDFS

[ https://issues.apache.org/jira/browse/BEAM-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509955#comment-15509955 ] Amit Sela commented on BEAM-645: [~eljefe6aa] do you run this example with other runners where the input is

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1181

See

[jira] [Commented] (BEAM-658) Support Read.Unbounded primitive

[ https://issues.apache.org/jira/browse/BEAM-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509674#comment-15509674 ] Amit Sela commented on BEAM-658: Suggested design:

[jira] [Created] (BEAM-658) Support Read.Unbounded primitive

Amit Sela created BEAM-658: -- Summary: Support Read.Unbounded primitive Key: BEAM-658 URL: https://issues.apache.org/jira/browse/BEAM-658 Project: Beam Issue Type: Sub-task Components:

[jira] [Created] (BEAM-657) Support Read.Bounded primitive

Amit Sela created BEAM-657: -- Summary: Support Read.Bounded primitive Key: BEAM-657 URL: https://issues.apache.org/jira/browse/BEAM-657 Project: Beam Issue Type: Sub-task Components:

[jira] [Commented] (BEAM-17) Add support for new Beam Source API

[ https://issues.apache.org/jira/browse/BEAM-17?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509658#comment-15509658 ] Amit Sela commented on BEAM-17: --- Same for streaming, with UnboundedSource, but the challenge there is greater

[jira] [Comment Edited] (BEAM-17) Add support for new Beam Source API

[ https://issues.apache.org/jira/browse/BEAM-17?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348444#comment-15348444 ] Amit Sela edited comment on BEAM-17 at 9/21/16 11:44 AM: - Since the SDK has moved to

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1180

See