[jira] [Assigned] (BEAM-440) Create.values() returns a type-unsafe Coder

[Manu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Manu Zhang reassigned BEAM-440:
---

Assignee: Manu Zhang

> Create.values() returns a type-unsafe Coder
> ---
>
> Key: BEAM-440
> URL: https://issues.apache.org/jira/browse/BEAM-440
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Daniel Halperin
>Assignee: Manu Zhang
>  Labels: newbie, starter
>
> Create.values() with no arguments will default to a VoidCoder, unless one is 
> set later with #setCoder(Coder).
> Although it will encode its input correctly, this seems like a bad choice in 
> many cases. E.g., with Flatten:
> PCollection> initial = p.apply("First", 
> Create.>of());
> PCollection> second =
> p.apply("Second", Create.of("a", "b")).apply(ParDo.of(new 
> MyAvroDoFn()));
> PCollectionList
> .of(initial).and(second)
> .apply(Flatten.>pCollections());
> This crashes trying to cast a KV from "Second" to a Void.class.
> 1. Suggest throwing a warning in #getDefaultOutputCoder when defaulting to 
> VoidCoder for an empty elements list. Should this be an error?
> 2. Suggest adding something like Create.empty(TypeDescriptor) to handle this 
> case properly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #748: Optimize Map and Flatmap when there are no...

[robertwb]

GitHub user robertwb opened a pull request:

https://github.com/apache/incubator-beam/pull/748

Optimize Map and Flatmap when there are no side inputs.

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

varargs and kwargs are expensive, even when they're empty.

This is especially true for otherwise one-argument Python calls
which are special cased in CPython.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/robertwb/incubator-beam fast-noarg

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/748.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #748


commit 257d3c449b68fbf842838c95bb2744bb517d02a9
Author: Robert Bradshaw 
Date:   2016-07-28T01:29:59Z

Optimize Map and Flatmap when there are no side inputs.

varargs and kwargs are expensive, even when they're empty.

This is especially true for otherwise one-argument Python calls
which are special cased in CPython.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (BEAM-495) Generalize FileChecksumMatcher used for all E2E test

[Mark Liu (JIRA)]

Mark Liu created BEAM-495:
-

 Summary: Generalize FileChecksumMatcher used for all E2E test
 Key: BEAM-495
 URL: https://issues.apache.org/jira/browse/BEAM-495
 Project: Beam
  Issue Type: Improvement
Reporter: Mark Liu
Assignee: Mark Liu


Refactor WordCountOnSuccessMatcher to be more general so that it can be reused 
by other tests.

Requirement:
Given input file path (accept glob) and expected checksum, generate checksum of 
file(s) and verify with expected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (BEAM-494) All E2E Tests Run Against All Runners In Postcommit

[Jason Kuster (JIRA)]

Jason Kuster created BEAM-494:
-

 Summary: All E2E Tests Run Against All Runners In Postcommit
 Key: BEAM-494
 URL: https://issues.apache.org/jira/browse/BEAM-494
 Project: Beam
  Issue Type: Improvement
Reporter: Jason Kuster
Assignee: Jason Kuster






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (BEAM-493) All Runners Run WordCount in Presubmit

[Jason Kuster (JIRA)]

Jason Kuster created BEAM-493:
-

 Summary: All Runners Run WordCount in Presubmit
 Key: BEAM-493
 URL: https://issues.apache.org/jira/browse/BEAM-493
 Project: Beam
  Issue Type: Improvement
Reporter: Jason Kuster
Assignee: Jason Kuster






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-156) Implement Quiescence Signalling in the InProcessPipelineRunner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396678#comment-15396678
 ] 

ASF GitHub Bot commented on BEAM-156:
-

GitHub user tgroh opened a pull request:

https://github.com/apache/incubator-beam/pull/745

[BEAM-156] Apply ExecutorUpdates in two Phases

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

This removes the need for an explicit break by ensuring that work added
by the monitor will not complete and add more work for the monitor to
complete.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tgroh/incubator-beam two_phase_executor_update

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/745.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #745


commit 4b932d93352b04f60392b6083ce5573c86535a65
Author: Thomas Groh 
Date:   2016-07-22T20:47:19Z

Apply ExecutorUpdates in two Phases

This removes the need for an explicit break by ensuring that work added
by the monitor will not complete and add more work for the monitor to
complete.




> Implement Quiescence Signalling in the InProcessPipelineRunner
> --
>
> Key: BEAM-156
> URL: https://issues.apache.org/jira/browse/BEAM-156
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> A pipeline is quiescent when the following two properties hold:
>   There are no triggers that can fire, given the current processing time and 
> watermark
>   All pending elements cannot make progress until a side input produces 
> additional output
> This is approximately equivalent to: If no more input is received, the 
> pipeline will not perform any additional processing absent advances in 
> processing time or event time
> See also: 
> https://docs.google.com/document/d/1fZUUbG2LxBtqCVabQshldXIhkMcXepsbv2vuuny8Ix4/edit#



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-156) Implement Quiescence Signalling in the InProcessPipelineRunner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396679#comment-15396679
 ] 

ASF GitHub Bot commented on BEAM-156:
-

GitHub user tgroh opened a pull request:

https://github.com/apache/incubator-beam/pull/746

Use AutoValue for StepTransformResult

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---
This is a cleanup PR for future changes to StepTransformResult, primarily 
as part of
BEAM-156


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tgroh/incubator-beam 
step_transform_result_autovalue

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/746.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #746


commit c1f66b67c82e149ef27d19c88993b0ab15d19b35
Author: Thomas Groh 
Date:   2016-07-26T16:38:13Z

Use AutoValue for StepTransformResult




> Implement Quiescence Signalling in the InProcessPipelineRunner
> --
>
> Key: BEAM-156
> URL: https://issues.apache.org/jira/browse/BEAM-156
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> A pipeline is quiescent when the following two properties hold:
>   There are no triggers that can fire, given the current processing time and 
> watermark
>   All pending elements cannot make progress until a side input produces 
> additional output
> This is approximately equivalent to: If no more input is received, the 
> pipeline will not perform any additional processing absent advances in 
> processing time or event time
> See also: 
> https://docs.google.com/document/d/1fZUUbG2LxBtqCVabQshldXIhkMcXepsbv2vuuny8Ix4/edit#



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #747: Remove unneccessary method in ReadEvaluato...

[tgroh]

GitHub user tgroh opened a pull request:

https://github.com/apache/incubator-beam/pull/747

Remove unneccessary method in ReadEvaluatorFactories

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

The getTransformEvaluator() method's only call is poll on the result of
getTransformEvaluatorQueue (which is only ever called by
getTransformEvaluator). Instead, move the construction to
getTransformEvaluator and call poll on the result.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tgroh/incubator-beam 
cleanup_read_evaluator_factories

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/747.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #747


commit 7008b72e6fa8ccfc3355b0dde1598190123638a7
Author: Thomas Groh 
Date:   2016-07-22T22:34:03Z

Remove unneccessary method in ReadEvaluatorFactories

The getTransformEvaluator() method's only call is poll on the result of
getTransformEvaluatorQueue (which is only ever called by
getTransformEvaluator). Instead, move the construction to
getTransformEvaluator and call poll on the result.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #745: [BEAM-156] Apply ExecutorUpdates in two Ph...

[tgroh]

GitHub user tgroh opened a pull request:

https://github.com/apache/incubator-beam/pull/745

[BEAM-156] Apply ExecutorUpdates in two Phases

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

This removes the need for an explicit break by ensuring that work added
by the monitor will not complete and add more work for the monitor to
complete.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tgroh/incubator-beam two_phase_executor_update

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/745.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #745


commit 4b932d93352b04f60392b6083ce5573c86535a65
Author: Thomas Groh 
Date:   2016-07-22T20:47:19Z

Apply ExecutorUpdates in two Phases

This removes the need for an explicit break by ensuring that work added
by the monitor will not complete and add more work for the monitor to
complete.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (BEAM-416) Jenkins Python Verify post commit tests are timing out

[Daniel Halperin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin updated BEAM-416:
-
Fix Version/s: (was: 0.2.0-incubating)
   Not applicable

> Jenkins Python Verify post commit tests are timing out
> --
>
> Key: BEAM-416
> URL: https://issues.apache.org/jira/browse/BEAM-416
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py
>Reporter: Ahmet Altay
>Assignee: Silviu Calinoiu
> Fix For: Not applicable
>
> Attachments: job_output (2)
>
>
> beam_PostCommit_PythonVerify is timing out at the e2e testing phase:
> Console output:
> https://builds.apache.org/view/Beam/job/beam_PostCommit_PythonVerify/8/console
> e2e test:
> https://pantheon.corp.google.com/dataflow/job/2016-07-01_08_02_45-15435546446836030984?project=apache-beam-testing
> Workers are failing to find the correct container image (from worker logs):
> Error syncing pod 6d3e3a71409d65aa43494143d705455b, skipping: failed to 
> "StartContainer" for "python" with ImagePullBackOff: "Back-off pulling image 
> \"dataflow.gcr.io/v1beta3/python:latest\""
> It might be related to this commit:
> https://github.com/apache/incubator-beam/commit/0bda677d47d5bd5d9c45b74e00e5c3fd113a4f81



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (BEAM-416) Jenkins Python Verify post commit tests are timing out

[Daniel Halperin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin reopened BEAM-416:
--

> Jenkins Python Verify post commit tests are timing out
> --
>
> Key: BEAM-416
> URL: https://issues.apache.org/jira/browse/BEAM-416
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py
>Reporter: Ahmet Altay
>Assignee: Silviu Calinoiu
> Fix For: Not applicable
>
> Attachments: job_output (2)
>
>
> beam_PostCommit_PythonVerify is timing out at the e2e testing phase:
> Console output:
> https://builds.apache.org/view/Beam/job/beam_PostCommit_PythonVerify/8/console
> e2e test:
> https://pantheon.corp.google.com/dataflow/job/2016-07-01_08_02_45-15435546446836030984?project=apache-beam-testing
> Workers are failing to find the correct container image (from worker logs):
> Error syncing pod 6d3e3a71409d65aa43494143d705455b, skipping: failed to 
> "StartContainer" for "python" with ImagePullBackOff: "Back-off pulling image 
> \"dataflow.gcr.io/v1beta3/python:latest\""
> It might be related to this commit:
> https://github.com/apache/incubator-beam/commit/0bda677d47d5bd5d9c45b74e00e5c3fd113a4f81



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (BEAM-416) Jenkins Python Verify post commit tests are timing out

[Daniel Halperin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin resolved BEAM-416.
--
Resolution: Fixed

> Jenkins Python Verify post commit tests are timing out
> --
>
> Key: BEAM-416
> URL: https://issues.apache.org/jira/browse/BEAM-416
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py
>Reporter: Ahmet Altay
>Assignee: Silviu Calinoiu
> Fix For: Not applicable
>
> Attachments: job_output (2)
>
>
> beam_PostCommit_PythonVerify is timing out at the e2e testing phase:
> Console output:
> https://builds.apache.org/view/Beam/job/beam_PostCommit_PythonVerify/8/console
> e2e test:
> https://pantheon.corp.google.com/dataflow/job/2016-07-01_08_02_45-15435546446836030984?project=apache-beam-testing
> Workers are failing to find the correct container image (from worker logs):
> Error syncing pod 6d3e3a71409d65aa43494143d705455b, skipping: failed to 
> "StartContainer" for "python" with ImagePullBackOff: "Back-off pulling image 
> \"dataflow.gcr.io/v1beta3/python:latest\""
> It might be related to this commit:
> https://github.com/apache/incubator-beam/commit/0bda677d47d5bd5d9c45b74e00e5c3fd113a4f81



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-372) CoderProperties: Test that the coder doesn't consume more bytes than it produces

[Daniel Halperin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin updated BEAM-372:
-
Issue Type: Test  (was: Bug)

> CoderProperties: Test that the coder doesn't consume more bytes than it 
> produces
> 
>
> Key: BEAM-372
> URL: https://issues.apache.org/jira/browse/BEAM-372
> Project: Beam
>  Issue Type: Test
>  Components: sdk-java-core
>Reporter: Ben Chambers
>Assignee: Chandni Singh
>Priority: Minor
>  Labels: beginner, newbie, starter
> Fix For: 0.2.0-incubating
>
>
> Add a test to CoderProperties that does the following:
> 1. Encode a value using the Coder in the nested context
> 2. Decode the value using the Coder in the nested context
> 3. Verify that the input stream wasn't requested to consume any extra bytes
> (This could possibly just be an enhancement to the existing round-trip 
> encode/decode test)
> When this fails it can lead to very difficult to debug situations in a coder 
> wrapped around the problematic coder. This would be an easy test that would 
> clearly fail *for the coder which was problematic*.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-285) Make MinLongFn and MaxLongFn mimic SumLongFn and use BinaryCombineLongFn

[Daniel Halperin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin updated BEAM-285:
-
Issue Type: Improvement  (was: Bug)

> Make MinLongFn and MaxLongFn mimic SumLongFn and use BinaryCombineLongFn
> 
>
> Key: BEAM-285
> URL: https://issues.apache.org/jira/browse/BEAM-285
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Mark Shields
>Assignee: Pei He
> Fix For: 0.2.0-incubating
>
>
> Ditto for the other 'optimized accumulator' combiner functions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

incubator-beam git commit: [maven-release-plugin] rollback changes from release preparation of v0.2.0-incubating-RC1

[dhalperi]

Repository: incubator-beam
Updated Branches:
  refs/heads/release-0.2.0-incubating 453f9838b -> a6633756e


[maven-release-plugin] rollback changes from release preparation of 
v0.2.0-incubating-RC1


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a6633756
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/a6633756
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/a6633756

Branch: refs/heads/release-0.2.0-incubating
Commit: a6633756edfe2a52e985f77a32afe7dc7ff5a1de
Parents: 453f983
Author: Dan Halperin 
Authored: Wed Jul 27 15:29:15 2016 -0700
Committer: Dan Halperin 
Committed: Wed Jul 27 15:29:15 2016 -0700

--
 examples/java/pom.xml   | 2 +-
 examples/java8/pom.xml  | 2 +-
 examples/pom.xml| 2 +-
 pom.xml | 4 ++--
 runners/core-java/pom.xml   | 2 +-
 runners/direct-java/pom.xml | 2 +-
 runners/flink/examples/pom.xml  | 2 +-
 runners/flink/pom.xml   | 2 +-
 runners/flink/runner/pom.xml| 2 +-
 runners/google-cloud-dataflow-java/pom.xml  | 2 +-
 runners/pom.xml | 2 +-
 runners/spark/pom.xml   | 2 +-
 sdks/java/build-tools/pom.xml   | 2 +-
 sdks/java/core/pom.xml  | 2 +-
 sdks/java/extensions/join-library/pom.xml   | 2 +-
 sdks/java/extensions/pom.xml| 2 +-
 sdks/java/io/google-cloud-platform/pom.xml  | 2 +-
 sdks/java/io/hdfs/pom.xml   | 2 +-
 sdks/java/io/jms/pom.xml| 2 +-
 sdks/java/io/kafka/pom.xml  | 2 +-
 sdks/java/io/pom.xml| 2 +-
 sdks/java/java8tests/pom.xml| 2 +-
 sdks/java/maven-archetypes/examples/pom.xml | 2 +-
 sdks/java/maven-archetypes/pom.xml  | 2 +-
 sdks/java/maven-archetypes/starter/pom.xml  | 2 +-
 sdks/java/microbenchmarks/pom.xml   | 2 +-
 sdks/java/pom.xml   | 2 +-
 sdks/pom.xml| 2 +-
 28 files changed, 29 insertions(+), 29 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/a6633756/examples/java/pom.xml
--
diff --git a/examples/java/pom.xml b/examples/java/pom.xml
index cdc1296..5044393 100644
--- a/examples/java/pom.xml
+++ b/examples/java/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-examples-parent
-0.2.0-incubating
+0.2.0-incubating-SNAPSHOT
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/a6633756/examples/java8/pom.xml
--
diff --git a/examples/java8/pom.xml b/examples/java8/pom.xml
index ef081b9..36d44c5 100644
--- a/examples/java8/pom.xml
+++ b/examples/java8/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-examples-parent
-0.2.0-incubating
+0.2.0-incubating-SNAPSHOT
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/a6633756/examples/pom.xml
--
diff --git a/examples/pom.xml b/examples/pom.xml
index beaa34a..ee12b69 100644
--- a/examples/pom.xml
+++ b/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-parent
-0.2.0-incubating
+0.2.0-incubating-SNAPSHOT
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/a6633756/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 1984562..8e8ef47 100644
--- a/pom.xml
+++ b/pom.xml
@@ -34,7 +34,7 @@
   http://beam.incubator.apache.org
   2016
 
-  0.2.0-incubating
+  0.2.0-incubating-SNAPSHOT
 
   
 
@@ -48,7 +48,7 @@
 
scm:git:https://git-wip-us.apache.org/repos/asf/incubator-beam.git
 
scm:git:https://git-wip-us.apache.org/repos/asf/incubator-beam.git
 
https://git-wip-us.apache.org/repos/asf?p=incubator-beam.git;a=summary
-v0.2.0-incubating-RC1
+release-0.2.0-incubating
   
 
   

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/a6633756/runners/core-java/pom.xml
--
diff --git a/runners/core-java/pom.xml b/runners/core-java/pom.xml
index bc42860..fc8be0a 100644
--- a/runners/core-java/pom.xml
+++ b/runners/core-java/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-runners-parent
-0.2.0-incubating
+0.2.0-incubating-SNAPSHOT
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/a6633756/runners/direct-java/pom.xml

incubator-beam git commit: [maven-release-plugin] prepare release v0.2.0-incubating-RC1

[dhalperi]

Repository: incubator-beam
Updated Branches:
  refs/heads/release-0.2.0-incubating db7dbeaf2 -> 453f9838b


[maven-release-plugin] prepare release v0.2.0-incubating-RC1


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/453f9838
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/453f9838
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/453f9838

Branch: refs/heads/release-0.2.0-incubating
Commit: 453f9838bec704f2bd20ad5b2ef4f82f5cd3cbca
Parents: db7dbea
Author: Dan Halperin 
Authored: Wed Jul 27 15:29:00 2016 -0700
Committer: Dan Halperin 
Committed: Wed Jul 27 15:29:00 2016 -0700

--
 examples/java/pom.xml   | 2 +-
 examples/java8/pom.xml  | 2 +-
 examples/pom.xml| 2 +-
 pom.xml | 4 ++--
 runners/core-java/pom.xml   | 2 +-
 runners/direct-java/pom.xml | 2 +-
 runners/flink/examples/pom.xml  | 2 +-
 runners/flink/pom.xml   | 2 +-
 runners/flink/runner/pom.xml| 2 +-
 runners/google-cloud-dataflow-java/pom.xml  | 2 +-
 runners/pom.xml | 2 +-
 runners/spark/pom.xml   | 2 +-
 sdks/java/build-tools/pom.xml   | 2 +-
 sdks/java/core/pom.xml  | 2 +-
 sdks/java/extensions/join-library/pom.xml   | 2 +-
 sdks/java/extensions/pom.xml| 2 +-
 sdks/java/io/google-cloud-platform/pom.xml  | 2 +-
 sdks/java/io/hdfs/pom.xml   | 2 +-
 sdks/java/io/jms/pom.xml| 2 +-
 sdks/java/io/kafka/pom.xml  | 2 +-
 sdks/java/io/pom.xml| 2 +-
 sdks/java/java8tests/pom.xml| 2 +-
 sdks/java/maven-archetypes/examples/pom.xml | 2 +-
 sdks/java/maven-archetypes/pom.xml  | 2 +-
 sdks/java/maven-archetypes/starter/pom.xml  | 2 +-
 sdks/java/microbenchmarks/pom.xml   | 2 +-
 sdks/java/pom.xml   | 2 +-
 sdks/pom.xml| 2 +-
 28 files changed, 29 insertions(+), 29 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/453f9838/examples/java/pom.xml
--
diff --git a/examples/java/pom.xml b/examples/java/pom.xml
index 5044393..cdc1296 100644
--- a/examples/java/pom.xml
+++ b/examples/java/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-examples-parent
-0.2.0-incubating-SNAPSHOT
+0.2.0-incubating
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/453f9838/examples/java8/pom.xml
--
diff --git a/examples/java8/pom.xml b/examples/java8/pom.xml
index 36d44c5..ef081b9 100644
--- a/examples/java8/pom.xml
+++ b/examples/java8/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-examples-parent
-0.2.0-incubating-SNAPSHOT
+0.2.0-incubating
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/453f9838/examples/pom.xml
--
diff --git a/examples/pom.xml b/examples/pom.xml
index ee12b69..beaa34a 100644
--- a/examples/pom.xml
+++ b/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-parent
-0.2.0-incubating-SNAPSHOT
+0.2.0-incubating
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/453f9838/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 8e8ef47..1984562 100644
--- a/pom.xml
+++ b/pom.xml
@@ -34,7 +34,7 @@
   http://beam.incubator.apache.org
   2016
 
-  0.2.0-incubating-SNAPSHOT
+  0.2.0-incubating
 
   
 
@@ -48,7 +48,7 @@
 
scm:git:https://git-wip-us.apache.org/repos/asf/incubator-beam.git
 
scm:git:https://git-wip-us.apache.org/repos/asf/incubator-beam.git
 
https://git-wip-us.apache.org/repos/asf?p=incubator-beam.git;a=summary
-release-0.2.0-incubating
+v0.2.0-incubating-RC1
   
 
   

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/453f9838/runners/core-java/pom.xml
--
diff --git a/runners/core-java/pom.xml b/runners/core-java/pom.xml
index fc8be0a..bc42860 100644
--- a/runners/core-java/pom.xml
+++ b/runners/core-java/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-runners-parent
-0.2.0-incubating-SNAPSHOT
+0.2.0-incubating
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/453f9838/runners/direct-java/pom.xml

[incubator-beam] Git Push Summary

[dhalperi]

Repository: incubator-beam
Updated Tags:  refs/tags/v0.2.0-incubating-RC1 [created] aa09bc7a8

incubator-beam git commit: [maven-release-plugin] prepare for next development iteration

[dhalperi]

Repository: incubator-beam
Updated Branches:
  refs/heads/master db7dbeaf2 -> af396bf8e


[maven-release-plugin] prepare for next development iteration


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/af396bf8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/af396bf8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/af396bf8

Branch: refs/heads/master
Commit: af396bf8e4cdc021cf8558a8cb3a7b8483afc7a2
Parents: db7dbea
Author: Dan Halperin 
Authored: Wed Jul 27 14:54:54 2016 -0700
Committer: Dan Halperin 
Committed: Wed Jul 27 14:54:54 2016 -0700

--
 examples/java/pom.xml   | 2 +-
 examples/java8/pom.xml  | 2 +-
 examples/pom.xml| 2 +-
 pom.xml | 2 +-
 runners/core-java/pom.xml   | 2 +-
 runners/direct-java/pom.xml | 2 +-
 runners/flink/examples/pom.xml  | 2 +-
 runners/flink/pom.xml   | 2 +-
 runners/flink/runner/pom.xml| 2 +-
 runners/google-cloud-dataflow-java/pom.xml  | 2 +-
 runners/pom.xml | 2 +-
 runners/spark/pom.xml   | 2 +-
 sdks/java/build-tools/pom.xml   | 2 +-
 sdks/java/core/pom.xml  | 2 +-
 sdks/java/extensions/join-library/pom.xml   | 2 +-
 sdks/java/extensions/pom.xml| 2 +-
 sdks/java/io/google-cloud-platform/pom.xml  | 2 +-
 sdks/java/io/hdfs/pom.xml   | 2 +-
 sdks/java/io/jms/pom.xml| 2 +-
 sdks/java/io/kafka/pom.xml  | 2 +-
 sdks/java/io/pom.xml| 2 +-
 sdks/java/java8tests/pom.xml| 2 +-
 sdks/java/maven-archetypes/examples/pom.xml | 2 +-
 sdks/java/maven-archetypes/pom.xml  | 2 +-
 sdks/java/maven-archetypes/starter/pom.xml  | 2 +-
 sdks/java/microbenchmarks/pom.xml   | 2 +-
 sdks/java/pom.xml   | 2 +-
 sdks/pom.xml| 2 +-
 28 files changed, 28 insertions(+), 28 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/af396bf8/examples/java/pom.xml
--
diff --git a/examples/java/pom.xml b/examples/java/pom.xml
index 5044393..8b98288 100644
--- a/examples/java/pom.xml
+++ b/examples/java/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-examples-parent
-0.2.0-incubating-SNAPSHOT
+0.3.0-incubating-SNAPSHOT
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/af396bf8/examples/java8/pom.xml
--
diff --git a/examples/java8/pom.xml b/examples/java8/pom.xml
index 36d44c5..609dcfa 100644
--- a/examples/java8/pom.xml
+++ b/examples/java8/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-examples-parent
-0.2.0-incubating-SNAPSHOT
+0.3.0-incubating-SNAPSHOT
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/af396bf8/examples/pom.xml
--
diff --git a/examples/pom.xml b/examples/pom.xml
index ee12b69..2efb146 100644
--- a/examples/pom.xml
+++ b/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-parent
-0.2.0-incubating-SNAPSHOT
+0.3.0-incubating-SNAPSHOT
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/af396bf8/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 8e8ef47..caec6d5 100644
--- a/pom.xml
+++ b/pom.xml
@@ -34,7 +34,7 @@
   http://beam.incubator.apache.org
   2016
 
-  0.2.0-incubating-SNAPSHOT
+  0.3.0-incubating-SNAPSHOT
 
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/af396bf8/runners/core-java/pom.xml
--
diff --git a/runners/core-java/pom.xml b/runners/core-java/pom.xml
index fc8be0a..d958dd2 100644
--- a/runners/core-java/pom.xml
+++ b/runners/core-java/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-runners-parent
-0.2.0-incubating-SNAPSHOT
+0.3.0-incubating-SNAPSHOT
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/af396bf8/runners/direct-java/pom.xml
--
diff --git a/runners/direct-java/pom.xml b/runners/direct-java/pom.xml
index 4afb0d8..0a2b4b9 100644
--- a/runners/direct-java/pom.xml
+++ b/runners/direct-java/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-runners-parent
-

incubator-beam git commit: [maven-release-plugin] prepare branch release-0.2.0-incubating

[dhalperi]

Repository: incubator-beam
Updated Branches:
  refs/heads/master c6e748c09 -> db7dbeaf2


[maven-release-plugin] prepare branch release-0.2.0-incubating


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/db7dbeaf
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/db7dbeaf
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/db7dbeaf

Branch: refs/heads/master
Commit: db7dbeaf205e27534e17f42a457ac1eacbd2aedd
Parents: c6e748c
Author: Dan Halperin 
Authored: Wed Jul 27 14:50:58 2016 -0700
Committer: Dan Halperin 
Committed: Wed Jul 27 14:50:58 2016 -0700

--
 examples/java/pom.xml  | 2 +-
 pom.xml| 4 ++--
 runners/google-cloud-dataflow-java/pom.xml | 2 +-
 sdks/java/io/jms/pom.xml   | 4 +---
 4 files changed, 5 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/db7dbeaf/examples/java/pom.xml
--
diff --git a/examples/java/pom.xml b/examples/java/pom.xml
index c1ea140..5044393 100644
--- a/examples/java/pom.xml
+++ b/examples/java/pom.xml
@@ -45,7 +45,7 @@
 maven-surefire-plugin
 
   
-
+
 
 
   

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/db7dbeaf/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 9d4974f..8e8ef47 100644
--- a/pom.xml
+++ b/pom.xml
@@ -48,7 +48,7 @@
 
scm:git:https://git-wip-us.apache.org/repos/asf/incubator-beam.git
 
scm:git:https://git-wip-us.apache.org/repos/asf/incubator-beam.git
 
https://git-wip-us.apache.org/repos/asf?p=incubator-beam.git;a=summary
-release-0.1.0-incubating
+release-0.2.0-incubating
   
 
   
@@ -98,7 +98,7 @@
 true
 
 
-
+
 
 
 1.8.1

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/db7dbeaf/runners/google-cloud-dataflow-java/pom.xml
--
diff --git a/runners/google-cloud-dataflow-java/pom.xml 
b/runners/google-cloud-dataflow-java/pom.xml
index f2c0fe8..da052ff 100644
--- a/runners/google-cloud-dataflow-java/pom.xml
+++ b/runners/google-cloud-dataflow-java/pom.xml
@@ -56,7 +56,7 @@
 maven-surefire-plugin
 
   
-
+
 true
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/db7dbeaf/sdks/java/io/jms/pom.xml
--
diff --git a/sdks/java/io/jms/pom.xml b/sdks/java/io/jms/pom.xml
index e0e0f36..fa56c7b 100644
--- a/sdks/java/io/jms/pom.xml
+++ b/sdks/java/io/jms/pom.xml
@@ -15,9 +15,7 @@
 See the License for the specific language governing permissions and
 limitations under the License.
 -->
-http://maven.apache.org/POM/4.0.0;
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
 
   4.0.0
 

[jira] [Assigned] (BEAM-450) Modules are shaded to the same path

[Daniel Halperin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin reassigned BEAM-450:


Assignee: Daniel Halperin  (was: Manu Zhang)

> Modules are shaded to the same path
> ---
>
> Key: BEAM-450
> URL: https://issues.apache.org/jira/browse/BEAM-450
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Affects Versions: 0.1.0-incubating, 0.2.0-incubating
>Reporter: Daniel Halperin
>Assignee: Daniel Halperin
>  Labels: newbie, starter
>
> Right now multiple modules are using the same repackaged path. We should be 
> using per-artifact paths so that they don't conflict.
> One proposal was simply to adopt 
> {{${project.groupId}.${project.artifactId}.repackaged}} as the shading 
> location. If it works.
> This is a good starter issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #744: [BEAM-450] Shade separately per artifact

[dhalperi]

GitHub user dhalperi opened a pull request:

https://github.com/apache/incubator-beam/pull/744

[BEAM-450] Shade separately per artifact

Prevents reusing the same path to shaded files across packages.

R: @lukecwik or @kennknowles 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dhalperi/incubator-beam beam-450

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/744.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #744


commit b10a427f4ac92d3217dce346a93ef063b4101e47
Author: Dan Halperin 
Date:   2016-07-26T06:54:34Z

[BEAM-450] Shade separately per artifact

Prevents conflicts among shaded files




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (BEAM-450) Modules are shaded to the same path

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396434#comment-15396434
 ] 

ASF GitHub Bot commented on BEAM-450:
-

GitHub user dhalperi opened a pull request:

https://github.com/apache/incubator-beam/pull/744

[BEAM-450] Shade separately per artifact

Prevents reusing the same path to shaded files across packages.

R: @lukecwik or @kennknowles 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dhalperi/incubator-beam beam-450

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/744.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #744


commit b10a427f4ac92d3217dce346a93ef063b4101e47
Author: Dan Halperin 
Date:   2016-07-26T06:54:34Z

[BEAM-450] Shade separately per artifact

Prevents conflicts among shaded files




> Modules are shaded to the same path
> ---
>
> Key: BEAM-450
> URL: https://issues.apache.org/jira/browse/BEAM-450
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Affects Versions: 0.1.0-incubating, 0.2.0-incubating
>Reporter: Daniel Halperin
>Assignee: Manu Zhang
>  Labels: newbie, starter
>
> Right now multiple modules are using the same repackaged path. We should be 
> using per-artifact paths so that they don't conflict.
> One proposal was simply to adopt 
> {{${project.groupId}.${project.artifactId}.repackaged}} as the shading 
> location. If it works.
> This is a good starter issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-433) Make Beam examples runners agnostic

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396415#comment-15396415
 ] 

ASF GitHub Bot commented on BEAM-433:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/742


> Make Beam examples runners agnostic
> ---
>
> Key: BEAM-433
> URL: https://issues.apache.org/jira/browse/BEAM-433
> Project: Beam
>  Issue Type: Improvement
>  Components: examples-java
>Reporter: Pei He
>Assignee: Pei He
>
> Beam examples are ported from Dataflow, and they heavily reference to 
> Dataflow classes.
> There are following cleanup tasks:
> 1. Remove Dataflow streaming and batch injector setup (Done).
> 2. Remove references to DataflowPipelineOptions.
> 3. Move cancel() from DataflowPipelineJob to PipelineResult.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] incubator-beam git commit: [BEAM-433] Remove references to DataflowPipelineOptions

[dhalperi]

Repository: incubator-beam
Updated Branches:
  refs/heads/master 2cbac53c5 -> c6e748c09


[BEAM-433] Remove references to DataflowPipelineOptions


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c11ebf70
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c11ebf70
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/c11ebf70

Branch: refs/heads/master
Commit: c11ebf70a9749f5099c1e0cd55bd6f0da40a8402
Parents: 2cbac53
Author: Pei He 
Authored: Wed Jul 27 12:29:44 2016 -0700
Committer: Dan Halperin 
Committed: Wed Jul 27 14:33:04 2016 -0700

--
 .../java/org/apache/beam/examples/common/PubsubFileInjector.java | 4 ++--
 .../org/apache/beam/examples/complete/TopWikipediaSessions.java  | 4 +---
 2 files changed, 3 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/c11ebf70/examples/java/src/main/java/org/apache/beam/examples/common/PubsubFileInjector.java
--
diff --git 
a/examples/java/src/main/java/org/apache/beam/examples/common/PubsubFileInjector.java
 
b/examples/java/src/main/java/org/apache/beam/examples/common/PubsubFileInjector.java
index 82d58b6..15eda06 100644
--- 
a/examples/java/src/main/java/org/apache/beam/examples/common/PubsubFileInjector.java
+++ 
b/examples/java/src/main/java/org/apache/beam/examples/common/PubsubFileInjector.java
@@ -17,12 +17,12 @@
  */
 package org.apache.beam.examples.common;
 
-import org.apache.beam.runners.dataflow.options.DataflowPipelineOptions;
 import org.apache.beam.sdk.Pipeline;
 import org.apache.beam.sdk.io.TextIO;
 import org.apache.beam.sdk.options.Description;
 import org.apache.beam.sdk.options.PipelineOptions;
 import org.apache.beam.sdk.options.PipelineOptionsFactory;
+import org.apache.beam.sdk.options.PubsubOptions;
 import org.apache.beam.sdk.options.Validation;
 import org.apache.beam.sdk.transforms.DoFn;
 import org.apache.beam.sdk.transforms.IntraBundleParallelization;
@@ -85,7 +85,7 @@ public class PubsubFileInjector {
 @Override
 public void startBundle(Context context) {
   this.pubsub =
-  
Transport.newPubsubClient(context.getPipelineOptions().as(DataflowPipelineOptions.class))
+  
Transport.newPubsubClient(context.getPipelineOptions().as(PubsubOptions.class))
   .build();
 }
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/c11ebf70/examples/java/src/main/java/org/apache/beam/examples/complete/TopWikipediaSessions.java
--
diff --git 
a/examples/java/src/main/java/org/apache/beam/examples/complete/TopWikipediaSessions.java
 
b/examples/java/src/main/java/org/apache/beam/examples/complete/TopWikipediaSessions.java
index 80b3ade..f8af02a 100644
--- 
a/examples/java/src/main/java/org/apache/beam/examples/complete/TopWikipediaSessions.java
+++ 
b/examples/java/src/main/java/org/apache/beam/examples/complete/TopWikipediaSessions.java
@@ -17,7 +17,6 @@
  */
 package org.apache.beam.examples.complete;
 
-import org.apache.beam.runners.dataflow.options.DataflowPipelineOptions;
 import org.apache.beam.sdk.Pipeline;
 import org.apache.beam.sdk.coders.TableRowJsonCoder;
 import org.apache.beam.sdk.io.TextIO;
@@ -207,9 +206,8 @@ public class TopWikipediaSessions {
 Options options = PipelineOptionsFactory.fromArgs(args)
 .withValidation()
 .as(Options.class);
-DataflowPipelineOptions dataflowOptions = 
options.as(DataflowPipelineOptions.class);
 
-Pipeline p = Pipeline.create(dataflowOptions);
+Pipeline p = Pipeline.create(options);
 
 double samplingThreshold = 0.1;
 

[GitHub] incubator-beam pull request #742: [BEAM-433] Remove references to DataflowPi...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/742


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[2/2] incubator-beam git commit: Closes #742

[dhalperi]

Closes #742


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c6e748c0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c6e748c0
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/c6e748c0

Branch: refs/heads/master
Commit: c6e748c09c18fad9b0c0158db48e329c76d66a08
Parents: 2cbac53 c11ebf7
Author: Dan Halperin 
Authored: Wed Jul 27 14:33:05 2016 -0700
Committer: Dan Halperin 
Committed: Wed Jul 27 14:33:05 2016 -0700

--
 .../java/org/apache/beam/examples/common/PubsubFileInjector.java | 4 ++--
 .../org/apache/beam/examples/complete/TopWikipediaSessions.java  | 4 +---
 2 files changed, 3 insertions(+), 5 deletions(-)
--

[1/2] incubator-beam git commit: Create DoFnInvoker instances in the package of the invoked DoFn

[kenn]

Repository: incubator-beam
Updated Branches:
  refs/heads/master 0866e4906 -> 2cbac53c5


Create DoFnInvoker instances in the package of the invoked DoFn

The DoFnInvoker implementation for a DoFn carries a field
with the type of the invoked DoFn, and that type needs to be
visible.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/681ad896
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/681ad896
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/681ad896

Branch: refs/heads/master
Commit: 681ad8968ef68a309e118ae0d87635535ec26b3f
Parents: e8695a1
Author: Kenneth Knowles 
Authored: Tue Jul 26 21:40:02 2016 -0700
Committer: Kenneth Knowles 
Committed: Wed Jul 27 14:00:05 2016 -0700

--
 .../beam/sdk/transforms/DoFnReflector.java  |  20 +++-
 .../beam/sdk/transforms/DoFnReflectorTest.java  |  75 +++-
 .../dofnreflector/DoFnReflectorTestHelper.java  | 116 +++
 3 files changed, 203 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/681ad896/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFnReflector.java
--
diff --git 
a/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFnReflector.java
 
b/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFnReflector.java
index 116b64d..0616eff 100644
--- 
a/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFnReflector.java
+++ 
b/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFnReflector.java
@@ -42,16 +42,18 @@ import com.google.common.reflect.TypeParameter;
 import com.google.common.reflect.TypeToken;
 
 import net.bytebuddy.ByteBuddy;
+import net.bytebuddy.NamingStrategy.SuffixingRandom;
 import net.bytebuddy.description.field.FieldDescription;
 import net.bytebuddy.description.method.MethodDescription;
 import net.bytebuddy.description.method.ParameterList;
 import net.bytebuddy.description.modifier.FieldManifestation;
 import net.bytebuddy.description.modifier.Visibility;
 import net.bytebuddy.description.type.TypeDescription;
+import net.bytebuddy.description.type.TypeDescription.Generic;
 import net.bytebuddy.dynamic.DynamicType;
 import net.bytebuddy.dynamic.loading.ClassLoadingStrategy;
 import net.bytebuddy.dynamic.scaffold.InstrumentedType;
-import net.bytebuddy.dynamic.scaffold.subclass.ConstructorStrategy;
+import net.bytebuddy.dynamic.scaffold.subclass.ConstructorStrategy.Default;
 import net.bytebuddy.implementation.Implementation;
 import net.bytebuddy.implementation.MethodCall.MethodLocator;
 import net.bytebuddy.implementation.StubMethod;
@@ -70,7 +72,6 @@ import net.bytebuddy.jar.asm.Label;
 import net.bytebuddy.jar.asm.MethodVisitor;
 import net.bytebuddy.jar.asm.Opcodes;
 import net.bytebuddy.matcher.ElementMatchers;
-
 import org.joda.time.Instant;
 
 import java.io.IOException;
@@ -89,7 +90,6 @@ import java.util.LinkedHashMap;
 import java.util.LinkedHashSet;
 import java.util.List;
 import java.util.Map;
-
 import javax.annotation.Nullable;
 
 
@@ -499,8 +499,20 @@ public abstract class DoFnReflector {
  */
 private Constructor> createInvokerConstructor(
 @SuppressWarnings("rawtypes") Class clazz) {
+
+  final TypeDescription clazzDescription = new 
TypeDescription.ForLoadedType(clazz);
+
   DynamicType.Builder builder = new ByteBuddy()
-  .subclass(DoFnInvoker.class, 
ConstructorStrategy.Default.NO_CONSTRUCTORS)
+  // Create subclasses inside the target class, to have access to
+  // private and package-private bits
+  .with(new SuffixingRandom("auxiliary") {
+@Override
+public String subclass(Generic superClass) {
+  return super.name(clazzDescription);
+}
+  })
+  // Create a subclass of DoFnInvoker
+  .subclass(DoFnInvoker.class, Default.NO_CONSTRUCTORS)
   .defineField(FN_DELEGATE_FIELD_NAME, clazz, Visibility.PRIVATE, 
FieldManifestation.FINAL)
   // Define a constructor to populate fields appropriately.
   .defineConstructor(Visibility.PUBLIC)

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/681ad896/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/DoFnReflectorTest.java
--
diff --git 
a/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/DoFnReflectorTest.java
 
b/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/DoFnReflectorTest.java
index cf9f8e8..3238f2c 100644
--- 
a/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/DoFnReflectorTest.java
+++ 

[GitHub] incubator-beam pull request #738: Create DoFnInvoker instances in the packag...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/738


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[2/2] incubator-beam git commit: This closes #738

[kenn]

This closes #738


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2cbac53c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2cbac53c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/2cbac53c

Branch: refs/heads/master
Commit: 2cbac53c5532b29218aadc351f6b9c87b89aa1e0
Parents: 0866e49 681ad89
Author: Kenneth Knowles 
Authored: Wed Jul 27 14:00:31 2016 -0700
Committer: Kenneth Knowles 
Committed: Wed Jul 27 14:00:31 2016 -0700

--
 .../beam/sdk/transforms/DoFnReflector.java  |  20 +++-
 .../beam/sdk/transforms/DoFnReflectorTest.java  |  75 +++-
 .../dofnreflector/DoFnReflectorTestHelper.java  | 116 +++
 3 files changed, 203 insertions(+), 8 deletions(-)
--

[jira] [Commented] (BEAM-13) Create JMS IO

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396344#comment-15396344
 ] 

ASF GitHub Bot commented on BEAM-13:


Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/299


> Create JMS IO
> -
>
> Key: BEAM-13
> URL: https://issues.apache.org/jira/browse/BEAM-13
> Project: Beam
>  Issue Type: New Feature
>  Components: sdk-java-extensions
>Reporter: Jean-Baptiste Onofré
>Assignee: Jean-Baptiste Onofré
>
> Work in progress: https://github.com/jbonofre/DataflowJavaSDK/tree/IO-JMS



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #299: [BEAM-13] Add JmsIO

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/299


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[1/2] incubator-beam git commit: Closes #299

[dhalperi]

Repository: incubator-beam
Updated Branches:
  refs/heads/master 76928d3bc -> 0866e4906


Closes #299


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/0866e490
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/0866e490
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/0866e490

Branch: refs/heads/master
Commit: 0866e49060ed8f7c9693275ac60ffb2e27f95742
Parents: 76928d3 2cf7556
Author: Dan Halperin 
Authored: Wed Jul 27 13:49:28 2016 -0700
Committer: Dan Halperin 
Committed: Wed Jul 27 13:49:28 2016 -0700

--
 sdks/java/io/jms/pom.xml| 136 +
 .../beam/sdk/io/jms/JmsCheckpointMark.java  |  82 +++
 .../java/org/apache/beam/sdk/io/jms/JmsIO.java  | 518 +++
 .../org/apache/beam/sdk/io/jms/JmsRecord.java   | 153 ++
 .../apache/beam/sdk/io/jms/package-info.java|  22 +
 .../org/apache/beam/sdk/io/jms/JmsIOTest.java   | 145 ++
 sdks/java/io/pom.xml|   1 +
 7 files changed, 1057 insertions(+)
--

[2/2] incubator-beam git commit: [BEAM-13] Add JmsIO

[dhalperi]

[BEAM-13] Add JmsIO


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2cf75568
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2cf75568
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/2cf75568

Branch: refs/heads/master
Commit: 2cf755686f2518ed9575ec5c087884be0e6ea678
Parents: 76928d3
Author: Jean-Baptiste Onofré 
Authored: Thu May 5 19:14:37 2016 +0200
Committer: Dan Halperin 
Committed: Wed Jul 27 13:49:28 2016 -0700

--
 sdks/java/io/jms/pom.xml| 136 +
 .../beam/sdk/io/jms/JmsCheckpointMark.java  |  82 +++
 .../java/org/apache/beam/sdk/io/jms/JmsIO.java  | 518 +++
 .../org/apache/beam/sdk/io/jms/JmsRecord.java   | 153 ++
 .../apache/beam/sdk/io/jms/package-info.java|  22 +
 .../org/apache/beam/sdk/io/jms/JmsIOTest.java   | 145 ++
 sdks/java/io/pom.xml|   1 +
 7 files changed, 1057 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/2cf75568/sdks/java/io/jms/pom.xml
--
diff --git a/sdks/java/io/jms/pom.xml b/sdks/java/io/jms/pom.xml
new file mode 100644
index 000..e0e0f36
--- /dev/null
+++ b/sdks/java/io/jms/pom.xml
@@ -0,0 +1,136 @@
+
+
+http://maven.apache.org/POM/4.0.0;
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+
+  4.0.0
+
+  
+org.apache.beam
+beam-sdks-java-io-parent
+0.2.0-incubating-SNAPSHOT
+../pom.xml
+  
+
+  jms
+  Apache Beam :: SDKs :: Java :: IO :: JMS
+  IO to read and write to JMS (Java Messaging Service)
+destinations (queues and topics). 
+
+  
+5.13.1
+1.1.1
+  
+
+  
+
+  
+org.apache.maven.plugins
+maven-compiler-plugin
+  
+  
+org.apache.maven.plugins
+maven-jar-plugin
+  
+  
+org.apache.maven.plugins
+maven-source-plugin
+  
+  
+org.apache.maven.plugins
+maven-checkstyle-plugin
+  
+
+  
+
+  
+
+  org.apache.beam
+  beam-sdks-java-core
+
+
+
+  org.slf4j
+  slf4j-api
+
+
+
+  joda-time
+  joda-time
+
+
+
+  com.google.guava
+  guava
+
+
+
+  org.apache.geronimo.specs
+  geronimo-jms_1.1_spec
+  ${geronimo-jms.version}
+
+
+
+  com.google.code.findbugs
+  annotations
+
+
+
+
+  org.apache.activemq
+  activemq-broker
+  ${activemq.version}
+  test
+
+
+  org.apache.activemq
+  activemq-kahadb-store
+  ${activemq.version}
+  test
+
+
+  org.apache.activemq
+  activemq-client
+  ${activemq.version}
+  test
+
+
+  org.apache.beam
+  beam-runners-direct-java
+  ${project.version}
+  test
+
+
+  junit
+  junit
+  test
+
+
+  org.hamcrest
+  hamcrest-all
+  test
+
+
+  org.slf4j
+  slf4j-jdk14
+  test
+
+  
+
+
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/2cf75568/sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java
--
diff --git 
a/sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java
 
b/sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java
new file mode 100644
index 000..81c2b82
--- /dev/null
+++ 
b/sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.jms;
+
+import org.apache.beam.sdk.coders.AvroCoder;
+import org.apache.beam.sdk.coders.DefaultCoder;
+import org.apache.beam.sdk.io.UnboundedSource;
+import 

[GitHub] incubator-beam pull request #743: [BEAM-492] Spark runner should call Pipeli...

[amitsela]

GitHub user amitsela opened a pull request:

https://github.com/apache/incubator-beam/pull/743

[BEAM-492] Spark runner should call Pipeline.run() instead of 
SparkRunner.run()

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/amitsela/incubator-beam BEAM-492

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/743.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #743


commit 6db286e91bb5a72d46f9ac71aad5f20799faa885
Author: Sela 
Date:   2016-07-27T20:11:37Z

Remove SparkStreamingPipelineOptions.

commit dcc107323ffdb6121b7e2ca11af4409cac4bb7a8
Author: Sela 
Date:   2016-07-27T20:11:56Z

Run pipeline with Pipeline.run().




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (BEAM-492) Spark runner should call Pipeline.run() instead of SparkRunner.run()

[Amit Sela (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396252#comment-15396252
 ] 

Amit Sela commented on BEAM-492:


I suggest to remove the SparkStreamingPipelineOptions while we're at it. We can 
move the timeout property to the SparkPipelineOptions, and ignore it in batch.

> Spark runner should call Pipeline.run() instead of SparkRunner.run()
> 
>
> Key: BEAM-492
> URL: https://issues.apache.org/jira/browse/BEAM-492
> Project: Beam
>  Issue Type: Bug
>Affects Versions: 0.1.0-incubating
>Reporter: Amit Sela
>Assignee: Amit Sela
>
> Currently, in some places (streaming examples, etc.), the Pipeline executes 
> by calling SparkRunner.run() instead of calling Pipeline.run() which is kind 
> of "backwards" to the Beam API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (BEAM-486) Cleanup NOTICE file

[JIRA]

 [ 
https://issues.apache.org/jira/browse/BEAM-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Baptiste Onofré resolved BEAM-486.
---
Resolution: Fixed

> Cleanup NOTICE file
> ---
>
> Key: BEAM-486
> URL: https://issues.apache.org/jira/browse/BEAM-486
> Project: Beam
>  Issue Type: Task
>  Components: project-management
>Reporter: Daniel Halperin
>Assignee: Jean-Baptiste Onofré
> Fix For: 0.2.0-incubating
>
>
> http://mail-archives.apache.org/mod_mbox/incubator-general/201606.mbox/%3ca5f50a0f-f1e1-4391-8188-391187b9e...@classsoftware.com%3E
> - NOTICE file contain unneeded text (i.e mentions  Apache v2.0 licence). 
> There no need to
> generally mention Apache 2.0 licences in NOTICE [2]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-486) Cleanup NOTICE file

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396229#comment-15396229
 ] 

ASF GitHub Bot commented on BEAM-486:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/727


> Cleanup NOTICE file
> ---
>
> Key: BEAM-486
> URL: https://issues.apache.org/jira/browse/BEAM-486
> Project: Beam
>  Issue Type: Task
>  Components: project-management
>Reporter: Daniel Halperin
>Assignee: Jean-Baptiste Onofré
> Fix For: 0.2.0-incubating
>
>
> http://mail-archives.apache.org/mod_mbox/incubator-general/201606.mbox/%3ca5f50a0f-f1e1-4391-8188-391187b9e...@classsoftware.com%3E
> - NOTICE file contain unneeded text (i.e mentions  Apache v2.0 licence). 
> There no need to
> generally mention Apache 2.0 licences in NOTICE [2]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-486) Cleanup NOTICE file

[JIRA]

 [ 
https://issues.apache.org/jira/browse/BEAM-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Baptiste Onofré updated BEAM-486:
--
Fix Version/s: 0.2.0-incubating

> Cleanup NOTICE file
> ---
>
> Key: BEAM-486
> URL: https://issues.apache.org/jira/browse/BEAM-486
> Project: Beam
>  Issue Type: Task
>  Components: project-management
>Reporter: Daniel Halperin
>Assignee: Jean-Baptiste Onofré
> Fix For: 0.2.0-incubating
>
>
> http://mail-archives.apache.org/mod_mbox/incubator-general/201606.mbox/%3ca5f50a0f-f1e1-4391-8188-391187b9e...@classsoftware.com%3E
> - NOTICE file contain unneeded text (i.e mentions  Apache v2.0 licence). 
> There no need to
> generally mention Apache 2.0 licences in NOTICE [2]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] incubator-beam git commit: [BEAM-486] Remove unnecessary mention of Apache v2.0 LICENSE

[jbonofre]

Repository: incubator-beam
Updated Branches:
  refs/heads/master 65045f98a -> 76928d3bc


[BEAM-486] Remove unnecessary mention of Apache v2.0 LICENSE


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/15b7f81f
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/15b7f81f
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/15b7f81f

Branch: refs/heads/master
Commit: 15b7f81f453966bf7c2066862bd639c97b3b0e6d
Parents: 65045f9
Author: Dan Halperin 
Authored: Mon Jul 25 16:42:23 2016 -0700
Committer: Jean-Baptiste Onofré 
Committed: Wed Jul 27 21:45:36 2016 +0200

--
 NOTICE | 1 -
 1 file changed, 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/15b7f81f/NOTICE
--
diff --git a/NOTICE b/NOTICE
index ef729ba..bd3400c 100644
--- a/NOTICE
+++ b/NOTICE
@@ -9,4 +9,3 @@ Google (http://www.google.com/).
 
 This product includes software developed at
 Google (http://www.google.com/).
-Licensed under the Apache v2.0 License.

[GitHub] incubator-beam pull request #727: [BEAM-486] Remove mention of Apache v2.0 L...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/727


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[2/2] incubator-beam git commit: [BEAM-486] This closes #727

[jbonofre]

[BEAM-486] This closes #727


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/76928d3b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/76928d3b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/76928d3b

Branch: refs/heads/master
Commit: 76928d3bc43d30c8fdea0847aec828a535e93d9f
Parents: 65045f9 15b7f81
Author: Jean-Baptiste Onofré 
Authored: Wed Jul 27 21:46:17 2016 +0200
Committer: Jean-Baptiste Onofré 
Committed: Wed Jul 27 21:46:17 2016 +0200

--
 NOTICE | 1 -
 1 file changed, 1 deletion(-)
--

[jira] [Assigned] (BEAM-486) Cleanup NOTICE file

[JIRA]

 [ 
https://issues.apache.org/jira/browse/BEAM-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Baptiste Onofré reassigned BEAM-486:
-

Assignee: Jean-Baptiste Onofré  (was: Daniel Halperin)

> Cleanup NOTICE file
> ---
>
> Key: BEAM-486
> URL: https://issues.apache.org/jira/browse/BEAM-486
> Project: Beam
>  Issue Type: Task
>  Components: project-management
>Reporter: Daniel Halperin
>Assignee: Jean-Baptiste Onofré
>
> http://mail-archives.apache.org/mod_mbox/incubator-general/201606.mbox/%3ca5f50a0f-f1e1-4391-8188-391187b9e...@classsoftware.com%3E
> - NOTICE file contain unneeded text (i.e mentions  Apache v2.0 licence). 
> There no need to
> generally mention Apache 2.0 licences in NOTICE [2]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (BEAM-488) Remove KEYS file

[JIRA]

 [ 
https://issues.apache.org/jira/browse/BEAM-488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Baptiste Onofré resolved BEAM-488.
---
Resolution: Fixed

> Remove KEYS file
> 
>
> Key: BEAM-488
> URL: https://issues.apache.org/jira/browse/BEAM-488
> Project: Beam
>  Issue Type: Task
>  Components: project-management
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>Assignee: Jean-Baptiste Onofré
> Fix For: 0.2.0-incubating
>
>
> http://mail-archives.apache.org/mod_mbox/incubator-general/201606.mbox/%3CCAAS6=7hVLcw6060Un7sXxk+WLLh08DFOSWktC0Aam4F=dye...@mail.gmail.com%3E
> > Bundling PGP keys inside a package is worse than worthless -- an attacker 
> > can
> just bundle spoofed keys with a bogus distro!  Keys need to be made available
> from a highly reliable, separate server: Download the main package from a
> mirror, get PGP keys from apache.org, pgp.mit.edu, etc. and verify.
> > 
> > The KEYS file within the Beam source tree should be deleted.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-488) Remove KEYS file

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396212#comment-15396212
 ] 

ASF GitHub Bot commented on BEAM-488:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/732


> Remove KEYS file
> 
>
> Key: BEAM-488
> URL: https://issues.apache.org/jira/browse/BEAM-488
> Project: Beam
>  Issue Type: Task
>  Components: project-management
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>Assignee: Jean-Baptiste Onofré
> Fix For: 0.2.0-incubating
>
>
> http://mail-archives.apache.org/mod_mbox/incubator-general/201606.mbox/%3CCAAS6=7hVLcw6060Un7sXxk+WLLh08DFOSWktC0Aam4F=dye...@mail.gmail.com%3E
> > Bundling PGP keys inside a package is worse than worthless -- an attacker 
> > can
> just bundle spoofed keys with a bogus distro!  Keys need to be made available
> from a highly reliable, separate server: Download the main package from a
> mirror, get PGP keys from apache.org, pgp.mit.edu, etc. and verify.
> > 
> > The KEYS file within the Beam source tree should be deleted.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #732: [BEAM-488] Remove KEYS file

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/732


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[1/2] incubator-beam git commit: [BEAM-488] Remove KEYS file

[jbonofre]

Repository: incubator-beam
Updated Branches:
  refs/heads/master b6c29e6b3 -> 65045f98a


[BEAM-488] Remove KEYS file

Per discussion, linked in JIRA:

> Bundling PGP keys inside a package is worse than worthless – an
> attacker can just bundle spoofed keys with a bogus distro! Keys need
> to be made available from a highly reliable, separate server: Download
> the main package from a mirror, get PGP keys from apache.org,
> pgp.mit.edu, etc. and verify.
>
> The KEYS file within the Beam source tree should be deleted.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9e567341
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9e567341
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/9e567341

Branch: refs/heads/master
Commit: 9e5673419c29b88a4ba5b1f22e62d048280673b6
Parents: b6c29e6
Author: Dan Halperin 
Authored: Mon Jul 25 23:24:10 2016 -0700
Committer: Jean-Baptiste Onofré 
Committed: Wed Jul 27 21:28:50 2016 +0200

--
 KEYS | 141 --
 1 file changed, 141 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/9e567341/KEYS
--
diff --git a/KEYS b/KEYS
deleted file mode 100644
index 1f14625..000
--- a/KEYS
+++ /dev/null
@@ -1,141 +0,0 @@
-This file contains the PGP keys of various developers.
-
-Users: pgp < KEYS
-   gpg --import KEYS
-Developers: 
-pgp -kxa  and append it to this file.
-(pgpk -ll  && pgpk -xa ) >> this file.
-(gpg --list-sigs 
- && gpg --armor --export ) >> this file.
- 
-
-pub   4096R/C8282E76 2009-09-08
-uid  Jean-Baptiste Onofré 
-sig 3C8282E76 2009-09-08  Jean-Baptiste Onofré 
-sub   4096R/9F043BBC 2009-09-08
-sig  C8282E76 2009-09-08  Jean-Baptiste Onofré 
-
--BEGIN PGP PUBLIC KEY BLOCK-
-Version: GnuPG v1
-
-mQINBEqmJkEBEADAAMOjOidXzoyK4FK9WhhRg2EEGX1gm5lK8PpJtk68Fqmz6xvv
-N8VJXMIJUgeD7M35zZSQUWJY43xEU8Yfn6oLL0KR0dIqVOclxE+7G8vxXFcIbRE9
-ziZFp7Z5yzsdzjiIzXv5MVQMczcAAMev/i0BnjiRy5Cg+k6kHXVpu/Gsn05JKPaG
-s7ZcfSxpboyS99MVKQvoFLE5Z/Shh4gFJn2rFInqK5EgVpoZbVyysF52nx0dti/e
-O0NjraQkrEDBWvsPt3cYZA0oP1gWiZiRvOLfAFIarf3poMDyoWBIwnbqb3Msv09j
-yDAmcGq9wsD3alHFHcRIiJl5SzFUStml1d5x/BvUl/Xc5VfHPi2ObKF3xOPGkyTf
-aZ6mYFLaRCAJ0v2MPW+4/grDXKsP8n8xPbE2VQvHBpxaZklD7q4Omn2d+m2sUOLX
-NRUo4n29NyfowAffBYl7ZqrYBBodR9YngWC9LpgM+APHyiw3HzauZ94bGy5Of3+L
-Yu6/riDcP4OXF6r6IH6KIsVqIkv5xzq7OGxxXmlhWg8ifNPLq5yNRccS0nWXc5BD
-/9q06ta/ceQGNkXL327XPuZC+lstWGAa4dKEosRDgcO0Pv2j2a3h8W8oHyxF+gEe
-O+9s0mGdQFxNiEA+JyeKCg+jvfx9Hv/2Syrlert76NEkfbaTFA7BJ4c3EQARAQAB
-tCtKZWFuLUJhcHRpc3RlIE9ub2Zyw6kgPGpib25vZnJlQGFwYWNoZS5vcmc+iQI2
-BBMBAgAgBQJKpiZBAhsDBgsJCAcDAgQVAggDBBYCAwECHgECF4AACgkQv/LuQsgo
-LnboyRAAguqFIpiKkCCR6TR0Y5UQDFhgEMhBreQKCEW0czbGoFnxfULV9H1kJRSB
-Vt0knecGaYS340WEmz4B7BMpkBCgaszgn66+fhacZTBd+Aff1k2lbhdMgdBvlPcm
-q9vFGtbE515j9bPHzsPRJ2wFWd6ot9wXiLD3RJLV6c7L3Egstu3qTp0tEoFHrQps
-qskGBl+mahhMyz3BUDlusavB0Y0tb6hhXCR79ErhjQrTgU947isztYWpgJlA40lx
-DW0hskZWbuGNXjxUJvTT3pKiYUN32WG+2CDNYHceuhsfRLxO/Wb4BKwwDaHWAlH9
-d5F9/vhdPObSv5GQbuUtmCEzeqADUd65jLLM7WSlvRJ+i4m0/TTeP8y4NfxlVbBP
-WuYrQW4gPmDKEDNvEec6PH6hhBfMLJz3M6o4huwLp2kQrq6wSTMDGIoxOLP0ae3c
-BMIuFM5EavLDJmuATUIWWyZt/c7mmAOOh5TGcFWTugnJ6l4FllOrFPiWyFsjMn+U
-zzzaeSkYmq/xZYxjRTdWjK5Zb5rbVuCx/q5VF9Awdy4EM6UXhaqWo06VyjWNOJ86
-wgres4+bVldB7+TiVi9iO6n80WNlPgIaQJlLc+FRsld4Er21kdXreX5doxFD5Iue
-S4y/pLwftHfx1xxj+p2jPJ49Hb0ddNr+XrsrO5txing2pNJgfH65Ag0ESqYmQQEQ
-AKPoXgIIKnyJiPvks7xBV+FqJPecVAx3SSlLyTfsh/jBat9QLd4hsfiZcv1ANZHB
-n4qDeGlsmJ6uDGv8wnUZQ2Im8Heje1h7dKeLNpNnxfBS9gn6e2bXKhAsJGUE7gip
-qVfijFnEY0Vj6Tztzq+Wyqg2Gbz+bJZMo1JVQiaAYyQeQlrOcoZcQHsA/Ol+y48h
-Le36A1TSIPMOSI4ZAZXkqxXAumEaMaz82EvV8KDH7Ijr23Y0wZjEUJ+dJQM9ssuE
-f9GMLIuCbmM/CJ5MCCwepGJd52ymllvgJTHC7B+BY/jKNMWHwAsMJ1oWcPlLzFQI
-Bmyy5RjKoMifzaoSo/hTWkiwcL2Vc+qU3b3/2eUtnCnBB/nkrZkJNNc+OV5YGBSP
-vNPaN43Gvjbvborv4PBvt7QhVjZYQemtXO2sWx1XWSFsucD2K4kJ8ipNWxVgIqDu
-J8SJOnGigX9hMpsZ2HVAwOeKP/jI90J3voKrCPLaKcL1Ip+b28k0aj7kl44YJqw4
-5pbRSx/v73bH4uleQiXSW+JczA+KLw7hX3tOWJEnLS2+Ig9sNUKYGZOg0nw613bN
-fZy8Cbx/UkT10Lznx9FW6MedGyJPYT4MJMMh/PnnsWv50jFnfu2rtnRXEOUXwujL
-fwrmCYbXHgE3Ka+fmRz8HxsyTmtqIHtPixw8RoqfoFfxABEBAAGJAh8EGAECAAkF
-AkqmJkECGwwACgkQv/LuQsgoLnb8AQ/+POsLFdqNqSKfwBXp1YOIEjNdbVjysQc6
-zC6LlMJXNSxAmUmol2g9bJYh9LdpvOTU3gfFgIanaGytC75U7/NOl0zEsN4IU18j
-CLBNaD5/Or1ciQ3CVrID/lPO8s0Hm0/cUPreEjJPPrrPbXG+i9bweg3Dtfy3+WQl
-PhfpvgudwtUjB3st2gztYipkUhmrH+STbbJZVJN5ZNL8mOoM5M2wGS+9VweOWbKe
-z0QeZ9hIPyQNMzTn1xlvRUVNTu8fz2FGvumrd+zgzYcpTE5VpFkOxxUayr3aWXSf
-Cak+HH0WjUDWc9/lJR4dVpwdjLonJfiC70W07J4CnNodYwnPUaGKTVYq3pvQzAPw

[jira] [Commented] (BEAM-433) Make Beam examples runners agnostic

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396211#comment-15396211
 ] 

ASF GitHub Bot commented on BEAM-433:
-

GitHub user peihe opened a pull request:

https://github.com/apache/incubator-beam/pull/742

[BEAM-433] Remove references to DataflowPipelineOptions




You can merge this pull request into a Git repository by running:

$ git pull https://github.com/peihe/incubator-beam rm-dataflow-options

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/742.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #742


commit 9aad5c6ec034a10ded8c07e251f69213a0b0bdc8
Author: Pei He 
Date:   2016-07-27T19:29:44Z

[BEAM-433] Remove references to DataflowPipelineOptions




> Make Beam examples runners agnostic
> ---
>
> Key: BEAM-433
> URL: https://issues.apache.org/jira/browse/BEAM-433
> Project: Beam
>  Issue Type: Improvement
>  Components: examples-java
>Reporter: Pei He
>Assignee: Pei He
>
> Beam examples are ported from Dataflow, and they heavily reference to 
> Dataflow classes.
> There are following cleanup tasks:
> 1. Remove Dataflow streaming and batch injector setup (Done).
> 2. Remove references to DataflowPipelineOptions.
> 3. Move cancel() from DataflowPipelineJob to PipelineResult.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] incubator-beam git commit: [BEAM-488] This closes #732

[jbonofre]

[BEAM-488] This closes #732


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/65045f98
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/65045f98
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/65045f98

Branch: refs/heads/master
Commit: 65045f98a09661e485389dddc5fda3f51825bf31
Parents: b6c29e6 9e56734
Author: Jean-Baptiste Onofré 
Authored: Wed Jul 27 21:30:16 2016 +0200
Committer: Jean-Baptiste Onofré 
Committed: Wed Jul 27 21:30:16 2016 +0200

--
 KEYS | 141 --
 1 file changed, 141 deletions(-)
--

[GitHub] incubator-beam pull request #742: [BEAM-433] Remove references to DataflowPi...

[peihe]

GitHub user peihe opened a pull request:

https://github.com/apache/incubator-beam/pull/742

[BEAM-433] Remove references to DataflowPipelineOptions




You can merge this pull request into a Git repository by running:

$ git pull https://github.com/peihe/incubator-beam rm-dataflow-options

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/742.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #742


commit 9aad5c6ec034a10ded8c07e251f69213a0b0bdc8
Author: Pei He 
Date:   2016-07-27T19:29:44Z

[BEAM-433] Remove references to DataflowPipelineOptions




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #741: Top improvements

[robertwb]

GitHub user robertwb opened a pull request:

https://github.com/apache/incubator-beam/pull/741

Top improvements

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/robertwb/incubator-beam top

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/741.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #741


commit 9151fcf861704c550c21f411a9047f955f179ec4
Author: Robert Bradshaw 
Date:   2016-07-27T17:09:49Z

Better top implementation.

When selecting the top k of n, it is common that k << n.
Using a heap is O(n log k) while select algorithms can
achieve O(n + k log k).

This also avoids the ugliness that heapq does not take the
comparator as an argument, resulting in _HeapItem classes that
were cumbersome and expensive to serialize.

commit e38d6f60f18bf6ad61f2d387c67efa2012af2e14
Author: Robert Bradshaw 
Date:   2016-07-27T19:23:02Z

Allow Top operations to take key argument rather than compare, and order
by the natural ordering if neither is specified.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (BEAM-433) Make Beam examples runners agnostic

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396202#comment-15396202
 ] 

ASF GitHub Bot commented on BEAM-433:
-

Github user peihe closed the pull request at:

https://github.com/apache/incubator-beam/pull/301


> Make Beam examples runners agnostic
> ---
>
> Key: BEAM-433
> URL: https://issues.apache.org/jira/browse/BEAM-433
> Project: Beam
>  Issue Type: Improvement
>  Components: examples-java
>Reporter: Pei He
>Assignee: Pei He
>
> Beam examples are ported from Dataflow, and they heavily reference to 
> Dataflow classes.
> There are following cleanup tasks:
> 1. Remove Dataflow streaming and batch injector setup (Done).
> 2. Remove references to DataflowPipelineOptions.
> 3. Move cancel() from DataflowPipelineJob to PipelineResult.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #301: [BEAM-433] Remove Dataflow references in E...

[peihe]

Github user peihe closed the pull request at:

https://github.com/apache/incubator-beam/pull/301


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (BEAM-354) Modify DatastoreIO to use Datastore v1beta3 API

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396163#comment-15396163
 ] 

ASF GitHub Bot commented on BEAM-354:
-

Github user vikkyrk closed the pull request at:

https://github.com/apache/incubator-beam/pull/725


> Modify DatastoreIO to use Datastore v1beta3 API
> ---
>
> Key: BEAM-354
> URL: https://issues.apache.org/jira/browse/BEAM-354
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-gcp
>Affects Versions: 0.2.0-incubating
>Reporter: Vikas Kedigehalli
>Assignee: Vikas Kedigehalli
> Fix For: 0.2.0-incubating
>
>
> Datastore v1beta2 API is getting deprecated in favor of v1beta3. Hence the 
> DatastoreIO needs to be migrated to use the new version. Also in the process 
> of doing so, this is a good time to add a level of indirection via a 
> PTranform such that future changes in Datastore API would not result in 
> changing user/pipeline code. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #725: [BEAM-354] Move V1Beta3 DatastoreSource in...

[vikkyrk]

Github user vikkyrk closed the pull request at:

https://github.com/apache/incubator-beam/pull/725


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #740: Remove DataflowJUnitTestRunner as integrat...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/740


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[2/2] incubator-beam git commit: Remove DataflowJUnitTestRunner as integration tests execute using surefire/failsafe

[lcwik]

Remove DataflowJUnitTestRunner as integration tests execute using 
surefire/failsafe

This closes #740


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b6c29e6b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/b6c29e6b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/b6c29e6b

Branch: refs/heads/master
Commit: b6c29e6b3047e3ec7698185c69993144a0fc8c88
Parents: ddd57d8 07ee019
Author: Luke Cwik 
Authored: Wed Jul 27 14:38:44 2016 -0400
Committer: Luke Cwik 
Committed: Wed Jul 27 14:38:44 2016 -0400

--
 .../sdk/testing/DataflowJUnitTestRunner.java| 130 ---
 1 file changed, 130 deletions(-)
--

[GitHub] incubator-beam pull request #728: CoderRegistry: make deprecated method priv...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/728


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[2/2] incubator-beam git commit: CoderRegistry: make deprecated method private

[dhalperi]

CoderRegistry: make deprecated method private

This was part of the public API signature in Dataflow, but need not be in Beam.

Also remove a few unneeded publics in the same file.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d1238eb4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/d1238eb4
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/d1238eb4

Branch: refs/heads/master
Commit: d1238eb4eb3aae571121221e2a0b93268f8ac45f
Parents: 4c0e11e
Author: Dan Halperin 
Authored: Mon Jul 25 17:12:01 2016 -0700
Committer: Dan Halperin 
Committed: Wed Jul 27 11:08:12 2016 -0700

--
 .../apache/beam/sdk/coders/CoderRegistry.java   | 29 ++--
 1 file changed, 15 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/d1238eb4/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/CoderRegistry.java
--
diff --git 
a/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/CoderRegistry.java 
b/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/CoderRegistry.java
index 50e4422..d9b988b 100644
--- a/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/CoderRegistry.java
+++ b/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/CoderRegistry.java
@@ -1,3 +1,4 @@
+
 /*
  * Licensed to the Apache Software Foundation (ASF) under one
  * or more contributor license agreements.  See the NOTICE file
@@ -73,11 +74,13 @@ import javax.annotation.Nullable;
  *   the default {@code Coder} type. The {@link Coder} class must satisfy 
the requirements
  *   of {@link CoderProviders#fromStaticMethods}.
  *   Fallback: A fallback {@link CoderProvider} is used to attempt to 
provide a {@link Coder}
- *   for any type. By default, this is {@link SerializableCoder#PROVIDER}, 
which can provide
- *   a {@link Coder} for any type that is serializable via Java 
serialization. The fallback
- *   {@link CoderProvider} can be get and set via {@link 
#getFallbackCoderProvider()}
- *   and {@link #setFallbackCoderProvider}. Multiple fallbacks can be 
chained together using
- *   {@link CoderProviders#firstOf}.
+ *   for any type. By default, there are two chained fallback coders:
+ *   {@link ProtoCoder#coderProvider}, which can provide a coder to 
efficiently serialize any
+ *   Protocol Buffers message, and then {@link 
SerializableCoder#PROVIDER}, which can provide a
+ *   {@link Coder} for any type that is serializable via Java 
serialization. The fallback
+ *   {@link CoderProvider} can be get and set respectively using
+ *   {@link #getFallbackCoderProvider()} and {@link 
#setFallbackCoderProvider}. Multiple
+ *   fallbacks can be chained together using {@link 
CoderProviders#firstOf}.
  * 
  */
 public class CoderRegistry implements CoderProvider {
@@ -366,7 +369,8 @@ public class CoderRegistry implements CoderProvider {
* providing a {@code Coder} for a type {@code T}, then the registry will 
attempt to create
* a {@link Coder} using this {@link CoderProvider}.
*
-   * By default, this is set to {@link SerializableCoder#PROVIDER}.
+   * By default, this is set to the chain of {@link 
ProtoCoder#coderProvider()} and
+   * {@link SerializableCoder#PROVIDER}.
*
* See {@link #getFallbackCoderProvider}.
*/
@@ -383,6 +387,8 @@ public class CoderRegistry implements CoderProvider {
 return fallbackCoderProvider;
   }
 
+  /
+
   /**
* Returns a {@code Map} from each of {@code baseClass}'s type parameters to 
the {@link Coder} to
* use by default for it, in the context of {@code subClass}'s 
specialization of
@@ -413,11 +419,8 @@ public class CoderRegistry implements CoderProvider {
* @param baseClass the base type, a parameterized class
* @param knownCoders a map corresponding to the set of known {@link Coder 
Coders} indexed by
* parameter name
-   *
-   * @deprecated this method is not part of the public interface and will be 
made private
*/
-  @Deprecated
-  public  Map getDefaultCoders(
+  private  Map getDefaultCoders(
   Class subClass,
   Class baseClass,
   Map knownCoders) {
@@ -525,8 +528,6 @@ public class CoderRegistry implements CoderProvider {
   }
 
 
-  /
-
   /**
* Thrown when a {@link Coder} cannot possibly encode a type, yet has been 
proposed as a
* {@link Coder} for that type.
@@ -535,13 +536,13 @@ public class CoderRegistry implements CoderProvider {
 private Coder 

[1/2] incubator-beam git commit: Closes #728

[dhalperi]

Repository: incubator-beam
Updated Branches:
  refs/heads/master 4c0e11e50 -> ddd57d8b9


Closes #728


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/ddd57d8b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/ddd57d8b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/ddd57d8b

Branch: refs/heads/master
Commit: ddd57d8b94ee13a4fa7ef22968cd266b6f9d4f27
Parents: 4c0e11e d1238eb
Author: Dan Halperin 
Authored: Wed Jul 27 11:08:12 2016 -0700
Committer: Dan Halperin 
Committed: Wed Jul 27 11:08:12 2016 -0700

--
 .../apache/beam/sdk/coders/CoderRegistry.java   | 29 ++--
 1 file changed, 15 insertions(+), 14 deletions(-)
--

[GitHub] incubator-beam pull request #740: Remove DataflowJUnitTestRunner as integrat...

[lukecwik]

GitHub user lukecwik opened a pull request:

https://github.com/apache/incubator-beam/pull/740

Remove DataflowJUnitTestRunner as integration tests execute using 
surefire/failsafe

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lukecwik/incubator-beam remove_junit_runner

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/740.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #740


commit d9b6903dbadd05e27fb3a2cb2a539374ef7fa487
Author: Luke Cwik 
Date:   2016-07-27T17:14:41Z

Remove DataflowJUnitTestRunner as integration tests execute using 
surefire/failsafe.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (BEAM-488) Remove KEYS file

[JIRA]

 [ 
https://issues.apache.org/jira/browse/BEAM-488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Baptiste Onofré updated BEAM-488:
--
Fix Version/s: 0.2.0-incubating

> Remove KEYS file
> 
>
> Key: BEAM-488
> URL: https://issues.apache.org/jira/browse/BEAM-488
> Project: Beam
>  Issue Type: Task
>  Components: project-management
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>Assignee: Jean-Baptiste Onofré
> Fix For: 0.2.0-incubating
>
>
> http://mail-archives.apache.org/mod_mbox/incubator-general/201606.mbox/%3CCAAS6=7hVLcw6060Un7sXxk+WLLh08DFOSWktC0Aam4F=dye...@mail.gmail.com%3E
> > Bundling PGP keys inside a package is worse than worthless -- an attacker 
> > can
> just bundle spoofed keys with a bogus distro!  Keys need to be made available
> from a highly reliable, separate server: Download the main package from a
> mirror, get PGP keys from apache.org, pgp.mit.edu, etc. and verify.
> > 
> > The KEYS file within the Beam source tree should be deleted.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (BEAM-488) Remove KEYS file

[JIRA]

 [ 
https://issues.apache.org/jira/browse/BEAM-488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Baptiste Onofré reassigned BEAM-488:
-

Assignee: Jean-Baptiste Onofré  (was: Daniel Halperin)

> Remove KEYS file
> 
>
> Key: BEAM-488
> URL: https://issues.apache.org/jira/browse/BEAM-488
> Project: Beam
>  Issue Type: Task
>  Components: project-management
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>Assignee: Jean-Baptiste Onofré
>
> http://mail-archives.apache.org/mod_mbox/incubator-general/201606.mbox/%3CCAAS6=7hVLcw6060Un7sXxk+WLLh08DFOSWktC0Aam4F=dye...@mail.gmail.com%3E
> > Bundling PGP keys inside a package is worse than worthless -- an attacker 
> > can
> just bundle spoofed keys with a bogus distro!  Keys need to be made available
> from a highly reliable, separate server: Download the main package from a
> mirror, get PGP keys from apache.org, pgp.mit.edu, etc. and verify.
> > 
> > The KEYS file within the Beam source tree should be deleted.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (BEAM-487) Add disclaimer to GitHub README.md

[JIRA]

 [ 
https://issues.apache.org/jira/browse/BEAM-487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Baptiste Onofré resolved BEAM-487.
---
Resolution: Fixed

> Add disclaimer to GitHub README.md
> --
>
> Key: BEAM-487
> URL: https://issues.apache.org/jira/browse/BEAM-487
> Project: Beam
>  Issue Type: Task
>  Components: project-management
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>Assignee: Jean-Baptiste Onofré
> Fix For: 0.2.0-incubating
>
>
> http://mail-archives.apache.org/mod_mbox/incubator-general/201606.mbox/%3c47b44d85-be01-42f9-96c6-43ff23e31...@apache.org%3E
> > 1. The DISCLAIMER file is sufficient for the purposes of a source release. 
> > But for the github
> audience, reaching https://github.com/apache/incubator-beam 
> ,
> there is no disclaimer. I think there should be a disclaimer on README.md, 
> and at least the
> first reference to beam should read “Apache Beam (incubating)”.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #733: [BEAM-487] README.md: add DISCLAIMER, incu...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/733


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[1/2] incubator-beam git commit: [BEAM-487] Update README.md: add DISCLAIMER, incubating, minor fixes

[jbonofre]

Repository: incubator-beam
Updated Branches:
  refs/heads/master b1f36df2d -> 4c0e11e50


[BEAM-487] Update README.md: add DISCLAIMER, incubating, minor fixes


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9329d2e5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9329d2e5
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/9329d2e5

Branch: refs/heads/master
Commit: 9329d2e5d3bfed480d3343e5db73c360dfffe38f
Parents: b1f36df
Author: Dan Halperin 
Authored: Mon Jul 25 23:30:44 2016 -0700
Committer: Jean-Baptiste Onofré 
Committed: Wed Jul 27 19:11:13 2016 +0200

--
 README.md | 22 +-
 1 file changed, 17 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/9329d2e5/README.md
--
diff --git a/README.md b/README.md
index 798c12b..e72aec2 100644
--- a/README.md
+++ b/README.md
@@ -17,10 +17,22 @@
 under the License.
 -->
 
-# Apache Beam
+# Apache Beam (incubating)
 
 [Apache Beam](http://beam.incubator.apache.org) is a unified model for 
defining both batch and streaming data-parallel processing pipelines, as well 
as a set of language-specific SDKs for constructing pipelines and Runners for 
executing them on distributed processing backends like [Apache 
Spark](http://spark.apache.org/), [Apache Flink](http://flink.apache.org), and 
[Google Cloud Dataflow](http://cloud.google.com/dataflow).
 
+```
+Apache Beam is an effort undergoing incubation at the Apache Software
+Foundation (ASF), sponsored by the Apache Incubator PMC.
+
+Incubation is required of all newly accepted projects until a further review
+indicates that the infrastructure, communications, and decision making process
+have stabilized in a manner consistent with other successful ASF projects.
+
+While incubation status is not necessarily a reflection of the completeness
+or stability of the code, it does indicate that the project has yet to be
+fully endorsed by the ASF.
+```
 
 ## Status
 
@@ -59,7 +71,7 @@ Beam supports multiple language specific SDKs for writing 
pipelines against the
 
 Currently, this repository contains the Beam Java SDK, which is in the process 
of evolving from the [Dataflow Java 
SDK](https://github.com/GoogleCloudPlatform/DataflowJavaSDK). The [Dataflow 
Python SDK](https://github.com/GoogleCloudPlatform/DataflowPythonSDK) will also 
become part of Beam in the near future.
 
-Have ideas for new SDKs or DSLs? See the 
[Jira](https://issues.apache.org/jira/browse/BEAM/component/12328909/).
+Have ideas for new SDKs or DSLs? See the 
[JIRA](https://issues.apache.org/jira/browse/BEAM/component/12328909/).
 
 
 ### Runners
@@ -67,11 +79,11 @@ Have ideas for new SDKs or DSLs? See the 
[Jira](https://issues.apache.org/jira/b
 Beam supports executing programs on multiple distributed processing backends 
through PipelineRunners. Currently, the following PipelineRunners are available:
 
 - The `DirectRunner` runs the pipeline on your local machine.
-- The `DataflowPipelineRunner` submits the pipeline to the [Google Cloud 
Dataflow](http://cloud.google.com/dataflow/).
+- The `DataflowRunner` submits the pipeline to the [Google Cloud 
Dataflow](http://cloud.google.com/dataflow/).
 - The `FlinkRunner` runs the pipeline on an Apache Flink cluster. The code has 
been donated from 
[dataArtisans/flink-dataflow](https://github.com/dataArtisans/flink-dataflow) 
and is now part of Beam.
 - The `SparkRunner` runs the pipeline on an Apache Spark cluster. The code has 
been donated from 
[cloudera/spark-dataflow](https://github.com/cloudera/spark-dataflow) and is 
now part of Beam.
 
-Have ideas for new Runners? See the 
[Jira](https://issues.apache.org/jira/browse/BEAM/component/12328916/).
+Have ideas for new Runners? See the 
[JIRA](https://issues.apache.org/jira/browse/BEAM/component/12328916/).
 
 
 ## Getting Started
@@ -92,7 +104,7 @@ To get involved in Apache Beam:
 
 * [Subscribe](mailto:user-subscr...@beam.incubator.apache.org) or 
[mail](mailto:u...@beam.incubator.apache.org) the 
[u...@beam.incubator.apache.org](http://mail-archives.apache.org/mod_mbox/incubator-beam-user/)
 list.
 * [Subscribe](mailto:dev-subscr...@beam.incubator.apache.org) or 
[mail](mailto:d...@beam.incubator.apache.org) the 
[d...@beam.incubator.apache.org](http://mail-archives.apache.org/mod_mbox/incubator-beam-dev/)
 list.
-* Report issues on [Jira](https://issues.apache.org/jira/browse/BEAM).
+* Report issues on [JIRA](https://issues.apache.org/jira/browse/BEAM).
 
 
 ## More Information

[jira] [Commented] (BEAM-328) CoderRegistry does not provide SerializableCoder for `T extends Serializable`

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395997#comment-15395997
 ] 

ASF GitHub Bot commented on BEAM-328:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/663


> CoderRegistry does not provide SerializableCoder for `T extends Serializable`
> -
>
> Key: BEAM-328
> URL: https://issues.apache.org/jira/browse/BEAM-328
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Manu Zhang
>Priority: Minor
>  Labels: starter
>
> When the type for which a coder is being inferred is a type variable with an 
> upper bound of {{Serializable}}, it is reasonable for the coder registry to 
> propagate this to the {{SerializableCoder.PROVIDER}}, which should be able to 
> succeed.
> Unfortunately, the particulars of the distinctions made between {{Type}}, 
> {{Class}}, {{TypeVariable}}, {{ParameterizedType}}, etc, go down a code path 
> where this is not the case. Instead, an error is raised that the type 
> variable has been subject to erasure.
> Originally reported at: 
> https://github.com/GoogleCloudPlatform/DataflowJavaSDK/issues/298



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-487) Add disclaimer to GitHub README.md

[JIRA]

 [ 
https://issues.apache.org/jira/browse/BEAM-487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Baptiste Onofré updated BEAM-487:
--
Fix Version/s: 0.2.0-incubating

> Add disclaimer to GitHub README.md
> --
>
> Key: BEAM-487
> URL: https://issues.apache.org/jira/browse/BEAM-487
> Project: Beam
>  Issue Type: Task
>  Components: project-management
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>Assignee: Jean-Baptiste Onofré
> Fix For: 0.2.0-incubating
>
>
> http://mail-archives.apache.org/mod_mbox/incubator-general/201606.mbox/%3c47b44d85-be01-42f9-96c6-43ff23e31...@apache.org%3E
> > 1. The DISCLAIMER file is sufficient for the purposes of a source release. 
> > But for the github
> audience, reaching https://github.com/apache/incubator-beam 
> ,
> there is no disclaimer. I think there should be a disclaimer on README.md, 
> and at least the
> first reference to beam should read “Apache Beam (incubating)”.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (BEAM-487) Add disclaimer to GitHub README.md

[JIRA]

 [ 
https://issues.apache.org/jira/browse/BEAM-487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Baptiste Onofré reassigned BEAM-487:
-

Assignee: Jean-Baptiste Onofré  (was: Daniel Halperin)

> Add disclaimer to GitHub README.md
> --
>
> Key: BEAM-487
> URL: https://issues.apache.org/jira/browse/BEAM-487
> Project: Beam
>  Issue Type: Task
>  Components: project-management
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>Assignee: Jean-Baptiste Onofré
> Fix For: 0.2.0-incubating
>
>
> http://mail-archives.apache.org/mod_mbox/incubator-general/201606.mbox/%3c47b44d85-be01-42f9-96c6-43ff23e31...@apache.org%3E
> > 1. The DISCLAIMER file is sufficient for the purposes of a source release. 
> > But for the github
> audience, reaching https://github.com/apache/incubator-beam 
> ,
> there is no disclaimer. I think there should be a disclaimer on README.md, 
> and at least the
> first reference to beam should read “Apache Beam (incubating)”.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] incubator-beam git commit: [BEAM-328] uses SerializableCoder for `T extends Serializable`

[kenn]

Repository: incubator-beam
Updated Branches:
  refs/heads/master e8695a1b6 -> b1f36df2d


[BEAM-328] uses SerializableCoder for `T extends Serializable`


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7bbac011
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7bbac011
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/7bbac011

Branch: refs/heads/master
Commit: 7bbac011afd61dc535de298675ef42cb9a3abb56
Parents: 68850b4
Author: manuzhang 
Authored: Thu Jun 16 12:09:22 2016 +0800
Committer: manuzhang 
Committed: Wed Jul 27 06:14:32 2016 +0800

--
 .../org/apache/beam/sdk/coders/CoderRegistry.java | 10 ++
 .../apache/beam/sdk/coders/CoderRegistryTest.java | 18 +++---
 .../apache/beam/sdk/values/TypedPValueTest.java   | 16 
 .../sdk/transforms/RemoveDuplicatesJava8Test.java | 10 --
 .../beam/sdk/transforms/WithKeysJava8Test.java| 11 +--
 5 files changed, 50 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7bbac011/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/CoderRegistry.java
--
diff --git 
a/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/CoderRegistry.java 
b/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/CoderRegistry.java
index 9c3fabe..50e4422 100644
--- a/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/CoderRegistry.java
+++ b/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/CoderRegistry.java
@@ -163,9 +163,9 @@ public class CoderRegistry implements CoderProvider {
*/
   public  void registerCoder(Class rawClazz, Coder coder) {
 checkArgument(
-  rawClazz.getTypeParameters().length == 0,
-  "CoderRegistry.registerCoder(Class, Coder) may not be used "
-  + "with unspecialized generic classes");
+rawClazz.getTypeParameters().length == 0,
+"CoderRegistry.registerCoder(Class, Coder) may not be used "
++ "with unspecialized generic classes");
 
 CoderFactory factory = CoderFactories.forCoder(coder);
 registerCoder(rawClazz, factory);
@@ -720,7 +720,9 @@ public class CoderRegistry implements CoderProvider {
   return getDefaultCoder(clazz);
 } else if (type instanceof ParameterizedType) {
   return getDefaultCoder((ParameterizedType) type, typeCoderBindings);
-} else if (type instanceof TypeVariable || type instanceof WildcardType) {
+} else if (type instanceof TypeVariable) {
+  return getDefaultCoder(TypeDescriptor.of(type).getRawType());
+} else if (type instanceof WildcardType) {
   // No default coder for an unknown generic type.
   throw new CannotProvideCoderException(
   String.format("Cannot provide a coder for type variable %s"

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7bbac011/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/CoderRegistryTest.java
--
diff --git 
a/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/CoderRegistryTest.java
 
b/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/CoderRegistryTest.java
index 37f0e10..817ea20 100644
--- 
a/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/CoderRegistryTest.java
+++ 
b/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/CoderRegistryTest.java
@@ -388,15 +388,27 @@ public class CoderRegistryTest {
 
 thrown.expect(CannotProvideCoderException.class);
 thrown.expectMessage(allOf(
-containsString("TestGenericT"),
-containsString("erasure"),
-
containsString("org.apache.beam.sdk.coders.CoderRegistryTest$TestGenericClass")));
+containsString("No CoderFactory has been registered"),
+containsString("does not have a @DefaultCoder annotation"),
+containsString("does not implement Serializable")));
 registry.getDefaultCoder(TypeDescriptor.of(
 TestGenericClass.class.getTypeParameters()[0]));
   }
 
   private static class TestGenericClass { }
 
+  @Test
+  public void testSerializableTypeVariableDefaultCoder() throws Exception {
+CoderRegistry registry = new CoderRegistry();
+
+TypeDescriptor type = TypeDescriptor.of(
+TestSerializableGenericClass.class.getTypeParameters()[0]);
+assertEquals(registry.getDefaultCoder(type),
+SerializableCoder.of(type));
+  }
+
+  private static class TestSerializableGenericClass {}
+
   /**
* In-context test that assures the functionality tested in
* {@link #testDefaultCoderAnnotationGeneric} is invoked in the right ways.

[GitHub] incubator-beam pull request #663: [BEAM-328] uses SerializableCoder for `T e...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/663


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[2/2] incubator-beam git commit: This closes #663

[kenn]

This closes #663


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b1f36df2
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/b1f36df2
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/b1f36df2

Branch: refs/heads/master
Commit: b1f36df2d185de553edc8b22bce237ad87b1b2bd
Parents: e8695a1 7bbac01
Author: Kenneth Knowles 
Authored: Wed Jul 27 10:03:53 2016 -0700
Committer: Kenneth Knowles 
Committed: Wed Jul 27 10:03:53 2016 -0700

--
 .../org/apache/beam/sdk/coders/CoderRegistry.java | 10 ++
 .../apache/beam/sdk/coders/CoderRegistryTest.java | 18 +++---
 .../apache/beam/sdk/values/TypedPValueTest.java   | 16 
 .../sdk/transforms/RemoveDuplicatesJava8Test.java | 10 --
 .../beam/sdk/transforms/WithKeysJava8Test.java| 11 +--
 5 files changed, 50 insertions(+), 15 deletions(-)
--

[jira] [Commented] (BEAM-23) User-facing state and timers for ParDo(DoFn)

[Kenneth Knowles (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-23?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395848#comment-15395848
 ] 

Kenneth Knowles commented on BEAM-23:
-

Design proposal is at https://s.apache.org/beam-state.

> User-facing state and timers for ParDo(DoFn)
> 
>
> Key: BEAM-23
> URL: https://issues.apache.org/jira/browse/BEAM-23
> Project: Beam
>  Issue Type: New Feature
>  Components: beam-model
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>  Labels: State
>
> To a key-partitioned ParDo, add the ability for a user's DoFn to, on a 
> per-key-and-window basis:
>  - read and write consistent state
>  - set a timer to request a callback at a particular moment (in any time 
> domain)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-484) Datastore Source should support Dynamic Splitting

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395774#comment-15395774
 ] 

ASF GitHub Bot commented on BEAM-484:
-

GitHub user vikkyrk opened a pull request:

https://github.com/apache/incubator-beam/pull/739

[BEAM-484] Datastore Read as a composite PTransform

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---
- Datastore Read as a composite PTransform:
Create.of(query) | ParDo.of(split query) | Reshard | ParDo.of(read from 
query)

- Dynamic rebalancing support comes for free as it happens after the 
sharding of queries (GBK)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vikkyrk/incubator-beam 
vikasrk/ds_as_ptransform

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/739.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #739


commit d8fab9fd351a4e1227623c6469137dec0d7f24e6
Author: Vikas Kedigehalli 
Date:   2016-07-26T16:54:43Z

Datastore Read as a composite PTransform




> Datastore Source should support Dynamic Splitting
> -
>
> Key: BEAM-484
> URL: https://issues.apache.org/jira/browse/BEAM-484
> Project: Beam
>  Issue Type: Improvement
>Reporter: Vikas Kedigehalli
>Assignee: Vikas Kedigehalli
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #739: [BEAM-484] Datastore Read as a composite P...

[vikkyrk]

GitHub user vikkyrk opened a pull request:

https://github.com/apache/incubator-beam/pull/739

[BEAM-484] Datastore Read as a composite PTransform

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---
- Datastore Read as a composite PTransform:
Create.of(query) | ParDo.of(split query) | Reshard | ParDo.of(read from 
query)

- Dynamic rebalancing support comes for free as it happens after the 
sharding of queries (GBK)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vikkyrk/incubator-beam 
vikasrk/ds_as_ptransform

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/739.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #739


commit d8fab9fd351a4e1227623c6469137dec0d7f24e6
Author: Vikas Kedigehalli 
Date:   2016-07-26T16:54:43Z

Datastore Read as a composite PTransform




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (BEAM-492) Spark runner should call Pipeline.run() instead of SparkRunner.run()

[Amit Sela (JIRA)]

Amit Sela created BEAM-492:
--

 Summary: Spark runner should call Pipeline.run() instead of 
SparkRunner.run()
 Key: BEAM-492
 URL: https://issues.apache.org/jira/browse/BEAM-492
 Project: Beam
  Issue Type: Bug
Affects Versions: 0.1.0-incubating
Reporter: Amit Sela
Assignee: Amit Sela


Currently, in some places (streaming examples, etc.), the Pipeline executes by 
calling SparkRunner.run() instead of calling Pipeline.run() which is kind of 
"backwards" to the Beam API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)