[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134706#comment-16134706
 ] 

ASF GitHub Bot commented on BEAM-79:


Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3726


> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129897#comment-16129897
 ] 

ASF GitHub Bot commented on BEAM-79:


GitHub user huafengw opened a pull request:

https://github.com/apache/beam/pull/3726

[BEAM-79] Add Gearpump runner profile to Java8 Archetype POM

Follow this checklist to help us incorporate your contribution quickly and 
easily:

 - [ ] Make sure there is a [JIRA 
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the 
change (usually before you start working on it).  Trivial changes like typos do 
not require a JIRA issue.  Your pull request should address just this issue, 
without pulling in other changes.
 - [ ] Each commit in the pull request should have a meaningful subject 
line and body.
 - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue.
 - [ ] Write a pull request description that is detailed enough to 
understand what the pull request does, how, and why.
 - [ ] Run `mvn clean verify` to make sure basic checks pass. A more 
thorough check will be performed on your pull request automatically.
 - [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/huafengw/beam example

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3726.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3726


commit 31810c459ea95cb7816261087a7e73dc0e0338c3
Author: huafengw 
Date:   2017-08-17T03:58:33Z

[BEAM-79] Add Gearpump runner profile to Java8 Archetype POM




> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-79) Gearpump runner

[Manu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117833#comment-16117833
 ] 

Manu Zhang commented on BEAM-79:


Hi [~takidau],

We've reached a milestone after merging gearpump-runner into master.  Future 
development will be under new JIRAs. Can we close this issue now ?

Thanks for asking on Twitter and initiating this work a year ago.

Manu

> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117579#comment-16117579
 ] 

ASF GitHub Bot commented on BEAM-79:


Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3636


> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105245#comment-16105245
 ] 

ASF GitHub Bot commented on BEAM-79:


Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3637


> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101071#comment-16101071
 ] 

ASF GitHub Bot commented on BEAM-79:


Github user manuzhang closed the pull request at:

https://github.com/apache/beam/pull/3611


> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099471#comment-16099471
 ] 

ASF GitHub Bot commented on BEAM-79:


GitHub user manuzhang opened a pull request:

https://github.com/apache/beam/pull/3637

[BEAM-79] Prepare for merging gearpump-runner

Follow this checklist to help us incorporate your contribution quickly and 
easily:

 - [ ] Make sure there is a [JIRA 
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the 
change (usually before you start working on it).  Trivial changes like typos do 
not require a JIRA issue.  Your pull request should address just this issue, 
without pulling in other changes.
 - [ ] Each commit in the pull request should have a meaningful subject 
line and body.
 - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue.
 - [ ] Write a pull request description that is detailed enough to 
understand what the pull request does, how, and why.
 - [ ] Run `mvn clean verify` to make sure basic checks pass. A more 
thorough check will be performed on your pull request automatically.
 - [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/beam gearpump-runner

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3637.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3637


commit e655f53c2e732bcd082449ba8d0b551f417e8aaa
Author: manuzhang 
Date:   2017-07-21T06:27:37Z

Revert accidental changes to sdks/java/pom.xml

commit daa7566939e212095ef745ff004580bfe4209b38
Author: manuzhang 
Date:   2017-07-21T06:58:13Z

Upgrade BEAM version to 2.2.0-SNAPSHOT in gearpump-runner

commit 49d4ed59567852e978d14729c89b9ea91bb96c32
Author: manuzhang 
Date:   2017-07-21T14:33:36Z

Add beam-runners-gearpump dependency to javadoc




> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099465#comment-16099465
 ] 

ASF GitHub Bot commented on BEAM-79:


GitHub user manuzhang opened a pull request:

https://github.com/apache/beam/pull/3636

[BEAM-79] merge gearpump-runner into master

Follow this checklist to help us incorporate your contribution quickly and 
easily:

 - [ ] Make sure there is a [JIRA 
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the 
change (usually before you start working on it).  Trivial changes like typos do 
not require a JIRA issue.  Your pull request should address just this issue, 
without pulling in other changes.
 - [ ] Each commit in the pull request should have a meaningful subject 
line and body.
 - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue.
 - [ ] Write a pull request description that is detailed enough to 
understand what the pull request does, how, and why.
 - [ ] Run `mvn clean verify` to make sure basic checks pass. A more 
thorough check will be performed on your pull request automatically.
 - [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/beam gearpump-runner

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3636.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3636


commit 9478f4117de3a2d0ea40614ed4cb801918610724
Author: manuzhang 
Date:   2016-03-15T08:15:16Z

[BEAM-79] add Gearpump runner

commit 02b2248a5b3c8a2c064547d7380bebc97f849bf1
Author: Kenneth Knowles 
Date:   2016-07-20T16:07:48Z

This closes #323

commit 2a0ba61e8507e1539115b583749a78f14d577bd8
Author: Kenneth Knowles 
Date:   2016-08-25T18:36:45Z

Merge branch master into gearpump-runner

commit 1672b5483e029292816397248dc6fe63bf51f4af
Author: manuzhang 
Date:   2016-07-23T06:10:15Z

move integration tests to profile

commit 276a2e106aa1a573fc2eb2426b640f63cf68
Author: manuzhang 
Date:   2016-07-28T08:30:13Z

add package-info.java

commit 40be715a696bb1218b209f7ad9a979b7e5d088d3
Author: Kenneth Knowles 
Date:   2016-08-10T17:26:57Z

Update Gearpump runner version to 0.3.0-incubating

commit bc1b354949416db3b52c4f37c66968bdb86f0813
Author: manuzhang 
Date:   2016-08-11T23:22:00Z

Rename DoFn to OldDoFn in Gearpump runner

commit 091a15a07c7625ae3009cefaecece3a29a34c109
Author: Kenneth Knowles 
Date:   2016-08-25T18:40:03Z

This closess #750

commit fb74c936ed92c7a8548c338cc03957794fc60902
Author: Dan Halperin 
Date:   2016-08-26T23:25:58Z

gearpump: switch to stable version

They have apparently deleted the SNAPSHOT jar and now builds are failing.

commit bf0a2edae11416a3cbddeaff2c0a70adc272c5fe
Author: Dan Halperin 
Date:   2016-08-27T00:46:42Z

Closes #895

commit 89921c41ca9d4c333af45efa32359a631214c1df
Author: bchambers 
Date:   2016-07-29T16:41:17Z

Remove Counter and associated code

Aggregator is the model level concept. Counter was specific to the
Dataflow Runner, and is now not needed as part of Beam.

commit 7fc2c6848f002ac8b2ccbe35e6b5a576777a7af9
Author: Mark Liu 
Date:   2016-08-03T00:25:14Z

[BEAM-495] Create General Verifier for File Checksum

commit b47549e4893a6d487c00ea0ba02619168a3f19f3
Author: Mark Liu 
Date:   2016-08-03T00:47:46Z

Add output checksum to  WordCountITOptions

commit 58cd781c82fa728f34f5ab0641f8f9b6edcf449c
Author: Ian Zhou 
Date:   2016-08-05T22:31:59Z

Added unit tests and error handling in removeTemporaryTables

commit 36a9aa232ea56de449930194788becce585212ef
Author: Thomas Groh 
Date:   2016-08-09T02:09:58Z

Improve Write Error Message

If provided with an Unbounded PCollection, Write will fail due to
restriction of calling finalize only once. This error message fails in a
deep stack trace based on it not being possible to apply a GroupByKey.
Instead, throw immediately on application with a specific error message.

commit d5641553cebb02f08ca7c1fe667948d39cb3962c
Author: Thomas Groh 
Date:   2016-08-09T17:47:09Z

Remove Streaming Write Overrides in 

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095850#comment-16095850
 ] 

ASF GitHub Bot commented on BEAM-79:


GitHub user manuzhang opened a pull request:

https://github.com/apache/beam/pull/3611

[BEAM-79] merge gearpump-runner into master

Follow this checklist to help us incorporate your contribution quickly and 
easily:

 - [ ] Make sure there is a [JIRA 
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the 
change (usually before you start working on it).  Trivial changes like typos do 
not require a JIRA issue.  Your pull request should address just this issue, 
without pulling in other changes.
 - [ ] Each commit in the pull request should have a meaningful subject 
line and body.
 - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue.
 - [ ] Write a pull request description that is detailed enough to 
understand what the pull request does, how, and why.
 - [ ] Run `mvn clean verify` to make sure basic checks pass. A more 
thorough check will be performed on your pull request automatically.
 - [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/beam gearpump-runner

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3611.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3611


commit 9478f4117de3a2d0ea40614ed4cb801918610724
Author: manuzhang 
Date:   2016-03-15T08:15:16Z

[BEAM-79] add Gearpump runner

commit 02b2248a5b3c8a2c064547d7380bebc97f849bf1
Author: Kenneth Knowles 
Date:   2016-07-20T16:07:48Z

This closes #323

commit 2a0ba61e8507e1539115b583749a78f14d577bd8
Author: Kenneth Knowles 
Date:   2016-08-25T18:36:45Z

Merge branch master into gearpump-runner

commit 1672b5483e029292816397248dc6fe63bf51f4af
Author: manuzhang 
Date:   2016-07-23T06:10:15Z

move integration tests to profile

commit 276a2e106aa1a573fc2eb2426b640f63cf68
Author: manuzhang 
Date:   2016-07-28T08:30:13Z

add package-info.java

commit 40be715a696bb1218b209f7ad9a979b7e5d088d3
Author: Kenneth Knowles 
Date:   2016-08-10T17:26:57Z

Update Gearpump runner version to 0.3.0-incubating

commit bc1b354949416db3b52c4f37c66968bdb86f0813
Author: manuzhang 
Date:   2016-08-11T23:22:00Z

Rename DoFn to OldDoFn in Gearpump runner

commit 091a15a07c7625ae3009cefaecece3a29a34c109
Author: Kenneth Knowles 
Date:   2016-08-25T18:40:03Z

This closess #750

commit fb74c936ed92c7a8548c338cc03957794fc60902
Author: Dan Halperin 
Date:   2016-08-26T23:25:58Z

gearpump: switch to stable version

They have apparently deleted the SNAPSHOT jar and now builds are failing.

commit bf0a2edae11416a3cbddeaff2c0a70adc272c5fe
Author: Dan Halperin 
Date:   2016-08-27T00:46:42Z

Closes #895

commit 89921c41ca9d4c333af45efa32359a631214c1df
Author: bchambers 
Date:   2016-07-29T16:41:17Z

Remove Counter and associated code

Aggregator is the model level concept. Counter was specific to the
Dataflow Runner, and is now not needed as part of Beam.

commit 7fc2c6848f002ac8b2ccbe35e6b5a576777a7af9
Author: Mark Liu 
Date:   2016-08-03T00:25:14Z

[BEAM-495] Create General Verifier for File Checksum

commit b47549e4893a6d487c00ea0ba02619168a3f19f3
Author: Mark Liu 
Date:   2016-08-03T00:47:46Z

Add output checksum to  WordCountITOptions

commit 58cd781c82fa728f34f5ab0641f8f9b6edcf449c
Author: Ian Zhou 
Date:   2016-08-05T22:31:59Z

Added unit tests and error handling in removeTemporaryTables

commit 36a9aa232ea56de449930194788becce585212ef
Author: Thomas Groh 
Date:   2016-08-09T02:09:58Z

Improve Write Error Message

If provided with an Unbounded PCollection, Write will fail due to
restriction of calling finalize only once. This error message fails in a
deep stack trace based on it not being possible to apply a GroupByKey.
Instead, throw immediately on application with a specific error message.

commit d5641553cebb02f08ca7c1fe667948d39cb3962c
Author: Thomas Groh 
Date:   2016-08-09T17:47:09Z

Remove Streaming Write Overrides in 

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077824#comment-16077824
 ] 

ASF GitHub Bot commented on BEAM-79:


GitHub user manuzhang opened a pull request:

https://github.com/apache/beam/pull/3515

[BEAM-79] Sync gearpump-runner with master and upgrade to gearpump 0.8.4

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/beam sync-master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3515.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3515


commit 7b8cd6401cb5ed6e184ed36571a89d3ae324dd5f
Author: Jeremie Lenfant-Engelmann 
Date:   2017-06-29T01:32:56Z

Properly convert milliseconds whether there's less than 3/more than 9 
digits.
TimeUtil did not properly convert (and returned null) when the number of
digits for fractions of seconds was less than 3 digits or more than 9 
digits.
The solution is to pad with zeros when there is less than 3 digits and to
truncate when there is more than 3.

commit f75dfe711b357800762a93dc5059d9a19da9ee61
Author: Luke Cwik 
Date:   2017-06-30T16:00:13Z

Properly convert milliseconds whether there's less than 3/more than 9 
digits.

This closes #3469

commit 56cb6c51748fde6ad56522733ab10edca062e802
Author: Kenneth Knowles 
Date:   2017-06-13T17:29:50Z

Add support for PipelineOptions parameters

This is a step towards eliminating catch-all context parameters and
making DoFns express their fine-grained data needs.

commit 0e429b33ff85eba08da5018c9febd0b99b44f720
Author: Kenneth Knowles 
Date:   2017-06-30T22:39:20Z

This closes #3355: Add support for PipelineOptions parameters

commit f99ab1a472868e4ce175a86e5d76823b1c09c10b
Author: Kenneth Knowles 
Date:   2017-07-01T04:42:17Z

Fix DoFn javadoc: StateSpec does not require a key

commit ce4e5174750ae2c4b1691c6994bd0f92e161
Author: Ismaël Mejía 
Date:   2017-06-04T20:55:05Z

Make modules that depend on Hadoop and Spark use the same version property

commit 11010cf08ec9ca90a843cc15ea8f0fd4910f7fab
Author: Jean-Baptiste Onofré 
Date:   2017-07-03T07:26:40Z

This closes #3291

commit 75475ef3dc23a09fa9bbba478d6fdbc468f7dd2e
Author: Ismaël Mejía 
Date:   2017-06-28T14:58:55Z

[BEAM-2530] Fix compilation of modules with Java 9 that depend on jdk.tools

commit 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243
Author: Jean-Baptiste Onofré 
Date:   2017-07-03T12:23:18Z

[BEAM-2530] This closes #3460

commit 68f1fb64fd2565e287e322d715ca778d01e7137b
Author: Ahmet Altay 
Date:   2017-07-01T00:37:33Z

For GCS operations use an http client with a default timeout value.

commit b8ac32641183b06a87517a169c8ecc5371996aed
Author: Ahmet Altay 
Date:   2017-07-04T18:07:31Z

This closes #3486

commit 51877a3405dbf778c3bb88f19bb194e54c3b3def
Author: Jean-Baptiste Onofré 
Date:   2017-07-05T14:47:29Z

[maven-release-plugin] prepare branch release-2.1.0

commit 7f0723cf7fd587c4f8dbe8a8b1c9d298a7b1e5e3
Author: Jean-Baptiste Onofré 
Date:   2017-07-05T14:47:38Z

[maven-release-plugin] prepare for next development iteration

commit 14fa7f79f0830739122b7573e032ad0aea172a98
Author: Jean-Baptiste Onofré 
Date:   2017-07-05T14:52:48Z

Update Python SDK version

commit 06897b1cc142f658437ac7779c849e5182e331f1
Author: Jason Kuster 
Date:   2017-06-09T08:39:15Z

Website Mergebot Job

Signed-off-by: Jason Kuster 

commit f0549b464737d0d11ae1da8e0534622b10247208
Author: Thomas Groh 
Date:   2017-07-05T16:22:20Z

This closes #3485

commit 6ca410a908f1f4e7ac1e141ee1335f7a537bb150
Author: Luke Cwik 
Date:   2017-07-05T17:38:44Z

[BEAM-2553] Update Maven exec plugin to 1.6.0 to incorporate messaging 
improvements

commit 39c80f640749fed5f55a0c8f25c849e5f7c3d179
Author: Luke Cwik 
Date:   2017-07-05T19:26:28Z

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070757#comment-16070757
 ] 

ASF GitHub Bot commented on BEAM-79:


Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3479


> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069701#comment-16069701
 ] 

ASF GitHub Bot commented on BEAM-79:


GitHub user manuzhang opened a pull request:

https://github.com/apache/beam/pull/3479

[BEAM-79] Merge master into gearpump-runner branch

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/beam sync-master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3479.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3479


commit cab4d8969e7f95b0ece59838ad2d578e75d38823
Author: Kenneth Knowles 
Date:   2017-06-22T03:25:31Z

DataflowRunner: Reject merging windowing for stateful ParDo

commit 5f8cfa7412e8f4ce97d6df4413fecbea22b4b5e1
Author: Kenneth Knowles 
Date:   2017-06-23T03:05:28Z

This closes #3419: DataflowRunner: Reject merging windowing for stateful 
ParDo

commit 649994b353afe28c917969609c7a1a47a4f39aaf
Author: Rune Fevang 
Date:   2017-06-15T11:51:12Z

Allow output from FinishBundle in DoFnTester

commit b3c36256e32f4a4a616d0e152686d29f1174c5cd
Author: Kenneth Knowles 
Date:   2017-06-23T03:07:48Z

This closes #3367: Allow output from FinishBundle in DoFnTester

commit 8dcda6e40355af13f4d92fcd44aae4539a225a4a
Author: Sourabh Bajaj 
Date:   2017-06-23T00:08:20Z

[BEAM-2497] Fix the reading of concat gzip files

commit 336b7f1cffc8709cd480a243a16de95bce39b53e
Author: Chamikara Jayalath 
Date:   2017-06-23T05:13:05Z

This closes #3428

commit f291713b28e3ba0246a8c0a710c71506cd0a0f91
Author: Etienne Chauchot 
Date:   2017-06-21T08:39:39Z

[BEAM-2489] Use dynamic ES port in HIFIOWithElasticTest

commit 6cef5c7b625c6b5c2078084ba611b1c2adccc998
Author: Jean-Baptiste Onofré 
Date:   2017-06-23T13:01:42Z

[BEAM-2489] This closes #3411

commit fd8f15f1ac761425dc791a455b042a8846081f48
Author: Mark Liu 
Date:   2017-06-22T21:04:00Z

[BEAM-2745] Add Jenkins Suite for Python Performance Test

commit af69e979ab3037274bdd78904bdeb5016563bcad
Author: Ahmet Altay 
Date:   2017-06-23T21:03:17Z

This closes #3426

commit 32095487e56b63b5c1aa690bb6e098375cb108d5
Author: Vikas Kedigehalli 
Date:   2017-06-23T18:50:12Z

Fix python fn API data plane remote grpc port access

commit 9acce7150ee8fa6c9e50049155ad7b85b646f98e
Author: Ahmet Altay 
Date:   2017-06-23T22:13:30Z

This closes #3431

commit 903da41ac5395e76c44ef8ae1c8a695569e23abb
Author: Robert Bradshaw 
Date:   2017-06-23T22:01:42Z

Avoid pickling the entire pipeline per-transform.

commit a90e40ae9f748c1c8392c1198469f3229a06ed70
Author: Robert Bradshaw 
Date:   2017-06-23T23:39:52Z

Closes #3433

commit e45f522d6e945899c20259ebf8faca105c2e552e
Author: Valentyn Tymofieiev 
Date:   2017-06-23T23:44:49Z

Fix a typo in function args

commit 16f87f49f20796e29d01ed363a9097ea5420583c
Author: Ahmet Altay 
Date:   2017-06-24T00:34:21Z

This closes #3435

commit 926f949580c3a21df72a8836feda1f6b947850ec
Author: Charles Chen 
Date:   2017-06-26T20:00:14Z

Remove old deprecated PubSub code

commit 1ea1de4aa9d32e3c5a596ccd7d84aff1cc2a7428
Author: Ahmet Altay 
Date:   2017-06-26T21:23:08Z

This closes #3440

commit bec32fe93c6b5c16563d7ea4b877a2dee3352fee
Author: Eugene Kirpichov 
Date:   2017-06-16T21:56:07Z

Reintroduces DoFn.ProcessContinuation (Dataflow worker compatibility part)

commit 2052cc7689b4ed53f817f56dd71a32235fb083ca
Author: Eugene Kirpichov 
Date:   2017-06-23T17:16:30Z

Bump Dataflow worker to 0623

commit 95e6bbe5065a983f9c71b79b20df1fda83c9fd1b
Author: Eugene Kirpichov 
Date:   2017-06-27T00:25:14Z

This closes #3383: [BEAM-2447] Reintroduces DoFn.ProcessContinuation 
(Dataflow worker compatibility part)

commit eb379e76adaa9c4b4e24a4b3c5757be8523d95c4
Author: Charles Chen 

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16060352#comment-16060352
 ] 

ASF GitHub Bot commented on BEAM-79:


Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3388


> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16053585#comment-16053585
 ] 

ASF GitHub Bot commented on BEAM-79:


GitHub user manuzhang opened a pull request:

https://github.com/apache/beam/pull/3388

[BEAM-79] Sync gearpump-runner branch with master

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/beam sync-master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3388.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3388


commit 8362bdb9cd35cc02ed179b3a64fd72f1264a99be
Author: JingsongLi 
Date:   2017-06-07T17:31:34Z

[BEAM-2423] Abstract StateInternalsTest for the different state internals

commit f9d51aa5cc718c44c82ec4773dbd16a57e7d394e
Author: Aljoscha Krettek 
Date:   2017-06-12T20:35:35Z

This closes #3313

commit 809f17876d847002ba76979cb3362451fa01c110
Author: Charles Chen 
Date:   2017-06-12T21:17:50Z

Reverse removal of NativeWrite evaluator in Python DirectRunner

commit 86e04893a14d9b6aa29b78e25dc064879b712fb3
Author: Ahmet Altay 
Date:   2017-06-12T22:43:44Z

This closes #3346

commit f0f98c70b0b01c43ce1b093c2035b20bd90ba907
Author: Thomas Groh 
Date:   2017-06-12T18:07:47Z

Cleanup Combine Tests with Context

Split out the "shared" bit of all the accumulators, so they show up as
an explicit component of the final result string. Update timestamped
creation logic.

commit ce3dd4583294fcbb65121f5fecfc49a8b85634ad
Author: Thomas Groh 
Date:   2017-06-12T22:48:38Z

This closes #3345

commit 1ac18b2eb1371422e60d50a8c3f37b3b24d59611
Author: Thomas Groh 
Date:   2017-06-12T23:55:59Z

Check for Deferral on Non-additional inputs

Because Side Inputs are represented within the expanded inputs, the
check that the transform is a Combine with Side Inputs would never be
hit. This ensures that we do not consider additional inputs during the
check to defer evaluation of the node.

commit 32d55323cd5a88f604ad9c763ab265de845df08c
Author: Kenneth Knowles 
Date:   2017-06-13T03:20:40Z

This closes #3347: Check for Deferral on Non-additional inputs

commit 6859f80400bb16bbca34ed282e7a5e8a1328f955
Author: Ismaël Mejía 
Date:   2017-06-05T21:20:27Z

[BEAM-2412] Update HBaseIO to use HBase client 1.2.6

commit fe3d55403334bf235952fff33e6a90c0f4445a0d
Author: Jean-Baptiste Onofré 
Date:   2017-06-13T05:59:30Z

[BEAM-2412] This closes #3327

commit 4c36508733a69fafce0f7dfb86c71eee5eb6bc84
Author: JingsongLi 
Date:   2017-06-07T06:34:25Z

Use CoderTypeSerializer and remove unuse code in FlinkStateInternals

commit 10b166b355a03daeae78dd1e71016fc72805939d
Author: JingsongLi 
Date:   2017-06-07T06:40:30Z

[BEAM-1483] Support SetState in Flink runner and fix MapState to be 
consistent with InMemoryStateInternals.

commit 4d18606378f43c7b0d3ac05d45ca6e0570e49eef
Author: JingsongLi 
Date:   2017-06-13T02:15:33Z

Add set and map readable test to StateInternalsTest

commit 7126fdc6ee5671e99a2dede3f25ba616aa0e8fa4
Author: Aljoscha Krettek 
Date:   2017-06-13T09:35:43Z

This closes #3309

commit e71eb66ae319bdf0cdad1fe9b54662962c8e8f16
Author: Robert Bradshaw 
Date:   2017-06-09T23:44:55Z

Actually test the fn_api_runner.

The test suite was not being run due to a typo.
Fix breakage due to changes in the code in the meantime.

commit bd83612f246c1654b80df19890a25f8de48a063a
Author: Robert Bradshaw 
Date:   2017-06-13T15:56:36Z

Closes #3339

commit b547b5a1eee61e293ff2f8ccfac57f308867328c
Author: Maria Garcia Herrero 
Date:   2017-06-10T06:34:59Z

Make unique test names for value-provider arguments

commit e906fe9c3a1e12f01d84e05fbb872d47b28d2fe9
Author: Ahmet Altay 
Date:   2017-06-13T16:34:16Z

This closes #3341

commit 

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040410#comment-16040410
 ] 

ASF GitHub Bot commented on BEAM-79:


Github user manuzhang closed the pull request at:

https://github.com/apache/beam/pull/3181


> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16036469#comment-16036469
 ] 

ASF GitHub Bot commented on BEAM-79:


GitHub user manuzhang opened a pull request:

https://github.com/apache/beam/pull/3292

[BEAM-79] Merge master into gearpump-runner

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/beam sync-master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3292.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3292


commit 66e564e507b1ad5b9e379e043a193b556e1f3e76
Author: Borisa Zivkovic 
Date:   2017-05-15T08:24:10Z

[BEAM-1702] Document how to use BigtableIO with local emulator

commit 8ef812defc5237934f4f3efa6bd8c4c4240cd7d0
Author: Dan Halperin 
Date:   2017-05-18T20:07:13Z

This closes #3141

commit a8d2125e5783a556056e88dad8fe3c0a397920d5
Author: huafengw 
Date:   2017-05-09T03:21:44Z

[BEAM-659] WindowFn#isCompatible should provide a meaningful reason

commit 8c572ef0b998a4fa529e2f4ba0e622b3db9f4ec6
Author: Kenneth Knowles 
Date:   2017-05-18T20:27:59Z

This closes #2985: WindowFn#isCompatible should provide a meaningful reason

commit 26a615e885b39f92dc81d383c8115906c280187a
Author: Andrew Martin 
Date:   2017-05-18T19:52:09Z

Remove incorrect `Message` bound on writeAvros

commit 6a7eeeb932ffb33c876c0d11d7f325c47606d8de
Author: Eugene Kirpichov 
Date:   2017-05-18T20:52:33Z

This closes #3176: Remove incorrect `Message` bound on writeAvros

commit 790e7fe6653b926044d3dfecdccbc2fda9c998f0
Author: Thomas Groh 
Date:   2017-03-21T22:06:58Z

Add ParDos

Add ParDoPayloadTranslator to PTransformTranslator

commit 2e0cf008303ff5ed48268550b9fe3cd1aa748c15
Author: Thomas Groh 
Date:   2017-05-18T22:41:17Z

This closes #2597

commit 7a2e829c849d3828353994b8eddca4784a900aca
Author: Eugene Kirpichov 
Date:   2017-05-15T22:54:03Z

Splits SplittableParDo into a core-construction part and a KWI-related part

commit c93b3aa3d2081558a776fa12c564efb5f15090ab
Author: Eugene Kirpichov 
Date:   2017-05-18T23:15:00Z

This closes #3156

commit 54808be14ff2e86088807adc3c0dc0965c8a6f03
Author: Dan Halperin 
Date:   2017-05-18T22:43:31Z

[BEAM-2318] Ensure that tables that don't exist are not created

We use the same test table name in two tests: one that expects the table 
not to exist, another that
creates the table. Obviously, the correctness of this will depend on the 
order in which unit tests
are executed.

Fix the flake by using different and more explicit test names.

commit f458065dad0429ba2639510ea460cf7f2679ed36
Author: Dan Halperin 
Date:   2017-05-18T23:41:41Z

This closes #3179

commit 1da8da79dcc623c53b35d97419566b736447f6b6
Author: Dan Halperin 
Date:   2017-05-18T20:19:29Z

[BEAM-2391] Clone Scan in HBaseReader

commit 49245080ad6393b5678077caa047f7a6ef0efffc
Author: Ismaël Mejía 
Date:   2017-05-19T08:50:06Z

This closes #3178

commit f969ea443c90345cf8c33f953c399e46223e8d03
Author: Luke Cwik 
Date:   2017-05-15T20:39:12Z

[BEAM-2252] Use pluginManagement instead of redefining maven-shade-plugin 
to inherit build order from root pom.xml

commit ea5b9dabbfb55ff47a4f5983a8cbf04b0949c77c
Author: Luke Cwik 
Date:   2017-05-19T17:37:10Z

[BEAM-2252] Use pluginManagement instead of redefining maven-shade-plugin 
to inherit build order from root pom.xml

This closes #3152

commit 3efb34b8322ac5933d9d71255d67e99d8e8361d7
Author: Ahmet Altay 
Date:   2017-05-19T02:00:00Z

update dataflow python container

commit a58275a2f904ab1304de7ca5295f27a4f6df6f1d
Author: Ahmet Altay 
Date:   2017-05-19T17:43:15Z

This closes #3182

commit bac7bde5f878e6b2104c2e7cf3d01894a8f1b71d
Author: Mark Liu 
Date:   2017-05-17T21:24:25Z

[BEAM-2164] Fix 

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016766#comment-16016766
 ] 

ASF GitHub Bot commented on BEAM-79:


GitHub user manuzhang opened a pull request:

https://github.com/apache/beam/pull/3181

[BEAM-79] Respect WindowFn#getOutputTime in gearpump-runner

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/beam fix_windowing_test

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3181.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3181


commit 7328ff21f2b805801123c0458d4fad22563ad384
Author: manuzhang 
Date:   2017-05-19T01:19:42Z

[BEAM-79] Respect WindowFn#getOutputTime in gearpump-runner




> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016380#comment-16016380
 ] 

ASF GitHub Bot commented on BEAM-79:


Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3172


> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16015131#comment-16015131
 ] 

ASF GitHub Bot commented on BEAM-79:


GitHub user manuzhang opened a pull request:

https://github.com/apache/beam/pull/3172

[BEAM-79] Sync gearpump-runner with master

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/beam sync-master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3172.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3172


commit 44c9385ddbcfbdf904c5d054754dca3ddde88ec2
Author: Dan Halperin 
Date:   2017-05-06T00:16:34Z

[BEAM-2190] pom.xml: do a better job of dependency management

Even if Beam appears to have the correct dependencies, we cannot
guarantee that modules that depend on us transitively get the right
dependencies. For example, even though grpc-protobuf-lite has
protobuf-lite excluded, and the Maven Enforcer banned-dependencies
check passes... if a user happens to get a transitive dependency on
grpc-all first, they may pull in grpc-protobuf from that other source
without the exclusion. Thus we need to exclude protobuf-lite from
grpc-all as well.

While we're here, also add guava-jdk5 to the set of banned dependencies,
though (as above) we cannot currently properly identify the places it
might be transitively exposed in a users' pom.xml.

commit 2b0e699b8f6a0aac26ac57c536e8d45cee1c3d2e
Author: Dan Halperin 
Date:   2017-05-08T18:35:56Z

This closes #2934

commit 5bac40efcc7bcd9d53183579ec6089c23aeda782
Author: Dan Halperin 
Date:   2017-05-06T02:06:03Z

[BEAM-2212] FileBasedSource: improve message when logging.

ValueProvider should not be printed, rather the string instead.

commit 9423babd8f827e843723c218441e9a91aaa7b361
Author: Dan Halperin 
Date:   2017-05-08T16:59:16Z

[BEAM-2212] FileBasedSource: refactor to remove uses of 
fileOrPatternSpec.get()

Makes it less likely to have errors from printing ValueProviders instead of 
runtime values

commit e1791c3f83a007013625b43f9976e7ffa64e8a49
Author: Dan Halperin 
Date:   2017-05-08T18:48:42Z

This closes #2956

commit 13e51c9cd114eec73c47b71f46214dd92ee81048
Author: Dan Halperin 
Date:   2017-05-06T02:49:29Z

Convert Executor Services to use Daemon Threads

This will cause the DirectRunner to automatically shut down when the
worker threads are shut down.

commit 63c6bea3bbdb52964ac54073326d4ad99ba7432c
Author: Thomas Groh 
Date:   2017-05-08T19:52:09Z

This closes #2954

commit 50a4c56de1d3aa0e51e8b2b08b71715be4c2ef80
Author: Sourabh Bajaj 
Date:   2017-05-07T20:55:20Z

[BEAM-2206] Move pipelineOptions into options modules

commit 604c2f59e2a928a95106814b0b55b8097dc39faa
Author: Ahmet Altay 
Date:   2017-05-08T19:56:15Z

This closes #2925

commit d3fc7d18a4a3c394d32d85ef34c08a85712a8b18
Author: Reuven Lax 
Date:   2017-05-08T16:06:55Z

TableDescription is allowed to be null.

commit c481396edc7a3e906cd79dcc2a356baf9c93895e
Author: Eugene Kirpichov 
Date:   2017-05-08T20:41:01Z

This closes #2953

commit 1fd15bd7441e1be72feb3f9535ffe536d730a531
Author: Reuven Lax 
Date:   2017-05-07T18:50:08Z

Shortens PubsubIO names for reading/writing messages.

commit d1914ed0c3195e1de2f707e2bb7bc57597b834fe
Author: Eugene Kirpichov 
Date:   2017-05-08T20:45:25Z

This closes #2950

commit b7635aadc2532b5c3bdc38f070865091ebf901c1
Author: Ahmet Altay 
Date:   2017-05-08T20:42:02Z

Remove apex README.md. This information is already in the website in
quickstart and apex runner pages.
https://github.com/apache/beam-site/pull/232 moves small bits of missing
content.

commit 962e8796d68f9a81b0ae3d1b30d7a4bcadbdaf60
Author: Ahmet Altay 
Date:   2017-05-08T21:28:36Z

This closes #2962

commit fba4c010ce90db8b5cdac489d47126ca3ee8db65
Author: Ahmet 

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997350#comment-15997350
 ] 

ASF GitHub Bot commented on BEAM-79:


Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/2888


> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996155#comment-15996155
 ] 

ASF GitHub Bot commented on BEAM-79:


GitHub user manuzhang opened a pull request:

https://github.com/apache/beam/pull/2888

[BEAM-79] Merge master into gearpump-runner branch

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/incubator-beam sync-master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/2888.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2888


commit d9595d35176d2572423f6be069982b05eea204dd
Author: Maria Garcia Herrero 
Date:   2017-04-26T23:22:09Z

Add extra message to deprecated annotation

commit 42e3a6f8588046f257e5c780dec12d16d0314a66
Author: Ahmet Altay 
Date:   2017-04-27T18:25:18Z

This closes #2719

commit 9174ebf48391f393761b2ce085d2f165ebb2c7bb
Author: Tibor Kiss 
Date:   2017-04-27T12:12:13Z

[BEAM-1695] Improve Python-SDK's programming guide

commit 884935cb98f0e29bd965fda9182970423d111d7e
Author: Ahmet Altay 
Date:   2017-04-27T19:03:22Z

This closes #2738

commit e047b69efab9c988011303cf2eda86ac408b38c2
Author: Dan Halperin 
Date:   2017-04-27T15:30:47Z

[BEAM-2096] Make DataflowMetrics more resilient

DataflowMetrics seems to have many hard-coded assumptions about what will be
returned by the Dataflow service that, which will likely break when users
use new types of metrics or if the Dataflow service makes minor adjustments
in our it sends metrics back to the user.

In order for code to continue working in these cases, handle errors by 
logging
and skipping.

commit fdf2de9994268f5e1d704aa30a4504ac6ee624b8
Author: Dan Halperin 
Date:   2017-04-27T20:10:32Z

This closes #2739

commit f93a27755921e73141fd97de042e6dcd04e10a47
Author: Dan Halperin 
Date:   2017-04-26T00:29:02Z

[BEAM-59] Add FileSystems#matchNewResource

The new FileSystems API needs a way to generate a ResourceId for a
resource that does not exist. This does not come up in sources, because
we typically are just matching existing files. However, sinks need the
ability to reference a new directory (say, in order to create it).

Couldn't think of anything better than a simple function that says
"treat this string as a full resource path with the specified type",
which is what FileSystems#matchNewResource is.

commit 14a90d7c5196d96de66e897687d28202d790439f
Author: Dan Halperin 
Date:   2017-04-27T21:56:24Z

This closes #2728

commit d3b399fa84bff45b0960368d32d4c69e433f4c94
Author: Thomas Groh 
Date:   2017-04-26T22:12:54Z

Remove Coder Encoding IDs

These were never safe to use, and will not be used once Coders have been
migrated to the Runner API representation.

commit c4936958834aa55f0f1fcb73fe9b8419c4c9eb9b
Author: Thomas Groh 
Date:   2017-04-27T22:22:55Z

This closes #2717

commit 1925a50d860b7b8f8422f1c2f251d0ea11def736
Author: Aviem Zur 
Date:   2017-04-27T20:48:02Z

[BEAM-2029] NullPointerException when using multi output ParDo in Spark 
runner in streaming mode.

commit a46eb1a80074dc9c382da4d71725ffc61f2e65ba
Author: Dan Halperin 
Date:   2017-04-27T22:33:39Z

This closes #2746

commit 9a3b830c8737859c6afaa6e6e1922b037a78741f
Author: Dan Halperin 
Date:   2017-04-27T22:35:10Z

[BEAM-59] LocalResources: helper utility to create LocalResourceId

Primarily for testing.

commit f71395bfd8f94cbe8d78098f5bb66a1dce65e74b
Author: Dan Halperin 
Date:   2017-04-27T23:30:50Z

This closes #2751

commit 95fdb8f4dbb6bb8ec6d80152aa85cc970ab13366
Author: Dan Halperin 
Date:   2017-04-27T20:51:53Z

WriteFiles: trivial message fix

commit 674ae9a475f1f236230697d5cb1c33d36fec49db
Author: Dan Halperin 
Date:   2017-04-28T00:20:54Z

This closes #2755

commit 06f4ef40bec7d6d3c5826fb9e0c164eb9c05efe7

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15975906#comment-15975906
 ] 

ASF GitHub Bot commented on BEAM-79:


GitHub user huafengw opened a pull request:

https://github.com/apache/beam/pull/2607

[BEAM-79] Upgrade Gearpump to a stable version

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/huafengw/beam upgrade

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/2607.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2607


commit 3ba9b229a0a2cfcd84d5b83f208748c060caf20d
Author: huafengw 
Date:   2017-04-20T01:36:45Z

[BEAM-79] Upgrade Gearpump to a stable version




> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930783#comment-15930783
 ] 

ASF GitHub Bot commented on BEAM-79:


Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/2241


> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905763#comment-15905763
 ] 

ASF GitHub Bot commented on BEAM-79:


Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/2150


> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893939#comment-15893939
 ] 

ASF GitHub Bot commented on BEAM-79:


GitHub user manuzhang opened a pull request:

https://github.com/apache/beam/pull/2150

[BEAM-79] Add SideInput support for GearpumpRunner

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/incubator-beam gearpump-runner-fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/2150.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2150


commit 08b18bf326a7ac1bae485fbdf73aeab0de46c716
Author: manuzhang 
Date:   2017-02-14T04:33:31Z

[BEAM-79] Add SideInput support for GearpumpRunner




> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868581#comment-15868581
 ] 

ASF GitHub Bot commented on BEAM-79:


Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/1935


> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-79) Gearpump runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839334#comment-15839334
 ] 

ASF GitHub Bot commented on BEAM-79:


Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/1828


> Gearpump runner
> ---
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-gearpump
>Reporter: Tyler Akidau
>Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF 
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of 
> low-level primitives a la MillWheel, with some higher level primitives like 
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)