[GitHub] beam pull request #4166: Fix test runtime class conflicts and get hbase test...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/4166 Fix test runtime class conflicts and get hbase tests passing. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam gradle Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4166.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4166 ---
[GitHub] beam pull request #4163: Update sdks/java/io/hadoop/jdk1.8-tests to run and ...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/4163 Update sdks/java/io/hadoop/jdk1.8-tests to run and pass for Gradle build. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam gradle Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4163.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4163 commit fc0f3f04958aa2ca2b928f78fd64fde914325322 Author: Luke Cwik <lc...@google.com> Date: 2017-11-22T02:02:55Z Update sdks/java/io/hadoop/jdk1.8-tests to run and pass for Gradle build. ---
[GitHub] beam pull request #4160: Add precommit integration tests to examples/java
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/4160 Add precommit integration tests to examples/java Also make Jenkins run a common root task which enumerates all the tasks that it should run. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam gradle Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4160.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4160 commit cb58720660fe6425836ba7a63fcd93534ad66a10 Author: Lukasz Cwik <lc...@google.com> Date: 2017-11-21T06:35:57Z Add precommit integration tests to examples/java Also make Jenkins run a common root task which enumerates all the tasks that it should run. ---
[GitHub] beam pull request #4154: Re-apply PR/3976 and disable the Jenkins Gradle bui...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/4154 Re-apply PR/3976 and disable the Jenkins Gradle build to fix the other precommits. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam gradle Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4154.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4154 commit 9a4f5eb93d4cbc80dbc4e07ba9626c17a693610d Author: Luke Cwik <lc...@google.com> Date: 2017-11-20T21:56:29Z Re-apply PR/3976 and disable the Jenkins Gradle build to fix the other precommits. ---
[GitHub] beam pull request #4146: Enable Java Gradle Precommit in Jenkins
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/4146 Enable Java Gradle Precommit in Jenkins Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam gradle Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4146.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4146 commit 08ab262664381033836918ce5f1ce7ed21a53274 Author: Luke Cwik <lc...@google.com> Date: 2017-11-17T23:05:09Z Enable Java Gradle Precommit in Jenkins ---
[GitHub] beam pull request #4140: [BEAM-3202] Ensure that PipelineOptions.getOptionsI...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/4140 [BEAM-3202] Ensure that PipelineOptions.getOptionsId is always populated. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam3202 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4140.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4140 commit 66ac83928219ff71c218a880770f8809f5e6c307 Author: Luke Cwik <lc...@google.com> Date: 2017-11-16T17:40:42Z [BEAM-3202] Ensure that PipelineOptions.getOptionsId is always populated. ---
[GitHub] beam pull request #4096: Enable building Apache Beam using Gradle
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/4096 Enable building Apache Beam using Gradle Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam gradle Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4096.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4096 commit cc9a58d80e7d66c1836376dcf72e54318e6c5031 Author: Luke Cwik <lc...@google.com> Date: 2017-10-31T19:00:56Z Move AVRO files to avro source location commit a55c109aaf2bfa3dbaea47fa593900f857768da5 Author: Luke Cwik <lc...@google.com> Date: 2017-10-31T19:04:45Z Initial gradle build definitions for all java based modules which is able to assemble successfully. commit 4727e1b71134c4956799d950f3805595aff76a77 Author: Luke Cwik <lc...@google.com> Date: 2017-10-31T19:12:31Z Add license headers to all build.gradle files. commit 7bd940aba0b5f8ec105e6c701d0d8c900cd42251 Author: Luke Cwik <lc...@google.com> Date: 2017-10-31T21:59:40Z Add provided scope plugin and migrate compileOnly/testCompileOnly to provided scope commit 9763c9222187a8753547ee1711c632e890058fac Author: Luke Cwik <lc...@google.com> Date: 2017-10-31T22:31:42Z Add the ability to list dependencies recursively. Also ban the guava-jdk5 and jdk.tools modules from all dependencies. commit b3ce651220b0d1efd60394530b31b4abcba1353e Author: Luke Cwik <lc...@google.com> Date: 2017-10-31T22:45:59Z Update comments commit 6763e6e501ed39d0df897acf07fb89aff3286248 Author: Luke Cwik <lc...@google.com> Date: 2017-10-31T23:05:34Z Configure the checkstyle plugin commit a4b23fed0e2990e9589a2a69f36008e8626a1eaf Author: Luke Cwik <lc...@google.com> Date: 2017-10-31T23:32:17Z Add support to run the Apache Rat plugin Add gradle generated files to .gitignore commit 6c4ef41e5c571fc21946196ca04cd0c963aeeceb Author: Pablo <pabl...@google.com> Date: 2017-11-01T20:43:20Z Adding Python gradle changes commit 3df139feb224d83bf937a4ca8a61b65bfd40316f Author: Luke Cwik <lc...@google.com> Date: 2017-11-01T20:51:30Z Fix python tasks to not re-use the default gradle lifecycle names. commit 2e62c20183efb2069d2430e1e4bdeaa22621b99a Author: Luke Cwik <lc...@google.com> Date: 2017-11-01T20:59:34Z Add test task dependencies for testCython and testGcp and also add lint to the check task commit 427feccad268517738a8efc0d6c80d87334b90e3 Author: Luke Cwik <lc...@google.com> Date: 2017-11-02T01:06:08Z Create build_rules file moving out specific plugins to them. This requires re-adding common lifecycle tasks to the Python SDK. commit 9810e8d23c8e12e9f8b759614a02e3772df25baf Author: Luke Cwik <lc...@google.com> Date: 2017-11-02T16:42:06Z Add missing build_rules file. commit c007eb85287f3ff7b24c72729f90223357cc2901 Author: Anton Kedin <ke...@kedin-macbookpro.roam.corp.google.com> Date: 2017-11-02T16:46:36Z Ignore test categories NeedsRunner, ValidatesRunner, and all *IT tests Match current Mvn build logic commit 96ac3c21d864849f75d1390c0f8ab7b9c0939766 Author: Scott Wegner <sweg...@google.com> Date: 2017-11-02T17:45:10Z Add code style validation and auto-formatting to Gradle build. Note: actual style validation is currently disabled since the checked-in format rules seem to be out-of-date. commit 011f07b27fc49af4c9fc3f37bc62575b5af9cfca Author: Anton Kedin <ke...@google.com> Date: 2017-11-02T17:58:45Z Add beam-sdks-java-core dependency to beam-local-artifact-service Fix test initialization commit a204d3c3be1a33c0f7a6f5069
[GitHub] beam pull request #4046: Reset Jenkins config
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/4046 ---
[GitHub] beam pull request #4046: Reset Jenkins config
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/4046 Reset Jenkins config Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam jenkins2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4046.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4046 commit b11b30bd36d028f3cf659b0cb33bdc43eabba4f3 Author: Luke Cwik <lc...@google.com> Date: 2017-10-26T23:51:29Z Reset Jenkins config ---
[GitHub] beam pull request #4045: Reset Jenkins config
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/4045 ---
[GitHub] beam pull request #4042: [BEAM-3105] Cancel Jenkins builds when an update to...
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/4042 ---
[GitHub] beam pull request #4045: Reset Jenkins config
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/4045 Reset Jenkins config Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam jenkins2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4045.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4045 commit b11b30bd36d028f3cf659b0cb33bdc43eabba4f3 Author: Luke Cwik <lc...@google.com> Date: 2017-10-26T23:51:29Z Reset Jenkins config ---
[GitHub] beam pull request #4042: [BEAM-3105] Cancel Jenkins builds when an update to...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/4042 [BEAM-3105] Cancel Jenkins builds when an update to a PR makes a prior build irrelevant Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam jenkins Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4042.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4042 ---
[GitHub] beam pull request #4016: [BEAM-2566] Decouple SDK harness from Dataflow runn...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/4016 [BEAM-2566] Decouple SDK harness from Dataflow runner by elevating experiments and SDK harness configuration to java-sdk-core. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- I'm entertaining name changes on the options interfaces or any suggestions on how to make it easier to migrate from DataflowWorkerLoggingOptions to SdkHarnessOptions for logging configuration. You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam2566 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4016.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4016 commit 8b524c64e203c26ddab05d8adb4f86711fecfadc Author: Luke Cwik <lc...@google.com> Date: 2017-10-19T18:26:04Z [BEAM-2566] Decouple SDK harness from Dataflow runner by elevating experiments and SDK harness configuration to java-sdk-core. ---
[GitHub] beam pull request #4011: [BEAM-2926] Migrate to using a trivial multimap mat...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/4011 [BEAM-2926] Migrate to using a trivial multimap materialization within the Java SDK. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam side_input Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4011.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4011 commit e83084491714335d1453a6a4959b6e8c2c9d3ac1 Author: Luke Cwik <lc...@google.com> Date: 2017-10-18T20:55:10Z [BEAM-2926] Migrate to using a trivial multimap materialization within the Java SDK. ---
[GitHub] beam pull request #4000: [BEAM-3063] Improve VoidCoder structural value to u...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/4000 [BEAM-3063] Improve VoidCoder structural value to use a single shared instance. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam3063 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4000.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4000 commit 0d8626e5a0c99420535375a2162e22e60eeb28df Author: Luke Cwik <lc...@google.com> Date: 2017-10-16T20:56:04Z [BEAM-3063] Improve VoidCoder structural value to use a single shared instance. ---
[GitHub] beam pull request #3992: [BEAM-1487, BEAM-3016] Address termination correctn...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3992 [BEAM-1487, BEAM-3016] Address termination correctness issues in BufferingStreamObserver & BeamFnLoggingClient The issue with BeamFnLoggingClient is that we can't arriveAndDeregister during termination since the onReadyHandler may also arrive at the same time which is why we swap to using forced termination. Also, I added code that would guarantee that log messages produced by the thread which is shutting down are guaranteed to make it (this was being caught occassionally by the testLogging test). The BufferingStreamObserver was incorrectly shutting down since it may attempt to enqueue something into a full queue with a reading thread that has already exitted for some reason so it would loop forever attempting to insert the poison pill. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3992.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3992 commit 9dd1c5be1c79de55a49e827e8cd9d2418a9a55b8 Author: Luke Cwik <lc...@google.com> Date: 2017-10-12T23:26:29Z [BEAM-1487, BEAM-3016] Address termination correctness issues in BufferingStreamObserver & BeamFnLoggingClient The issue with BeamFnLoggingClient is that we can't arriveAndDeregister during termination since the onReadyHandler may also arrive at the same time which is why we swap to using forced termination. Also, I added code that would guarantee that log messages produced by the thread which is shutting down are guaranteed to make it (this was being caught occassionally by the testLogging test). The BufferingStreamObserver was incorrectly shutting down since it may attempt to enqueue something into a full queue with a reading thread that has already exitted for some reason so it would loop forever attempting to insert the poison pill. ---
[GitHub] beam pull request #3987: [BEAM-3048] Remove RAND_RANGE in WindowedWordCount
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3987 [BEAM-3048] Remove RAND_RANGE in WindowedWordCount Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3987.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3987 commit dd79b80a911f98b18f246bb1acead789a751ed4a Author: Luke Cwik <lc...@google.com> Date: 2017-10-12T16:49:06Z [BEAM-3048] Remove RAND_RANGE in WindowedWordCount ---
[GitHub] beam pull request #3944: [BEAM-3016] Fix blocking issue within run() when ch...
GitHub user lukecwik reopened a pull request: https://github.com/apache/beam/pull/3944 [BEAM-3016] Fix blocking issue within run() when channel terminates while blocking within DirectStreamObserver. Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam fn_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3944.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3944 commit 43964a933ae2d3ce72b7658dce885fde1c8e9865 Author: Luke Cwik <lc...@google.com> Date: 2017-10-06T16:09:17Z [BEAM-3016] Fix blocking issue within run() when channel terminates while blocking within DirectStreamObserver. ---
[GitHub] beam pull request #3944: [BEAM-3016] Fix blocking issue within run() when ch...
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3944 ---
[GitHub] beam pull request #3952: [BEAM-2926] Add initial support to be able to read ...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3952 [BEAM-2926] Add initial support to be able to read singleton and iterable side inputs to Java SDK harness. Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam fn_api2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3952.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3952 commit c49c564daa2a32d7930b819175e77d414e617419 Author: Luke Cwik <lc...@google.com> Date: 2017-10-05T23:54:19Z [BEAM-2926] Add initial support to be able to read singleton and iterable side inputs to Java SDK harness. ---
[GitHub] beam pull request #3944: [BEAM-3016] Fix blocking issue within run() when ch...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3944 [BEAM-3016] Fix blocking issue within run() when channel terminates while blocking within DirectStreamObserver. Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam fn_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3944.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3944 commit d4e6c4d8d055e395d28e9d9c9ad8740c82698082 Author: Luke Cwik <lc...@google.com> Date: 2017-10-04T23:34:54Z [BEAM-3016] Fix blocking issue within run() when channel terminates while blocking within DirectStreamObserver. ---
[GitHub] beam pull request #3931: [BEAM-3007] Add test which covers PCollection consu...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3931 [BEAM-3007] Add test which covers PCollection consumed by Flatten and another primitive. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam3007 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3931.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3931 commit e0e10874a067cc69b3cea7afa7f8d420783dd2cc Author: Luke Cwik <lc...@google.com> Date: 2017-10-02T18:44:22Z [BEAM-3007] Add test which covers PCollection consumed by Flatten and another primitive. ---
[GitHub] beam pull request #3913: [BEAM-2959] Encapsulate enums within a message so t...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3913 [BEAM-2959] Encapsulate enums within a message so that C++/Python have meaningful namespaces when importing. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam fn_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3913.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3913 commit 40c0a74e4a9426b79ecddc2f5d29cae41e0445ce Author: Luke Cwik <lc...@google.com> Date: 2017-09-27T20:33:16Z [BEAM-2959] Encapsulate enums within a message so that C++/Python have meaningful namespaces when importing. ---
[GitHub] beam pull request #3904: [BEAM-2959] Fix proto enums to use "YYY_UNSPECIFIED...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3904 [BEAM-2959] Fix proto enums to use "YYY_UNSPECIFIED" as the first declared enum. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam fn_api2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3904.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3904 commit a0c55c8e52af83cf1c7fdb98bb862387f791f0a9 Author: Luke Cwik <lc...@google.com> Date: 2017-09-15T18:23:19Z [BEAM-2959] Fix proto enums to use "YYY_UNSPECIFIED" as the first declared enum. ---
[GitHub] beam pull request #3856: [BEAM-2959] Fix proto enums to use "YYY_UNSPECIFIED...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3856 [BEAM-2959] Fix proto enums to use "YYY_UNSPECIFIED" as the first declared field. Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam fn_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3856.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3856 commit 2d2fb623d69f97108944948444dfc05c7af59ed3 Author: Luke Cwik <lc...@google.com> Date: 2017-09-15T18:23:19Z [BEAM-2959] Fix proto enums to use "YYY_UNSPECIFIED" as the first declared enum. ---
[GitHub] beam pull request #3849: [BEAM-2956] Attempt to correctly report the Dataflo...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3849 [BEAM-2956] Attempt to correctly report the Dataflow distribution in GCP related modules. Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3849.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3849 commit e9176e9939f0dd61789718203cca95ed760e0954 Author: Luke Cwik <lc...@google.com> Date: 2017-09-14T00:00:04Z [BEAM-2956] Attempt to correctly report the Dataflow distribution in GCP related modules. ---
[GitHub] beam pull request #3832: [BEAM-2862] Add PTransform overrides specific to ex...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3832 [BEAM-2862] Add PTransform overrides specific to execution of StatefulDoFns over the Fn API using the DataflowRunner. Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3832.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3832 commit 58c62b32f9e1ba186b53783e7c814ea5c0f85535 Author: Luke Cwik <lc...@google.com> Date: 2017-09-09T01:26:46Z [BEAM-2862] Add PTransform overrides specific to execution of StatefulDoFns over the Fn API using the DataflowRunner. ---
[GitHub] beam pull request #3824: Add ThrowingBiConsumer to the set of functional int...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3824 Add ThrowingBiConsumer to the set of functional interfaces Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3824.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3824 commit 60df3e9b18df0c384085efe11536fccb0e2e1728 Author: Luke Cwik <lc...@google.com> Date: 2017-09-08T16:41:27Z Add ThrowingBiConsumer to the set of functional interfaces ---
[GitHub] beam pull request #3795: [BEAM-1347] Wire up the BeamFnStateGrpcClientCache ...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3795 [BEAM-1347] Wire up the BeamFnStateGrpcClientCache implementation into the ProcessBundleHandler Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- Add a BeamFnStateClient that is dependent on whether the State API service descriptor is populated. You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3795.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3795 commit 9e2e999f6ee5e5ffa2e0fc4937d4f4b717f19259 Author: Luke Cwik <lc...@google.com> Date: 2017-08-30T20:56:40Z [BEAM-1347] Wire up the BeamFnStateGrpcClientCache implementation into the ProcessBundleHandler Add a BeamFnStateClient that is dependent on whether the State API service descriptor is populated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3788: [BEAM-1347] Implement a BeamFnStateClient which com...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3788 [BEAM-1347] Implement a BeamFnStateClient which communicates over gRPC. Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3788.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3788 commit 27ae6d75641eb7fdeeecd55bc913692dde630afb Author: Luke Cwik <lc...@google.com> Date: 2017-08-30T01:31:39Z [BEAM-1347] Implement a BeamFnStateClient which communicates over gRPC. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3783: [BEAM-1347] Create value state, combining state, an...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3783 [BEAM-1347] Create value state, combining state, and bag state views over the BagUserState. Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- Also bind the state persistence to the end of finishBundle. You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3783.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3783 commit 1edb7f160444fad08fc1a24689344d2ea76faad6 Author: Luke Cwik <lc...@google.com> Date: 2017-08-29T17:45:04Z [BEAM-1347] Create value state, combining state, and bag state views over the BagUserState. Also bind the state persistence to the end of finishBundle. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3766: Ensure Dataflow service version matches Google API ...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3766 Ensure Dataflow service version matches Google API service version 1.22.0 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3766.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3766 commit 4a2b45ff396c23a2a85e2bdafd82db9faa3501bf Author: Luke Cwik <lc...@google.com> Date: 2017-08-25T21:53:32Z Ensure Dataflow service version matches Google API service version 1.22.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3760: [BEAM-1347] Add a BagUserState implementation over ...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3760 [BEAM-1347] Add a BagUserState implementation over the BeamFnStateClient Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3760.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3760 commit 3e601f0d5a49b013ca83c63712fafe039d9ea7bc Author: Luke Cwik <lc...@google.com> Date: 2017-08-25T01:34:47Z [BEAM-1347] Add a BagUserState implementation over the BeamFnStateClient --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3761: [BEAM-1347] Remove unused FakeStepContext now that ...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3761 [BEAM-1347] Remove unused FakeStepContext now that FnApiDoFnRunner has its own implementation Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3761.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3761 commit ed8eb70c32a9cb101ef51f756e0da7aaf2e23669 Author: Luke Cwik <lc...@google.com> Date: 2017-08-25T01:31:50Z [BEAM-1347] Remove unused FakeStepContext now that FnApiDoFnRunner has its own implementation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3736: [BEAM-1347] Provide an abstraction which creates an...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3736 [BEAM-1347] Provide an abstraction which creates an Iterator view over the Beam Fn State API Combining this with the DataStreams.DataStreamDecoder converts the Beam Fn State API into a an input stream backed by multiple logical chunks. Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3736.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3736 commit 34f3646ec1c39a6fe0a3e8d4a2dfdcbc2b319827 Author: Luke Cwik <lc...@google.com> Date: 2017-08-17T18:49:35Z [BEAM-1347] Provide an abstraction which creates an Iterator view over the Beam Fn State API Combining this with the DataStreams.DataStreamDecoder converts the Beam Fn State API into a an input stream backed by multiple logical chunks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3724: [BEAM-1347] Convert an InputStream into an Iterable...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3724 [BEAM-1347] Convert an InputStream into an Iterable using the Beam Fn data specification Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- This is towards sharing common code that supports the Beam Fn State API and the Beam Fn Data API. You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3724.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3724 commit b20fea885cd9e2c98f13f4d024f780c312ff1d80 Author: Luke Cwik <lc...@google.com> Date: 2017-08-16T23:44:59Z [BEAM-1347] Convert an InputStream into an Iterable using the Beam Fn data specification This is towards sharing common code that supports the Beam Fn State API and the Beam Fn Data API. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3723: [BEAM-1347] Plumb through a yet to be created state...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3723 [BEAM-1347] Plumb through a yet to be created state client through PTransformRunnerFactory Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3723.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3723 commit 5d5f509747f1206cc19538d7fb1db2237fa6aeff Author: Luke Cwik <lc...@google.com> Date: 2017-07-07T21:02:58Z [BEAM-1347] Plumb through a yet to be created state client through PTransformRunnerFactory --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3638: [BEAM-1347] Add utility to be able to model inbound...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3638 [BEAM-1347] Add utility to be able to model inbound reading as a single input stream Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- This is towards having both Beam Fn State and Beam Fn Data APIs share code for consuming from a logical input stream. You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3638.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3638 commit 1709961e6cd8755ad8ee13d93ba3dc4a6c56d3d7 Author: Luke Cwik <lc...@google.com> Date: 2017-07-25T16:02:41Z [BEAM-1347] Add utility to be able to model inbound reading as a single input stream --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3622: [BEAM-2642] Update Google Auth to 0.7.1
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3622 [BEAM-2642] Update Google Auth to 0.7.1 Google auth libraries 0.6.1 and 0.7.1 have the same versions of dependencies. Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- This is a really annoying bug for Dataflow customers, if a 2.1.0 RC3 gets built. I would like to include this PR into it. You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam2642release Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3622.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3622 commit a98031c28c352cfc8c16f53fc7a58a7aed390b64 Author: Luke Cwik <lc...@google.com> Date: 2017-07-19T18:36:35Z [BEAM-2642] Update Google Auth to 0.7.1 Google auth libraries 0.6.1 and 0.7.1 have the same versions of dependencies. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3621: [BEAM-2662] Fix Java quickstart for Spark to includ...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3621 [BEAM-2662] Fix Java quickstart for Spark to include Spark version Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam2662release Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3621.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3621 commit 9ebda844f9366f8b242208b7bf0b95c68d0990b9 Author: Luke Cwik <lc...@google.com> Date: 2017-07-23T21:00:17Z [BEAM-2662] Fix Java quickstart for Spark to include Spark version --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3620: [BEAM-2662] Fix Java quickstart for Spark to includ...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3620 [BEAM-2662] Fix Java quickstart for Spark to include Spark version Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam2662 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3620.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3620 commit a7c18495f91c9d7537271c9595b3ac4d24003ece Author: Luke Cwik <lc...@google.com> Date: 2017-07-23T21:00:17Z [BEAM-2662] Fix Java quickstart for Spark to include Spark version --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3596: [BEAM-2642] Update Google Auth to 0.7.1
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3596 [BEAM-2642] Update Google Auth to 0.7.1 Google auth libraries 0.6.1 and 0.7.1 have the same versions of dependencies. Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam2642 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3596.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3596 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3495: [BEAM-2552] Update Dataflow Java container version ...
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3495 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3495: [BEAM-2552] Update Dataflow Java container version ...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3495 [BEAM-2552] Update Dataflow Java container version for 2.1.0 release Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam release Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3495.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3495 commit aea93fdd1b1a8770eff0159968ba7fc6c569a47b Author: Luke Cwik <lc...@google.com> Date: 2017-07-05T15:51:46Z [BEAM-2552] Update Dataflow container version for 2.1.0 release --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3494: [BEAM-2553] Update Maven exec plugin to 1.6.0 to in...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3494 [BEAM-2553] Update Maven exec plugin to 1.6.0 to incorporate messaging improvements Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam mvn_exec Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3494.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3494 commit 6ca410a908f1f4e7ac1e141ee1335f7a537bb150 Author: Luke Cwik <lc...@google.com> Date: 2017-07-05T17:38:44Z [BEAM-2553] Update Maven exec plugin to 1.6.0 to incorporate messaging improvements --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3482: [BEAM-1348] Remove deprecated concepts in Fn API (n...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3482 [BEAM-1348] Remove deprecated concepts in Fn API (now replaced with Runner API concepts). Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam fn_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3482.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3482 commit f1301bed0e1ef531c44c649745b76cd80c6cc6bf Author: Luke Cwik <lc...@google.com> Date: 2017-06-30T17:21:55Z [BEAM-1348] Remove deprecated concepts in Fn API (now replaced with Runner API concepts). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3432: [BEAM-1347] Create a DoFnRunner specific for the Fn...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3432 [BEAM-1347] Create a DoFnRunner specific for the Fn API Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- This allows me to work on the Fn State API without needing to use StateInternals which brings in a bunch of structure which is orthogonal to how stuff works with the Fn State API (specifically regarding caching and state keys). You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3432.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3432 commit 0825eb4a871616239dd4984b791f34731279049e Author: Luke Cwik <lc...@google.com> Date: 2017-06-23T21:31:58Z [BEAM-1347] Rename DoFnRunnerFactory to FnApiDoFnRunner. commit b7ebfca05bf4d8ccced64f9552f7b03d01689dc3 Author: Luke Cwik <lc...@google.com> Date: 2017-06-23T21:34:36Z [BEAM-1347] Add DoFnRunner specific to Fn Api. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3430: [BEAM-1187] Improve logging to contain the number o...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3430 [BEAM-1187] Improve logging to contain the number of retries done due to IOException and unsuccessful response codes. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam1187 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3430.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3430 commit f20a47c450f5a6b05cbb410506b233fee71dbb0c Author: Luke Cwik <lc...@google.com> Date: 2017-06-23T16:32:49Z [BEAM-1187] Improve logging to contain the number of retries done due to IOException and unsuccessful response codes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam-site pull request #257: [BEAM-2500] List Amazon S3 File System as a pla...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam-site/pull/257 [BEAM-2500] List Amazon S3 File System as a planned I/O. You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam-site asf-site Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam-site/pull/257.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #257 commit a1643b1360a3628e6fd6ed6812df5cfb360e8873 Author: Luke Cwik <lc...@google.com> Date: 2017-06-22T17:52:03Z [BEAM-2500] List Amazon S3 File System as a planned I/O. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3397: [BEAM-1348] Fix type error introduced into Python S...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3397 [BEAM-1348] Fix type error introduced into Python SDK because of PR/3268 Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3397.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3397 commit 595ca1ec84134328be5be3c8ae21a5a43a5a7166 Author: Luke Cwik <lc...@google.com> Date: 2017-06-19T22:48:06Z [BEAM-1348] Fix type error introduced into Python SDK because of PR/3268 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3375: [BEAM-1347] Break apart ProcessBundleHandler to use...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3375 [BEAM-1347] Break apart ProcessBundleHandler to use service locator pattern based upon URNs. This cleans up ProcessBundleHandler and allows for separate improvements of the various PTransform handler factories. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam_fn_api2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3375.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3375 commit c1006b18263296ec43b96167f3d4eaeb66048255 Author: Luke Cwik <lc...@google.com> Date: 2017-06-15T23:36:22Z [BEAM-1347] Break apart ProcessBundleHandler to use service locator pattern based upon URNs. This cleans up ProcessBundleHandler and allows for separate improvements of the various PTransform handler factories. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3366: [BEAM-1348] Mark Runner API like types declared wit...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3366 [BEAM-1348] Mark Runner API like types declared within Fn API as deprecated Add additional documentation. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam_fn_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3366.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3366 commit 74a738848a3ab089e347b859d0df6350c2daaf65 Author: Luke Cwik <lc...@google.com> Date: 2017-06-15T03:13:01Z [BEAM-1348] Mark Runner API like types declared within Fn API as deprecated. Add additional documentation. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3312: Swap to use an Impulse primitive + DoFn for Create ...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3312 Swap to use an Impulse primitive + DoFn for Create when executing with the Fn API. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam fn_api2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3312.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3312 commit 0d7cabef297bc340910e2cc1728d7b9743e3f44b Author: Luke Cwik <lc...@google.com> Date: 2017-06-07T15:03:43Z [BEAM-2421] Make ReflectiveRootOverride pass output PCollection to override implementations commit f3c3391f33c00ab925a37e66040b977b154e50b1 Author: Luke Cwik <lc...@google.com> Date: 2017-06-07T15:53:14Z [BEAM-2421] Swap to use an Impulse primitive + DoFn for Create when executing with the Fn API. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3297: [BEAM-1347] Migrate to Runner API constructs within...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3297 [BEAM-1347] Migrate to Runner API constructs within the Java SDK harness Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam fn_api2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3297.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3297 commit c5e90f1ba428b7ae35b5179616ab625aae6db91c Author: Luke Cwik <lc...@google.com> Date: 2017-06-05T18:01:54Z [BEAM-1347] Migrate to Runner API constructs within the Java SDK harness --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3296: [BEAM-1347] Add additional logging
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3296 [BEAM-1347] Add additional logging Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam fn_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3296.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3296 commit ae858a54357252a6b3487c5cf25f3fa0a492f77a Author: Luke Cwik <lc...@google.com> Date: 2017-06-05T16:28:17Z [BEAM-1347] Add additional logging --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3271: [BEAM-1348] Add Runner API constructs to ProcessBun...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3271 [BEAM-1348] Add Runner API constructs to ProcessBundleDescriptor. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- This is towards removing the Fn API constructs once code has been migrated. You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam migrate_to_runner_constructs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3271.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3271 commit d944a8684d1c47327e15dbdde15650d06db1ece5 Author: Luke Cwik <lc...@google.com> Date: 2017-05-31T17:10:06Z [BEAM-1348] Add Runner API constructs to ProcessBundleDescriptor. This is towards removing the Fn API constructs once code has been migrated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3268: [BEAM-1348] Model the Fn State Api as per https://s...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3268 [BEAM-1348] Model the Fn State Api as per https://s.apache.org/beam-fn-state-api-and-bundle-processing Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- These types/services are unimplemented and this change is towards prototyping the client/service using this definition. You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam state_api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3268.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3268 commit 21e4f3582e9dcf8f6ee2a6eac3201e8c0cb928ea Author: Luke Cwik <lc...@google.com> Date: 2017-05-31T15:54:16Z [BEAM-1348] Model the Fn State Api as per https://s.apache.org/beam-fn-state-api-and-bundle-processing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3260: [BEAM-1347] Remove the usage of a thread local on a...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3260 [BEAM-1347] Remove the usage of a thread local on a potentially hot path Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam remove_thread_local Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3260.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3260 commit 25478d15b8907c834ad1bfa21ce1d40af5c67435 Author: Luke Cwik <lc...@google.com> Date: 2017-05-30T20:15:31Z [BEAM-1347] Remove the usage of a thread local on a potentially hot path --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3223: [BEAM-2354] Add a ReadStringsFromPubSub/WriteString...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3223 [BEAM-2354] Add a ReadStringsFromPubSub/WriteStringsToPubSub PTransform Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- I hid existing PubSubSource/PubSubSink which is backwards incompatible but the PTransform didn't work anyways. You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam pubsub Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3223.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3223 commit 6d52705a1406db3965024203abb9cc9d897cff95 Author: Luke Cwik <lc...@google.com> Date: 2017-05-25T01:14:59Z [BEAM-2354] Add a ReadStringsFromPubSub/WriteStringsToPubSub PTransform --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3212: [BEAM-2354] Enable a native pubsub source within th...
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3212 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3212: [BEAM-2354] Enable a native pubsub source within th...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3212 [BEAM-2354] Enable a native pubsub source within the Dataflow Runner Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam pubsub Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3212.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3212 commit 7b46aee52f3ef8f3b11dff9eb2345e0bf65fdb3d Author: Luke Cwik <lc...@google.com> Date: 2017-05-23T21:39:41Z [BEAM-2354] Enable a native pubsub source within the Dataflow Runner --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3159: [BEAM-2252] Use resource transformer plugin to use ...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3159 [BEAM-2252] Use resource transformer plugin to use versions defined in root pom.xml within maven archetype poms Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [X] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [X] Make sure tests pass via `mvn clean verify`. - [X] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam2253 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3159.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3159 commit d90fe13c0da4f34860f3574ab1d586a4fb418417 Author: Luke Cwik <lc...@google.com> Date: 2017-05-15T22:43:00Z [BEAM-2253] Use resource transformer plugin to use versions defined in root pom.xml within maven archetype poms --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3152: [BEAM-2252] Use pluginManagement instead of redefin...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3152 [BEAM-2252] Use pluginManagement instead of redefining maven-shade-plugin to inherit build order from root pom.xml Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam2252 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3152.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3152 commit ca62f3311d6a622ce3895a716da97187bbd0709a Author: Luke Cwik <lc...@google.com> Date: 2017-05-15T20:39:12Z [BEAM-2252] Use pluginManagement instead of redefining maven-shade-plugin to inherit build order from root pom.xml --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3143: [BEAM-2299] Run maven install on Windows machine fo...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3143 [BEAM-2299] Run maven install on Windows machine for build/test coverage on Windows. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam windows Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3143.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3143 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3134: [BEAM-1345] Remove FileSystems.setDefaultConfigInWo...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3134 [BEAM-1345] Remove FileSystems.setDefaultConfigInWorkers since Dataflow no longer depends on this Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam cleanup Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3134.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3134 commit eb6d85808f748b40d15e4111e503ce1d82077892 Author: Luke Cwik <lc...@google.com> Date: 2017-05-12T23:44:10Z [BEAM-1345] Remove FileSystems.setDefaultConfigInWorkers since Dataflow no longer depends on this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3120: Cherry pick #2854: [BEAM-2095] Made SourceRDD.Bound...
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3120 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3120: Cherry pick #2854: [BEAM-2095] Made SourceRDD.Bound...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3120 Cherry pick #2854: [BEAM-2095] Made SourceRDD.Bounded iterator's hasNext idempotent Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam cp2854 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3120.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3120 commit 538c6bf0d4cad186eed575e686918775fd50d9e6 Author: Stas Levin <stasle...@apache.org> Date: 2017-05-01T04:30:49Z [BEAM-2095] Made SourceRDD hasNext idempotent --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3119: Cherry pick #2854: [BEAM-2095] Made SourceRDD.Bound...
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3119 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3119: Cherry pick #2854: [BEAM-2095] Made SourceRDD.Bound...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3119 Cherry pick #2854: [BEAM-2095] Made SourceRDD.Bounded iterator's hasNext idempotent Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam cp2854 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3119.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3119 commit 56262d8f3064031606e01801eb36322cb3a38b95 Author: Davor Bonaci <da...@google.com> Date: 2017-05-05T22:53:46Z Update version number from 0.7.0-SNAPSHOT to 2.0.0-SNAPSHOT commit 1a77e208e440ef34b7d0f7e1104d5c8c5ee04474 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-05T23:32:54Z This cherry-picks #2926 commit b9c8cfe227d7a6bcb258b93717969a78a31dac07 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-06T00:24:12Z This closes #2932 commit d2fa51b78892f7ebf13da1a5fc7bb45755440a5f Author: Davor Bonaci <da...@google.com> Date: 2017-05-05T23:17:43Z Cherry-pick pull request #2911 into release-2.0.0 commit 67ea7ae4d2144525582c7de03b17d06daa9f35bb Author: Davor Bonaci <da...@google.com> Date: 2017-05-05T23:20:28Z Cherry-pick pull request #2907 into release-2.0.0 branch commit 96aeb97cc41b4a93bec7d72cff4887e9f358eef2 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-06T00:26:41Z This closes #2931 commit 1ad3f84c68235eeae6927f283180829de2f0aa33 Author: Davor Bonaci <da...@google.com> Date: 2017-05-06T00:36:09Z Set Dataflow runner's worker container image for version 2.0.0 commit f97e52b3b677a5a35bac7a2012366837b7bb15cb Author: Davor Bonaci <da...@google.com> Date: 2017-05-06T01:46:31Z [maven-release-plugin] prepare release v2.0.0-RC1 commit 3b7a62301fba55f3caf68d8876b39c53e80f171a Author: Davor Bonaci <da...@google.com> Date: 2017-05-07T02:16:30Z [maven-release-plugin] rollback changes from release preparation of v2.0.0-RC1 commit a71f6cd481800a977022ce8cddce5ef9640bbe4e Author: Kenneth Knowles <k...@google.com> Date: 2017-05-06T12:13:34Z Register the TestDataflowRunner via registrar commit d305d6d382179c2eadd77dac3c603d73f65c80e4 Author: Kenneth Knowles <k...@google.com> Date: 2017-05-06T12:22:03Z Move TestDataflowRunner into dataflow package commit 6eab5c9465bda3da4d8a1ea9f73a74e9c8faec85 Author: chinmaykolhatkar <chin...@apache.org> Date: 2017-03-01T11:29:46Z [BEAM-831] ParDo Fusion of Apex Runner commit 3f5282d515fa53516fda6d0376cc912560fd6d85 Author: Thomas Weise <t...@apache.org> Date: 2017-05-05T13:45:34Z [BEAM-831] Fix chaining, add test. closes #2216 commit bec30b3beaa241483814e859d781f1e04479394b Author: Ahmet Altay <al...@google.com> Date: 2017-05-08T06:31:42Z Cherry-pick pull request #2946 in 2.0.0 release branch. Fix typo in datastore_wordcount.py. commit 021468e03e5a5b0851e21f333ebc07060dc471cd Author: Ahmet Altay <al...@google.com> Date: 2017-05-08T17:25:40Z This closes #2955 commit 72241117cbf2d9682054a69ea895e4c6f6a93146 Author: Sourabh Bajaj <sourabhba...@google.com> Date: 2017-05-07T20:55:20Z [BEAM-2206] Move pipelineOptions into options modules commit c4f234c8cfb349d877eeb5c62eec7d80e844be07 Author: Sourabh Bajaj <sourabhba...@google.com> Date: 2017-05-08T01:08:49Z Only cythonize files within apache_beam commit 741bf7442d88a9e30064bd132046e5db55e7a740 Author: Ahmet Altay <al...@google.com> Date: 2017-05-08T21:02:54Z This closes #2964 commit 25cda3abb0f6442f02ab6f31f0a4850be66d09d9 Author: Sourabh Bajaj <sourabhba...@google.com> Date: 2017-05-06T02:15:48Z Update python dataflow worker commit 265405bc85f9f705776e88680e3af26fab4e7de3 Author: Ahmet Altay <al...@google.com> Date: 2017-05-08T21:06:17Z This closes #2943 commit e0faeeef80211ddbc632e622ecefc1e005c5ca29 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-06T02:06:03Z [BEAM-2212] FileBasedSource: improve message when logging. ValueProvider should not be printed, rather the string instead. commit bff819a9858c79c6c3232b4c03f262421d325c00 Author: Dan Halperin <dhalp...@google.com> Date:
[GitHub] beam pull request #3100: Cherry pick #3093: [BEAM-2256] Add the last previou...
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3100 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3100: Cherry pick #3093: [BEAM-2256] Add the last previou...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3100 Cherry pick #3093: [BEAM-2256] Add the last previous range filter Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam2256 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3100.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3100 commit 1ba1e031fe7494b94b10674bfb07b975894fde13 Author: Jean-Baptiste Onofré <jbono...@apache.org> Date: 2017-05-11T20:09:50Z [BEAM-2256] Add the last previous range filter --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3066: Cherry-pick #3062: Skip generating empty jars for p...
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3066 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3066: Cherry-pick #3063: Skip generating empty jars for p...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3066 Cherry-pick #3063: Skip generating empty jars for parent poms Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam pr3063 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3066.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3066 commit 194b99f3693e0a6f8dc9686cdd1f5d534300563b Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-05-10T21:29:45Z Skip generating empty jars for parent poms --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3061: Cherry-pick #3034: [BEAM-2242] Ensure that jars are...
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3061 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3061: Cherry-pick #3034: [BEAM-2242] Ensure that jars are...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3061 Cherry-pick #3034: [BEAM-2242] Ensure that jars are shaded correctly by running the jar plugin before the shade plugin Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam2242 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3061.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3061 commit 4324d9d99bd3444ca0f9fbde48b13ca214b7c318 Author: Luke Cwik <lc...@google.com> Date: 2017-05-10T02:07:36Z [BEAM-2242] Ensure that jars are shaded correctly by running the jar plugin before the shade plugin This required listing the maven jar plugin earlier within the file so that the jar/test jar ran before shade within the package phase. I also update the maven shade plugin version in archetypes to pickup the fixes in the service file transformer and dropped the usage of finalName in the shade plugin so all artifacts are named the same way. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3034: [BEAM-2242] Ensure that the test jar is shaded corr...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3034 [BEAM-2242] Ensure that the test jar is shaded correctly This required listing the maven jar plugin earlier within the file so that the jar/test jar ran before shade within the package phase. I also update the maven shade plugin version in archetypes to pickup the fixes in the service file transformer and dropped the usage of finalName in the shade plugin so all artifacts are named the same way. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam beam2242 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3034.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3034 commit ea938d06f7f431c94d5635e9b1b885283c5cf604 Author: Luke Cwik <lc...@google.com> Date: 2017-05-10T02:07:36Z [BEAM-2242] Ensure that the test jar is shaded correctly This required listing the maven jar plugin earlier within the file so that the jar/test jar ran before shade within the package phase. I also update the maven shade plugin version in archetypes to pickup the fixes in the service file transformer and dropped the usage of finalName in the shade plugin so all artifacts are named the same way. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2980: PR to run tests for #2933 using updated Dataflow wo...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2980 PR to run tests for #2933 using updated Dataflow worker image Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam finish-pr-2933 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2980.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2980 commit 71ea42f5d0bdc1335bf5b9e0cb0b7fcf7db439c2 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-05T23:20:37Z Remove explicit used of nested contexts. find . -type f -name '*.java' | xargs sed -i '' 's/\([.]..code[(].*\), *context.nested..[)]/\1)/' find . -type f -name '*.java' | xargs sed -i '' 's/\([.]..code[(].*\), *nestedContext[)]/\1)/' find . -type f -name '*.java' | xargs sed -i '' 's/\([.]..code[(].*\), *Context.NESTED[)]/\1)/' find . -type f -name '*.java' | xargs sed -i '' 's/\([.]..code[(].*\), *[^ ]*.Context.NESTED[)]/\1)/' Added back explicit context in CoGbkResult.java due to compile error. commit 1f930b7888a923e3d8b7fbb42d26484f6a711123 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-05T23:36:47Z Remove contexts from coders where they'll never be used. commit 98ec267b5a0e95eeb2c27b4969ffec03e0e360a8 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-06T00:24:02Z automated context removal or redirection commit fb50c3ff26db4ecac8abaebaffc407651b294ac9 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-06T00:27:13Z get it compiling commit 44ee9b87b603a1905e136cf7356453e3d26ed5ca Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-06T00:35:35Z Remove en/decodeOuter and default encode/decode methods. Now only the context-free encode() and decode() methods are abstract. commit 7b3f2914a4195052d7e9d68e4e6c82e4ce0d8dc2 Author: Lukasz Cwik <lc...@google.com> Date: 2017-05-08T02:41:07Z fixup! Swap to use encode/decode in anonymous inner class coder and @AutoValue coder commit 0c502352cf5d2f52f454de30dbeac2f483dbdd2d Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-08T17:48:40Z Reviewer comments + a couple of extra fixes. All compiles. commit 0acdef7dd0417e613284f62a824c2e11b51ce999 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-08T18:27:40Z lint error commit 52d594ec7650899f3aa3064a5c698a3378052a65 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-08T18:54:48Z checkstyle commit 6b81523e392af31829ae12f28a1ae0b1aba8d550 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-08T21:56:25Z Use latest dataflow worker. commit ac626f04cd8ce5aa0862723cd208b5e7d017aaa8 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-08T22:04:36Z Explicitly mark Coder context as experimental as well as deprecated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2941: PR to run tests for #2930 on Jenkins
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/2941 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2941: PR to run tests for #2930 on Jenkins
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2941 PR to run tests for #2930 on Jenkins Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam finish-pr-2930 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2941.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2941 commit 78a99be1c46656996081744e52dac3d619cb4fe7 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-05T22:18:52Z Remove context from size estimate operations. commit 1f59e0a99f86c5b2d78adaebda1818f270cba775 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-05T22:23:03Z Remove uses of context in coder size estimation calls. find . -type f -name '*.java' | xargs sed -i '' 's/\([a-zA-Z]*[bB]yteSize[a-zA-Z]*[(].*\), [^,]*[Cc]ontext[^,()]*\([(][)]\)*[)]/\1)/' plus a one-off Python script and some manual fixups. commit 0593da952214f581dbe0c816d12e39fd9ca73c54 Author: Luke Cwik <lc...@google.com> Date: 2017-05-07T02:23:20Z fixup! Fix test issues with PR 2930 and migrate to new Dataflow worker image --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2929: [BEAM-2187] Fix build break in SparkRuntimeContextT...
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/2929 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2929: [BEAM-2187] Fix build break in SparkRuntimeContextT...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2929 [BEAM-2187] Fix build break in SparkRuntimeContextTest Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2929.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2929 commit 56a83c73b9b7ab0760c421cf154caff16953500a Author: Luke Cwik <lc...@google.com> Date: 2017-05-05T22:09:26Z [BEAM-2187] Fix build break in SparkRuntimeContextTest --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2910: [BEAM-2174] Update CoderRegistry to allow creating ...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2910 [BEAM-2174] Update CoderRegistry to allow creating coders through CoderFactory for a wider range of types Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- This removes getInstanceComponents and hard codes the inference for commonly "known" types into Create removing the additionally complexity that was brought along with it. Also: * @DefaultCoder to become a CoderFactory * Created a ProtoCoderFactory which delegates to the ProtoCoder for all Message types * Created a SerializableCoderFactory which delegates to the SerializableCoder for all Serializable types * Created a WritableCoderFactory which delegates to the Hadoop WritableCoder for all Hadoop Writable types * Renamed getDefaultCoder to getCoder on CoderRegistry You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam coder_registrar Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2910.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2910 commit 917cc45411c602143450e109390b6a7750fc2f06 Author: Luke Cwik <lc...@google.com> Date: 2017-05-05T02:55:47Z [BEAM-2174] Update CoderRegistry to allow creating coders through CoderFactory for a wider range of types --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2905: [BEAM-1871] Move out test only utilities used only ...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2905 [BEAM-1871] Move out test only utilities used only in examples/java out of sdks/java/core. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- Also removes some minor usages on google.api.client.util package from sdks/java/core You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam thin_sdk_core Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2905.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2905 commit e446a21a021a3d2f837d7138bd1870892f3f50aa Author: Luke Cwik <lc...@google.com> Date: 2017-05-04T23:14:34Z [BEAM-1871] Move out test only utilities used only in examples/java out of sdks/java/core. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2881: [BEAM-2135] Update Dataflow to support serializing/...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2881 [BEAM-2135] Update Dataflow to support serializing/deserializing custom user types configured via Jackson modules This also updates the runner harness and existing tests to use a properly constructed ObjectMapper for PipelineOptions. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam dataflow_custom_user_options Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2881.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2881 commit 7deb633f94b97fab1461772ebba3608404f46d17 Author: Luke Cwik <lc...@google.com> Date: 2017-05-04T00:19:00Z [BEAM-2135] Update Dataflow to support serializing/deserializing custom user types configured via Jackson modules This also updates the runner harness and existing tests to use a properly constructed ObjectMapper for PipelineOptions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2880: [BEAM-2165] Update Apex to support serializing/dese...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2880 [BEAM-2165] Update Apex to support serializing/deserializing custom user types configured via Jackson modules Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam apex_custom_user_options Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2880.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2880 commit 03dc5f3c355ffe91b61fb1e86bc2faab40dce7ff Author: Luke Cwik <lc...@google.com> Date: 2017-05-04T00:17:11Z [BEAM-2165] Update Apex to support serializing/deserializing custom user types configured via Jackson modules --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2879: [BEAM-2165] Update Flink to support serializing/des...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2879 [BEAM-2165] Update Flink to support serializing/deserializing custom user types configured via Jackson modules Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam flink_custom_user_options Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2879.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2879 commit 81c3f3cab7b897ed7aec384318e79ec638008f6c Author: Luke Cwik <lc...@google.com> Date: 2017-05-04T00:12:20Z [BEAM-2165] Update Flink to support serializing/deserializing custom user types configured via Jackson modules --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2877: [BEAM-2135] Update Spark to support serializing/des...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2877 [BEAM-2135] Update Spark to support serializing/deserializing custom user types configured via Jackson modules Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam spark_custom_user_options Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2877.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2877 commit 289cb3fc1cc5d0b885c803637e4c8bbbfd68ad27 Author: Luke Cwik <lc...@google.com> Date: 2017-05-04T00:15:52Z [BEAM-2135] Update Spark to support serializing/deserializing custom user types configured via Jackson modules --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2862: [BEAM-1871] Move GCP specific serialization CloudOb...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2862 [BEAM-1871] Move GCP specific serialization CloudObject and supporting translation code to Dataflow runner module Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam thin_sdk_core Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2862.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2862 commit c85b29f7b2c6d87dbddf5ba8d2c23db2d73bf2b3 Author: Luke Cwik <lc...@google.com> Date: 2017-05-03T17:03:45Z [BEAM-1871] Move GCP specific serialization CloudObject and supporting translation code to Dataflow runner module --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2844: [BEAM-2135] Fix pointers to sdks/java/io/hadoop-fil...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2844 [BEAM-2135] Fix pointers to sdks/java/io/hadoop-file-system Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam hdfs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2844.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2844 commit 6a1a40099e18e8191a5b042a530bc300ed6d2528 Author: Luke Cwik <lc...@google.com> Date: 2017-05-02T21:03:57Z [BEAM-2135] Fix pointers to sdks/java/io/hadoop-file-system --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2827: [BEAM-2135] Move gcp-core to google-cloud-platform-...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2827 [BEAM-2135] Move gcp-core to google-cloud-platform-core Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2827.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2827 commit a4ebe994865784012c063ca82df1b273b9163094 Author: Luke Cwik <lc...@google.com> Date: 2017-05-02T16:11:32Z [BEAM-2135] Move gcp-core to google-cloud-platform-core --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2825: Remove Coder.asCloudObject and related methods
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2825 Remove Coder.asCloudObject and related methods Rebase #2806 on top of apache/master If this passes, will merge in #2806 You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam finish-pr-2806 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2825.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2825 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2804: [BEAM-1871] Remove deprecated org.apache.beam.sdk.o...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2804 [BEAM-1871] Remove deprecated org.apache.beam.sdk.options.GcsOptions after updating Dataflow worker Worker has been migrated to org.apache.beam.sdk.extensions.gcp.options.GcsOptions Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam thin_sdk_core Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2804.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2804 commit d739cbdbe3b73445c0491d9d10338dbb88ad7166 Author: Luke Cwik <lc...@google.com> Date: 2017-05-01T20:07:11Z [BEAM-1871] Remove deprecated org.apache.beam.sdk.options.GcsOptions after updating Dataflow worker Worker has been migrated to org.apache.beam.sdk.extensions.gcp.options.GcsOptions --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2803: [BEAM-1871] Hide CloudResourceManager client in sdk...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2803 [BEAM-1871] Hide CloudResourceManager client in sdks/java/extensions/gcp-core Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam thin_sdk_core2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2803.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2803 commit 2390374a961f41adb0569717203f160daa85f5a9 Author: Luke Cwik <lc...@google.com> Date: 2017-05-01T20:30:36Z [BEAM-1871] Hide CloudResourceManager client in sdks/java/extensions/gcp-core --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2794: [BEAM-2130] Ensure the options id is never null.
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2794 [BEAM-2130] Ensure the options id is never null. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam options_id Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2794.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2794 commit 43b8dabcd249a83d69a072898ee5c812ceeb03c7 Author: Lukasz Cwik <lc...@google.com> Date: 2017-04-30T23:43:33Z [BEAM-2130] Ensure the options id is never null. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2777: [BEAM-2005, BEAM-2030, BEAM-2031, BEAM-2032, BEAM-2...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2777 [BEAM-2005, BEAM-2030, BEAM-2031, BEAM-2032, BEAM-2033, BEAM-2070] Base implementation of HadoopFileSystem. TODO: * Add multiplexing FileSystem that is able to route requests based upon the base URI when configured for multiple file systems. * Take a look at copy/rename again to see if we can do better than moving all the bytes through the local machine. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam hdfs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2777.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2777 commit c792da69467ccb0a486fe74ff7df0e5e9cb54abf Author: Luke Cwik <lc...@google.com> Date: 2017-04-29T02:07:59Z [BEAM-2005, BEAM-2030, BEAM-2031, BEAM-2032, BEAM-2033, BEAM-2070] Base implementation of HadoopFileSystem. TODO: * Add multiplexing FileSystem that is able to route requests based upon the base URI when configured for multiple file systems. * Take a look at copy/rename again to see if we can do better than moving all the bytes through the local machine. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2776: [BEAM-2005, BEAM-2030, BEAM-2031, BEAM-2032, BEAM-2...
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/2776 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2776: [BEAM-2005, BEAM-2030, BEAM-2031, BEAM-2032, BEAM-2...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2776 [BEAM-2005, BEAM-2030, BEAM-2031, BEAM-2032, BEAM-2033, BEAM-2070] Base implementation of HadoopFileSystem. TODO: * Add multiplexing FileSystem that is able to route requests based upon the base URI when configured for multiple file systems. * Take a look at copy/rename again to see if we can do better than moving all the bytes through the local machine. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam hdfs2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2776.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2776 commit ecfb534c79cb38bdf378dffe25d6a7ee3e20e5c6 Author: Luke Cwik <lc...@google.com> Date: 2017-04-29T01:37:03Z [BEAM-2005, BEAM-2030, BEAM-2031, BEAM-2032, BEAM-2033, BEAM-2070] Base implementation of HDFS. TODO: * Add multiplexing FileSystem that is able to route requests based upon the base URI when configured for multiple file systems. * Take a look at copy/rename again to see if we can do better than moving all the bytes through the local machine. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2772: [BEAM-2005] Swap to use Lists within MatchResult in...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2772 [BEAM-2005] Swap to use Lists within MatchResult instead of arrays. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- Everywhere in our codebase we go through and want to have lists as the return type. This also AutoValue's them. You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam hdfs2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2772.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2772 commit f9fe921cb64a19a092590644e1d94a4b202db3c9 Author: Luke Cwik <lc...@google.com> Date: 2017-04-28T23:27:53Z [BEAM-2005] Swap to use Lists within MatchResult instead of arrays. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2765: [BEAM-2119] Fix FileSystems to have the local files...
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2765 [BEAM-2119] Fix FileSystems to have the local filesystem installed by default. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2765.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2765 commit 3423bfd48a3f44e0b58eaef870b6adae45db3af6 Author: Luke Cwik <lc...@google.com> Date: 2017-04-28T17:55:52Z [BEAM-2119] Fix FileSystems to have the local filesystem installed by default. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
