[GitHub] beam pull request #4122: [BEAM-3172] Fix flakiness in sideinputs_test
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/4122 [BEAM-3172] Fix flakiness in sideinputs_test This change also does a slight clean-up of the exception handling logic. R: @chamikaramj You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam sideinputs-test Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4122.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4122 ---
[GitHub] beam pull request #4114: [BEAM-3172] CP PR 4113: Properly handle side input ...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/4114 [BEAM-3172] CP PR 4113: Properly handle side input exception when all reader threads complete R: @reuvenlax CC: @chamikaramj You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam sideinputs-cp Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4114.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4114 commit 8b6909e231289143c121bce5b38de76f82f56c32 Author: Charles Chen <c...@google.com> Date: 2017-11-10T19:28:43Z CP PR 4113: Properly handle side input exception when all reader threads complete ---
[GitHub] beam pull request #4112: [BEAM-3172] Properly handle side input exception wh...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/4112 [BEAM-3172] Properly handle side input exception when all reader threads complete R: @chamikaramj You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam side-input-exception Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4112.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4112 commit 8afe87c9a7eda6ac7b9e51e9d4e2919e5a1dd9f3 Author: Charles Chen <c...@google.com> Date: 2017-11-10T19:28:43Z Properly handle side input exception when all reader threads complete ---
[GitHub] beam pull request #3975: Do not merge: scratch work 3
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3975 Do not merge: scratch work 3 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam streaming-test Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3975.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3975 commit c2b3f3222baf5dd7b1f107dc0bfcc7f8b5d49111 Author: Charles Chen <c...@google.com> Date: 2017-04-21T05:51:27Z Don't buffer elements in GroupByKeyOnly. commit 3383c260855150dc9f0182a15f37136ea1b1c93c Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:50:02Z streaming test commit 038f3f33297421237814124887564c6f2620711a Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:53:09Z Working TestStream evaluator commit fc661f6e22fb2d4b648af345527ed67a32776754 Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:54:09Z Slightly better watermark propagation, debugging print cleanup. commit 053c7ae370df66d224469129f91fe8028ff3bcf4 Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:55:42Z Factor out GroupAlsoByWindows so that it can be overridden by the DirectRunner. commit 8854c53c1bc75631bd3b09b3f7ed6412eb93f844 Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:56:53Z Skeleton implementation of GroupAlsoByKeyEvaluator. commit ae0d80d0b945e8c50426696495e893565dc7db4a Author: Charles Chen <c...@google.com> Date: 2017-04-24T15:16:05Z Rename old non-keyed state to legacy_state, introduce KeyedWorkItem, prepare to expose DirectUnmergedState API. commit 8be01a96055fb92013f6fc5aed8edc2af6e75e42 Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:57:30Z Mostly working streaming GroupAlsoByWindow. commit bf38fe7afd219144e068d5dbd7035449f83bb723 Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:58:08Z Introduce keyed bundles, KeyedWorkItems. commit 97de55600417145702576424dc7512de3f981838 Author: Charles Chen <c...@google.com> Date: 2017-05-24T22:01:59Z Fix rebase to head after Beam 2.0 release. commit 341fa9245e5d67cc5fab4244bfeca5ea49c72231 Author: Charles Chen <c...@google.com> Date: 2017-05-25T21:22:03Z Clarify transform_keyed_states. commit 1e83d5e900c7039ccf89d989081ae50b3b3e510f Author: Charles Chen <c...@google.com> Date: 2017-05-26T00:38:31Z Support timer firings. commit 256309f2f3b01b230b258b8001a494d75e3e1713 Author: Charles Chen <c...@google.com> Date: 2017-05-26T01:08:44Z Reintroduce hacks from head. commit 25ce9f18a124e8d0aa80e05a516699da6a067c19 Author: Charles Chen <c...@google.com> Date: 2017-05-26T21:12:59Z Add debugging output. commit 7be85c67c5c405c16c146e5605dd2a5bb9bd3bdd Author: Charles Chen <c...@google.com> Date: 2017-05-26T22:56:32Z Don't hold input watermark on pending elements. commit 1df07a5b46811d0d3a383b156121d77f5f37aa57 Author: Charles Chen <c...@google.com> Date: 2017-05-30T18:26:03Z Finish moving earliest hold extraction to evaluators, dbg output++ commit e52ba42a9efe0961ed57da52e93fdd8e5970eedc Author: Charles Chen <c...@google.com> Date: 2017-05-31T22:46:06Z Batch WordCount works in streaming mode!! Fixes earliest hold to be < instead of <=. commit e5bb716e00ac02679edc3a357153009b862bb4d0 Author: Charles Chen <c...@google.com> Date: 2017-06-05T22:20:37Z Working batch and streaming mode execution, except for certain empty side inputs. commit 91873cc1422d5bc7812464feea6789a1c4242577 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-06-07T15:48
[GitHub] beam pull request #3874: Update Beam Python version for 2.1.1 release
Github user charlesccychen closed the pull request at: https://github.com/apache/beam/pull/3874 ---
[GitHub] beam pull request #3875: Fix Jenkins presubmits by reverting #3255
GitHub user charlesccychen opened a pull request:
https://github.com/apache/beam/pull/3875
Fix Jenkins presubmits by reverting #3255
Currently, presubmits at head are broken with the following error:
```
[EnvInject] - Variables injected successfully.
Parsing POMs
Downloaded artifact
http://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed:
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_MavenInstall@2 at
hudson.remoting.Channel@7b1b4155:beam7: hudson.remoting.ProxyException:
hudson.maven.MavenModuleSetBuild$MavenExecutionException:
org.apache.maven.project.ProjectBuildingException: Some problems were
encountered while processing the POMs:
[ERROR] 'dependencies.dependency.version' for
org.apache.beam:beam-runners-flink_2.10:jar is missing. @ line 68, column 21
[WARNING] 'artifactId' contains an expression but should be a constant. @
org.apache.beam:beam-runners-flink_${flink.scala.version}:2.2.0-SNAPSHOT,
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_MavenInstall@2/runners/flink/pom.xml,
line 29, column 15
at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:985)
at
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:690)
at
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
at hudson.model.Run.execute(Run.java:1735)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:542)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
Caused by: hudson.remoting.ProxyException:
hudson.maven.MavenModuleSetBuild$MavenExecutionException:
org.apache.maven.project.ProjectBuildingException: Some problems were
encountered while processing the POMs:
[...]
```
https://builds.apache.org/job/beam_PreCommit_Java_MavenInstall/14503/console
This change reverts the suspected cause #3255 to unbreak development at
head.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/charlesccychen/beam fix-master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/3875.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3875
commit 66dca6c33d5a4100a4aa4af1101d74902e347671
Author: Charles Chen <c...@google.com>
Date: 2017-09-20T22:17:34Z
Revert "[BEAM-2377] Allow cross compilation (2.10,2.11) for flink runner"
This reverts commit ab975317e1aa532053b68ccc105e13afff0c0b1a.
---
[GitHub] beam pull request #3874: Update Beam Python version for 2.1.1 release
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3874 Update Beam Python version for 2.1.1 release R: @robertwb You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam update-211-version Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3874.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3874 commit 9b99e4c89a980f4cd81b3713f9ba2a2e9072772b Author: Charles Chen <c...@google.com> Date: 2017-09-20T21:57:40Z Update Beam Python version to 2.1.1 ---
[GitHub] beam pull request #3872: Exclude incompatible six release, part 2
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3872 Exclude incompatible six release, part 2 R: @robertwb You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-six-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3872.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3872 commit cb7e2a75d731334c445149df5bcdf10da4462005 Author: Charles Chen <c...@google.com> Date: 2017-09-20T19:26:39Z Exclude incompatible six release, part 2 ---
[GitHub] beam pull request #3742: Avoid passing extra argument to object() constructo...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3742 Avoid passing extra argument to object() constructor in GcsIO This change fixes the following warning in gcsio_test: ``` apache_beam/io/gcp/gcsio.py:108: DeprecationWarning: object() takes no parameters return super(GcsIO, cls).__new__(cls, storage_client) ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-gcsio-super-constructor Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3742.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3742 commit 848be78875075e113a3bef8430890e5c9f696422 Author: Charles Chen <c...@google.com> Date: 2017-08-21T22:42:59Z Avoid passing extra argument to object() constructor in GcsIO --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3741: [BEAM-2785] Allow termination of DirectRunner execu...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3741 [BEAM-2785] Allow termination of DirectRunner execution with Ctrl-C R: @aaltay You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam directrunner-ctrl-c Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3741.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3741 commit de71861122a076dd483ad8b051cf550c7c58f1f3 Author: Charles Chen <c...@google.com> Date: 2017-08-21T21:18:45Z Allow termination of DirectRunner execution with Ctrl-C --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3646: Do not merge: scratch work 2
Github user charlesccychen closed the pull request at: https://github.com/apache/beam/pull/3646 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3646: Do not merge: scratch work 2
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3646 Do not merge: scratch work 2 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam streaming-test Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3646.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3646 commit c2b3f3222baf5dd7b1f107dc0bfcc7f8b5d49111 Author: Charles Chen <c...@google.com> Date: 2017-04-21T05:51:27Z Don't buffer elements in GroupByKeyOnly. commit 3383c260855150dc9f0182a15f37136ea1b1c93c Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:50:02Z streaming test commit 038f3f33297421237814124887564c6f2620711a Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:53:09Z Working TestStream evaluator commit fc661f6e22fb2d4b648af345527ed67a32776754 Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:54:09Z Slightly better watermark propagation, debugging print cleanup. commit 053c7ae370df66d224469129f91fe8028ff3bcf4 Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:55:42Z Factor out GroupAlsoByWindows so that it can be overridden by the DirectRunner. commit 8854c53c1bc75631bd3b09b3f7ed6412eb93f844 Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:56:53Z Skeleton implementation of GroupAlsoByKeyEvaluator. commit ae0d80d0b945e8c50426696495e893565dc7db4a Author: Charles Chen <c...@google.com> Date: 2017-04-24T15:16:05Z Rename old non-keyed state to legacy_state, introduce KeyedWorkItem, prepare to expose DirectUnmergedState API. commit 8be01a96055fb92013f6fc5aed8edc2af6e75e42 Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:57:30Z Mostly working streaming GroupAlsoByWindow. commit bf38fe7afd219144e068d5dbd7035449f83bb723 Author: Charles Chen <c...@google.com> Date: 2017-05-24T21:58:08Z Introduce keyed bundles, KeyedWorkItems. commit 97de55600417145702576424dc7512de3f981838 Author: Charles Chen <c...@google.com> Date: 2017-05-24T22:01:59Z Fix rebase to head after Beam 2.0 release. commit 341fa9245e5d67cc5fab4244bfeca5ea49c72231 Author: Charles Chen <c...@google.com> Date: 2017-05-25T21:22:03Z Clarify transform_keyed_states. commit 1e83d5e900c7039ccf89d989081ae50b3b3e510f Author: Charles Chen <c...@google.com> Date: 2017-05-26T00:38:31Z Support timer firings. commit 256309f2f3b01b230b258b8001a494d75e3e1713 Author: Charles Chen <c...@google.com> Date: 2017-05-26T01:08:44Z Reintroduce hacks from head. commit 25ce9f18a124e8d0aa80e05a516699da6a067c19 Author: Charles Chen <c...@google.com> Date: 2017-05-26T21:12:59Z Add debugging output. commit 7be85c67c5c405c16c146e5605dd2a5bb9bd3bdd Author: Charles Chen <c...@google.com> Date: 2017-05-26T22:56:32Z Don't hold input watermark on pending elements. commit 1df07a5b46811d0d3a383b156121d77f5f37aa57 Author: Charles Chen <c...@google.com> Date: 2017-05-30T18:26:03Z Finish moving earliest hold extraction to evaluators, dbg output++ commit e52ba42a9efe0961ed57da52e93fdd8e5970eedc Author: Charles Chen <c...@google.com> Date: 2017-05-31T22:46:06Z Batch WordCount works in streaming mode!! Fixes earliest hold to be < instead of <=. commit e5bb716e00ac02679edc3a357153009b862bb4d0 Author: Charles Chen <c...@google.com> Date: 2017-06-05T22:20:37Z Working batch and streaming mode execution, except for certain empty side inputs. commit 91873cc1422d5bc7812464feea6789a1c4242577 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-06-07T15:48
[GitHub] beam pull request #3594: Do not submit: scratch work
Github user charlesccychen closed the pull request at: https://github.com/apache/beam/pull/3594 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3594: Do not submit: scratch work
GitHub user charlesccychen reopened a pull request: https://github.com/apache/beam/pull/3594 Do not submit: scratch work Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam test-branch Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3594.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3594 commit 200b55c1a9cacddf501869d3127f34e735e68f37 Author: Charles Chen <c...@google.com> Date: 2017-07-19T16:50:42Z Test commit commit 213cff3a659b785a25d6496b8cdbfc292d1fc6e9 Author: Charles Chen <c...@google.com> Date: 2017-07-19T20:17:22Z Delete stuff. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3594: Do not submit: scratch work
Github user charlesccychen closed the pull request at: https://github.com/apache/beam/pull/3594 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3594: Do not submit: scratch work
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3594 Do not submit: scratch work Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam test-branch Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3594.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3594 commit 200b55c1a9cacddf501869d3127f34e735e68f37 Author: Charles Chen <c...@google.com> Date: 2017-07-19T16:50:42Z Test commit --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3157: [BEAM-1265] Do not merge: streaming scratch work
Github user charlesccychen closed the pull request at: https://github.com/apache/beam/pull/3157 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3499: [BEAM-1265] Fix PValue input in _PubSubReadEvaluato...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3499 [BEAM-1265] Fix PValue input in _PubSubReadEvaluator This incorporates @davidcavazos' change proposed in #3483 You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-te-inputs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3499.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3499 commit 4dfd37fed7540b2748c60debc63367ab8f711f91 Author: Charles Chen <c...@google.com> Date: 2017-07-05T23:18:51Z Fix PValue input in _PubSubReadEvaluator --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3454: Add PubSub I/O support to Python DirectRunner
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3454 Add PubSub I/O support to Python DirectRunner Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam directrunner-pubsub Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3454.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3454 commit 5c0686d1148605cc4bbe5932b1949d8e58bdc55b Author: Charles Chen <c...@google.com> Date: 2017-06-27T01:03:53Z Add PubSub I/O support to Python DirectRunner --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3444: Implement streaming GroupByKey in Python DirectRunn...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3444 Implement streaming GroupByKey in Python DirectRunner Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam streaming-gbk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3444.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3444 commit a39d4c02105754710ec5350d448d37cb20c9fcc4 Author: Charles Chen <c...@google.com> Date: 2017-06-26T23:54:00Z Implement streaming GroupByKey in Python DirectRunner --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3440: Remove old deprecated PubSub code
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3440 Remove old deprecated PubSub code Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam remove-old-pubsub Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3440.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3440 commit 926f949580c3a21df72a8836feda1f6b947850ec Author: Charles Chen <c...@google.com> Date: 2017-06-26T20:00:14Z Remove old deprecated PubSub code --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3423: Clean up test_stream_test and remove stray print st...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3423 Clean up test_stream_test and remove stray print statement Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam teststream-evaluator Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3423.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3423 commit 2c88af6dd37e6eb3bbe4133e3ee2cbdeed68be72 Author: Charles Chen <c...@google.com> Date: 2017-06-22T18:29:54Z Clean up test_stream_test and remove stray print statement --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3409: Allow production of unprocessed bundles, introduce ...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3409 Allow production of unprocessed bundles, introduce TestStream evaluator in DirectRunner This change depends on #3405. You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam teststream-evaluator Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3409.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3409 commit 34f0cb5502224b92a40fde1cae7fcf4c6179a2b9 Author: Charles Chen <c...@google.com> Date: 2017-06-20T22:22:58Z Use state / timer API for DirectRunner timer firings commit 1ab35ace53b9ed113f21c9750e2598f789b357d0 Author: Charles Chen <c...@google.com> Date: 2017-06-21T00:16:20Z Allow production of unprocessed bundles, introduce TestStream evaluator in DirectRunner --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3405: Use state / timer API for DirectRunner timer firing...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3405 Use state / timer API for DirectRunner timer firings Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam use-state-timers Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3405.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3405 commit bd77b17f8f49b63e1caf5dbed9ff719cbc8c4422 Author: Charles Chen <c...@google.com> Date: 2017-06-20T22:22:58Z Use state / timer API for DirectRunner timer firings --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3373: Populate PBegin input when decoding from Runner API
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3373 Populate PBegin input when decoding from Runner API Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-runner-api-input Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3373.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3373 commit 843f3f5a7ad7cf5297b3078aa9e929efcdc7092d Author: Charles Chen <c...@google.com> Date: 2017-06-15T22:27:18Z Populate PBegin input when decoding from Runner API --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3370: Introduce pending bundles and RootBundleProvider in...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3370 Introduce pending bundles and RootBundleProvider in DirectRunner Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam root-bundle-provider Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3370.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3370 commit 9b1343449ec0dec140b2befbd53df3d2924a1fca Author: Charles Chen <c...@google.com> Date: 2017-06-15T21:27:47Z Introduce pending bundles and RootBundleProvider in DirectRunner --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3362: Migrate DirectRunner evaluators to use Beam state A...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3362 Migrate DirectRunner evaluators to use Beam state API This change depends on #3318. You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam refactor-direct-state Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3362.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3362 commit fabbd85afd418c6b08b22df584e3d96fb77eec00 Author: Charles Chen <c...@google.com> Date: 2017-06-07T23:09:10Z Choose GroupAlsoByWindows implementation based on streaming flag commit 4226e55ca25a7e5a864439d2900b48abf14a100e Author: Charles Chen <c...@google.com> Date: 2017-06-13T21:04:27Z Address reviewer comments commit 150477182c76c79e0271de3ab6ba90c1006d362a Author: Charles Chen <c...@google.com> Date: 2017-06-14T23:14:50Z Migrate DirectRunner evaluators to use Beam state API --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3354: Move Runner API protos to portability/api
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3354 Move Runner API protos to portability/api Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-api-dir Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3354.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3354 commit 6c7ec5bedddc4a62d389078387362b4f6992e2cc Author: Charles Chen <c...@google.com> Date: 2017-06-13T22:03:15Z Move Runner API protos to portability/api --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3346: Reverse removal of NativeWrite evaluator in Python ...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3346 Reverse removal of NativeWrite evaluator in Python DirectRunner Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-nativewrite Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3346.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3346 commit 5a5c8208a0e0e4b55e8c5fe8ba3a6228e6c5856a Author: Charles Chen <c...@google.com> Date: 2017-06-12T21:17:50Z Reverse removal of NativeWrite evaluator in Python DirectRunner --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3322: Refine Python DirectRunner watermark advancement be...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3322 Refine Python DirectRunner watermark advancement behavior This change helps prepare for streaming pipeline execution. You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam streaming-watermarks Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3322.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3322 commit 591612d4271d03bd5ac8250cc8ac018b24cdd1a1 Author: Charles Chen <c...@google.com> Date: 2017-06-08T00:46:36Z Refine Python DirectRunner watermark advancement behavior This change helps prepare for streaming pipeline execution. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3321: Remove support for NativeSinks from the Python Dire...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3321 Remove support for NativeSinks from the Python DirectRunner You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam remove-nativewrite-dr Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3321.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3321 commit f2e3088633fef10f19bfd11ff9b508930916a740 Author: Charles Chen <c...@google.com> Date: 2017-06-08T00:00:57Z Remove support for NativeSinks from the Python DirectRunner --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3318: Choose GroupAlsoByWindows implementation based on s...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3318 Choose GroupAlsoByWindows implementation based on streaming flag This change depends on https://github.com/apache/beam/pull/3315. You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam streaming-gabw-refactor Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3318.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3318 commit 113a7d9591a91d6066f2e4f2b11095d755e55193 Author: Charles Chen <c...@google.com> Date: 2017-06-07T23:08:43Z Move Runner API protos to portability/runners/api This fixes a circular import issue between transforms/ and runners/ commit 1ce756980b7c98cd2146b9261a4d6ce374e5e913 Author: Charles Chen <c...@google.com> Date: 2017-06-07T23:09:10Z Choose GroupAlsoByWindows implementation based on streaming flag --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3315: Move Runner API protos to portability/runners/api
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3315 Move Runner API protos to portability/runners/api This fixes a circular import issue between transforms/ and runners/ Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-circular-api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3315.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3315 commit 113a7d9591a91d6066f2e4f2b11095d755e55193 Author: Charles Chen <c...@google.com> Date: 2017-06-07T23:08:43Z Move Runner API protos to portability/runners/api This fixes a circular import issue between transforms/ and runners/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3299: Migrate Python tests to not depend on fix sharding ...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3299 Migrate Python tests to not depend on fix sharding for file output Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-python-sharding Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3299.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3299 commit 64fc974d941aae32b5dd7f1bf6943d2e33dd1f41 Author: Charles Chen <c...@google.com> Date: 2017-06-05T23:31:13Z Migrate Python tests to not depend on fix sharding for file output --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3242: Fix Python Dataflow execution errors due to #3223
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3242 Fix Python Dataflow execution errors due to #3223 PR #3223 should have first updated the Dataflow Beam worker container before changing the public API in `gcp/pubsub.py`. This partially rolls back the deletion part of that change to fix the current breakage at head. You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-python-errors Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3242.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3242 commit bc91313896a72b653b0715d79dfcd8bbd395beb4 Author: Charles Chen <c...@google.com> Date: 2017-05-26T21:24:55Z Fix Python Dataflow execution errors due to #3223 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3198: Update Dataflow BATCH_ENVIRONMENT_MAJOR_VERSION to ...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3198 Update Dataflow BATCH_ENVIRONMENT_MAJOR_VERSION to 6 Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam update-major-version Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3198.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3198 commit 4cee177c8a798015189c1be5061164b22c3ccfe3 Author: Charles Chen <c...@google.com> Date: 2017-05-22T20:03:38Z Update Dataflow BATCH_ENVIRONMENT_MAJOR_VERSION to 6 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3157: [BEAM-1265] Do not merge: streaming scratch work
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3157 [BEAM-1265] Do not merge: streaming scratch work Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam streaming-test Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3157.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3157 commit f78e47ca47e19ac2684c13bf13dbb66e65f04048 Author: Charles Chen <c...@google.com> Date: 2017-04-19T01:20:07Z streaming test commit dd3966a572b73ffa9421a176cc03f096707cdb0d Author: Charles Chen <c...@google.com> Date: 2017-04-20T05:54:02Z Working TestStream evaluator commit 68b5a84595f4ca9adc131fe2fcee38f91078a047 Author: Charles Chen <c...@google.com> Date: 2017-04-20T08:59:05Z Slightly better watermark propagation, debugging print cleanup. commit 48d4f1ef68829e98e964070f60ce9470528fcb0e Author: Charles Chen <c...@google.com> Date: 2017-04-20T09:44:19Z Factor out GroupAlsoByWindows so that it can be overridden by the DirectRunner. commit b5e7073b66c02f634a4bdec8f787b38bb923d52a Author: Charles Chen <c...@google.com> Date: 2017-04-21T05:51:27Z Don't buffer elements in GroupByKeyOnly. commit f81c2d0b536e2715fe4cd278359eed5c6a855dc5 Author: Charles Chen <c...@google.com> Date: 2017-04-21T17:33:03Z Skeleton implementation of GroupAlsoByKeyEvaluator. commit 05bd9c677c89d3d3413c205a2965ec0cc6827786 Author: Charles Chen <c...@google.com> Date: 2017-04-22T08:32:25Z Mostly working streaming GroupAlsoByWindow. commit f9197ff807d27e61eef337b261f826d5f4b43908 Author: Charles Chen <c...@google.com> Date: 2017-04-24T08:58:58Z Introduce keyed bundles, KeyedWorkItems. commit c4b6faf9a4b8231616ec35ec2fd766b863e7a76e Author: Charles Chen <c...@google.com> Date: 2017-04-24T15:16:05Z Rename old non-keyed state to legacy_state, introduce KeyedWorkItem, prepare to expose DirectUnmergedState API. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3148: Remove "Dataflow" from apache_beam __init__.py file
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3148 Remove "Dataflow" from apache_beam __init__.py file Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam remove-dataflow Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3148.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3148 commit 6d0451a99455453c736c69c4255a1e20d491d1f4 Author: Charles Chen <c...@google.com> Date: 2017-05-15T18:26:59Z Remove "Dataflow" from apache_beam __init__.py file --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3095: [BEAM-1345] Clearly delineate public API in apache_...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3095 [BEAM-1345] Clearly delineate public API in apache_beam/options Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-all-3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3095.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3095 commit c7b671f81a7c3654d4a8d25dff2054bc87350fe3 Author: Charles Chen <c...@google.com> Date: 2017-05-11T20:31:18Z [BEAM-1345] Clearly delineate public API in apache_beam/options --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3086: [BEAM-1340] Move assert_that, equal_to, is_empty to...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3086 [BEAM-1340] Move assert_that, equal_to, is_empty to apache_beam.testing.util Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-assertthat-util Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3086.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3086 commit 0870b2335c5228507bbf216af731e80420ed8624 Author: Charles Chen <c...@google.com> Date: 2017-05-11T18:15:25Z Move assert_that, equal_to, is_empty to apache_beam.testing.util --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3075: [BEAM-1340] Add __all__ tags to modules in package ...
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3075 [BEAM-1340] Add __all__ tags to modules in package apache_beam/tranforms Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-all-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3075.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3075 commit d7479ac3964b0563443bd45940f1bb9ec81bd72a Author: Charles Chen <c...@google.com> Date: 2017-05-11T06:06:36Z [BEAM-1340] Add __all__ tags to modules in package apache_beam/transforms --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3021: Update Apache Beam Python version to 2.0.0
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3021 Update Apache Beam Python version to 2.0.0 You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam update-python-200-version Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3021.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3021 commit 62e3520c91e7a7bc4f711d7a3f254408dc49a93e Author: Charles Chen <c...@google.com> Date: 2017-05-09T23:23:48Z Update Apache Beam Python version to 2.0.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3020: Update Apache Beam Python version to 2.1.0.dev
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/3020 Update Apache Beam Python version to 2.1.0.dev You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam update-python-master-version Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3020.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3020 commit 566469eff3095ffd186a04cb46e5e4cbd7d05724 Author: Charles Chen <c...@google.com> Date: 2017-05-09T23:22:19Z Update Apache Beam Python version to 2.1.0.dev --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2571: Remove overloading of __call__ in DirectRunner
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/2571 Remove overloading of __call__ in DirectRunner This change makes the usage of CallableTask easier to understand--previously, there was no easy way to discover where the __call__ method of CallableTasks was invoked since Python code, being untyped, is not amenable to this type of static analysis. You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-call Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2571.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2571 commit 434c2211f6298435678effe69862896f3f8f72df Author: Charles Chen <c...@google.com> Date: 2017-04-18T08:42:57Z Remove overloading of __call__ in DirectRunner --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2427: Test
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/2427 Test Do not submit You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam py_custom_create Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2427.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2427 commit 21f1bbd3efeaa0ada3859791bac808ef3264cdc8 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-04-03T17:01:45Z Create commit 2b6c29fcdd0f499e80fe03b167e06090e521fc9f Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-04-04T17:31:41Z Create as custom source commit 35fc478d2fdf524e463cb256c6ac515806f15c7b Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-04-04T20:10:35Z dill trace enable/disable commit 6d5f336753e1816aba5875277f50eca524b28899 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-04-04T21:11:12Z Fixes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2427: Test
Github user charlesccychen closed the pull request at: https://github.com/apache/beam/pull/2427 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2389: Add TestStream to Python SDK
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/2389 Add TestStream to Python SDK The TestStream will be used for verifying streaming runner semantics. You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam test-stream Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2389.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2389 commit 4238de506ae9a9290bfdb9360f96a90e97f54958 Author: Charles Chen <c...@google.com> Date: 2017-03-31T01:20:04Z Add TestStream to Python SDK The TestStream will be used for verifying streaming runner semantics. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2027: Add Windowing snippets for Python SDK.
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/2027 Add Windowing snippets for Python SDK. You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam add-python-snippets Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2027.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2027 commit a6a86c9b20eed2a4bbf64b38446dc74e765feb70 Author: Charles Chen <c...@google.com> Date: 2017-02-17T00:47:04Z Add Windowing snippets for Python SDK. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #1733: Improve performance of fileio._CompressedFile
GitHub user charlesccychen opened a pull request: https://github.com/apache/beam/pull/1733 Improve performance of fileio._CompressedFile This change fixes the timing out of "python setup.py test" for the Python SDK. You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/beam fix-fileio-compressed Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/1733.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1733 commit 3b687f54fffbddad312ea1d1a81a437b9ec91c0c Author: Charles Chen <c...@google.com> Date: 2017-01-04T23:48:30Z Improve performance of fileio._CompressedFile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
