[
https://issues.apache.org/jira/browse/BEAM-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196303#comment-15196303
]
ASF GitHub Bot commented on BEAM-22:
------------------------------------
GitHub user tgroh opened a pull request:
https://github.com/apache/incubator-beam/pull/53
[BEAM-22] Add ExecutorServiceParallelExecutor as an InProcessExecutor
This is responsible for scheduling transform evaluations and
communicating results back to the evaluation context. The executor
handle PTransforms that block arbitarily waiting for additional input.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tgroh/incubator-beam ippr_executor
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/53.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #53
----
commit bac398b5ac8c7c29a85f9bb56886e8e9429c213c
Author: Thomas Groh <[email protected]>
Date: 2016-02-27T01:29:43Z
Add ExecutorServiceParallelExecutor as an InProcessExecutor
This is responsible for scheduling transform evaluations and
communicating results back to the evaluation context. The executor
handle PTransforms that block arbitarily waiting for additional input.
----
> DirectPipelineRunner: support for unbounded collections
> -------------------------------------------------------
>
> Key: BEAM-22
> URL: https://issues.apache.org/jira/browse/BEAM-22
> Project: Beam
> Issue Type: Improvement
> Components: runner-direct
> Reporter: Davor Bonaci
> Assignee: Thomas Groh
>
> DirectPipelineRunner currently runs over bounded PCollections only, and
> implements only a portion of the Beam Model.
> We should improve it to faithfully implement the full Beam Model, such as add
> ability to run over unbounded PCollections, and better resemble execution
> model in a distributed system.
> This further enables features such as a testing source which may simulate
> late data and test triggers in the pipeline. Finally, we may want to expose
> an option to select between "debug" (single threaded), "chaos monkey" (test
> as many model requirements as possible), and "performance" (multi-threaded).
> more testing (chaos monkey)
> Once this is done, we should update this StackOverflow question:
> http://stackoverflow.com/questions/35350113/testing-triggers-with-processing-time/35401426#35401426
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
