GitHub user StephanEwen opened a pull request:
https://github.com/apache/flink/pull/487
[FLINK-1671] Add different execution modes to APIs and optimizer.
This allows users to specify how programs should be executed:
- PIPELINED: Uses pipelining for shuffling, except where the data
exchange is deadlock prone.
- BATCH: Used batch for shuffles, broadcasts, rebalancing.
- BATCH_FORCED: Uses batch everywhere, effectively prohibits chaining
(debug only, IMHO)
- PIPELINED_FORCED: Uses pipelining everywhere, also in deadlock prone
situations.
I fixed and improved a lot of comments on-the-fly while implementing this.
Because of that, the PR is slightly bigger.
NOTE: This is only the API, and optimizer implementation of selecting the
proper data exchange mode. The runtime implementation of blocking data exchange
is a separate pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StephanEwen/incubator-flink modes
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/487.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #487
----
commit 62a41f82598a75af76956f971bf05a994a073b7b
Author: Stephan Ewen <[email protected]>
Date: 2015-03-10T14:14:09Z
[optimizer] Remove obsolete and unused fields from OptimizerNode
commit 11a7449714771532fd8b101af07ef3e0bd289a21
Author: Stephan Ewen <[email protected]>
Date: 2015-03-10T17:22:20Z
[FLINK-1671] [jobmanager] Rename JobManager's ExecutionMode to
JobManagerMode
This is done to avoid name conflicts with the overloaded type name
ExecutionMode.
commit 6f9ab0b9d3370ca2dd18de371d1fae025e60561e
Author: Stephan Ewen <[email protected]>
Date: 2015-03-10T17:36:52Z
[FLINK-1671] [tests] Rename "ExecutionMode" in MultiProgramTestBase to
"TestExecutionMode"
... to avoid name clashes with the program execution mode.
commit 537b06602b194e708d61203e5a8b43ecaf96024e
Author: Stephan Ewen <[email protected]>
Date: 2015-03-10T19:25:01Z
[FLINK-1671] [core] Add execution mode to execution config
commit 566a2bbc38e6b06a43b4efe99834b9aadec062d6
Author: Stephan Ewen <[email protected]>
Date: 2015-03-11T22:35:11Z
[FLINK-1671] [optimizer] Add data exchange mode to optimizer classes
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
