[
https://issues.apache.org/jira/browse/FLINK-1709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367450#comment-14367450
]
ASF GitHub Bot commented on FLINK-1709:
---------------------------------------
GitHub user uce opened a pull request:
https://github.com/apache/flink/pull/498
[FLINK-1709] [wip] Add SlotCountExceedingParallelismTest
This is the initial simple case. I'll push the other variants to this
branch.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/uce/incubator-flink flink-1709-slot_dop
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/498.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #498
----
commit ca66bd17b06f0de675de91fc6441a4851a71fdcb
Author: Ufuk Celebi <[email protected]>
Date: 2015-03-18T13:19:05Z
[FLINK-1724] [tests] Respect number of task managers in TestingCluster
Starting a task manager via TestingUtils does not respect the number of
configured task managers and mis-configures the task managers to use local
network communication (LocalConnectionManager instead of
NettyConnectionManager).
commit d311039077f6413035a821cf07826e5002bf5759
Author: Ufuk Celebi <[email protected]>
Date: 2015-03-18T13:20:59Z
[FLINK-1724] [tests] Fix test kit actor configuration typo
commit cee335c95f1b0339a81e26d085127ac7f71ef15a
Author: Ufuk Celebi <[email protected]>
Date: 2015-03-18T13:23:53Z
[jobmanager] Add subtask index to state transition debug msg
commit 1ed82b19b80e24b40b55e87fd7deb09675b3ea64
Author: Ufuk Celebi <[email protected]>
Date: 2015-03-18T13:24:54Z
[jobmanager] Fix typo in ResultPartition deployment descriptor
commit 0c7c80481a63be40e53666c470a34868d578e289
Author: Ufuk Celebi <[email protected]>
Date: 2015-03-18T16:26:42Z
[jobmanager] Fix missing queue scheduling mode
commit ce7cbe65bba297d6f65e718ed24a601f85c3bfcd
Author: Ufuk Celebi <[email protected]>
Date: 2015-03-18T16:29:58Z
[FLINK-1709] Add initial SlotCountExceedingParallelismTest
----
> Add support for programs with slot count exceeding parallelism
> --------------------------------------------------------------
>
> Key: FLINK-1709
> URL: https://issues.apache.org/jira/browse/FLINK-1709
> Project: Flink
> Issue Type: Improvement
> Affects Versions: master
> Reporter: Ufuk Celebi
>
> Currently, we can't run programs with higher parallelism than available slots.
> For example, if we currently have a map-reduce program and 4 task slots
> configured (e.g. 2 task managers with 2 slots per task manager), the map and
> reduce tasks will be scheduled with pipelined results and the same
> parallelism in shared slots. Setting the parallelism to more than available
> slots will result in a NoResourcesAvailableException.
> As a first step to support these kinds of programs, we can add initial
> support for this when running in batch mode (after
> https://github.com/apache/flink/pull/471 is merged).
> This is easier than the original pipelined scenario, because the map tasks
> can be deployed after each other to produce the blocking result. The blocking
> result can then be consumed after all map tasks produced their result. The
> mechanism in #471 to deploy result receivers can be used for this and should
> not need any modifications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
