[
https://issues.apache.org/jira/browse/FLINK-937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034389#comment-14034389
]
Daniel Warneke commented on FLINK-937:
--------------------------------------
As far as I know, queues are rather an operational feature cluster
administrators can use to handle different job priorities or isolate different
types of workloads. I don’t see how they could help to find a reasonable
default degree of parallelism. To the best of my knowledge, the only YARN APIs
we can rely on are:
- Min/max size of containers
- Total number of nodes in the cluster (idle or not)
Therefore, I basically see two possibilities for the default behavior:
- Start large: Determine number of nodes and use them all (Robert’s initial
suggestions?)
- Start small: Make default degree of parallelism 1 or 2 (default behavior of
MapReduce)
I have a slight preference for option 2, but I would choose either one over the
“assistant”…
> Change the YARN Client to allocate all cluster resources, if no argument given
> ------------------------------------------------------------------------------
>
> Key: FLINK-937
> URL: https://issues.apache.org/jira/browse/FLINK-937
> Project: Flink
> Issue Type: Improvement
> Components: YARN Client
> Reporter: Robert Metzger
> Assignee: Robert Metzger
> Fix For: 0.6-incubating
>
>
> In order to further improve the user experience, I would like to change the
> YARN client's behavior to allocate as many cluster resources as possible, if
> the user does not specify differently.
> The majority of users have exclusive access to the cluster.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
