[
https://issues.apache.org/jira/browse/YARN-8255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16490231#comment-16490231
]
Suma Shivaprasad commented on YARN-8255:
----------------------------------------
One usecase I can think of is in the case of Spark applications which have
Dynamic Allocation enabled on the Spark driver and executors need to flex up
and down based on driver's discretion. There could two be components here - one
for Spark Driver and one for Spark Executor and the driver needs to flex the
executor component instances up and down based on workload at that point in
time /idle timeout.
Thoughts [~eyang] [~leftnoteasy] [~billie.rinaldi]
> Allow option to disable flex for a service component
> -----------------------------------------------------
>
> Key: YARN-8255
> URL: https://issues.apache.org/jira/browse/YARN-8255
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn-native-services
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Priority: Major
>
> YARN-8080 implements restart capabilities for service component instances.
> YARN service components should add an option to disallow flexing to support
> workloads which are essentially batch/iterative jobs which terminate with
> restart_policy=NEVER/ON_FAILURE. This could be disabled by default for
> components where restart_policy=NEVER/ON_FAILURE and enabled by default when
> restart_policy=ALWAYS(which is the default restart_policy) unless explicitly
> set at the service spec.
> The option could be exposed as part of the component spec as "allow_flexing".
> cc [~billie.rinaldi] [~gsaha] [~eyang] [~csingh] [~wangda]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
