[
https://issues.apache.org/jira/browse/YARN-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15939440#comment-15939440
]
Yufei Gu commented on YARN-5654:
--------------------------------
Uploaded patch v2.
# Remove class {{ResourceSchedulerWrapper}}. We realize that better to inherit
Scheduler Wrapper from the Real scheduler class instead of
AbstractYarnScheduler after an offline discussion with [~rkanter], so that any
public interface changes of FairScheduler could reflect into the scheduler
wrapper. CapacityScheduler did in that way for a long time. The only downside
is we don't automatically have the wrapper for FifoScheduler.
# There are lots of duplicated code in {{SLSCapacityScheduler}} and
{{SLSFairScheduler}}(used to be in {{ResourceSchedulerWrapper}}). Most of them
are related to scheduler metrics. Make more sense to move them into
{{SchedulerMetrics}}. This not only makes extension of SLSFairScheduler and
SLSCapacityScheduler easier, e.g., right now both SLS scheduler wrapper extend
functions {{allocate()}} and {{handle()}}, we need to extend more functions
potentially. This also makes writing {{SLSFifoScheduler}} easier. File
YARN-6383 for FifoScheduler.
# Parameterize unit tests TestAMSimulator and TestNMSimulator so that they can
support both capacity scheduler and fair scheduler.
# This patch make FairScheduler in SLS work generally. But there are still some
bugs to fix, e.g. the queue metrics and cluster metrics doesn't work well. It
is too much to put them in this jira. I will file followup jira for them.
> Not be able to run SLS with FairScheduler
> -----------------------------------------
>
> Key: YARN-5654
> URL: https://issues.apache.org/jira/browse/YARN-5654
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Wangda Tan
> Assignee: Yufei Gu
> Attachments: YARN-5654.002.patch, YARN-5654.1.patch
>
>
> With the config:
> https://github.com/leftnoteasy/yarn_application_synthesizer/tree/master/configs/hadoop-conf-fs
> And data:
> https://github.com/leftnoteasy/yarn_application_synthesizer/tree/master/data/scheduler-load-test-data
> Capacity Scheduler runs fine, but Fair Scheduler cannot be successfully run.
> It reports NPE from RMAppAttemptImpl
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
