[
https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16766008#comment-16766008
]
Zhaohui Xin edited comment on YARN-8655 at 2/12/19 1:09 PM:
------------------------------------------------------------
[~wilfreds], I accidentally discovered this problem in our production cluster
about a few months ago. *I think it's enough to satisfy fair share starvation,
so I removed min share starvation to fix this problem finally.*
I just learned that the community will also abolish min share in the future.
After YARN-9066, this issue will no longer be needed.
Thanks for your reply. :D
was (Author: uranus):
[~wilfreds], I accidentally discovered this problem in our production cluster
about a few months ago. *I think it's enough to satisfy fair share starvation,
so I removed min share starvation to fix this problem finally.*
I just learned that the community will also abolish this in the future. After
[YARN-9066|https://issues.apache.org/jira/browse/YARN-9066], this issue will no
longer be needed.
Thanks for your reply. :D
> FairScheduler: FSStarvedApps is not thread safe
> -----------------------------------------------
>
> Key: YARN-8655
> URL: https://issues.apache.org/jira/browse/YARN-8655
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Affects Versions: 3.0.0
> Reporter: Zhaohui Xin
> Assignee: Zhaohui Xin
> Priority: Major
> Attachments: YARN-8655.002.patch, YARN-8655.patch
>
>
> *FSStarvedApps is not thread safe, this may make one starve app is processed
> for two times continuously.*
> For example, when app1 is *fair share starved*, it has been added to
> appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
> update to app1. At the moment, app1 is *starved by min share*, so this app
> is added to appsToProcess again! Because appBeingProcessed is null and
> appsToProcess also have not this one.
> {code:java}
> void addStarvedApp(FSAppAttempt app) {
> if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
> appsToProcess.add(app);
> }
> }
> FSAppAttempt take() throws InterruptedException {
> // Reset appBeingProcessed before the blocking call
> appBeingProcessed = null;
> // Blocking call to fetch the next starved application
> FSAppAttempt app = appsToProcess.take();
> appBeingProcessed = app;
> return app;
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
