[jira] [Updated] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe
[
https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhaohui Xin updated YARN-8655:
--
Attachment: YARN-8655.002.patch
> FairScheduler: FSStarvedApps is not thread safe
> ---
>
> Key: YARN-8655
> URL: https://issues.apache.org/jira/browse/YARN-8655
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Affects Versions: 3.0.0
>Reporter: Zhaohui Xin
>Assignee: Zhaohui Xin
>Priority: Major
> Attachments: YARN-8655.002.patch, YARN-8655.patch
>
>
> *FSStarvedApps is not thread safe, this may make one starve app is processed
> for two times continuously.*
> For example, when app1 is *fair share starved*, it has been added to
> appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
> update to app1. At the moment, app1 is *starved by min share*, so this app
> is added to appsToProcess again! Because appBeingProcessed is null and
> appsToProcess also have not this one.
> {code:java}
> void addStarvedApp(FSAppAttempt app) {
> if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
> appsToProcess.add(app);
> }
> }
> FSAppAttempt take() throws InterruptedException {
> // Reset appBeingProcessed before the blocking call
> appBeingProcessed = null;
> // Blocking call to fetch the next starved application
> FSAppAttempt app = appsToProcess.take();
> appBeingProcessed = app;
> return app;
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe
[
https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhaohui Xin updated YARN-8655:
--
Description:
*FSStarvedApps is not thread safe, this may make one starve app is processed
for two times continuously.*
For example, when app1 is *fair share starved*, it has been added to
appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
update to app1. At the moment, app1 is *starved by min share*, so this app is
added to appsToProcess again! Because appBeingProcessed is null and
appsToProcess also have not this one.
{code:java}
void addStarvedApp(FSAppAttempt app) {
if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
appsToProcess.add(app);
}
}
FSAppAttempt take() throws InterruptedException {
// Reset appBeingProcessed before the blocking call
appBeingProcessed = null;
// Blocking call to fetch the next starved application
FSAppAttempt app = appsToProcess.take();
appBeingProcessed = app;
return app;
}
{code}
was:
*FSStarvedApps is not thread safe, this may make one starve app is processed
for two times continuously.*
For example, when app1 is fair share starved, it has been added to
appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
update to app1. At the moment, app1 is starved by min share, so this app is
added to appsToProcess again! Because appBeingProcessed is null and
appsToProcess also have not this one.
{code:java}
void addStarvedApp(FSAppAttempt app) {
if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
appsToProcess.add(app);
}
}
FSAppAttempt take() throws InterruptedException {
// Reset appBeingProcessed before the blocking call
appBeingProcessed = null;
// Blocking call to fetch the next starved application
FSAppAttempt app = appsToProcess.take();
appBeingProcessed = app;
return app;
}
{code}
> FairScheduler: FSStarvedApps is not thread safe
> ---
>
> Key: YARN-8655
> URL: https://issues.apache.org/jira/browse/YARN-8655
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Affects Versions: 3.0.0
>Reporter: Zhaohui Xin
>Assignee: Zhaohui Xin
>Priority: Major
> Attachments: YARN-8655.patch
>
>
> *FSStarvedApps is not thread safe, this may make one starve app is processed
> for two times continuously.*
> For example, when app1 is *fair share starved*, it has been added to
> appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
> update to app1. At the moment, app1 is *starved by min share*, so this app
> is added to appsToProcess again! Because appBeingProcessed is null and
> appsToProcess also have not this one.
> {code:java}
> void addStarvedApp(FSAppAttempt app) {
> if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
> appsToProcess.add(app);
> }
> }
> FSAppAttempt take() throws InterruptedException {
> // Reset appBeingProcessed before the blocking call
> appBeingProcessed = null;
> // Blocking call to fetch the next starved application
> FSAppAttempt app = appsToProcess.take();
> appBeingProcessed = app;
> return app;
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe
[
https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhaohui Xin updated YARN-8655:
--
Issue Type: Sub-task (was: Bug)
Parent: YARN-5990
> FairScheduler: FSStarvedApps is not thread safe
> ---
>
> Key: YARN-8655
> URL: https://issues.apache.org/jira/browse/YARN-8655
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Affects Versions: 3.0.0
>Reporter: Zhaohui Xin
>Assignee: Zhaohui Xin
>Priority: Major
> Attachments: YARN-8655.patch
>
>
> *FSStarvedApps is not thread safe, this may make one starve app is processed
> for two times continuously.*
> For example, when app1 is fair share starved, it has been added to
> appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
> update to app1. At the moment, app1 is starved by min share, so this app is
> added to appsToProcess again! Because appBeingProcessed is null and
> appsToProcess also have not this one.
> {code:java}
> void addStarvedApp(FSAppAttempt app) {
> if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
> appsToProcess.add(app);
> }
> }
> FSAppAttempt take() throws InterruptedException {
> // Reset appBeingProcessed before the blocking call
> appBeingProcessed = null;
> // Blocking call to fetch the next starved application
> FSAppAttempt app = appsToProcess.take();
> appBeingProcessed = app;
> return app;
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe
[
https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhaohui Xin updated YARN-8655:
--
Description:
*FSStarvedApps is not thread safe, this may make one starve app is processed
for two times continuously.*
For example, when app1 is fair share starved, it has been added to
appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
update to app1. At the moment, app1 is starved by min share, so this app is
added to appsToProcess again! Because appBeingProcessed is null and
appsToProcess also have not this one.
{code:java}
void addStarvedApp(FSAppAttempt app) {
if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
appsToProcess.add(app);
}
}
FSAppAttempt take() throws InterruptedException {
// Reset appBeingProcessed before the blocking call
appBeingProcessed = null;
// Blocking call to fetch the next starved application
FSAppAttempt app = appsToProcess.take();
appBeingProcessed = app;
return app;
}
{code}
was:
*FSStarvedApps is not thread safe, this may make one starve app is processed
for two times continuously.*
For example, when app1 is fair share starved, it has been added to
appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
update to app1. At the moment, app1 is starved by min share, so this app is
added to appsToProcess again! Because appBeingProcessed is null and
appsToProcess also have not this one.
> FairScheduler: FSStarvedApps is not thread safe
> ---
>
> Key: YARN-8655
> URL: https://issues.apache.org/jira/browse/YARN-8655
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.0.0
>Reporter: Zhaohui Xin
>Priority: Major
> Attachments: YARN-8655.patch
>
>
> *FSStarvedApps is not thread safe, this may make one starve app is processed
> for two times continuously.*
> For example, when app1 is fair share starved, it has been added to
> appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
> update to app1. At the moment, app1 is starved by min share, so this app is
> added to appsToProcess again! Because appBeingProcessed is null and
> appsToProcess also have not this one.
>
>
> {code:java}
> void addStarvedApp(FSAppAttempt app) {
> if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
> appsToProcess.add(app);
> }
> }
> FSAppAttempt take() throws InterruptedException {
> // Reset appBeingProcessed before the blocking call
> appBeingProcessed = null;
> // Blocking call to fetch the next starved application
> FSAppAttempt app = appsToProcess.take();
> appBeingProcessed = app;
> return app;
> }
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe
[
https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhaohui Xin updated YARN-8655:
--
Description:
*FSStarvedApps is not thread safe, this may make one starve app is processed
for two times continuously.*
For example, when app1 is fair share starved, it has been added to
appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
update to app1. At the moment, app1 is starved by min share, so this app is
added to appsToProcess again! Because appBeingProcessed is null and
appsToProcess also have not this one.
{code:java}
void addStarvedApp(FSAppAttempt app) {
if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
appsToProcess.add(app);
}
}
FSAppAttempt take() throws InterruptedException {
// Reset appBeingProcessed before the blocking call
appBeingProcessed = null;
// Blocking call to fetch the next starved application
FSAppAttempt app = appsToProcess.take();
appBeingProcessed = app;
return app;
}
{code}
was:
*FSStarvedApps is not thread safe, this may make one starve app is processed
for two times continuously.*
For example, when app1 is fair share starved, it has been added to
appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
update to app1. At the moment, app1 is starved by min share, so this app is
added to appsToProcess again! Because appBeingProcessed is null and
appsToProcess also have not this one.
{code:java}
void addStarvedApp(FSAppAttempt app) {
if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
appsToProcess.add(app);
}
}
FSAppAttempt take() throws InterruptedException {
// Reset appBeingProcessed before the blocking call
appBeingProcessed = null;
// Blocking call to fetch the next starved application
FSAppAttempt app = appsToProcess.take();
appBeingProcessed = app;
return app;
}
{code}
> FairScheduler: FSStarvedApps is not thread safe
> ---
>
> Key: YARN-8655
> URL: https://issues.apache.org/jira/browse/YARN-8655
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.0.0
>Reporter: Zhaohui Xin
>Priority: Major
> Attachments: YARN-8655.patch
>
>
> *FSStarvedApps is not thread safe, this may make one starve app is processed
> for two times continuously.*
> For example, when app1 is fair share starved, it has been added to
> appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
> update to app1. At the moment, app1 is starved by min share, so this app is
> added to appsToProcess again! Because appBeingProcessed is null and
> appsToProcess also have not this one.
> {code:java}
> void addStarvedApp(FSAppAttempt app) {
> if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
> appsToProcess.add(app);
> }
> }
> FSAppAttempt take() throws InterruptedException {
> // Reset appBeingProcessed before the blocking call
> appBeingProcessed = null;
> // Blocking call to fetch the next starved application
> FSAppAttempt app = appsToProcess.take();
> appBeingProcessed = app;
> return app;
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe
[
https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhaohui Xin updated YARN-8655:
--
Description:
*FSStarvedApps is not thread safe, this may make one starve app is processed
for two times continuously.*
For example, when app1 is fair share starved, it has been added to
appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
update to app1. At the moment, app1 is starved by min share, so this app is
added to appsToProcess again! Because appBeingProcessed is null and
appsToProcess also have not this one.
{code:java}
void addStarvedApp(FSAppAttempt app) {
if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
appsToProcess.add(app);
}
}
FSAppAttempt take() throws InterruptedException {
// Reset appBeingProcessed before the blocking call
appBeingProcessed = null;
// Blocking call to fetch the next starved application
FSAppAttempt app = appsToProcess.take();
appBeingProcessed = app;
return app;
}
{code}
was:
*FSStarvedApps is not thread safe, this may make one starve app is processed
for two times continuously.*
For example, when app1 is fair share starved, it has been added to
appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
update to app1. At the moment, app1 is starved by min share, so this app is
added to appsToProcess again! Because appBeingProcessed is null and
appsToProcess also have not this one.
{code:java}
void addStarvedApp(FSAppAttempt app) {
if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
appsToProcess.add(app);
}
}
FSAppAttempt take() throws InterruptedException {
// Reset appBeingProcessed before the blocking call
appBeingProcessed = null;
// Blocking call to fetch the next starved application
FSAppAttempt app = appsToProcess.take();
appBeingProcessed = app;
return app;
}
{code}
> FairScheduler: FSStarvedApps is not thread safe
> ---
>
> Key: YARN-8655
> URL: https://issues.apache.org/jira/browse/YARN-8655
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.0.0
>Reporter: Zhaohui Xin
>Priority: Major
> Attachments: YARN-8655.patch
>
>
> *FSStarvedApps is not thread safe, this may make one starve app is processed
> for two times continuously.*
> For example, when app1 is fair share starved, it has been added to
> appsToProcess. After that, app1 is taken but appBeingProcessed is not yet
> update to app1. At the moment, app1 is starved by min share, so this app is
> added to appsToProcess again! Because appBeingProcessed is null and
> appsToProcess also have not this one.
>
> {code:java}
> void addStarvedApp(FSAppAttempt app) {
> if (!app.equals(appBeingProcessed) && !appsToProcess.contains(app)) {
> appsToProcess.add(app);
> }
> }
> FSAppAttempt take() throws InterruptedException {
> // Reset appBeingProcessed before the blocking call
> appBeingProcessed = null;
> // Blocking call to fetch the next starved application
> FSAppAttempt app = appsToProcess.take();
> appBeingProcessed = app;
> return app;
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe
[ https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhaohui Xin updated YARN-8655: -- Description: *FSStarvedApps is not thread safe, this may make one starve app is processed for two times continuously.* For example, when app1 is fair share starved, it has been added to appsToProcess. After that, app1 is taken but appBeingProcessed is not yet update to app1. At the moment, app1 is starved by min share, so this app is added to appsToProcess again! Because appBeingProcessed is null and appsToProcess also have not this one. was: *FSStarvedApps is not thread safe, this may make one starve app is processed for two times continuously.* For example, when app1 is fair share starved, it has been added to appsToProcess. After that, app1 is taken but appBeingProcessed is not yet update to app1. At the moment, app1 is starved by min share, so this app is added to appsToProcess again! Because appBeingProcessed is null and appsToProcess also have not this one! > FairScheduler: FSStarvedApps is not thread safe > --- > > Key: YARN-8655 > URL: https://issues.apache.org/jira/browse/YARN-8655 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 3.0.0 >Reporter: Zhaohui Xin >Priority: Major > Attachments: YARN-8655.patch > > > *FSStarvedApps is not thread safe, this may make one starve app is processed > for two times continuously.* > For example, when app1 is fair share starved, it has been added to > appsToProcess. After that, app1 is taken but appBeingProcessed is not yet > update to app1. At the moment, app1 is starved by min share, so this app is > added to appsToProcess again! Because appBeingProcessed is null and > appsToProcess also have not this one. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe
[ https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhaohui Xin updated YARN-8655: -- Description: *FSStarvedApps is not thread safe, this may make one starve app is processed for two times continuously.* For example, when app1 is fair share starved, it has been added to appsToProcess. After that, app1 is taken but appBeingProcessed is not yet update to app1. At the moment, app1 is starved by min share, so this app is added to appsToProcess again! Because appBeingProcessed is null and appsToProcess also have not this one! was: FSStarvedApps is not thread safe, this may make one starve app For example, when app1 is fair share starved, it has been added to appsToProcess. After that, app1 is taken but appBeingProcessed is not yet update to app1. At the moment, app1 is starved by min share, so this app is added to appsToProcess again! Because appBeingProcessed is null and appsToProcess also have not this one! > FairScheduler: FSStarvedApps is not thread safe > --- > > Key: YARN-8655 > URL: https://issues.apache.org/jira/browse/YARN-8655 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 3.0.0 >Reporter: Zhaohui Xin >Priority: Major > Attachments: YARN-8655.patch > > > *FSStarvedApps is not thread safe, this may make one starve app is processed > for two times continuously.* > For example, when app1 is fair share starved, it has been added to > appsToProcess. After that, app1 is taken but appBeingProcessed is not yet > update to app1. At the moment, app1 is starved by min share, so this app is > added to appsToProcess again! Because appBeingProcessed is null and > appsToProcess also have not this one! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe
[ https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhaohui Xin updated YARN-8655: -- Summary: FairScheduler: FSStarvedApps is not thread safe (was: FSStarvedApps is not thread safe) > FairScheduler: FSStarvedApps is not thread safe > --- > > Key: YARN-8655 > URL: https://issues.apache.org/jira/browse/YARN-8655 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 3.0.0 >Reporter: Zhaohui Xin >Priority: Major > Attachments: YARN-8655.patch > > > FSStarvedApps is not thread safe, this may make one starve app > For example, when app1 is fair share starved, it has been added to > appsToProcess. After that, app1 is taken but appBeingProcessed is not yet > update to app1. At the moment, app1 is starved by min share, so this app is > added to appsToProcess again! Because appBeingProcessed is null and > appsToProcess also have not this one! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
