[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer
[ https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated YARN-4396: Attachment: YARN-4396.005.patch Thanks [~templedf] for the comments. Your comments look good to me. Make two changes in the latets patch: * Change log level from {{debug}} to {{trace}}. * Do not print the log when the locality doesn't change. Thanks for reviewing! > Log the trance information on FSAppAttempt#assignContainer > -- > > Key: YARN-4396 > URL: https://issues.apache.org/jira/browse/YARN-4396 > Project: Hadoop YARN > Issue Type: Improvement > Components: applications, fairscheduler >Affects Versions: 2.7.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Labels: oct16-easy > Attachments: YARN-4396.001.patch, YARN-4396.002.patch, > YARN-4396.003.patch, YARN-4396.004.patch, YARN-4396.005.patch > > > When I configure the yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack to open this function, I have no > detail info of assigning container's locality. And it's important because it > will lead some delay scheduling and will have an influence on my cluster. If > I know these info, I can adjust param in cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer
[ https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated YARN-4396: - Labels: oct16-easy (was: oct-16-easy) > Log the trance information on FSAppAttempt#assignContainer > -- > > Key: YARN-4396 > URL: https://issues.apache.org/jira/browse/YARN-4396 > Project: Hadoop YARN > Issue Type: Improvement > Components: applications, fairscheduler >Affects Versions: 2.7.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Labels: oct16-easy > Attachments: YARN-4396.001.patch, YARN-4396.002.patch, > YARN-4396.003.patch, YARN-4396.004.patch > > > When I configure the yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack to open this function, I have no > detail info of assigning container's locality. And it's important because it > will lead some delay scheduling and will have an influence on my cluster. If > I know these info, I can adjust param in cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer
[ https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-4396: - Labels: oct-16-easy (was: ) > Log the trance information on FSAppAttempt#assignContainer > -- > > Key: YARN-4396 > URL: https://issues.apache.org/jira/browse/YARN-4396 > Project: Hadoop YARN > Issue Type: Improvement > Components: applications, fairscheduler >Affects Versions: 2.7.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Labels: oct-16-easy > Attachments: YARN-4396.001.patch, YARN-4396.002.patch, > YARN-4396.003.patch, YARN-4396.004.patch > > > When I configure the yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack to open this function, I have no > detail info of assigning container's locality. And it's important because it > will lead some delay scheduling and will have an influence on my cluster. If > I know these info, I can adjust param in cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer
[ https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-4396: - Component/s: (was: scheduler) fairscheduler > Log the trance information on FSAppAttempt#assignContainer > -- > > Key: YARN-4396 > URL: https://issues.apache.org/jira/browse/YARN-4396 > Project: Hadoop YARN > Issue Type: Improvement > Components: applications, fairscheduler >Affects Versions: 2.7.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Labels: oct-16-easy > Attachments: YARN-4396.001.patch, YARN-4396.002.patch, > YARN-4396.003.patch, YARN-4396.004.patch > > > When I configure the yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack to open this function, I have no > detail info of assigning container's locality. And it's important because it > will lead some delay scheduling and will have an influence on my cluster. If > I know these info, I can adjust param in cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer
[ https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Yiqun updated YARN-4396: Attachment: YARN-4396.004.patch Simplified the debug infos. > Log the trance information on FSAppAttempt#assignContainer > -- > > Key: YARN-4396 > URL: https://issues.apache.org/jira/browse/YARN-4396 > Project: Hadoop YARN > Issue Type: Improvement > Components: applications, scheduler >Affects Versions: 2.7.1 >Reporter: Lin Yiqun >Assignee: Lin Yiqun > Attachments: YARN-4396.001.patch, YARN-4396.002.patch, > YARN-4396.003.patch, YARN-4396.004.patch > > > When I configure the yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack to open this function, I have no > detail info of assigning container's locality. And it's important because it > will lead some delay scheduling and will have an influence on my cluster. If > I know these info, I can adjust param in cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer
[ https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Yiqun updated YARN-4396: Attachment: YARN-4396.003.patch Add the debug infos in {{getAllowedLocalityLevelByTime}} and {{getAllowedLocalityLevel}} methods.A sample info: * getAllowedLocalityLevelByTime: {code} 2015-11-27 09:54:21,553 DEBUG org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt: Waiting time is 32570 ms, more than nodeLocalityDelay time 1000 ms, change allowedLocality from NODE_LOCAL to RACK_LOCAL, priority:10, app attempt id:appattempt_1448589183973_0001_01 2015-11-27 09:54:21,553 DEBUG org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt: Assign container on qihe129040 node, requestType:OFF_SWITCH, allowedLocality:RACK_LOCAL, schedulingOpportunities:0, priority:10, app attempt id:appattempt_1448589183973_0001_01 {code} * getAllowedLocalityLevel: {code} 2015-11-27 09:42:47,132 DEBUG org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt: SchedulingOpportunities count is 2, more than nodeLocalityThreshold num 0.899761581421, change allowedLocality from NODE_LOCAL to RACK_LOCAL, priority:10, app attempt id:appattempt_1448588357362_0001_01 2015-11-27 09:42:47,132 DEBUG org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt: Assign container on qihe129040 node, requestType:OFF_SWITCH, allowedLocality:RACK_LOCAL, schedulingOpportunities:0, priority:10, app attempt id:appattempt_1448588357362_0001_01 {code} > Log the trance information on FSAppAttempt#assignContainer > -- > > Key: YARN-4396 > URL: https://issues.apache.org/jira/browse/YARN-4396 > Project: Hadoop YARN > Issue Type: Improvement > Components: applications, scheduler >Affects Versions: 2.7.1 >Reporter: Lin Yiqun >Assignee: Lin Yiqun > Attachments: YARN-4396.001.patch, YARN-4396.002.patch, > YARN-4396.003.patch > > > When I configure the yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack to open this function, I have no > detail info of assigning container's locality. And it's important because it > will lead some delay scheduling and will have an influence on my cluster. If > I know these info, I can adjust param in cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer
[ https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Yiqun updated YARN-4396: Attachment: YARN-4396.002.patch Modify some debug infos. > Log the trance information on FSAppAttempt#assignContainer > -- > > Key: YARN-4396 > URL: https://issues.apache.org/jira/browse/YARN-4396 > Project: Hadoop YARN > Issue Type: Improvement > Components: applications, scheduler >Affects Versions: 2.7.1 >Reporter: Lin Yiqun >Assignee: Lin Yiqun > Attachments: YARN-4396.001.patch, YARN-4396.002.patch > > > When I configure the yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack to open this function, I have no > detail info of assigning container's locality. And it's important because it > will lead some delay scheduling and will have an influence on my cluster. If > I know these info, I can adjust param in cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer
[ https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Yiqun updated YARN-4396: Attachment: YARN-4396.001.patch > Log the trance information on FSAppAttempt#assignContainer > -- > > Key: YARN-4396 > URL: https://issues.apache.org/jira/browse/YARN-4396 > Project: Hadoop YARN > Issue Type: Improvement > Components: applications, scheduler >Affects Versions: 2.7.1 >Reporter: Lin Yiqun >Assignee: Lin Yiqun > Attachments: YARN-4396.001.patch > > > When I configure the yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack to open this function, I have no > detail info of assigning container's locality. And it's important because it > will lead some delay scheduling and will have an influence on my cluster. If > I know these info, I can adjust param in cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)