[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer

2016-10-27 Thread Yiqun Lin (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yiqun Lin updated YARN-4396:

Attachment: YARN-4396.005.patch

Thanks [~templedf] for the comments. Your comments look good to me. Make two 
changes in the latets patch:

* Change log level from {{debug}} to {{trace}}.
* Do not print the log when the locality doesn't change.

Thanks for reviewing!

> Log the trance information on FSAppAttempt#assignContainer
> --
>
> Key: YARN-4396
> URL: https://issues.apache.org/jira/browse/YARN-4396
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, fairscheduler
>Affects Versions: 2.7.1
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
>  Labels: oct16-easy
> Attachments: YARN-4396.001.patch, YARN-4396.002.patch, 
> YARN-4396.003.patch, YARN-4396.004.patch, YARN-4396.005.patch
>
>
> When I configure the yarn.scheduler.fair.locality.threshold.node and 
> yarn.scheduler.fair.locality.threshold.rack to open this function, I have no 
> detail info of assigning container's locality. And it's important because it 
> will lead some delay scheduling and will have an influence on my cluster. If 
> I know these info, I can adjust param in cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer

2016-10-27 Thread Ray Chiang (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ray Chiang updated YARN-4396:
-
Labels: oct16-easy  (was: oct-16-easy)

> Log the trance information on FSAppAttempt#assignContainer
> --
>
> Key: YARN-4396
> URL: https://issues.apache.org/jira/browse/YARN-4396
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, fairscheduler
>Affects Versions: 2.7.1
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
>  Labels: oct16-easy
> Attachments: YARN-4396.001.patch, YARN-4396.002.patch, 
> YARN-4396.003.patch, YARN-4396.004.patch
>
>
> When I configure the yarn.scheduler.fair.locality.threshold.node and 
> yarn.scheduler.fair.locality.threshold.rack to open this function, I have no 
> detail info of assigning container's locality. And it's important because it 
> will lead some delay scheduling and will have an influence on my cluster. If 
> I know these info, I can adjust param in cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer

2016-10-27 Thread Billie Rinaldi (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Billie Rinaldi updated YARN-4396:
-
Labels: oct-16-easy  (was: )

> Log the trance information on FSAppAttempt#assignContainer
> --
>
> Key: YARN-4396
> URL: https://issues.apache.org/jira/browse/YARN-4396
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, fairscheduler
>Affects Versions: 2.7.1
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
>  Labels: oct-16-easy
> Attachments: YARN-4396.001.patch, YARN-4396.002.patch, 
> YARN-4396.003.patch, YARN-4396.004.patch
>
>
> When I configure the yarn.scheduler.fair.locality.threshold.node and 
> yarn.scheduler.fair.locality.threshold.rack to open this function, I have no 
> detail info of assigning container's locality. And it's important because it 
> will lead some delay scheduling and will have an influence on my cluster. If 
> I know these info, I can adjust param in cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer

2016-10-27 Thread Billie Rinaldi (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Billie Rinaldi updated YARN-4396:
-
Component/s: (was: scheduler)
 fairscheduler

> Log the trance information on FSAppAttempt#assignContainer
> --
>
> Key: YARN-4396
> URL: https://issues.apache.org/jira/browse/YARN-4396
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, fairscheduler
>Affects Versions: 2.7.1
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
>  Labels: oct-16-easy
> Attachments: YARN-4396.001.patch, YARN-4396.002.patch, 
> YARN-4396.003.patch, YARN-4396.004.patch
>
>
> When I configure the yarn.scheduler.fair.locality.threshold.node and 
> yarn.scheduler.fair.locality.threshold.rack to open this function, I have no 
> detail info of assigning container's locality. And it's important because it 
> will lead some delay scheduling and will have an influence on my cluster. If 
> I know these info, I can adjust param in cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer

2015-11-27 Thread Lin Yiqun (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lin Yiqun updated YARN-4396:

Attachment: YARN-4396.004.patch

Simplified the debug infos.

> Log the trance information on FSAppAttempt#assignContainer
> --
>
> Key: YARN-4396
> URL: https://issues.apache.org/jira/browse/YARN-4396
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, scheduler
>Affects Versions: 2.7.1
>Reporter: Lin Yiqun
>Assignee: Lin Yiqun
> Attachments: YARN-4396.001.patch, YARN-4396.002.patch, 
> YARN-4396.003.patch, YARN-4396.004.patch
>
>
> When I configure the yarn.scheduler.fair.locality.threshold.node and 
> yarn.scheduler.fair.locality.threshold.rack to open this function, I have no 
> detail info of assigning container's locality. And it's important because it 
> will lead some delay scheduling and will have an influence on my cluster. If 
> I know these info, I can adjust param in cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer

2015-11-26 Thread Lin Yiqun (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lin Yiqun updated YARN-4396:

Attachment: YARN-4396.003.patch

Add the debug infos in {{getAllowedLocalityLevelByTime}} and 
{{getAllowedLocalityLevel}} methods.A sample info:
* getAllowedLocalityLevelByTime:
{code}
2015-11-27 09:54:21,553 DEBUG 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt: 
Waiting time is 32570 ms, more than nodeLocalityDelay time 1000 ms, change 
allowedLocality from NODE_LOCAL to RACK_LOCAL, priority:10, app attempt 
id:appattempt_1448589183973_0001_01
2015-11-27 09:54:21,553 DEBUG 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt: 
Assign container on qihe129040 node, requestType:OFF_SWITCH, 
allowedLocality:RACK_LOCAL, schedulingOpportunities:0, priority:10, app attempt 
id:appattempt_1448589183973_0001_01
{code}

* getAllowedLocalityLevel:
{code}
2015-11-27 09:42:47,132 DEBUG 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt: 
SchedulingOpportunities count is 2, more than nodeLocalityThreshold num 
0.899761581421, change allowedLocality from NODE_LOCAL to RACK_LOCAL, 
priority:10, app attempt id:appattempt_1448588357362_0001_01
2015-11-27 09:42:47,132 DEBUG 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt: 
Assign container on qihe129040 node, requestType:OFF_SWITCH, 
allowedLocality:RACK_LOCAL, schedulingOpportunities:0, priority:10, app attempt 
id:appattempt_1448588357362_0001_01
{code}

> Log the trance information on FSAppAttempt#assignContainer
> --
>
> Key: YARN-4396
> URL: https://issues.apache.org/jira/browse/YARN-4396
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, scheduler
>Affects Versions: 2.7.1
>Reporter: Lin Yiqun
>Assignee: Lin Yiqun
> Attachments: YARN-4396.001.patch, YARN-4396.002.patch, 
> YARN-4396.003.patch
>
>
> When I configure the yarn.scheduler.fair.locality.threshold.node and 
> yarn.scheduler.fair.locality.threshold.rack to open this function, I have no 
> detail info of assigning container's locality. And it's important because it 
> will lead some delay scheduling and will have an influence on my cluster. If 
> I know these info, I can adjust param in cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer

2015-11-26 Thread Lin Yiqun (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lin Yiqun updated YARN-4396:

Attachment: YARN-4396.001.patch

> Log the trance information on FSAppAttempt#assignContainer
> --
>
> Key: YARN-4396
> URL: https://issues.apache.org/jira/browse/YARN-4396
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, scheduler
>Affects Versions: 2.7.1
>Reporter: Lin Yiqun
>Assignee: Lin Yiqun
> Attachments: YARN-4396.001.patch
>
>
> When I configure the yarn.scheduler.fair.locality.threshold.node and 
> yarn.scheduler.fair.locality.threshold.rack to open this function, I have no 
> detail info of assigning container's locality. And it's important because it 
> will lead some delay scheduling and will have an influence on my cluster. If 
> I know these info, I can adjust param in cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer

2015-11-26 Thread Lin Yiqun (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lin Yiqun updated YARN-4396:

Attachment: YARN-4396.002.patch

Modify some debug infos.

> Log the trance information on FSAppAttempt#assignContainer
> --
>
> Key: YARN-4396
> URL: https://issues.apache.org/jira/browse/YARN-4396
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, scheduler
>Affects Versions: 2.7.1
>Reporter: Lin Yiqun
>Assignee: Lin Yiqun
> Attachments: YARN-4396.001.patch, YARN-4396.002.patch
>
>
> When I configure the yarn.scheduler.fair.locality.threshold.node and 
> yarn.scheduler.fair.locality.threshold.rack to open this function, I have no 
> detail info of assigning container's locality. And it's important because it 
> will lead some delay scheduling and will have an influence on my cluster. If 
> I know these info, I can adjust param in cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)