[jira] [Commented] (YARN-8479) The capacity scheduler logs too frequently seriously affecting performance

2018-06-29 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16527488#comment-16527488 ] YunFan Zhou commented on YARN-8479: --- Thank [~cheersyang] I totally agree with you and i

[jira] [Updated] (YARN-8479) The capacity scheduler logs too frequently seriously affecting performance

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-8479: -- Attachment: image-2018-06-29-14-16-06-332.png > The capacity scheduler logs too frequently seriously aff

[jira] [Updated] (YARN-8479) The capacity scheduler logs too frequently seriously affecting performance

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-8479: -- Description: The capacity scheduler logs too frequently, seriously affecting performance. As a result o

[jira] [Updated] (YARN-8479) The capacity scheduler logs too frequently seriously affecting performance

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-8479: -- Description: The capacity scheduler logs too frequently, seriously affecting performance. As a result o

[jira] [Updated] (YARN-8479) The capacity scheduler logs too frequently seriously affecting performance

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-8479: -- Description: The capacity scheduler logs too frequently, seriously affecting performance. As a result o

[jira] [Assigned] (YARN-8479) The capacity scheduler logs too frequently seriously affecting performance

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou reassigned YARN-8479: - Assignee: YunFan Zhou > The capacity scheduler logs too frequently seriously affecting performanc

[jira] [Created] (YARN-8479) The capacity scheduler logs too frequently seriously affecting performance

2018-06-28 Thread YunFan Zhou (JIRA)
YunFan Zhou created YARN-8479: - Summary: The capacity scheduler logs too frequently seriously affecting performance Key: YARN-8479 URL: https://issues.apache.org/jira/browse/YARN-8479 Project: Hadoop YARN

[jira] [Updated] (YARN-8478) The capacity scheduler logs too frequently seriously affecting performance

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-8478: -- Summary: The capacity scheduler logs too frequently seriously affecting performance (was: The capacity

[jira] [Updated] (YARN-8478) The capacity scheduler logs too frequently, seriously affecting performance

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-8478: -- Description: The capacity scheduler logs too frequently, seriously affecting performance. As a result o

[jira] [Assigned] (YARN-8478) The capacity scheduler logs too frequently seriously affecting performance

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou reassigned YARN-8478: - Assignee: YunFan Zhou > The capacity scheduler logs too frequently seriously affecting performanc

[jira] [Updated] (YARN-8478) The capacity scheduler logs too frequently, seriously affecting performance

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-8478: -- Attachment: image-2018-06-29-14-08-50-981.png > The capacity scheduler logs too frequently, seriously af

[jira] [Created] (YARN-8478) The capacity scheduler logs too frequently, seriously affecting performance

2018-06-28 Thread YunFan Zhou (JIRA)
YunFan Zhou created YARN-8478: - Summary: The capacity scheduler logs too frequently, seriously affecting performance Key: YARN-8478 URL: https://issues.apache.org/jira/browse/YARN-8478 Project: Hadoop YAR

[jira] [Resolved] (YARN-8476) Should check the resource of assignment is greater than Resources.none() before submitResourceCommitRequest

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou resolved YARN-8476. --- Resolution: Won't Fix > Should check the resource of assignment is greater than Resources.none() > be

[jira] [Updated] (YARN-8476) Should check the resource of assignment is greater than Resources.none() before submitResourceCommitRequest

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-8476: -- Description: (was: Hi, [~leftnoteasy] We recently merge https://issues.apache.org/jira/browse/YARN-5

[jira] [Updated] (YARN-8476) Should check the resource of assignment is greater than Resources.none() before submitResourceCommitRequest

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-8476: -- Priority: Minor (was: Blocker) > Should check the resource of assignment is greater than Resources.none

[jira] [Updated] (YARN-8476) Should check the resource of assignment is greater than Resources.none() before submitResourceCommitRequest

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-8476: -- Description: Hi, [~leftnoteasy] We recently merge https://issues.apache.org/jira/browse/YARN-5139 into

[jira] [Updated] (YARN-8476) Should check the resource of assignment is greater than Resources.none() before submitResourceCommitRequest

2018-06-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-8476: -- Description: Hi, [~leftnoteasy] We recently merge https://issues.apache.org/jira/browse/YARN-5139 into

[jira] [Created] (YARN-8476) Should check the resource of assignment is greater than Resources.none() before submitResourceCommitRequest

2018-06-28 Thread YunFan Zhou (JIRA)
YunFan Zhou created YARN-8476: - Summary: Should check the resource of assignment is greater than Resources.none() before submitResourceCommitRequest Key: YARN-8476 URL: https://issues.apache.org/jira/browse/YARN-8476

[jira] [Commented] (YARN-7535) We should display origin value of demand in fair scheduler page

2017-11-26 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16266004#comment-16266004 ] YunFan Zhou commented on YARN-7535: --- Hi, [~yufeigu] [~templedf] I'm sorry to bother you,

[jira] [Assigned] (YARN-7535) We should display origin value of demand in fair scheduler page

2017-11-26 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou reassigned YARN-7535: - Assignee: YunFan Zhou > We should display origin value of demand in fair scheduler page >

[jira] [Commented] (YARN-7534) Fair scheduler assign resources may exceed maxResources

2017-11-20 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16259011#comment-16259011 ] YunFan Zhou commented on YARN-7534: --- [~wilfreds] You can take it for free! > Fair schedu

[jira] [Updated] (YARN-7535) We should display origin value of demand in fair scheduler page

2017-11-20 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7535: -- Description: The value of *demand* of leaf queue that we now view on the fair scheduler page shows only

[jira] [Created] (YARN-7535) Should display origin value of demand in fair scheduler page

2017-11-20 Thread YunFan Zhou (JIRA)
YunFan Zhou created YARN-7535: - Summary: Should display origin value of demand in fair scheduler page Key: YARN-7535 URL: https://issues.apache.org/jira/browse/YARN-7535 Project: Hadoop YARN Iss

[jira] [Updated] (YARN-7535) We should display origin value of demand in fair scheduler page

2017-11-20 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7535: -- Summary: We should display origin value of demand in fair scheduler page (was: Should display origin val

[jira] [Comment Edited] (YARN-7534) Fair scheduler assign resources may exceed maxResources

2017-11-19 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16258849#comment-16258849 ] YunFan Zhou edited comment on YARN-7534 at 11/20/17 6:34 AM: - [

[jira] [Comment Edited] (YARN-7534) Fair scheduler assign resources may exceed maxResources

2017-11-19 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16258849#comment-16258849 ] YunFan Zhou edited comment on YARN-7534 at 11/20/17 6:33 AM: - [

[jira] [Commented] (YARN-7534) Fair scheduler assign resources may exceed maxResources

2017-11-19 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16258849#comment-16258849 ] YunFan Zhou commented on YARN-7534: --- [~templedf] For example, a queue resource usage time

[jira] [Updated] (YARN-7534) Fair scheduler assign resources may exceed maxResources

2017-11-19 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7534: -- Description: The logic we're scheduling now is to check whether the resources used by the queue has excee

[jira] [Updated] (YARN-7534) Fair scheduler assign resources may exceed maxResources

2017-11-19 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7534: -- Description: The logic we're scheduling now is to check whether the resources used by the queue has excee

[jira] [Updated] (YARN-7534) Fair scheduler assign resources may exceed maxResources

2017-11-19 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7534: -- Description: The logic we're scheduling now is to check whether the resources used by the queue has excee

[jira] [Updated] (YARN-7534) Fair scheduler assign resources may exceed maxResources

2017-11-19 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7534: -- Description: The logic we're scheduling now is to check whether the resources used by the queue before as

[jira] [Created] (YARN-7534) Fair scheduler dispatch resources may exceed maxResources

2017-11-19 Thread YunFan Zhou (JIRA)
YunFan Zhou created YARN-7534: - Summary: Fair scheduler dispatch resources may exceed maxResources Key: YARN-7534 URL: https://issues.apache.org/jira/browse/YARN-7534 Project: Hadoop YARN Issue T

[jira] [Updated] (YARN-7534) Fair scheduler assign resources may exceed maxResources

2017-11-19 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7534: -- Summary: Fair scheduler assign resources may exceed maxResources (was: Fair scheduler dispatch resources

[jira] [Created] (YARN-7444) Add non-exclusive node labels for fair scheduler

2017-11-05 Thread YunFan Zhou (JIRA)
YunFan Zhou created YARN-7444: - Summary: Add non-exclusive node labels for fair scheduler Key: YARN-7444 URL: https://issues.apache.org/jira/browse/YARN-7444 Project: Hadoop YARN Issue Type: Impr

[jira] [Commented] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-09-08 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158318#comment-16158318 ] YunFan Zhou commented on YARN-7107: --- Hi, [~templedf]. I have uploaded my v1 patch. Please

[jira] [Updated] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-09-08 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7107: -- Attachment: (was: YARN-7107.001.preview.patch) > add ability in Fair Scheduler to configure whether d

[jira] [Updated] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-09-08 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7107: -- Attachment: YARN-7107.001.patch > add ability in Fair Scheduler to configure whether disable a queue > --

[jira] [Updated] (YARN-7167) Replace the MutableCounter type data in "QueueMetrics*" because the RM restart results in data disturbances on ganglia

2017-09-06 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7167: -- Description: Replace the MutableCounter type data in "*QueueMetrics**" because the RM restart results in

[jira] [Created] (YARN-7167) Replace the MutableCounter type data in "QueueMetrics*" because the RM restart results in data disturbances on ganglia

2017-09-06 Thread YunFan Zhou (JIRA)
YunFan Zhou created YARN-7167: - Summary: Replace the MutableCounter type data in "QueueMetrics*" because the RM restart results in data disturbances on ganglia Key: YARN-7167 URL: https://issues.apache.org/jira/browse

[jira] [Updated] (YARN-7155) Do not sort the inactive queue when assigning container

2017-09-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7155: -- Summary: Do not sort the inactive queue when assigning container (was: Do not sort the queues without an

[jira] [Created] (YARN-7155) Do not sort the queues without any application when assigning container

2017-09-04 Thread YunFan Zhou (JIRA)
YunFan Zhou created YARN-7155: - Summary: Do not sort the queues without any application when assigning container Key: YARN-7155 URL: https://issues.apache.org/jira/browse/YARN-7155 Project: Hadoop YARN

[jira] [Commented] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-09-03 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151842#comment-16151842 ] YunFan Zhou commented on YARN-7107: --- Hi, [~templedf] I think we can split the queue into

[jira] [Commented] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-09-03 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151841#comment-16151841 ] YunFan Zhou commented on YARN-7107: --- Thank [~templedf] for your review, I benefited a lot

[jira] [Commented] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-09-03 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151827#comment-16151827 ] YunFan Zhou commented on YARN-7107: --- [~templedf] Please help me review my code, thank you

[jira] [Updated] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-09-03 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7107: -- Attachment: (was: YARN-7107.001.preview.patch) > add ability in Fair Scheduler to configure whether d

[jira] [Updated] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-09-03 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7107: -- Attachment: YARN-7107.001.preview.patch > add ability in Fair Scheduler to configure whether disable a qu

[jira] [Updated] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-09-03 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7107: -- Attachment: YARN-7107.001.preview.patch > add ability in Fair Scheduler to configure whether disable a qu

[jira] [Updated] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-09-03 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7107: -- Attachment: (was: YARN-7107.001.preview.patch) > add ability in Fair Scheduler to configure whether d

[jira] [Updated] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-09-02 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7107: -- Attachment: YARN-7107.001.preview.patch > add ability in Fair Scheduler to configure whether disable a qu

[jira] [Comment Edited] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-08-30 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16148376#comment-16148376 ] YunFan Zhou edited comment on YARN-7107 at 8/31/17 3:01 AM: {no

[jira] [Commented] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-08-30 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16148376#comment-16148376 ] YunFan Zhou commented on YARN-7107: --- {noformat} The trick is how to zero out non-sibling

[jira] [Commented] (YARN-2497) Changes for fair scheduler to support allocate resource respect labels

2017-08-30 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16147013#comment-16147013 ] YunFan Zhou commented on YARN-2497: --- Thanks for working on this, [~templedf]. some nits:

[jira] [Comment Edited] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-08-30 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146890#comment-16146890 ] YunFan Zhou edited comment on YARN-7107 at 8/30/17 8:33 AM: Tha

[jira] [Commented] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-08-30 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146890#comment-16146890 ] YunFan Zhou commented on YARN-7107: --- Thank [~templedf], Here are some attributes I think

[jira] [Commented] (YARN-7110) NodeManager always crash for spark shuffle service out of memory

2017-08-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143954#comment-16143954 ] YunFan Zhou commented on YARN-7110: --- Thank [~jlowe]. Is there a plan for this issue? It s

[jira] [Updated] (YARN-7110) NodeManager always crash for spark shuffle service out of memory

2017-08-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7110: -- Description: NM often crash due to the Spark shuffle service, I can saw many error log messages before

[jira] [Comment Edited] (YARN-7110) NodeManager always crash for spark shuffle service out of memory

2017-08-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143894#comment-16143894 ] YunFan Zhou edited comment on YARN-7110 at 8/28/17 3:27 PM: [~j

[jira] [Commented] (YARN-7110) NodeManager always crash for spark shuffle service out of memory

2017-08-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143894#comment-16143894 ] YunFan Zhou commented on YARN-7110: --- [~jlowe] Thank Jason. I think this is more than just

[jira] [Comment Edited] (YARN-7110) NodeManager always crash for spark shuffle service out of memory

2017-08-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143894#comment-16143894 ] YunFan Zhou edited comment on YARN-7110 at 8/28/17 3:26 PM: [~j

[jira] [Updated] (YARN-7110) NodeManager always crash for spark shuffle service out of memory

2017-08-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7110: -- Description: NM often crash due to the Spark shuffle service, I can saw many error log messages before

[jira] [Updated] (YARN-7110) NodeManager always crash for spark shuffle service out of memory

2017-08-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7110: -- Attachment: screenshot-1.png > NodeManager always crash for spark shuffle service out of memory > ---

[jira] [Updated] (YARN-7110) NodeManager always crash for spark shuffle service out of memory

2017-08-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7110: -- Description: NM often crash due to the Spark shuffle service, I can saw many error log messages before

[jira] [Updated] (YARN-7110) NodeManager always crash for spark shuffle service out of memory

2017-08-28 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7110: -- Component/s: nodemanager > NodeManager always crash for spark shuffle service out of memory > ---

[jira] [Created] (YARN-7110) NodeManager always crash for spark shuffle service out of memory

2017-08-28 Thread YunFan Zhou (JIRA)
YunFan Zhou created YARN-7110: - Summary: NodeManager always crash for spark shuffle service out of memory Key: YARN-7110 URL: https://issues.apache.org/jira/browse/YARN-7110 Project: Hadoop YARN

[jira] [Updated] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-08-27 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7107: -- Labels: fairscheduler (was: ) > add ability in Fair Scheduler to configure whether disable a queue > ---

[jira] [Assigned] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-08-27 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou reassigned YARN-7107: - Assignee: YunFan Zhou > add ability in Fair Scheduler to configure whether disable a queue > -

[jira] [Updated] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-08-27 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7107: -- Description: In a production environment, emergency situations (such as the need to calculate the import

[jira] [Updated] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-08-27 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7107: -- Description: In a production environment, emergency situations (such as the need to calculate the import

[jira] [Updated] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-08-27 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-7107: -- Description: In a production environment, emergency situations (such as the need to calculate the import

[jira] [Created] (YARN-7107) add ability in Fair Scheduler to configure whether disable a queue

2017-08-27 Thread YunFan Zhou (JIRA)
YunFan Zhou created YARN-7107: - Summary: add ability in Fair Scheduler to configure whether disable a queue Key: YARN-7107 URL: https://issues.apache.org/jira/browse/YARN-7107 Project: Hadoop YARN

[jira] [Commented] (YARN-2162) add ability in Fair Scheduler to optionally configure maxResources in terms of percentage

2017-08-27 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143132#comment-16143132 ] YunFan Zhou commented on YARN-2162: --- [~yufeigu] Thank Yufei. It's a very nice feature, an

[jira] [Commented] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-17 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16131803#comment-16131803 ] YunFan Zhou commented on YARN-6361: --- [~yufeigu] Thank you Yufei. It was in our productio

[jira] [Commented] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-14 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126727#comment-16126727 ] YunFan Zhou commented on YARN-6361: --- [~yufeigu] [~sunilg] Could you please help me review

[jira] [Comment Edited] (YARN-6981) The “Total Resource” displayed in RM node label page is inaccurate when refresh node resources

2017-08-11 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16123041#comment-16123041 ] YunFan Zhou edited comment on YARN-6981 at 8/11/17 8:44 AM: Tha

[jira] [Commented] (YARN-6981) The “Total Resource” displayed in RM node label page is inaccurate when refresh node resources

2017-08-11 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16123041#comment-16123041 ] YunFan Zhou commented on YARN-6981: --- Thanks [~sunilg], we can reproduce the problem by fo

[jira] [Updated] (YARN-6981) The “Total Resource” displayed in RM node label page is inaccurate when refresh node resources

2017-08-11 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6981: -- Attachment: screenshot-3.png > The “Total Resource” displayed in RM node label page is inaccurate when >

[jira] [Updated] (YARN-6981) The “Total Resource” displayed in RM node label page is inaccurate when refresh node resources

2017-08-11 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6981: -- Attachment: screenshot-1.png > The “Total Resource” displayed in RM node label page is inaccurate when >

[jira] [Updated] (YARN-6981) The “Total Resource” displayed in RM node label page is inaccurate when refresh node resources

2017-08-11 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6981: -- Attachment: screenshot-2.png > The “Total Resource” displayed in RM node label page is inaccurate when >

[jira] [Updated] (YARN-6981) The “Total Resource” displayed in RM node label page is inaccurate when refresh node resources

2017-08-10 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6981: -- Component/s: fairscheduler > The “Total Resource” displayed in RM node label page is inaccurate when > r

[jira] [Updated] (YARN-6981) The “Total Resource” displayed in RM node label page is inaccurate when refresh node resources

2017-08-10 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6981: -- Description: The *Total Resource* displayed in RM node label page is inaccurate when refresh node resourc

[jira] [Updated] (YARN-6981) The “Total Resource” displayed in RM node label page is inaccurate when refresh node resources

2017-08-10 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6981: -- Description: The “Total Resource” displayed in RM node label page is inaccurate when refresh node resourc

[jira] [Updated] (YARN-6981) The “Total Resource” displayed in RM node label page is inaccurate when refresh node resources

2017-08-10 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6981: -- Description: The “Total Resource” displayed in RM node label page is inaccurate when refresh node resourc

[jira] [Updated] (YARN-6981) The “Total Resource” displayed in RM node label page is inaccurate when refresh node resources

2017-08-10 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6981: -- Summary: The “Total Resource” displayed in RM node label page is inaccurate when refresh node resources

[jira] [Updated] (YARN-6981) The “Total Resource” displayed in RM node label page is inaccurate when refresh node resources

2017-08-10 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6981: -- Description: The “Total Resource” displayed in RM node label page is > The “Total Resource” displayed in

[jira] [Created] (YARN-6981) The “Total Resource” displayed in RM node page is inaccurate when refresh node resources

2017-08-10 Thread YunFan Zhou (JIRA)
YunFan Zhou created YARN-6981: - Summary: The “Total Resource” displayed in RM node page is inaccurate when refresh node resources Key: YARN-6981 URL: https://issues.apache.org/jira/browse/YARN-6981 Projec

[jira] [Commented] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-09 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119528#comment-16119528 ] YunFan Zhou commented on YARN-6361: --- [~yufeigu][~Naganarasimha] Hi, Yufei, Naganarasimha.

[jira] [Issue Comment Deleted] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-09 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6361: -- Comment: was deleted (was: [~yufeigu] Thank you very much, Yufei. I have to say that this JIRA's descri

[jira] [Issue Comment Deleted] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-09 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6361: -- Comment: was deleted (was: [~yufeigu] [~Naganarasimha] Hi, Yufei, Naganarasimha. I've found that this opt

[jira] [Updated] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-09 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6361: -- Attachment: (was: YARN-6361.001.pre.patch) > FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage

[jira] [Issue Comment Deleted] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-09 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6361: -- Comment: was deleted (was: For personal environmental reasons, I uploaded an incomplete patch. And I wi

[jira] [Updated] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-09 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6361: -- Attachment: YARN-6361.001.patch > FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with b

[jira] [Commented] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-08 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118524#comment-16118524 ] YunFan Zhou commented on YARN-6361: --- [~yufeigu] [~Naganarasimha] Hi, Yufei, Naganarasimha

[jira] [Commented] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-06 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115832#comment-16115832 ] YunFan Zhou commented on YARN-6361: --- For personal environmental reasons, I uploaded an i

[jira] [Comment Edited] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-06 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115449#comment-16115449 ] YunFan Zhou edited comment on YARN-6361 at 8/6/17 3:04 PM: --- [~yuf

[jira] [Updated] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-06 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6361: -- Attachment: YARN-6361.001.pre.patch > FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high wi

[jira] [Comment Edited] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-05 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115449#comment-16115449 ] YunFan Zhou edited comment on YARN-6361 at 8/5/17 4:54 PM: --- [~yuf

[jira] [Comment Edited] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-05 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115449#comment-16115449 ] YunFan Zhou edited comment on YARN-6361 at 8/5/17 4:52 PM: --- [~yuf

[jira] [Commented] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-05 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115449#comment-16115449 ] YunFan Zhou commented on YARN-6361: --- [~yufeigu] Thank you very much, Yufei. I have to s

[jira] [Comment Edited] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115205#comment-16115205 ] YunFan Zhou edited comment on YARN-6361 at 8/5/17 1:49 AM: --- [~yuf

[jira] [Updated] (YARN-6361) FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues

2017-08-04 Thread YunFan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6361: -- Priority: Major (was: Minor) > FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big

  1   2   >