[jira] [Updated] (YARN-5222) DockerContainerExecutor dosn't set work directory

2016-06-09 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-5222: -- Attachment: YARN-5222.patch > DockerContainerExecutor dosn't set work directory >

[jira] [Created] (YARN-5222) DockerContainerExecutor:DockerContainerExecutor dosn't set work directory

2016-06-09 Thread zhengchenyu (JIRA)
zhengchenyu created YARN-5222: - Summary: DockerContainerExecutor:DockerContainerExecutor dosn't set work directory Key: YARN-5222 URL: https://issues.apache.org/jira/browse/YARN-5222 Project: Hadoop YARN

[jira] [Updated] (YARN-5222) DockerContainerExecutor dosn't set work directory

2016-06-09 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-5222: -- Summary: DockerContainerExecutor dosn't set work directory (was:

[jira] [Updated] (YARN-5222) DockerContainerExecutor dosn't set work directory

2016-06-12 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-5222: -- Attachment: (was: YARN-5222.patch) > DockerContainerExecutor dosn't set work directory >

[jira] [Resolved] (YARN-5846) Improve the fairscheduler attemptScheduler

2017-01-18 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu resolved YARN-5846. --- Resolution: Duplicate Fix Version/s: 2.7.1 Release Note: please pay attention to YARN-5139

[jira] [Commented] (YARN-5846) Improve the fairscheduler attemptScheduler

2017-01-18 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829221#comment-15829221 ] zhengchenyu commented on YARN-5846: --- please pay attention to YARN-5139 > Improve the fairscheduler

[jira] [Updated] (YARN-6091) the AppMaster register failed when use Docker on LinuxContainer

2017-01-17 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-6091: -- Affects Version/s: (was: 2.8.0) 2.8.1 > the AppMaster register failed when

[jira] [Updated] (YARN-5846) Improve the fairscheduler attemptScheduler

2016-11-09 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-5846: -- Priority: Critical (was: Minor) > Improve the fairscheduler attemptScheduler >

[jira] [Issue Comment Deleted] (YARN-5846) Improve the fairscheduler attemptScheduler

2016-11-07 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-5846: -- Comment: was deleted (was: Yeah! I thinks my suggestion may be a new scheduler. And YARN-5139 is indeed

[jira] [Commented] (YARN-5846) Improve the fairscheduler attemptScheduler

2016-11-07 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646635#comment-15646635 ] zhengchenyu commented on YARN-5846: --- Yeah! I thinks my suggestion may be a new scheduler. And YARN-5139

[jira] [Commented] (YARN-5846) Improve the fairscheduler attemptScheduler

2016-11-07 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646638#comment-15646638 ] zhengchenyu commented on YARN-5846: --- Yeah! I thinks my suggestion may be a new scheduler. And YARN-5139

[jira] [Comment Edited] (YARN-5846) Improve the fairscheduler attemptScheduler

2016-11-07 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646638#comment-15646638 ] zhengchenyu edited comment on YARN-5846 at 11/8/16 6:41 AM: Yeah! I thinks my

[jira] [Created] (YARN-5846) Improve the fairscheduler attemptScheduler

2016-11-07 Thread zhengchenyu (JIRA)
zhengchenyu created YARN-5846: - Summary: Improve the fairscheduler attemptScheduler Key: YARN-5846 URL: https://issues.apache.org/jira/browse/YARN-5846 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-5188) FairScheduler performance bug

2016-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15697478#comment-15697478 ] zhengchenyu commented on YARN-5188: --- the dead lock is same with Yarn-4090. getQueueUserAcls got the

[jira] [Commented] (YARN-5188) FairScheduler performance bug

2016-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15697477#comment-15697477 ] zhengchenyu commented on YARN-5188: --- the dead lock is same with Yarn-4090. getQueueUserAcls got the

[jira] [Comment Edited] (YARN-4090) Make Collections.sort() more efficient in FSParentQueue.java

2016-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695578#comment-15695578 ] zhengchenyu edited comment on YARN-4090 at 11/26/16 6:30 AM: - here we see a

[jira] [Issue Comment Deleted] (YARN-5188) FairScheduler performance bug

2016-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-5188: -- Comment: was deleted (was: the dead lock is same with Yarn-4090. getQueueUserAcls got the object lock

[jira] [Updated] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-11-24 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-5936: -- Description: When using LinuxContainer, the setting that

[jira] [Updated] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-11-24 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-5936: -- Description: When using LinuxContainer, the setting that

[jira] [Created] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-11-24 Thread zhengchenyu (JIRA)
zhengchenyu created YARN-5936: - Summary: when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers Key: YARN-5936 URL: https://issues.apache.org/jira/browse/YARN-5936

[jira] [Updated] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-11-24 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-5936: -- Description: When using LinuxContainer, the setting that

[jira] [Updated] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-11-24 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-5936: -- Description: When using LinuxContainer, the setting that

[jira] [Updated] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-11-24 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-5936: -- Description: When using LinuxContainer, the setting that

[jira] [Commented] (YARN-4090) Make Collections.sort() more efficient in FSParentQueue.java

2016-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695579#comment-15695579 ] zhengchenyu commented on YARN-4090: --- here we see a dead block: "IPC Server handler 98 on 8032" is

[jira] [Commented] (YARN-4090) Make Collections.sort() more efficient in FSParentQueue.java

2016-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695578#comment-15695578 ] zhengchenyu commented on YARN-4090: --- here we see a dead block: "IPC Server handler 98 on 8032" is

[jira] [Issue Comment Deleted] (YARN-4090) Make Collections.sort() more efficient in FSParentQueue.java

2016-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-4090: -- Comment: was deleted (was: here we see a dead block: "IPC Server handler 98 on 8032" is waiting for

[jira] [Comment Edited] (YARN-4090) Make Collections.sort() more efficient in FSParentQueue.java

2016-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695578#comment-15695578 ] zhengchenyu edited comment on YARN-4090 at 11/25/16 11:01 AM: -- here we see a

[jira] [Comment Edited] (YARN-4090) Make Collections.sort() more efficient in FSParentQueue.java

2016-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695578#comment-15695578 ] zhengchenyu edited comment on YARN-4090 at 11/25/16 11:25 AM: -- here we see a

[jira] [Comment Edited] (YARN-4090) Make Collections.sort() more efficient in FSParentQueue.java

2016-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695578#comment-15695578 ] zhengchenyu edited comment on YARN-4090 at 11/25/16 10:59 AM: -- here we see a

[jira] [Comment Edited] (YARN-4090) Make Collections.sort() more efficient in FSParentQueue.java

2016-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695578#comment-15695578 ] zhengchenyu edited comment on YARN-4090 at 11/25/16 11:04 AM: -- here we see a

[jira] [Comment Edited] (YARN-4090) Make Collections.sort() more efficient in FSParentQueue.java

2016-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695578#comment-15695578 ] zhengchenyu edited comment on YARN-4090 at 11/25/16 11:26 AM: -- here we see a

[jira] [Comment Edited] (YARN-4090) Make Collections.sort() more efficient in FSParentQueue.java

2016-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695578#comment-15695578 ] zhengchenyu edited comment on YARN-4090 at 11/25/16 11:26 AM: -- here we see a

[jira] [Commented] (YARN-5827) DockerLinuxContainer can't limit the cpu usage

2016-11-20 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15680979#comment-15680979 ] zhengchenyu commented on YARN-5827: --- Thanks! I tried Yarn-4553! I think that's a reasonable way to solve

[jira] [Created] (YARN-5919) FairSharePolicy is not reasonable when sorting the application

2016-11-21 Thread zhengchenyu (JIRA)
zhengchenyu created YARN-5919: - Summary: FairSharePolicy is not reasonable when sorting the application Key: YARN-5919 URL: https://issues.apache.org/jira/browse/YARN-5919 Project: Hadoop YARN

[jira] [Commented] (YARN-5827) DockerLinuxContainer can't limit the cpu usage

2016-11-21 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15683593#comment-15683593 ] zhengchenyu commented on YARN-5827: --- Oh, my god! I found "/usr/lib/systemd/system/docker.service" has

[jira] [Commented] (YARN-5827) DockerLinuxContainer can't limit the cpu usage

2016-11-20 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682331#comment-15682331 ] zhengchenyu commented on YARN-5827: --- Sorry, I reconfigurated and restarted docker by your way. still

[jira] [Created] (YARN-5827) DockerLinuxContainer can't limit the cpu usage

2016-11-03 Thread zhengchenyu (JIRA)
zhengchenyu created YARN-5827: - Summary: DockerLinuxContainer can't limit the cpu usage Key: YARN-5827 URL: https://issues.apache.org/jira/browse/YARN-5827 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-6091) the AppMaster register failed when use Docker on LinuxContainer

2017-01-13 Thread zhengchenyu (JIRA)
zhengchenyu created YARN-6091: - Summary: the AppMaster register failed when use Docker on LinuxContainer Key: YARN-6091 URL: https://issues.apache.org/jira/browse/YARN-6091 Project: Hadoop YARN

[jira] [Commented] (YARN-6091) the AppMaster register failed when use Docker on LinuxContainer

2017-01-13 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15821472#comment-15821472 ] zhengchenyu commented on YARN-6091: --- Here is the sever where appliation is running normally {code} [right

[jira] [Created] (YARN-5964) fairscheduler use too many object lock, leads to low performance

2016-12-04 Thread zhengchenyu (JIRA)
zhengchenyu created YARN-5964: - Summary: fairscheduler use too many object lock, leads to low performance Key: YARN-5964 URL: https://issues.apache.org/jira/browse/YARN-5964 Project: Hadoop YARN

[jira] [Commented] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-12-04 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15721247#comment-15721247 ] zhengchenyu commented on YARN-5936: --- I think two reason both affect the performance, but I can't evaluate

[jira] [Commented] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-12-04 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15721341#comment-15721341 ] zhengchenyu commented on YARN-5936: --- You didn't catch my meaning! In fact, I knew the reason of the

[jira] [Commented] (YARN-4090) Make Collections.sort() more efficient in FSParentQueue.java

2017-01-09 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811409#comment-15811409 ] zhengchenyu commented on YARN-4090: --- Take attention to the YARN-5188 which has better performance. Our

[jira] [Commented] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-12-19 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763042#comment-15763042 ] zhengchenyu commented on YARN-5936: --- I don't care the performance loss of cpu bandwith, because I know it

[jira] [Commented] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-12-19 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763020#comment-15763020 ] zhengchenyu commented on YARN-5936: --- But your "time" command is only related to it's own program, every

[jira] [Commented] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-12-19 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763023#comment-15763023 ] zhengchenyu commented on YARN-5936: --- But your "time" command is only related to it's own program, every

[jira] [Commented] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-12-19 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763021#comment-15763021 ] zhengchenyu commented on YARN-5936: --- But your "time" command is only related to it's own program, every

[jira] [Issue Comment Deleted] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-12-19 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-5936: -- Comment: was deleted (was: But your "time" command is only related to it's own program, every program

[jira] [Issue Comment Deleted] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-12-19 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-5936: -- Comment: was deleted (was: But your "time" command is only related to it's own program, every program

[jira] [Resolved] (YARN-5964) Lower the granularity of locks in FairScheduler

2016-12-18 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu resolved YARN-5964. --- Resolution: Duplicate Release Note: This issue is duplicated. Please pay attention to YARN-3091

[jira] [Commented] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2016-12-18 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15760010#comment-15760010 ] zhengchenyu commented on YARN-5936: --- Because your program is a single-thread. > when cpu strict mode is

[jira] [Commented] (YARN-5964) Lower the granularity of locks in FairScheduler

2016-12-18 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15759968#comment-15759968 ] zhengchenyu commented on YARN-5964: --- Please pay attention to YARN-3091 > Lower the granularity of locks

[jira] [Commented] (YARN-3091) [Umbrella] Improve and fix locks of RM scheduler

2016-12-19 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15760615#comment-15760615 ] zhengchenyu commented on YARN-3091: --- Sorry, although this issue is completed, I still wanna ask for

[jira] [Commented] (YARN-3091) [Umbrella] Improve and fix locks of RM scheduler

2016-12-19 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15760613#comment-15760613 ] zhengchenyu commented on YARN-3091: --- Sorry, although this issue is completed, I still wanna ask for

[jira] [Issue Comment Deleted] (YARN-3091) [Umbrella] Improve and fix locks of RM scheduler

2016-12-19 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-3091: -- Comment: was deleted (was: Sorry, although this issue is completed, I still wanna ask for advice. I

[jira] [Commented] (YARN-5964) Lower the granularity of locks in FairScheduler

2016-12-07 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15730808#comment-15730808 ] zhengchenyu commented on YARN-5964: --- When I found this problem, the continuous scheduling turned off. In

[jira] [Commented] (YARN-6407) Improve and fix locks of RM scheduler

2017-03-30 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948731#comment-15948731 ] zhengchenyu commented on YARN-6407: --- [~yufeigu] [~ka...@cloudera.com] We can't true off continuous

[jira] [Comment Edited] (YARN-6407) Improve and fix locks of RM scheduler

2017-03-30 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948731#comment-15948731 ] zhengchenyu edited comment on YARN-6407 at 3/30/17 10:26 AM: - [~yufeigu]

[jira] [Comment Edited] (YARN-6407) Improve and fix locks of RM scheduler

2017-03-30 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948731#comment-15948731 ] zhengchenyu edited comment on YARN-6407 at 3/30/17 10:26 AM: - [~yufeigu]

[jira] [Commented] (YARN-6407) Improve and fix locks of RM scheduler

2017-03-30 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948718#comment-15948718 ] zhengchenyu commented on YARN-6407: --- [~dan...@cloudera.com] Yes, I think so. Though this work is hard, I

[jira] [Created] (YARN-6407) Improve and fix locks of RM scheduler

2017-03-29 Thread zhengchenyu (JIRA)
zhengchenyu created YARN-6407: - Summary: Improve and fix locks of RM scheduler Key: YARN-6407 URL: https://issues.apache.org/jira/browse/YARN-6407 Project: Hadoop YARN Issue Type: Bug

[jira] [Comment Edited] (YARN-6407) Improve and fix locks of RM scheduler

2017-03-29 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946630#comment-15946630 ] zhengchenyu edited comment on YARN-6407 at 3/29/17 6:53 AM: [~vinodkv] Can you

[jira] [Commented] (YARN-6407) Improve and fix locks of RM scheduler

2017-03-29 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946630#comment-15946630 ] zhengchenyu commented on YARN-6407: --- [~vinodkv] > Improve and fix locks of RM scheduler >

[jira] [Commented] (YARN-6407) Improve and fix locks of RM scheduler

2017-03-30 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950187#comment-15950187 ] zhengchenyu commented on YARN-6407: --- [~ka...@cloudera.com] We have used delay scheduling to raise the

[jira] [Commented] (YARN-6091) the AppMaster register failed when use Docker on LinuxContainer

2017-04-05 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958176#comment-15958176 ] zhengchenyu commented on YARN-6091: --- I just changed the judging condition of pclose's return code, then

[jira] [Commented] (YARN-6407) Improve and fix locks of RM scheduler

2017-04-06 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958524#comment-15958524 ] zhengchenyu commented on YARN-6407: --- For exmaple, addNode and removeNode can't happen at the same time

[jira] [Commented] (YARN-2694) Ensure only single node labels specified in resource request / host, and node label expression only specified when resourceName=ANY

2017-07-03 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073156#comment-16073156 ] zhengchenyu commented on YARN-2694: --- [~naganarasimha...@apache.org] Thanks for your explaination, I know

[jira] [Comment Edited] (YARN-2694) Ensure only single node labels specified in resource request / host, and node label expression only specified when resourceName=ANY

2017-07-04 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073156#comment-16073156 ] zhengchenyu edited comment on YARN-2694 at 7/4/17 5:59 AM: ---

[jira] [Comment Edited] (YARN-6396) Call verifyAndCreateRemoteLogDir at service initialization instead of application initialization to decrease load for name node

2017-07-02 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071869#comment-16071869 ] zhengchenyu edited comment on YARN-6396 at 7/3/17 2:58 AM: --- The invocation

[jira] [Commented] (YARN-6299) FairSharePolicy is incorrect when demand is less than min share

2017-07-03 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071998#comment-16071998 ] zhengchenyu commented on YARN-6299: --- In FSAppAttempt, getMinShare return Resources.none(). So demand

[jira] [Commented] (YARN-6396) Call verifyAndCreateRemoteLogDir at service initialization instead of application initialization to decrease load for name node

2017-07-02 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071869#comment-16071869 ] zhengchenyu commented on YARN-6396: --- The invocation 'createAppDir' in initAppAggregator will also visit

[jira] [Commented] (YARN-5188) FairScheduler performance bug

2017-07-04 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074232#comment-16074232 ] zhengchenyu commented on YARN-5188: --- [~chenfolin] we use read-write lock to solve this problem! >

[jira] [Commented] (YARN-6568) A queue which runs a long time job couldn't acquire any container for long time.

2017-05-10 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004604#comment-16004604 ] zhengchenyu commented on YARN-6568: --- I solve this problem, when I set the timeout for queue. In my first

[jira] [Commented] (YARN-6568) A queue which runs a long time job couldn't acquire any container for long time.

2017-05-09 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002155#comment-16002155 ] zhengchenyu commented on YARN-6568: --- [~yufeigu] First, in FairSharePolicy.FairShareComparator.compare,

[jira] [Comment Edited] (YARN-6568) A queue which runs a long time job couldn't acquire any container for long time.

2017-05-09 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003908#comment-16003908 ] zhengchenyu edited comment on YARN-6568 at 5/10/17 1:57 AM: [~yufeigu] Sorry, I

[jira] [Commented] (YARN-6568) A queue which runs a long time job couldn't acquire any container for long time.

2017-05-09 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003908#comment-16003908 ] zhengchenyu commented on YARN-6568: --- [~yufeigu] Sorry, I didn't express definitely! I said that the

[jira] [Comment Edited] (YARN-6568) A queue which runs a long time job couldn't acquire any container for long time.

2017-05-09 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003908#comment-16003908 ] zhengchenyu edited comment on YARN-6568 at 5/10/17 1:56 AM: [~yufeigu] Sorry, I

[jira] [Created] (YARN-6728) Job will run slow when the performance of defaultFs degrades and the log-aggregation is enable.

2017-06-22 Thread zhengchenyu (JIRA)
zhengchenyu created YARN-6728: - Summary: Job will run slow when the performance of defaultFs degrades and the log-aggregation is enable. Key: YARN-6728 URL: https://issues.apache.org/jira/browse/YARN-6728

[jira] [Updated] (YARN-6728) Job will run slow when the performance of defaultFs degrades and the log-aggregation is enable.

2017-06-22 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-6728: -- Description: In our cluster, I found many map keep "NEW" state for several minutes. Here I got the

[jira] [Updated] (YARN-6728) Job will run slow when the performance of defaultFs degrades and the log-aggregation is enable.

2017-06-22 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-6728: -- Description: In our cluster, I found many map keep "NEW" state for several minutes. Here I got the

[jira] [Updated] (YARN-6728) Job will run slow when the performance of defaultFs degrades and the log-aggregation is enable.

2017-06-22 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-6728: -- Description: In our cluster, I found many map keep "NEW" state for several minutes. Here I got the

[jira] [Commented] (YARN-6728) Job will run slow when the performance of defaultFs degrades and the log-aggregation is enable.

2017-06-22 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059056#comment-16059056 ] zhengchenyu commented on YARN-6728: --- Our partner [~maobaolong] advice to remove

[jira] [Updated] (YARN-6728) Job will run slow when the performance of defaultFs degrades and the log-aggregation is enable.

2017-06-22 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-6728: -- Description: In our cluster, I found many map keep "NEW" state for several minutes. Here I got the

[jira] [Updated] (YARN-6728) Job will run slow when the performance of defaultFs degrades and the log-aggregation is enable.

2017-06-23 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-6728: -- Attachment: YARN-6728.patch.00_branch-2.7 > Job will run slow when the performance of defaultFs degrades

[jira] [Comment Edited] (YARN-6728) Job will run slow when the performance of defaultFs degrades and the log-aggregation is enable.

2017-06-23 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059056#comment-16059056 ] zhengchenyu edited comment on YARN-6728 at 6/23/17 7:00 AM: Our partner

[jira] [Commented] (YARN-2694) Ensure only single node labels specified in resource request / host, and node label expression only specified when resourceName=ANY

2017-06-18 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16053411#comment-16053411 ] zhengchenyu commented on YARN-2694: --- I don't know why disabled the function of supporting multi-nodelabel

[jira] [Comment Edited] (YARN-2694) Ensure only single node labels specified in resource request / host, and node label expression only specified when resourceName=ANY

2017-06-18 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16053411#comment-16053411 ] zhengchenyu edited comment on YARN-2694 at 6/19/17 2:21 AM: I don't know why

[jira] [Created] (YARN-6568) A queue which runs a long time job couldn't acquire any container for long time.

2017-05-08 Thread zhengchenyu (JIRA)
zhengchenyu created YARN-6568: - Summary: A queue which runs a long time job couldn't acquire any container for long time. Key: YARN-6568 URL: https://issues.apache.org/jira/browse/YARN-6568 Project:

[jira] [Updated] (YARN-7278) LinuxContainer in docker mode will be failed when nodemanager restart, because timeout for docker is too slow.

2017-10-02 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-7278: -- Affects Version/s: (was: 2.7.1) 2.8.0 > LinuxContainer in docker mode will be

[jira] [Commented] (YARN-7278) LinuxContainer in docker mode will be failed when nodemanager restart, because timeout for docker is too slow.

2017-10-02 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189158#comment-16189158 ] zhengchenyu commented on YARN-7278: --- [~ebadger] Yes, you are right. I changed affects versions to 2.8.0.

[jira] [Commented] (YARN-6223) [Umbrella] Natively support GPU configuration/discovery/scheduling/isolation on YARN

2017-09-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178812#comment-16178812 ] zhengchenyu commented on YARN-6223: --- I thinks NM recovery for gpu doesn't take effect. Becuase

[jira] [Created] (YARN-7278) LinuxContainer in docker mode will be failed when nodemanager restart, because timeout for docker is too slow.

2017-09-30 Thread zhengchenyu (JIRA)
zhengchenyu created YARN-7278: - Summary: LinuxContainer in docker mode will be failed when nodemanager restart, because timeout for docker is too slow. Key: YARN-7278 URL:

[jira] [Updated] (YARN-7278) LinuxContainer in docker mode will be failed when nodemanager restart, because timeout for docker is too slow.

2017-09-30 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-7278: -- Description: In our cluster, nodemanagere recovery is turn on, and we use LinuxConainer with docker

[jira] [Commented] (YARN-6223) [Umbrella] Natively support GPU configuration/discovery/scheduling/isolation on YARN

2017-09-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178761#comment-16178761 ] zhengchenyu commented on YARN-6223: --- I think there is a bug in below code. {code} try {

[jira] [Commented] (YARN-7560) Resourcemanager hangs when resourceUsedWithWeightToResourceRatio return a overflow value

2017-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16265918#comment-16265918 ] zhengchenyu commented on YARN-7560: --- [~yufeigu] the sum of all queue's minRes is over int.max >

[jira] [Created] (YARN-7560) Resourcemanager hangs when resourceUsedWithWeightToResourceRatio return a overflow value

2017-11-25 Thread zhengchenyu (JIRA)
zhengchenyu created YARN-7560: - Summary: Resourcemanager hangs when resourceUsedWithWeightToResourceRatio return a overflow value Key: YARN-7560 URL: https://issues.apache.org/jira/browse/YARN-7560

[jira] [Updated] (YARN-7560) Resourcemanager hangs when resourceUsedWithWeightToResourceRatio return a overflow value

2017-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-7560: -- Attachment: YARN-7560.patch.00 > Resourcemanager hangs when resourceUsedWithWeightToResourceRatio

[jira] [Updated] (YARN-7560) Resourcemanager hangs when resourceUsedWithWeightToResourceRatio return a overflow value

2017-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-7560: -- Description: In our cluster, we changed the configuration, then refreshQueues, we found the

[jira] [Updated] (YARN-7560) Resourcemanager hangs when resourceUsedWithWeightToResourceRatio return a overflow value

2017-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-7560: -- Attachment: YARN-7560.000.patch > Resourcemanager hangs when resourceUsedWithWeightToResourceRatio

[jira] [Updated] (YARN-7560) Resourcemanager hangs when resourceUsedWithWeightToResourceRatio return a overflow value

2017-11-25 Thread zhengchenyu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated YARN-7560: -- Fix Version/s: (was: 2.7.5) 3.0.0 > Resourcemanager hangs when

  1   2   3   >