[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.

[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Summary: Log aggregation create dir throw failed to setup application log directory, but the log dir not

[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the dir existed.

[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Attachment: image-2021-04-13-15-34-22-732.png > Log aggregation create dir throw failed to setup application

[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the dir existed.

[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Description: !image-2021-04-13-15-34-22-732.png|width=756,height=166! > Log aggregation create dir throw

[jira] [Commented] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.

[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320016#comment-17320016 ] Qi Zhu commented on YARN-10734: --- cc [~pbacsko] [~gandras] [~ebadger] [~epayne]   What's your opinion about

[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.

[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Issue Type: Bug (was: Improvement) > Log aggregation create dir throw failed to setup application log

[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.

[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Description: As follows log aggregation create dir throw failed to setup application log directory :

[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.

[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Description: As follows log aggregation create dir throw failed to setup application log directory :

[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.

[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Description:   !image-2021-04-13-15-34-22-732.png|width=756,height=166!

[jira] [Created] (YARN-10738) When multi thread scheduling with multi node, we should shuffle to prevent hot accessing nodes.

Qi Zhu created YARN-10738: - Summary: When multi thread scheduling with multi node, we should shuffle to prevent hot accessing nodes. Key: YARN-10738 URL: https://issues.apache.org/jira/browse/YARN-10738

[jira] [Created] (YARN-10737) Fix typos in CapacityScheduler#schedule.

Qi Zhu created YARN-10737: - Summary: Fix typos in CapacityScheduler#schedule. Key: YARN-10737 URL: https://issues.apache.org/jira/browse/YARN-10737 Project: Hadoop YARN Issue Type: Improvement

[jira] [Updated] (YARN-10738) When multi thread scheduling with multi node, we should shuffle to prevent hot accessing nodes.

[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10738: -- Description: Now the multi threading scheduling with multi node is not reasonable. In large clusters, it will

[jira] [Commented] (YARN-10738) When multi thread scheduling with multi node, we should shuffle with a gap to prevent hot accessing nodes.

[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322004#comment-17322004 ] Qi Zhu commented on YARN-10738: --- I have changed to *MultiNodeLookupPolicy * implementation in latest PR ,

[jira] [Updated] (YARN-10738) When multi thread scheduling with multi node, we should shuffle to prevent hot accessing nodes.

[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10738: -- Description: Now the multi threading scheduling  > When multi thread scheduling with multi node, we should

[jira] [Updated] (YARN-10738) When multi thread scheduling with multi node, we should shuffle to prevent hot accessing nodes.

[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10738: -- Target Version/s: 3.4.0 > When multi thread scheduling with multi node, we should shuffle to prevent > hot

[jira] [Commented] (YARN-10738) When multi thread scheduling with multi node, we should shuffle to prevent hot accessing nodes.

[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321936#comment-17321936 ] Qi Zhu commented on YARN-10738: --- Thanks a lot [~bibinchundatt] for reply. :D I will move it to 

[jira] [Updated] (YARN-10738) When multi thread scheduling with multi node, we should shuffle with a gap to prevent hot accessing nodes.

[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10738: -- Summary: When multi thread scheduling with multi node, we should shuffle with a gap to prevent hot accessing

[jira] [Updated] (YARN-10738) When multi thread scheduling with multi node, we should shuffle to prevent hot accessing nodes.

[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10738: -- Description: Now the multi threading scheduling with multi node is not reasonable. In large clusters, it will

[jira] [Commented] (YARN-10738) When multi thread scheduling with multi node, we should shuffle to prevent hot accessing nodes.

[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321926#comment-17321926 ] Qi Zhu commented on YARN-10738: --- cc [~ztang]  [~snemeth] [~pbacsko] [~gandras] [~ebadger] [~epayne]  

[jira] [Commented] (YARN-10738) When multi thread scheduling with multi node, we should shuffle with a gap to prevent hot accessing nodes.

[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322560#comment-17322560 ] Qi Zhu commented on YARN-10738: --- [~Jim_Brennan] Could you help review this, when you are free? Thanks. >

[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.

[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Description: As follows log aggregation create dir throw failed to setup application log directory :

[jira] [Commented] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.

[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320665#comment-17320665 ] Qi Zhu commented on YARN-10734: --- I find it duplicated with YARN-8418 , so i close it now. Thanks. > Log

[jira] [Resolved] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.

[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu resolved YARN-10734. --- Resolution: Duplicate > Log aggregation create dir throw failed to setup application log directory, > but

[jira] [Commented] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app

[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320677#comment-17320677 ] Qi Zhu commented on YARN-8418: -- [~bibinchundatt] [~rohithsharma]  Could you help backport to hadoop 2.

[jira] [Commented] (YARN-10503) Support queue capacity in terms of absolute resources with custom resourceType.

[ https://issues.apache.org/jira/browse/YARN-10503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17318431#comment-17318431 ] Qi Zhu commented on YARN-10503: --- Thanks [~ebadger] for commit to 3.4 and branch-3.3. > Support queue

[jira] [Commented] (YARN-10732) Disallow restarting a queue while it is in DRAINING state on CS reinitialization

[ https://issues.apache.org/jira/browse/YARN-10732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319516#comment-17319516 ] Qi Zhu commented on YARN-10732: --- Thanks [~gandras] for good finding. The patch LGTM. +1   > Disallow

[jira] [Created] (YARN-10743) Add a policy for not aggregating for container log size limit killed container.

Qi Zhu created YARN-10743: - Summary: Add a policy for not aggregating for container log size limit killed container. Key: YARN-10743 URL: https://issues.apache.org/jira/browse/YARN-10743 Project: Hadoop YARN

[jira] [Updated] (YARN-10743) Add a policy for not aggregating for containers which are killed because exceeding container log size limit.

[ https://issues.apache.org/jira/browse/YARN-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10743: -- Summary: Add a policy for not aggregating for containers which are killed because exceeding container log size

[jira] [Commented] (YARN-10743) Add a policy for not aggregating for containers which are killed because exceeding container log size limit.

[ https://issues.apache.org/jira/browse/YARN-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17325109#comment-17325109 ] Qi Zhu commented on YARN-10743: --- Thanks [~Jim_Brennan] for reply. But in our cluster, some flink log size

[jira] [Comment Edited] (YARN-10743) Add a policy for not aggregating for containers which are killed because exceeding container log size limit.

[ https://issues.apache.org/jira/browse/YARN-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17325109#comment-17325109 ] Qi Zhu edited comment on YARN-10743 at 4/19/21, 3:21 PM: - Thanks [~Jim_Brennan] 

[jira] [Comment Edited] (YARN-10743) Add a policy for not aggregating for containers which are killed because exceeding container log size limit.

[ https://issues.apache.org/jira/browse/YARN-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17325109#comment-17325109 ] Qi Zhu edited comment on YARN-10743 at 4/19/21, 3:22 PM: - Thanks [~Jim_Brennan] 

[jira] [Comment Edited] (YARN-10743) Add a policy for not aggregating for containers which are killed because exceeding container log size limit.

[ https://issues.apache.org/jira/browse/YARN-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17325432#comment-17325432 ] Qi Zhu edited comment on YARN-10743 at 4/20/21, 3:04 AM: - Thanks [~ebadger] for

[jira] [Updated] (YARN-10723) Change CS nodes page in UI to support custom resource.

[ https://issues.apache.org/jira/browse/YARN-10723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10723: -- Attachment: YARN-10723.005.patch > Change CS nodes page in UI to support custom resource. >

[jira] [Updated] (YARN-10743) Add a policy for not aggregating for containers which are killed because exceeding container log size limit.

[ https://issues.apache.org/jira/browse/YARN-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10743: -- Attachment: image-2021-04-20-10-41-01-057.png > Add a policy for not aggregating for containers which are

[jira] [Commented] (YARN-10743) Add a policy for not aggregating for containers which are killed because exceeding container log size limit.

[ https://issues.apache.org/jira/browse/YARN-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17325432#comment-17325432 ] Qi Zhu commented on YARN-10743: --- Thanks [~ebadger] for reply. One case in our cluster : If the container

[jira] [Comment Edited] (YARN-10743) Add a policy for not aggregating for containers which are killed because exceeding container log size limit.

[ https://issues.apache.org/jira/browse/YARN-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17325432#comment-17325432 ] Qi Zhu edited comment on YARN-10743 at 4/20/21, 3:02 AM: - Thanks [~ebadger] for

[jira] [Commented] (YARN-10723) Change CS nodes page in UI to support custom resource.

[ https://issues.apache.org/jira/browse/YARN-10723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17325526#comment-17325526 ] Qi Zhu commented on YARN-10723: --- [~ebadger] The test error is not related to this, passed locally. >

[jira] [Commented] (YARN-10723) Change CS nodes page in UI to support custom resource.

[ https://issues.apache.org/jira/browse/YARN-10723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17325438#comment-17325438 ] Qi Zhu commented on YARN-10723: --- [~ebadger] Sure, i uploaded 005 patch to trigger jenkins. > Change CS

[jira] [Commented] (YARN-10715) Remove hardcoded resource values (e.g. GPU/FPGA) in code.

[ https://issues.apache.org/jira/browse/YARN-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17325442#comment-17325442 ] Qi Zhu commented on YARN-10715: --- Thanks [~ebadger] for reply. It make sense to me. I will close it now. :)

[jira] [Commented] (YARN-10723) Change CS nodes page in UI to support custom resource.

[ https://issues.apache.org/jira/browse/YARN-10723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17326227#comment-17326227 ] Qi Zhu commented on YARN-10723: --- Thanks [~ebadger] for commit. > Change CS nodes page in UI to support

[jira] [Updated] (YARN-10707) Support custom resources in ResourceUtilization, and update Node GPU Utilization to use.

[ https://issues.apache.org/jira/browse/YARN-10707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10707: -- Attachment: YARN-10707.007.patch > Support custom resources in ResourceUtilization, and update Node GPU >

[jira] [Commented] (YARN-10707) Support custom resources in ResourceUtilization, and update Node GPU Utilization to use.

[ https://issues.apache.org/jira/browse/YARN-10707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17326521#comment-17326521 ] Qi Zhu commented on YARN-10707: --- Updated it in latest patch. > Support custom resources in

[jira] [Commented] (YARN-10707) Support custom resources in ResourceUtilization, and update Node GPU Utilization to use.

[ https://issues.apache.org/jira/browse/YARN-10707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17326452#comment-17326452 ] Qi Zhu commented on YARN-10707: --- Thanks [~ebadger] for very good suggestions. It make sense to me, i will

[jira] [Comment Edited] (YARN-10739) GenericEventHandler.printEventQueueDetails cause RM recovery cost too much time

[ https://issues.apache.org/jira/browse/YARN-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323774#comment-17323774 ] Qi Zhu edited comment on YARN-10739 at 4/16/21, 12:37 PM: -- Thanks [~zhanqi.cai]

[jira] [Assigned] (YARN-10739) GenericEventHandler.printEventQueueDetails cause RM recovery cost too much time

[ https://issues.apache.org/jira/browse/YARN-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu reassigned YARN-10739: - Assignee: Qi Zhu > GenericEventHandler.printEventQueueDetails cause RM recovery cost too much > time >

[jira] [Updated] (YARN-10739) GenericEventHandler.printEventQueueDetails cause RM recovery cost too much time

[ https://issues.apache.org/jira/browse/YARN-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10739: -- Attachment: YARN-10739.003.patch > GenericEventHandler.printEventQueueDetails cause RM recovery cost too much

[jira] [Comment Edited] (YARN-10739) GenericEventHandler.printEventQueueDetails cause RM recovery cost too much time

[ https://issues.apache.org/jira/browse/YARN-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323833#comment-17323833 ] Qi Zhu edited comment on YARN-10739 at 4/16/21, 1:58 PM: - [~zhanqi.cai] I don't

[jira] [Commented] (YARN-10739) GenericEventHandler.printEventQueueDetails cause RM recovery cost too much time

[ https://issues.apache.org/jira/browse/YARN-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323833#comment-17323833 ] Qi Zhu commented on YARN-10739: --- [~zhanqi.cai] I don't think wait 30s is a good choice after

[jira] [Comment Edited] (YARN-10739) GenericEventHandler.printEventQueueDetails cause RM recovery cost too much time

[ https://issues.apache.org/jira/browse/YARN-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323774#comment-17323774 ] Qi Zhu edited comment on YARN-10739 at 4/16/21, 12:26 PM: -- Thanks [~zhanqi.cai]

[jira] [Comment Edited] (YARN-10739) GenericEventHandler.printEventQueueDetails cause RM recovery cost too much time

[ https://issues.apache.org/jira/browse/YARN-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323774#comment-17323774 ] Qi Zhu edited comment on YARN-10739 at 4/16/21, 12:36 PM: -- Thanks [~zhanqi.cai]

[jira] [Comment Edited] (YARN-10739) GenericEventHandler.printEventQueueDetails cause RM recovery cost too much time

[ https://issues.apache.org/jira/browse/YARN-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323774#comment-17323774 ] Qi Zhu edited comment on YARN-10739 at 4/16/21, 12:33 PM: -- Thanks [~zhanqi.cai]

[jira] [Commented] (YARN-10739) GenericEventHandler.printEventQueueDetails cause RM recovery cost too much time

[ https://issues.apache.org/jira/browse/YARN-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323774#comment-17323774 ] Qi Zhu commented on YARN-10739: --- Thanks [~zhanqi.cai] for reporting this, the patch LGTM generally. I just

[jira] [Commented] (YARN-10739) GenericEventHandler.printEventQueueDetails cause RM recovery cost too much time

[ https://issues.apache.org/jira/browse/YARN-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323863#comment-17323863 ] Qi Zhu commented on YARN-10739: --- Trigger the jenkins.  > GenericEventHandler.printEventQueueDetails cause

[jira] [Commented] (YARN-10723) Change CS nodes page in UI to support custom resource.

[ https://issues.apache.org/jira/browse/YARN-10723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324158#comment-17324158 ] Qi Zhu commented on YARN-10723: --- Thanks a lot [~ebadger] for confirm. > Change CS nodes page in UI to

[jira] [Commented] (YARN-9869) Create scheduling policy to auto-adjust queue elasticity based on cluster demand

[ https://issues.apache.org/jira/browse/YARN-9869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324495#comment-17324495 ] Qi Zhu commented on YARN-9869: -- cc [~jhung] Is this going on ? I think it's a very good improvement.:D

[jira] [Comment Edited] (YARN-10178) Global Scheduler async thread crash caused by 'Comparison method violates its general contract'

[ https://issues.apache.org/jira/browse/YARN-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17297785#comment-17297785 ] Qi Zhu edited comment on YARN-10178 at 4/8/21, 2:23 PM: Thanks [~pbacsko] for

[jira] [Commented] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used

[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283691#comment-17283691 ] Qi Zhu commented on YARN-10532: --- !image-2021-02-12-21-32-02-267.png|width=1085,height=764! cc [~gandras] 

[jira] [Updated] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used

[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10532: -- Attachment: image-2021-02-12-21-32-02-267.png > Capacity Scheduler Auto Queue Creation: Allow auto delete

[jira] [Comment Edited] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used

[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283691#comment-17283691 ] Qi Zhu edited comment on YARN-10532 at 2/12/21, 1:34 PM: -

[jira] [Comment Edited] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used

[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283691#comment-17283691 ] Qi Zhu edited comment on YARN-10532 at 2/12/21, 2:15 PM: -

[jira] [Comment Edited] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used

[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283691#comment-17283691 ] Qi Zhu edited comment on YARN-10532 at 2/12/21, 2:15 PM: -

[jira] [Updated] (YARN-10609) Update the document for YARN-10531(Be able to disable user limit factor for CapacityScheduler Leaf Queue)

[ https://issues.apache.org/jira/browse/YARN-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10609: -- Attachment: YARN-10609.002.patch > Update the document for YARN-10531(Be able to disable user limit factor for

[jira] [Updated] (YARN-10609) Update the document for YARN-10531(Be able to disable user limit factor for CapacityScheduler Leaf Queue)

[ https://issues.apache.org/jira/browse/YARN-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10609: -- Attachment: YARN-10609.003.patch > Update the document for YARN-10531(Be able to disable user limit factor for

[jira] [Comment Edited] (YARN-10609) Update the document for YARN-10531(Be able to disable user limit factor for CapacityScheduler Leaf Queue)

[ https://issues.apache.org/jira/browse/YARN-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285229#comment-17285229 ] Qi Zhu edited comment on YARN-10609 at 2/16/21, 2:39 PM: - Thanks [~bteke]. :D I

[jira] [Commented] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used

[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285256#comment-17285256 ] Qi Zhu commented on YARN-10532: --- [~gandras] [~bteke] [~snemeth] I have added a log for sending a deletion

[jira] [Commented] (YARN-10609) Update the document for YARN-10531(Be able to disable user limit factor for CapacityScheduler Leaf Queue)

[ https://issues.apache.org/jira/browse/YARN-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285229#comment-17285229 ] Qi Zhu commented on YARN-10609: --- Thanks [~bteke]. :D I am appreciate for your patient review. This is a

[jira] [Commented] (YARN-10609) Update the document for YARN-10531(Be able to disable user limit factor for CapacityScheduler Leaf Queue)

[ https://issues.apache.org/jira/browse/YARN-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285191#comment-17285191 ] Qi Zhu commented on YARN-10609: --- [~bteke]  [~gandras] [~snemeth]  I have updated it in latest patch, if

[jira] [Updated] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used

[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10532: -- Attachment: YARN-10532.021.patch > Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue

[jira] [Comment Edited] (YARN-10627) Extend logging to give more information about weight mode

[ https://issues.apache.org/jira/browse/YARN-10627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285189#comment-17285189 ] Qi Zhu edited comment on YARN-10627 at 2/16/21, 1:11 PM: - Thanks [~bteke] for

[jira] [Commented] (YARN-10623) Capacity scheduler should support refresh queue automatically by a thread policy.

[ https://issues.apache.org/jira/browse/YARN-10623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285186#comment-17285186 ] Qi Zhu commented on YARN-10623: --- [~gandras] [~bteke] [~pbacsko] [~ztang] [~shuzirra] Could you help review

[jira] [Commented] (YARN-10627) Extend logging to give more information about weight mode

[ https://issues.apache.org/jira/browse/YARN-10627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285189#comment-17285189 ] Qi Zhu commented on YARN-10627: --- Thanks [~bteke] for this issue. I also think more information is helpful

[jira] [Commented] (YARN-10548) Decouple AM runner logic from SLSRunner

[ https://issues.apache.org/jira/browse/YARN-10548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286482#comment-17286482 ] Qi Zhu commented on YARN-10548: --- Thanks [~snemeth] for the contribution, the patch LGTM.   > Decouple AM

[jira] [Commented] (YARN-9615) Add dispatcher metrics to RM

[ https://issues.apache.org/jira/browse/YARN-9615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286324#comment-17286324 ] Qi Zhu commented on YARN-9615: -- cc [~bteke] [~gandras]  [~pbacsko]  If you any advice about this? Thanks.

[jira] [Updated] (YARN-9615) Add dispatcher metrics to RM

[ https://issues.apache.org/jira/browse/YARN-9615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-9615: - Attachment: YARN-9615.002.patch > Add dispatcher metrics to RM > > >

[jira] [Commented] (YARN-9615) Add dispatcher metrics to RM

[ https://issues.apache.org/jira/browse/YARN-9615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286468#comment-17286468 ] Qi Zhu commented on YARN-9615: -- [~jhung] [~bibinchundatt]  Update a patch, to fix the finding bugs, and the

[jira] [Commented] (YARN-10258) Add metrics for 'ApplicationsRunning' in NodeManager

[ https://issues.apache.org/jira/browse/YARN-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285803#comment-17285803 ] Qi Zhu commented on YARN-10258: --- Thank you [~gb.ana...@gmail.com] for your contribution. Patch LGTM.  >

[jira] [Created] (YARN-10632) Make maximum depth allowed configurable.

Qi Zhu created YARN-10632: - Summary: Make maximum depth allowed configurable. Key: YARN-10632 URL: https://issues.apache.org/jira/browse/YARN-10632 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Comment Edited] (YARN-10178) Global Scheduler async thread crash caused by 'Comparison method violates its general contract'

[ https://issues.apache.org/jira/browse/YARN-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279673#comment-17279673 ] Qi Zhu edited comment on YARN-10178 at 2/17/21, 12:50 PM: -- The test failed is

[jira] [Updated] (YARN-10609) Update the document for YARN-10531(Be able to disable user limit factor for CapacityScheduler Leaf Queue)

[ https://issues.apache.org/jira/browse/YARN-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10609: -- Attachment: YARN-10609.005.patch > Update the document for YARN-10531(Be able to disable user limit factor for

[jira] [Updated] (YARN-10632) Make maximum depth allowed configurable.

[ https://issues.apache.org/jira/browse/YARN-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10632: -- Fix Version/s: (was: 3.4.0) > Make maximum depth allowed configurable. >

[jira] [Commented] (YARN-10609) Update the document for YARN-10531(Be able to disable user limit factor for CapacityScheduler Leaf Queue)

[ https://issues.apache.org/jira/browse/YARN-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286252#comment-17286252 ] Qi Zhu commented on YARN-10609: --- Thanks a lot [~bteke]  for last check, i have fixed it in latest patch.

[jira] [Updated] (YARN-10609) Update the document for YARN-10531(Be able to disable user limit factor for CapacityScheduler Leaf Queue)

[ https://issues.apache.org/jira/browse/YARN-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10609: -- Attachment: YARN-10609.004.patch > Update the document for YARN-10531(Be able to disable user limit factor for

[jira] [Commented] (YARN-10632) Make maximum depth allowed configurable.

[ https://issues.apache.org/jira/browse/YARN-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286359#comment-17286359 ] Qi Zhu commented on YARN-10632: --- Thanks [~gandras] for reply. It makes sense to me. We can revisit this

[jira] [Comment Edited] (YARN-10632) Make maximum depth allowed configurable.

[ https://issues.apache.org/jira/browse/YARN-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286302#comment-17286302 ] Qi Zhu edited comment on YARN-10632 at 2/18/21, 6:30 AM: - cc [~bteke]  [~gandras] 

[jira] [Comment Edited] (YARN-10632) Make maximum depth allowed configurable.

[ https://issues.apache.org/jira/browse/YARN-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286302#comment-17286302 ] Qi Zhu edited comment on YARN-10632 at 2/18/21, 6:53 AM: - cc [~bteke]  [~gandras] 

[jira] [Commented] (YARN-10632) Make maximum depth allowed configurable.

[ https://issues.apache.org/jira/browse/YARN-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286302#comment-17286302 ] Qi Zhu commented on YARN-10632: --- cc [~bteke]  [~gandras]  [~snemeth] [~pbacsko]  I think for some queue

[jira] [Commented] (YARN-10623) Capacity scheduler should support refresh queue automatically by a thread policy.

[ https://issues.apache.org/jira/browse/YARN-10623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286567#comment-17286567 ] Qi Zhu commented on YARN-10623: --- Thanks [~gandras] for your reply. Our production cluster, have more than

[jira] [Comment Edited] (YARN-9615) Add dispatcher metrics to RM

[ https://issues.apache.org/jira/browse/YARN-9615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286468#comment-17286468 ] Qi Zhu edited comment on YARN-9615 at 2/18/21, 4:12 PM: [~jhung] [~bibinchundatt]

[jira] [Comment Edited] (YARN-10623) Capacity scheduler should support refresh queue automatically by a thread policy.

[ https://issues.apache.org/jira/browse/YARN-10623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286567#comment-17286567 ] Qi Zhu edited comment on YARN-10623 at 2/18/21, 4:11 PM: - Thanks [~gandras] for

[jira] [Updated] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used

[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10532: -- Attachment: YARN-10532.017.patch > Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue

[jira] [Updated] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used

[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10532: -- Attachment: YARN-10532.018.patch > Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue

[jira] [Commented] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used

[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280065#comment-17280065 ] Qi Zhu commented on YARN-10532: --- Thanks a lot [~gandras] for patient review. In latest patch, I have fixed 

[jira] [Commented] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used

[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280215#comment-17280215 ] Qi Zhu commented on YARN-10532: --- Fix the checkstyle in latest patch. > Capacity Scheduler Auto Queue

[jira] [Updated] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used

[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10532: -- Attachment: YARN-10532.019.patch > Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue

[jira] [Resolved] (YARN-8557) Exclude lagged/unhealthy/decommissioned nodes in async allocating thread

[ https://issues.apache.org/jira/browse/YARN-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu resolved YARN-8557. -- Resolution: Duplicate > Exclude lagged/unhealthy/decommissioned nodes in async allocating thread >

[jira] [Commented] (YARN-10593) Fix incorrect string comparison in GpuDiscoverer

[ https://issues.apache.org/jira/browse/YARN-10593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280350#comment-17280350 ] Qi Zhu commented on YARN-10593: --- [~pbacsko] May i take this? I can help to fix. > Fix incorrect string

[jira] [Commented] (YARN-8557) Exclude lagged/unhealthy/decommissioned nodes in async allocating thread

[ https://issues.apache.org/jira/browse/YARN-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280347#comment-17280347 ] Qi Zhu commented on YARN-8557: -- Close this, i included this in -YARN-10352.- > Exclude

[jira] [Commented] (YARN-10624) Support max queues limit configuration in new auto created queue, consistent with old auto created.

[ https://issues.apache.org/jira/browse/YARN-10624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284141#comment-17284141 ] Qi Zhu commented on YARN-10624: ---  [~snemeth] [~gandras] [~bteke]   Could you help review this? Thanks. >

[jira] [Updated] (YARN-10624) Support max queues limit configuration in new auto created queue, consistent with old auto created.

[ https://issues.apache.org/jira/browse/YARN-10624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10624: -- Description: Since old created leaf queue has the max leaf queues limit, i think we also should support this

[jira] [Updated] (YARN-10624) Support max queues limit configuration in new auto created queue, consistent with old auto created.

[ https://issues.apache.org/jira/browse/YARN-10624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10624: -- Summary: Support max queues limit configuration in new auto created queue, consistent with old auto created.

[jira] [Comment Edited] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used

[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283691#comment-17283691 ] Qi Zhu edited comment on YARN-10532 at 2/13/21, 2:11 AM: -

<    4   5   6   7   8   9   10   11   >