[jira] [Assigned] (YARN-10657) We should make max application per queue to support node label.

2021-06-09 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu reassigned YARN-10657: - Assignee: Andras Gyori > We should make max application per queue to support node label. >

[jira] [Commented] (YARN-10657) We should make max application per queue to support node label.

2021-06-09 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360109#comment-17360109 ] Qi Zhu commented on YARN-10657: --- [~gandras] Of course you can take it, and i will help review. :D Assigned

[jira] [Assigned] (YARN-10657) We should make max application per queue to support node label.

2021-06-09 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu reassigned YARN-10657: - Assignee: (was: Qi Zhu) > We should make max application per queue to support node label. >

[jira] [Commented] (YARN-10801) Fix Auto Queue template to properly set all configuration properties

2021-06-09 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360017#comment-17360017 ] Qi Zhu commented on YARN-10801: --- Thanks [~gandras] for update. The latest patch LGTM. > Fix Auto Queue

[jira] [Comment Edited] (YARN-10801) Fix Auto Queue template to properly set all configuration properties

2021-06-08 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359366#comment-17359366 ] Qi Zhu edited comment on YARN-10801 at 6/8/21, 1:42 PM: Thanks [~gandras] for

[jira] [Commented] (YARN-10801) Fix Auto Queue template to properly set all configuration properties

2021-06-08 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359366#comment-17359366 ] Qi Zhu commented on YARN-10801: --- Thanks [~gandras] for patch LGTM now. I have a question about the code,

[jira] [Commented] (YARN-10807) Parents node labels are incorrectly added to child queues in weight mode

2021-06-08 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359343#comment-17359343 ] Qi Zhu commented on YARN-10807: --- Thanks [~bteke] for patch and [~gandras] for review. Committed to trunk.

[jira] [Commented] (YARN-10807) Parents node labels are incorrectly added to child queues in weight mode

2021-06-07 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358980#comment-17358980 ] Qi Zhu commented on YARN-10807: --- Thanks [~bteke] for update. The patch LGTM.   > Parents node labels are

[jira] [Comment Edited] (YARN-10807) Parents node labels are incorrectly added to child queues in weight mode

2021-06-07 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358486#comment-17358486 ] Qi Zhu edited comment on YARN-10807 at 6/7/21, 9:54 AM: Thanks [~bteke] for this

[jira] [Commented] (YARN-10807) Parents node labels are incorrectly added to child queues in weight mode

2021-06-07 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358486#comment-17358486 ] Qi Zhu commented on YARN-10807: --- Thanks [~bteke] for this work. If we can skip the not existed also in sum

[jira] [Commented] (YARN-10789) RM HA startup can fail due to race conditions in ZKConfigurationStore

2021-06-03 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356504#comment-17356504 ] Qi Zhu commented on YARN-10789: --- Thanks [~tarunparimi] for this work. The latest patch LGTM. +1 > RM HA

[jira] [Commented] (YARN-10796) Capacity Scheduler: dynamic queue cannot scale out properly if its capacity is 0%

2021-06-03 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356214#comment-17356214 ] Qi Zhu commented on YARN-10796: --- Thanks [~pbacsko] the latest patch LGTM +1. And i agree with you the

[jira] [Commented] (YARN-10522) Document for Flexible Auto Queue Creation in Capacity Scheduler.

2021-06-03 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356213#comment-17356213 ] Qi Zhu commented on YARN-10522: --- Thanks for [~bteke] taking this. I assigned it to you.   > Document for

[jira] [Assigned] (YARN-10522) Document for Flexible Auto Queue Creation in Capacity Scheduler.

2021-06-03 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu reassigned YARN-10522: - Assignee: Benjamin Teke > Document for Flexible Auto Queue Creation in Capacity Scheduler. >

[jira] [Assigned] (YARN-10522) Document for Flexible Auto Queue Creation in Capacity Scheduler.

2021-06-03 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu reassigned YARN-10522: - Assignee: (was: Ankit Kumar) > Document for Flexible Auto Queue Creation in Capacity Scheduler. >

[jira] [Commented] (YARN-10795) Improve Capacity Scheduler reinitialisation performance

2021-05-31 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354809#comment-17354809 ] Qi Zhu commented on YARN-10795: --- Thanks [~gandras] for this work. It will be very helpful to clusters with

[jira] [Commented] (YARN-10781) The Thread of the NM aggregate log is exhausted and no other Application can aggregate the log

2021-05-25 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351456#comment-17351456 ] Qi Zhu commented on YARN-10781: --- [~zhangxiping] If you enabled rolling log aggregation for long running

[jira] [Commented] (YARN-10786) Federation:We can't access the AM page while using federation

2021-05-25 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350939#comment-17350939 ] Qi Zhu commented on YARN-10786: --- Thanks [~Song Jiacheng] for contribution. The patch LGTM. +1 cc 

[jira] [Commented] (YARN-10786) Federation:We can't access the AM page while using federation

2021-05-25 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350861#comment-17350861 ] Qi Zhu commented on YARN-10786: --- Thanks [~Song Jiacheng] for report this. Can you add some image to

[jira] [Commented] (YARN-10770) container-executor permission is wrong in SecureContainer.md

2021-05-25 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350807#comment-17350807 ] Qi Zhu commented on YARN-10770: --- Thanks [~aajisaka] for good finding, [~sahuja] for patch. The patch LGTM

[jira] [Created] (YARN-10785) Yarn NodeManager aux-services should support trim.

2021-05-24 Thread Qi Zhu (Jira)
Qi Zhu created YARN-10785: - Summary: Yarn NodeManager aux-services should support trim. Key: YARN-10785 URL: https://issues.apache.org/jira/browse/YARN-10785 Project: Hadoop YARN Issue Type: Bug

[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue

2021-05-24 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350482#comment-17350482 ] Qi Zhu commented on YARN-10771: --- Thanks [~chaosju] for contribution and [~pbacsko] for review. The test is

[jira] [Commented] (YARN-10783) Allow definition of auto queue template properties in root

2021-05-24 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350262#comment-17350262 ] Qi Zhu commented on YARN-10783: --- Thanks [~gandras] for this. The patch LGTM +1. > Allow definition of

[jira] [Comment Edited] (YARN-10781) The Thread of the NM aggregate log is exhausted and no other Application can aggregate the log

2021-05-24 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350255#comment-17350255 ] Qi Zhu edited comment on YARN-10781 at 5/24/21, 6:02 AM: - [~zhangxiping] It only

[jira] [Commented] (YARN-10781) The Thread of the NM aggregate log is exhausted and no other Application can aggregate the log

2021-05-24 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350255#comment-17350255 ] Qi Zhu commented on YARN-10781: --- [~zhangxiping] It only init app and create the thread pool, when AM

[jira] [Commented] (YARN-10324) Fetch data from NodeManager may case read timeout when disk is busy

2021-05-21 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17349124#comment-17349124 ] Qi Zhu commented on YARN-10324: --- [~yaoguangdong] I'm not sure if you removed the original 003, and

[jira] [Commented] (YARN-10324) Fetch data from NodeManager may case read timeout when disk is busy

2021-05-21 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17349111#comment-17349111 ] Qi Zhu commented on YARN-10324: --- [~yaoguangdong] You should submitted it, make it patch available, then the

[jira] [Updated] (YARN-10324) Fetch data from NodeManager may case read timeout when disk is busy

2021-05-21 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10324: -- Attachment: image-2021-05-21-17-48-03-476.png > Fetch data from NodeManager may case read timeout when disk is

[jira] [Commented] (YARN-10657) We should make max application per queue to support node label.

2021-05-21 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17349094#comment-17349094 ] Qi Zhu commented on YARN-10657: --- Thanks [~gandras] for reply. We now can close before we can discuss a

[jira] [Commented] (YARN-10779) Add option to disable lowercase conversion in GetApplicationsRequestPBImpl and ApplicationSubmissionContextPBImpl

2021-05-21 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17349073#comment-17349073 ] Qi Zhu commented on YARN-10779: --- Thanks [~pbacsko] for reply. I also agree that it only affect the

[jira] [Commented] (YARN-10781) The Thread of the NM aggregate log is exhausted and no other Application can aggregate the log

2021-05-21 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17349065#comment-17349065 ] Qi Zhu commented on YARN-10781: --- Thanks [~zhangxiping] for this. If you mean when the Spark dynamic

[jira] [Commented] (YARN-10779) Add option to disable lowercase conversion in GetApplicationsRequestPBImpl and ApplicationSubmissionContextPBImpl

2021-05-21 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17349052#comment-17349052 ] Qi Zhu commented on YARN-10779: --- Thanks [~gandras] for reminder. If we should enable use to reinitialize

[jira] [Commented] (YARN-10779) Add option to disable lowercase conversion in GetApplicationsRequestPBImpl and ApplicationSubmissionContextPBImpl

2021-05-20 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348941#comment-17348941 ] Qi Zhu commented on YARN-10779: --- Thanks [~pbacsko] for this work. The patch LGTM, just to fix the only one

[jira] [Commented] (YARN-10543) Timeline Server V1.5 not supporting audit log

2021-05-19 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348066#comment-17348066 ] Qi Zhu commented on YARN-10543: --- Thanks [~gb.ana...@gmail.com] for patch. The patch generally LGTM. But

[jira] [Comment Edited] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue

2021-05-19 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347313#comment-17347313 ] Qi Zhu edited comment on YARN-10771 at 5/20/21, 2:29 AM: - Thanks [~chaosju] for

[jira] [Commented] (YARN-10701) The yarn.resource-types should support multi types without trimmed.

2021-05-19 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347679#comment-17347679 ] Qi Zhu commented on YARN-10701: --- The test is not related this jira. Committed to branch-3.3. Thanks

[jira] [Commented] (YARN-10701) The yarn.resource-types should support multi types without trimmed.

2021-05-19 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347445#comment-17347445 ] Qi Zhu commented on YARN-10701: --- Submitted backport-3.3 patch to trigger jenkins. > The

[jira] [Commented] (YARN-10701) The yarn.resource-types should support multi types without trimmed.

2021-05-19 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347438#comment-17347438 ] Qi Zhu commented on YARN-10701: --- Thanks [~weichiu] for reminder. I will help to backport to branch-3.3. >

[jira] [Reopened] (YARN-10701) The yarn.resource-types should support multi types without trimmed.

2021-05-19 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu reopened YARN-10701: --- > The yarn.resource-types should support multi types without trimmed. >

[jira] [Comment Edited] (YARN-10774) Federation: Normalize the yarn federation queue name

2021-05-19 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347427#comment-17347427 ] Qi Zhu edited comment on YARN-10774 at 5/19/21, 9:01 AM: - [~luoyuan] Now fs

[jira] [Commented] (YARN-10774) Federation: Normalize the yarn federation queue name

2021-05-19 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347427#comment-17347427 ] Qi Zhu commented on YARN-10774: --- [~luoyuan] Now fs support both root.XXX and xxx but cs still not support

[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue

2021-05-18 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347313#comment-17347313 ] Qi Zhu commented on YARN-10771: --- Thanks [~chaosju] for update. The patch LGTM now. Waiting [~pbacsko] for

[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue

2021-05-18 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347005#comment-17347005 ] Qi Zhu commented on YARN-10771: --- Thanks [~chaosju] for update. The patch LGTM, go on to fix the

[jira] [Commented] (YARN-8564) Add queue level application lifetime monitor in FairScheduler

2021-05-18 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346911#comment-17346911 ] Qi Zhu commented on YARN-8564: -- Thanks [~tarunparimi] for reminder. I reopened it, you can take it if you

[jira] [Reopened] (YARN-8564) Add queue level application lifetime monitor in FairScheduler

2021-05-18 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu reopened YARN-8564: -- > Add queue level application lifetime monitor in FairScheduler >

[jira] [Comment Edited] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue

2021-05-17 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346575#comment-17346575 ] Qi Zhu edited comment on YARN-10771 at 5/18/21, 3:59 AM: - Thanks [~chaosju] for

[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue

2021-05-17 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346575#comment-17346575 ] Qi Zhu commented on YARN-10771: --- Thanks [~chaosju] for patch. I think we should change:"rm event queue

[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue

2021-05-17 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346181#comment-17346181 ] Qi Zhu commented on YARN-10771: --- Thanks [~chaosju] for this. This is useful for user to know the event

[jira] [Commented] (YARN-10555) Missing access check before getAppAttempts

2021-05-17 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346018#comment-17346018 ] Qi Zhu commented on YARN-10555: --- Thanks [~aajisaka] for backport. > Missing access check before

[jira] [Commented] (YARN-10555) missing access check before getAppAttempts

2021-05-17 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345995#comment-17345995 ] Qi Zhu commented on YARN-10555: --- It was merged by [~aajisaka] , i just make it fixed. Thanks

[jira] [Commented] (YARN-10763) add the speed of containers assigned metrics to ClusterMetrics

2021-05-16 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345849#comment-17345849 ] Qi Zhu commented on YARN-10763: --- Thanks [~chaosju] for update. The latest patch LGTM +1.   > add the

[jira] [Resolved] (YARN-10545) Improve the readability of diagnostics log in yarn-ui2 web page.

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu resolved YARN-10545. --- Fix Version/s: 3.4.0 Resolution: Fixed > Improve the readability of diagnostics log in yarn-ui2 web

[jira] [Assigned] (YARN-10545) Improve the readability of diagnostics log in yarn-ui2 web page.

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu reassigned YARN-10545: - Assignee: akiyamaneko > Improve the readability of diagnostics log in yarn-ui2 web page. >

[jira] [Commented] (YARN-9698) [Umbrella] Tools to help migration from Fair Scheduler to Capacity Scheduler

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344630#comment-17344630 ] Qi Zhu commented on YARN-9698: -- Thanks [~pbacsko] for reminder. I agree with you that we can creating a new 

[jira] [Commented] (YARN-9615) Add dispatcher metrics to RM

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344531#comment-17344531 ] Qi Zhu commented on YARN-9615: -- [~chaosju] Sure.:D > Add dispatcher metrics to RM >

[jira] [Commented] (YARN-10764) Add rm dispatcher event metrics in SLS

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344490#comment-17344490 ] Qi Zhu commented on YARN-10764: --- I think we should add the event related metrics to SLS, such as : # The

[jira] [Comment Edited] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344483#comment-17344483 ] Qi Zhu edited comment on YARN-10761 at 5/14/21, 9:26 AM: - Thanks [~snemeth] for

[jira] [Comment Edited] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344483#comment-17344483 ] Qi Zhu edited comment on YARN-10761 at 5/14/21, 9:19 AM: - Thanks [~snemeth] for

[jira] [Comment Edited] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344483#comment-17344483 ] Qi Zhu edited comment on YARN-10761 at 5/14/21, 9:16 AM: - Thanks [~snemeth] for

[jira] [Commented] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344483#comment-17344483 ] Qi Zhu commented on YARN-10761: --- Thanks [~snemeth] for reminder. Sorry for the commit. The YARN-9615 is

[jira] [Comment Edited] (YARN-10324) Fetch data from NodeManager may case read timeout when disk is busy

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344478#comment-17344478 ] Qi Zhu edited comment on YARN-10324 at 5/14/21, 9:01 AM: - Hi [~yaoguangdong] 

[jira] [Commented] (YARN-10324) Fetch data from NodeManager may case read timeout when disk is busy

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344478#comment-17344478 ] Qi Zhu commented on YARN-10324: --- Hi [~yaoguangdong]  Thanks for this work. I have added you to the

[jira] [Assigned] (YARN-10324) Fetch data from NodeManager may case read timeout when disk is busy

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu reassigned YARN-10324: - Assignee: Yao Guangdong > Fetch data from NodeManager may case read timeout when disk is busy >

[jira] [Commented] (YARN-10766) [UI2] Bump moment-timezone to 0.5.33

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344462#comment-17344462 ] Qi Zhu commented on YARN-10766: --- Thanks [~gandras] for patch. LGTM +1 > [UI2] Bump moment-timezone to

[jira] [Commented] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344372#comment-17344372 ] Qi Zhu commented on YARN-10761: --- Thanks [~ebadger] [~gandras]  [~chaosju] for review. Merged to trunk.  

[jira] [Commented] (YARN-10737) Fix typos in CapacityScheduler#schedule.

2021-05-13 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344319#comment-17344319 ] Qi Zhu commented on YARN-10737: --- Thanks [~hexiaoqiao]  [@fdalsotto|https://github.com/fdalsotto] for

[jira] [Commented] (YARN-10632) Make maximum depth allowed configurable.

2021-05-13 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343950#comment-17343950 ] Qi Zhu commented on YARN-10632: --- Fixed the checkstyle and java doc in latest patch. > Make maximum depth

[jira] [Updated] (YARN-10632) Make maximum depth allowed configurable.

2021-05-13 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10632: -- Attachment: YARN-10632.004.patch > Make maximum depth allowed configurable. >

[jira] [Updated] (YARN-10632) Make maximum depth allowed configurable.

2021-05-13 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10632: -- Attachment: YARN-10632.003.patch > Make maximum depth allowed configurable. >

[jira] [Commented] (YARN-10632) Make maximum depth allowed configurable.

2021-05-13 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343782#comment-17343782 ] Qi Zhu commented on YARN-10632: --- [~gandras]  I have updated it in latest patch. > Make maximum depth

[jira] [Commented] (YARN-10632) Make maximum depth allowed configurable.

2021-05-13 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343750#comment-17343750 ] Qi Zhu commented on YARN-10632: --- Thanks [~gandras] for reminder. I will change it based YARN-10571. >

[jira] [Commented] (YARN-10517) QueueMetrics has incorrect Allocated Resource when labelled partitions updated

2021-05-12 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343084#comment-17343084 ] Qi Zhu commented on YARN-10517: --- Thanks [~zhanqi.cai] for confirm. cc [~pbacsko]  [~ebadger] [~epayne] >

[jira] [Commented] (YARN-10759) Encapsulate queue config modes

2021-05-10 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342269#comment-17342269 ] Qi Zhu commented on YARN-10759: --- Thanks [~gandras] for this work. Very good work, i am very appreciate you

[jira] [Commented] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-10 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341986#comment-17341986 ] Qi Zhu commented on YARN-10761: --- Thanks [~gandras] for your review.  > Add more event type to RM

[jira] [Updated] (YARN-10764) Add rm dispatcher event metrics in SLS

2021-05-08 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10764: -- Description: We should use SLS to confirm if we can get performance improvement of event consume time etc. >

[jira] [Created] (YARN-10764) Add rm dispatcher event metrics in SLS

2021-05-08 Thread Qi Zhu (Jira)
Qi Zhu created YARN-10764: - Summary: Add rm dispatcher event metrics in SLS Key: YARN-10764 URL: https://issues.apache.org/jira/browse/YARN-10764 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Commented] (YARN-10763) add the speed of containers assigned metrics to ClusterMetrics

2021-05-07 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340883#comment-17340883 ] Qi Zhu commented on YARN-10763: --- Thanks [~chaosju] for report. If you can use aggregateContainersAllocated

[jira] [Commented] (YARN-10738) When multi thread scheduling with multi node, we should shuffle with a gap to prevent hot accessing nodes.

2021-05-07 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340599#comment-17340599 ] Qi Zhu commented on YARN-10738: --- Thanks a lot [~bibinchundatt] for reply and value information. For above

[jira] [Updated] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-06 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10761: -- Attachment: YARN-10761.003.patch > Add more event type to RM Dispatcher event metrics. >

[jira] [Commented] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-06 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340582#comment-17340582 ] Qi Zhu commented on YARN-10761: --- Fixed checkstyle in latest patch. > Add more event type to RM Dispatcher

[jira] [Commented] (YARN-10755) Multithreaded loading Apps from zk statestore

2021-05-06 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340570#comment-17340570 ] Qi Zhu commented on YARN-10755: --- Thanks [~chaosju] report , and [~BilwaST] for taking this. I will help

[jira] [Commented] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-06 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340529#comment-17340529 ] Qi Zhu commented on YARN-10761: --- Thanks a lot [~ebadger] for review. I have changed the two create to one,

[jira] [Updated] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-06 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10761: -- Attachment: YARN-10761.002.patch > Add more event type to RM Dispatcher event metrics. >

[jira] [Commented] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-06 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340081#comment-17340081 ] Qi Zhu commented on YARN-10761: --- [~ebadger] [~pbacsko] [~gandras] [~bilwa_st] Could you help review this?

[jira] [Updated] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-06 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10761: -- Attachment: image-2021-05-06-16-39-28-362.png > Add more event type to RM Dispatcher event metrics. >

[jira] [Updated] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-06 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10761: -- Attachment: image-2021-05-06-16-38-51-406.png > Add more event type to RM Dispatcher event metrics. >

[jira] [Created] (YARN-10761) Add more event type to RM Dispatcher event metrics.

2021-05-06 Thread Qi Zhu (Jira)
Qi Zhu created YARN-10761: - Summary: Add more event type to RM Dispatcher event metrics. Key: YARN-10761 URL: https://issues.apache.org/jira/browse/YARN-10761 Project: Hadoop YARN Issue Type:

[jira] [Comment Edited] (YARN-9927) RM multi-thread event processing mechanism

2021-05-06 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339383#comment-17339383 ] Qi Zhu edited comment on YARN-9927 at 5/6/21, 6:41 AM: --- Great review and

[jira] [Comment Edited] (YARN-9927) RM multi-thread event processing mechanism

2021-05-04 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339383#comment-17339383 ] Qi Zhu edited comment on YARN-9927 at 5/5/21, 3:05 AM: --- Great review and

[jira] [Commented] (YARN-9927) RM multi-thread event processing mechanism

2021-05-04 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339383#comment-17339383 ] Qi Zhu commented on YARN-9927: -- Great review and investigation! Thanks very much  [~ebadger] [~ebadger] . I

[jira] [Commented] (YARN-10517) QueueMetrics has incorrect Allocated Resource when labelled partitions updated

2021-05-04 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339378#comment-17339378 ] Qi Zhu commented on YARN-10517: --- Thanks [~zhanqi.cai] for report. Could you apply the latest patch to your

[jira] [Commented] (YARN-10524) Support multi resource type based weight mode in CS.

2021-05-04 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339376#comment-17339376 ] Qi Zhu commented on YARN-10524: --- Thanks [~gandras] for concern.  I think YARN-9936 will cover all this use

[jira] [Commented] (YARN-10592) Support QueueCapacities to use vector based multi resource types, and update absolute related to use first.

2021-05-04 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339375#comment-17339375 ] Qi Zhu commented on YARN-10592: --- [~gandras] I think  YARN-9936 would cover this. > Support

[jira] [Commented] (YARN-9443) Fast RM Failover using Ratis (Raft protocol)

2021-04-29 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335205#comment-17335205 ] Qi Zhu commented on YARN-9443: -- [~prabhujoseph] [~ztang] [~ebadger] [~epayne] Is this going on, now the

[jira] [Comment Edited] (YARN-10738) When multi thread scheduling with multi node, we should shuffle with a gap to prevent hot accessing nodes.

2021-04-28 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335104#comment-17335104 ] Qi Zhu edited comment on YARN-10738 at 4/29/21, 2:48 AM: - Thanks [~Jim_Brennan] 

[jira] [Comment Edited] (YARN-10738) When multi thread scheduling with multi node, we should shuffle with a gap to prevent hot accessing nodes.

2021-04-28 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335104#comment-17335104 ] Qi Zhu edited comment on YARN-10738 at 4/29/21, 2:46 AM: - Thanks [~Jim_Brennan] 

[jira] [Comment Edited] (YARN-10738) When multi thread scheduling with multi node, we should shuffle with a gap to prevent hot accessing nodes.

2021-04-28 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335104#comment-17335104 ] Qi Zhu edited comment on YARN-10738 at 4/29/21, 2:45 AM: - Thanks [~Jim_Brennan] 

[jira] [Commented] (YARN-10738) When multi thread scheduling with multi node, we should shuffle with a gap to prevent hot accessing nodes.

2021-04-28 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335104#comment-17335104 ] Qi Zhu commented on YARN-10738: --- Thanks [~Jim_Brennan] for review and very patient investigation. The

[jira] [Commented] (YARN-10707) Support custom resources in ResourceUtilization, and update Node GPU Utilization to use.

2021-04-28 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17334756#comment-17334756 ] Qi Zhu commented on YARN-10707: --- The failed time out test is not related, passed locally. > Support custom

[jira] [Updated] (YARN-10707) Support custom resources in ResourceUtilization, and update Node GPU Utilization to use.

2021-04-28 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10707: -- Attachment: YARN-10707.011.patch > Support custom resources in ResourceUtilization, and update Node GPU >

  1   2   3   4   5   6   7   8   9   10   >