[jira] [Commented] (YARN-8737) Race condition in ParentQueue when reinitializing and sorting child queues in the meanwhile

2020-09-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204467#comment-17204467 ] Tao Yang commented on YARN-8737: Hi, [~Amithsha], [~wangda], [~bteke]. Sorry for missing this issue so

[jira] [Commented] (YARN-10319) Record Last N Scheduler Activities from ActivitiesManager

2020-07-04 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151440#comment-17151440 ] Tao Yang commented on YARN-10319: - Thanks [~prabhujoseph] for updating the patch. The latest patch LGTM.

[jira] [Commented] (YARN-10319) Record Last N Scheduler Activities from ActivitiesManager

2020-07-01 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149285#comment-17149285 ] Tao Yang commented on YARN-10319: - Thanks [~adam.antal] for the review and comments, [~prabhujoseph],

[jira] [Commented] (YARN-10319) Record Last N Scheduler Activities from ActivitiesManager

2020-06-30 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149030#comment-17149030 ] Tao Yang commented on YARN-10319: - Thanks for updating the patch and sorry for missing the last comment,

[jira] [Commented] (YARN-10319) Record Last N Scheduler Activities from ActivitiesManager

2020-06-23 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143416#comment-17143416 ] Tao Yang commented on YARN-10319: - Thanks [~prabhujoseph] for this improvement. I agree that it may be

[jira] [Commented] (YARN-8011) TestOpportunisticContainerAllocatorAMService#testContainerPromoteAndDemoteBeforeContainerStart fails sometimes in trunk

2020-06-11 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17133859#comment-17133859 ] Tao Yang commented on YARN-8011: Thanks [~Jim_Brennan] for the feedback and contribution. The patch for

[jira] [Updated] (YARN-8011) TestOpportunisticContainerAllocatorAMService#testContainerPromoteAndDemoteBeforeContainerStart fails sometimes in trunk

2020-06-11 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8011: --- Fix Version/s: 2.10.1 >

[jira] [Commented] (YARN-10293) Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement (YARN-10259)

2020-06-11 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17133848#comment-17133848 ] Tao Yang commented on YARN-10293: - I think this patch is fine enough, and would like to commit the latest

[jira] [Commented] (YARN-10293) Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement (YARN-10259)

2020-06-09 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17129091#comment-17129091 ] Tao Yang commented on YARN-10293: - Thanks [~prabhujoseph] for updating the patch. LGTM now, [~wangda], do

[jira] [Commented] (YARN-10293) Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement (YARN-10259)

2020-06-07 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17127867#comment-17127867 ] Tao Yang commented on YARN-10293: - Thanks [~prabhujoseph] for updating the patch. Another concern in UT

[jira] [Commented] (YARN-10293) Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement (YARN-10259)

2020-06-04 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126407#comment-17126407 ] Tao Yang commented on YARN-10293: - Thanks [~prabhujoseph] for this effort. I'm fine, please go ahead.

[jira] [Commented] (YARN-10293) Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement (YARN-10259)

2020-06-02 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124527#comment-17124527 ] Tao Yang commented on YARN-10293: - Thanks [~wangda] for your confirmation. I think the proposed change

[jira] [Commented] (YARN-10293) Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement (YARN-10259)

2020-06-02 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17123686#comment-17123686 ] Tao Yang commented on YARN-10293: - Hi, [~prabhujoseph], [~wangda] This problem is similar to YARN-9598,

[jira] [Commented] (YARN-9050) [Umbrella] Usability improvements for scheduler activities

2020-03-13 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059185#comment-17059185 ] Tao Yang commented on YARN-9050: Thanks [~cheersyang] very much for your help and patience, very

[jira] [Commented] (YARN-10192) CapacityScheduler stuck in loop rejecting allocation proposals

2020-03-11 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057537#comment-17057537 ] Tao Yang commented on YARN-10192: - Hi, [~wangda]. I'm not sure about this issue, we have found some

[jira] [Commented] (YARN-10151) Disable Capacity Scheduler's move app between queue functionality

2020-02-18 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039665#comment-17039665 ] Tao Yang commented on YARN-10151: - Hi, [~leftnoteasy] FYI, a related issue which can make that happen

[jira] [Commented] (YARN-9567) Add diagnostics for outstanding resource requests on app attempts page

2020-02-04 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029648#comment-17029648 ] Tao Yang commented on YARN-9567: Thanks [~cheersyang] for the review. It seems that wrong file was taken

[jira] [Updated] (YARN-9567) Add diagnostics for outstanding resource requests on app attempts page

2020-02-04 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9567: --- Attachment: YARN-9567.004.patch > Add diagnostics for outstanding resource requests on app attempts page >

[jira] [Commented] (YARN-9538) Document scheduler/app activities and REST APIs

2020-01-19 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019278#comment-17019278 ] Tao Yang commented on YARN-9538: Thanks [~cheersyang] for the review. Attached v4 patch to fix failures

[jira] [Updated] (YARN-9538) Document scheduler/app activities and REST APIs

2020-01-19 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9538: --- Attachment: YARN-9538.004.patch > Document scheduler/app activities and REST APIs >

[jira] [Commented] (YARN-9567) Add diagnostics for outstanding resource requests on app attempts page

2020-01-19 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019208#comment-17019208 ] Tao Yang commented on YARN-9567: Thanks [~cheersyang] for the review. I have attached V3 patch with

[jira] [Updated] (YARN-9567) Add diagnostics for outstanding resource requests on app attempts page

2020-01-19 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9567: --- Attachment: scheduler-activities-example.png > Add diagnostics for outstanding resource requests on app

[jira] [Updated] (YARN-9567) Add diagnostics for outstanding resource requests on app attempts page

2020-01-19 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9567: --- Attachment: app-activities-example.png > Add diagnostics for outstanding resource requests on app attempts

[jira] [Updated] (YARN-9567) Add diagnostics for outstanding resource requests on app attempts page

2020-01-19 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9567: --- Attachment: YARN-9567.003.patch > Add diagnostics for outstanding resource requests on app attempts page >

[jira] [Updated] (YARN-9567) Add diagnostics for outstanding resource requests on app attempts page

2020-01-19 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9567: --- Attachment: (was: YARN-9567.003.patch) > Add diagnostics for outstanding resource requests on app

[jira] [Updated] (YARN-9567) Add diagnostics for outstanding resource requests on app attempts page

2020-01-19 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9567: --- Attachment: YARN-9567.003.patch > Add diagnostics for outstanding resource requests on app attempts page >

[jira] [Commented] (YARN-7007) NPE in RM while using YarnClient.getApplications()

2020-01-10 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-7007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012615#comment-17012615 ] Tao Yang commented on YARN-7007: Already cherry-picked this fix to branch-2.8 > NPE in RM while using

[jira] [Updated] (YARN-7007) NPE in RM while using YarnClient.getApplications()

2020-01-10 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-7007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-7007: --- Fix Version/s: 2.8.6 > NPE in RM while using YarnClient.getApplications() >

[jira] [Commented] (YARN-7007) NPE in RM while using YarnClient.getApplications()

2020-01-10 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-7007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012554#comment-17012554 ] Tao Yang commented on YARN-7007: [~fly_in_gis], thanks for the feedback, I will cherry-pick this fix to

[jira] [Commented] (YARN-9567) Add diagnostics for outstanding resource requests on app attempts page

2020-01-09 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011789#comment-17011789 ] Tao Yang commented on YARN-9567: Thanks [~cheersyang] for the review. {quote} 1. since this is a CS only

[jira] [Updated] (YARN-9538) Document scheduler/app activities and REST APIs

2020-01-09 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9538: --- Attachment: YARN-9538.003.patch > Document scheduler/app activities and REST APIs >

[jira] [Commented] (YARN-9538) Document scheduler/app activities and REST APIs

2020-01-09 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011781#comment-17011781 ] Tao Yang commented on YARN-9538: Attached v3 patch in which most comments are addressed, updates need more

[jira] [Commented] (YARN-9538) Document scheduler/app activities and REST APIs

2020-01-08 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011505#comment-17011505 ] Tao Yang commented on YARN-9538: Thanks [~cheersyang] for finding out mistakes and providing better

[jira] [Commented] (YARN-9538) Document scheduler/app activities and REST APIs

2020-01-08 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011387#comment-17011387 ] Tao Yang commented on YARN-9538: Attached v2 patch which have been checked via hugo in my local test

[jira] [Updated] (YARN-9538) Document scheduler/app activities and REST APIs

2020-01-08 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9538: --- Attachment: YARN-9538.002.patch > Document scheduler/app activities and REST APIs >

[jira] [Commented] (YARN-9050) [Umbrella] Usability improvements for scheduler activities

2020-01-07 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010339#comment-17010339 ] Tao Yang commented on YARN-9050: Glad to hear that 3.3.0 release is on the way and thanks for reminding

[jira] [Updated] (YARN-10059) Final states of failed-to-localize containers are not recorded in NM state store

2019-12-24 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-10059: Attachment: YARN-10059.001.patch > Final states of failed-to-localize containers are not recorded in NM

[jira] [Updated] (YARN-10059) Final states of failed-to-localize containers are not recorded in NM state store

2019-12-24 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-10059: Attachment: (was: YARN-10059.001.patch) > Final states of failed-to-localize containers are not

[jira] [Commented] (YARN-10059) Final states of failed-to-localize containers are not recorded in NM state store

2019-12-23 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002700#comment-17002700 ] Tao Yang commented on YARN-10059: - Attached v1 patch for review. > Final states of failed-to-localize

[jira] [Updated] (YARN-10059) Final states of failed-to-localize containers are not recorded in NM state store

2019-12-23 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-10059: Attachment: YARN-10059.001.patch > Final states of failed-to-localize containers are not recorded in NM

[jira] [Created] (YARN-10059) Final states of failed-to-localize containers are not recorded in NM state store

2019-12-23 Thread Tao Yang (Jira)
Tao Yang created YARN-10059: --- Summary: Final states of failed-to-localize containers are not recorded in NM state store Key: YARN-10059 URL: https://issues.apache.org/jira/browse/YARN-10059 Project: Hadoop

[jira] [Updated] (YARN-9838) Fix resource inconsistency for queues when moving app with reserved container to another queue

2019-11-22 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9838: --- Fix Version/s: 3.1.4 3.2.2 2.9.3 3.3.0 > Fix

[jira] [Updated] (YARN-9838) Fix resource inconsistency for queues when moving app with reserved container to another queue

2019-11-21 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9838: --- Summary: Fix resource inconsistency for queues when moving app with reserved container to another queue

[jira] [Commented] (YARN-9635) Nodes page displayed duplicate nodes

2019-11-14 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974098#comment-16974098 ] Tao Yang commented on YARN-9635: Hi, [~jiwq]. I think the description of conf in NodeManager.md is not

[jira] [Commented] (YARN-9958) Remove the invalid lock in ContainerExecutor

2019-11-14 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974079#comment-16974079 ] Tao Yang commented on YARN-9958: Thanks [~jiwq] for this improvement. Patch LGTM, the related r/w lock

[jira] [Comment Edited] (YARN-7621) Support submitting apps with queue path for CapacityScheduler

2019-10-23 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16957845#comment-16957845 ] Tao Yang edited comment on YARN-7621 at 10/23/19 12:51 PM: --- Hi, [~cane]. Sorry

[jira] [Commented] (YARN-7621) Support submitting apps with queue path for CapacityScheduler

2019-10-23 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16957845#comment-16957845 ] Tao Yang commented on YARN-7621: Hi, [~cane]. Sorry for the late reply. It's make perfect sense for me to

[jira] [Comment Edited] (YARN-7621) Support submitting apps with queue path for CapacityScheduler

2019-10-23 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16957845#comment-16957845 ] Tao Yang edited comment on YARN-7621 at 10/23/19 12:48 PM: --- Hi, [~cane]. Sorry

[jira] [Commented] (YARN-8737) Race condition in ParentQueue when reinitializing and sorting child queues in the meanwhile

2019-10-15 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952049#comment-16952049 ] Tao Yang commented on YARN-8737: Thanks [~cheersyang] for the review. Submitted already. > Race condition

[jira] [Commented] (YARN-8737) Race condition in ParentQueue when reinitializing and sorting child queues in the meanwhile

2019-10-14 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951552#comment-16951552 ] Tao Yang commented on YARN-8737: Thanks [~Amithsha] for the feedback. Sorry to have forgot this issue for

[jira] [Comment Edited] (YARN-9838) Using the CapacityScheduler,Apply "movetoqueue" on the application which CS reserved containers for,will cause "Num Container" and "Used Resource" in ResourceUsage

2019-10-13 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950671#comment-16950671 ] Tao Yang edited comment on YARN-9838 at 10/14/19 3:17 AM: -- Thanks [~jiulongZhu] 

[jira] [Commented] (YARN-9838) Using the CapacityScheduler,Apply "movetoqueue" on the application which CS reserved containers for,will cause "Num Container" and "Used Resource" in ResourceUsage metri

2019-10-13 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950671#comment-16950671 ] Tao Yang commented on YARN-9838: Thanks [~jiulongZhu] for updating the patch. LGTM, +1 for the patch.

[jira] [Comment Edited] (YARN-9838) Using the CapacityScheduler,Apply "movetoqueue" on the application which CS reserved containers for,will cause "Num Container" and "Used Resource" in ResourceUsage

2019-10-11 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949330#comment-16949330 ] Tao Yang edited comment on YARN-9838 at 10/11/19 10:02 AM: --- Thanks [~jiulongZhu]

[jira] [Updated] (YARN-9838) Using the CapacityScheduler,Apply "movetoqueue" on the application which CS reserved containers for,will cause "Num Container" and "Used Resource" in ResourceUsage metrics

2019-10-11 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9838: --- Issue Type: Bug (was: Improvement) > Using the CapacityScheduler,Apply "movetoqueue" on the application

[jira] [Updated] (YARN-9838) Using the CapacityScheduler,Apply "movetoqueue" on the application which CS reserved containers for,will cause "Num Container" and "Used Resource" in ResourceUsage metrics

2019-10-11 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9838: --- Fix Version/s: (was: 2.7.3) > Using the CapacityScheduler,Apply "movetoqueue" on the application which CS

[jira] [Commented] (YARN-9838) Using the CapacityScheduler,Apply "movetoqueue" on the application which CS reserved containers for,will cause "Num Container" and "Used Resource" in ResourceUsage metri

2019-10-11 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949330#comment-16949330 ] Tao Yang commented on YARN-9838: Thanks [~jiulongZhu] for fixing this issue. The patch is LGTM in

[jira] [Comment Edited] (YARN-8995) Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time.

2019-09-06 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924672#comment-16924672 ] Tao Yang edited comment on YARN-8995 at 9/7/19 12:33 AM: - Thanks [~jhung] for

[jira] [Commented] (YARN-8995) Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time.

2019-09-06 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924672#comment-16924672 ] Tao Yang commented on YARN-8995: Thanks [~jhung] for fixing this problem, sorry for missing changes about

[jira] [Commented] (YARN-9817) Fix failing testcases due to not initialized AsyncDispatcher - ArithmeticException: / by zero

2019-09-06 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924659#comment-16924659 ] Tao Yang commented on YARN-9817: Thanks [~Prabhu Joseph] for raising this issue. Patch LGTM, committing

[jira] [Commented] (YARN-9795) ClusterMetrics to include AM allocation delay

2019-09-05 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923891#comment-16923891 ] Tao Yang commented on YARN-9795: +1 for the latest patch. I will commit this if no further comments from

[jira] [Commented] (YARN-9795) ClusterMetrics to include AM allocation delay

2019-09-05 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923882#comment-16923882 ] Tao Yang commented on YARN-9795: Thanks [~fengnanli] for the update. A small suggestion is to remove null

[jira] [Commented] (YARN-9795) ClusterMetrics to include AM allocation delay

2019-09-04 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923024#comment-16923024 ] Tao Yang commented on YARN-9795: Thanks [~fengnanli] for this improvement. Patch almost LGTM, IMO,

[jira] [Commented] (YARN-8995) Log the event type of the too big AsyncDispatcher event queue size, and add the information to the metrics.

2019-09-04 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922996#comment-16922996 ] Tao Yang commented on YARN-8995: Hi, [~zhuqi], I found another place need to be improved. {{ if (qSize %

[jira] [Commented] (YARN-8995) Log the event type of the too big AsyncDispatcher event queue size, and add the information to the metrics.

2019-09-04 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922279#comment-16922279 ] Tao Yang commented on YARN-8995: Confirmed that latest patch should not fail like that. Now the patch

[jira] [Commented] (YARN-8995) Log the event type of the too big AsyncDispatcher event queue size, and add the information to the metrics.

2019-09-04 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921981#comment-16921981 ] Tao Yang commented on YARN-8995: Hi, [~zhuqi]. I noticed

[jira] [Commented] (YARN-8995) Log the event type of the too big AsyncDispatcher event queue size, and add the information to the metrics.

2019-09-01 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920568#comment-16920568 ] Tao Yang commented on YARN-8995: Thanks [~zhuqi] for the update. Patch LGTM, could you please also fix the

[jira] [Commented] (YARN-9540) TestRMAppTransitions fails intermittently

2019-08-30 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919658#comment-16919658 ] Tao Yang commented on YARN-9540: Thanks [~abmodi], [~adam.antal] for the review and commit. >

[jira] [Commented] (YARN-9798) ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails intermittently

2019-08-30 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919654#comment-16919654 ] Tao Yang commented on YARN-9798: Thanks [~abmodi] for the review. The frequency is only 1 or 2 failures

[jira] [Updated] (YARN-9798) ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails intermittently

2019-08-30 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9798: --- Attachment: (was: YARN-9798.001.patch) >

[jira] [Updated] (YARN-9798) ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails intermittently

2019-08-30 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9798: --- Attachment: YARN-9798.001.patch > ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails

[jira] [Commented] (YARN-9714) ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919204#comment-16919204 ] Tao Yang commented on YARN-9714: Thanks [~rohithsharma], [~bibinchundatt] for the review and commit! >

[jira] [Resolved] (YARN-9803) NPE while accessing Scheduler UI

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang resolved YARN-9803. Resolution: Duplicate Hi, [~yifan.stan]. This is a duplicate of YARN-9685, closing it as duplicate. > NPE

[jira] [Comment Edited] (YARN-9540) TestRMAppTransitions fails intermittently

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919097#comment-16919097 ] Tao Yang edited comment on YARN-9540 at 8/30/19 2:00 AM: - Hi, [~adam.antal]. The

[jira] [Commented] (YARN-9540) TestRMAppTransitions fails intermittently

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919097#comment-16919097 ] Tao Yang commented on YARN-9540: Hi, [~adam.antal]. The cause is that the assertion which will make sure

[jira] [Commented] (YARN-9664) Improve response of scheduler/app activities for better understanding

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918511#comment-16918511 ] Tao Yang commented on YARN-9664: Thanks [~cheersyang] for the review and commit! > Improve response of

[jira] [Commented] (YARN-9538) Document scheduler/app activities and REST APIs

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918510#comment-16918510 ] Tao Yang commented on YARN-9538: Thanks [~cheersyang] for reminding me, I will do that later. > Document

[jira] [Updated] (YARN-9540) TestRMAppTransitions fails intermittently

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9540: --- Attachment: YARN-9540.001.patch > TestRMAppTransitions fails intermittently >

[jira] [Assigned] (YARN-9540) TestRMAppTransitions fails intermittently

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang reassigned YARN-9540: -- Assignee: Tao Yang (was: Prabhu Joseph) > TestRMAppTransitions fails intermittently >

[jira] [Commented] (YARN-9799) TestRMAppTransitions#testAppFinishedFinished fails intermittently

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918506#comment-16918506 ] Tao Yang commented on YARN-9799: Thanks [~Prabhu Joseph] for reminding me, I'll fix this issue over there.

[jira] [Commented] (YARN-9664) Improve response of scheduler/app activities for better understanding

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918469#comment-16918469 ] Tao Yang commented on YARN-9664: Hi, [~cheersyang]. {quote} UT seems not related to this patch, Tao Yang,

[jira] [Updated] (YARN-9799) TestRMAppTransitions#testAppFinishedFinished fails intermittently

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9799: --- Attachment: YARN-9799.001.patch > TestRMAppTransitions#testAppFinishedFinished fails intermittently >

[jira] [Created] (YARN-9799) TestRMAppTransitions#testAppFinishedFinished fails intermittently

2019-08-29 Thread Tao Yang (Jira)
Tao Yang created YARN-9799: -- Summary: TestRMAppTransitions#testAppFinishedFinished fails intermittently Key: YARN-9799 URL: https://issues.apache.org/jira/browse/YARN-9799 Project: Hadoop YARN

[jira] [Commented] (YARN-9714) ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918446#comment-16918446 ] Tao Yang commented on YARN-9714: There is an intermittent UT failure in the latest jenkins report, I have

[jira] [Updated] (YARN-9798) ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails intermittently

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9798: --- Attachment: YARN-9798.001.patch > ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails

[jira] [Created] (YARN-9798) ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails intermittently

2019-08-29 Thread Tao Yang (Jira)
Tao Yang created YARN-9798: -- Summary: ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails intermittently Key: YARN-9798 URL: https://issues.apache.org/jira/browse/YARN-9798 Project:

[jira] [Commented] (YARN-9664) Improve response of scheduler/app activities for better understanding

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918321#comment-16918321 ] Tao Yang commented on YARN-9664: Thanks [~cheersyang] for the advice. Attached v3 patch. > Improve

[jira] [Updated] (YARN-9664) Improve response of scheduler/app activities for better understanding

2019-08-29 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9664: --- Attachment: YARN-9664.003.patch > Improve response of scheduler/app activities for better understanding >

[jira] [Commented] (YARN-9714) ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby

2019-08-28 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918273#comment-16918273 ] Tao Yang commented on YARN-9714: Hi, [~rohithsharma]. UT log is filled with these errors:

[jira] [Commented] (YARN-9664) Improve response of scheduler/app activities for better understanding

2019-08-28 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918247#comment-16918247 ] Tao Yang commented on YARN-9664: Thanks [~cheersyang] for the review. {quote} ActivitiesUtils Line 56: I

[jira] [Commented] (YARN-9664) Improve response of scheduler/app activities for better understanding

2019-08-28 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917516#comment-16917516 ] Tao Yang commented on YARN-9664: Hi, [~cheersyang], it indeed changes a lot and most of them are

[jira] [Commented] (YARN-8995) Log the event type of the too big AsyncDispatcher event queue size, and add the information to the metrics.

2019-08-28 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917507#comment-16917507 ] Tao Yang commented on YARN-8995: Hi, [~zhuqi]. The latest patch seems not applicable for trunk now, could

[jira] [Updated] (YARN-9714) ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby

2019-08-28 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9714: --- Attachment: YARN-9714.005.patch > ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to

[jira] [Commented] (YARN-9714) ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby

2019-08-28 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917482#comment-16917482 ] Tao Yang commented on YARN-9714: {quote} Instead of comparing, how about checking for

[jira] [Updated] (YARN-9714) ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby

2019-08-26 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9714: --- Attachment: YARN-9714.004.patch > ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to

[jira] [Commented] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource

2019-08-26 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916291#comment-16916291 ] Tao Yang commented on YARN-8917: Thanks [~rohithsharma], [~leftnoteasy], [~sunilg] for the review and

[jira] [Commented] (YARN-9714) ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby

2019-08-26 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916290#comment-16916290 ] Tao Yang commented on YARN-9714: TestZKRMStateStore#testZKRootPathAcls UT failure is caused by itself,

[jira] [Commented] (YARN-8193) YARN RM hangs abruptly (stops allocating resources) when running successive applications.

2019-08-26 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-8193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915685#comment-16915685 ] Tao Yang commented on YARN-8193: Hi, [~sunilg], [~leftnoteasy]. Any updates or plans about this fix on

[jira] [Commented] (YARN-9779) NPE while allocating a container

2019-08-26 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915684#comment-16915684 ] Tao Yang commented on YARN-9779: Sorry for the late reply. I think this issue is duplicate with YARN-8193.

[jira] [Issue Comment Deleted] (YARN-9779) NPE while allocating a container

2019-08-26 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9779: --- Comment: was deleted (was: Sorry for the late reply. I think this issue is duplicate with YARN-8193.) > NPE

[jira] [Updated] (YARN-9714) ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby

2019-08-26 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-9714: --- Attachment: YARN-9714.003.patch > ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to

  1   2   3   4   5   6   7   8   9   10   >