[jira] [Commented] (YARN-10178) Global Scheduler async thread crash caused by 'Comparison method violates its general contract'

2021-09-14 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415305#comment-17415305 ] Qi Zhu commented on YARN-10178: --- Thanks [~wangda] for reply. cc [~MatthewSharp] [~it_singer]  I am glad

[jira] [Commented] (YARN-10935) AM Total Queue Limit goes below per-user AM Limit if parent is full.

2021-09-14 Thread Eric Badger (Jira)
[ https://issues.apache.org/jira/browse/YARN-10935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415243#comment-17415243 ] Eric Badger commented on YARN-10935: [~epayne], +1 the patch looks good to me. However, trunk

[jira] [Created] (YARN-10949) Simplify AbstractCSQueue#updateMaxAppRelatedField and find a more meaningful name for this method

2021-09-14 Thread Szilard Nemeth (Jira)
Szilard Nemeth created YARN-10949: - Summary: Simplify AbstractCSQueue#updateMaxAppRelatedField and find a more meaningful name for this method Key: YARN-10949 URL: https://issues.apache.org/jira/browse/YARN-10949

[jira] [Created] (YARN-10954) Remove commented code block from CSQueueUtils#loadCapacitiesByLabelsFromConf

2021-09-14 Thread Szilard Nemeth (Jira)
Szilard Nemeth created YARN-10954: - Summary: Remove commented code block from CSQueueUtils#loadCapacitiesByLabelsFromConf Key: YARN-10954 URL: https://issues.apache.org/jira/browse/YARN-10954

[jira] [Created] (YARN-10953) Make CapacityScheduler#getOrCreateQueueFromPlacementContext more easy to comprehend

2021-09-14 Thread Szilard Nemeth (Jira)
Szilard Nemeth created YARN-10953: - Summary: Make CapacityScheduler#getOrCreateQueueFromPlacementContext more easy to comprehend Key: YARN-10953 URL: https://issues.apache.org/jira/browse/YARN-10953

[jira] [Created] (YARN-10952) Move CapacityScheduler#updatePlacementRules elsewhere

2021-09-14 Thread Szilard Nemeth (Jira)
Szilard Nemeth created YARN-10952: - Summary: Move CapacityScheduler#updatePlacementRules elsewhere Key: YARN-10952 URL: https://issues.apache.org/jira/browse/YARN-10952 Project: Hadoop YARN

[jira] [Created] (YARN-10951) CapacityScheduler: Move all fields and initializer code that belongs to async scheduling to a new class

2021-09-14 Thread Szilard Nemeth (Jira)
Szilard Nemeth created YARN-10951: - Summary: CapacityScheduler: Move all fields and initializer code that belongs to async scheduling to a new class Key: YARN-10951 URL:

[jira] [Created] (YARN-10950) Code cleanup in QueueCapacities

2021-09-14 Thread Szilard Nemeth (Jira)
Szilard Nemeth created YARN-10950: - Summary: Code cleanup in QueueCapacities Key: YARN-10950 URL: https://issues.apache.org/jira/browse/YARN-10950 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Created] (YARN-10948) Rename SchedulerQueue#activeQueue to activateQueue

2021-09-14 Thread Szilard Nemeth (Jira)
Szilard Nemeth created YARN-10948: - Summary: Rename SchedulerQueue#activeQueue to activateQueue Key: YARN-10948 URL: https://issues.apache.org/jira/browse/YARN-10948 Project: Hadoop YARN

[jira] [Created] (YARN-10947) Simplify AbstractCSQueue#initializeQueueState

2021-09-14 Thread Szilard Nemeth (Jira)
Szilard Nemeth created YARN-10947: - Summary: Simplify AbstractCSQueue#initializeQueueState Key: YARN-10947 URL: https://issues.apache.org/jira/browse/YARN-10947 Project: Hadoop YARN Issue

[jira] [Created] (YARN-10946) AbstractCSQueue: Create separate class for constructing Queue API objects

2021-09-14 Thread Szilard Nemeth (Jira)
Szilard Nemeth created YARN-10946: - Summary: AbstractCSQueue: Create separate class for constructing Queue API objects Key: YARN-10946 URL: https://issues.apache.org/jira/browse/YARN-10946 Project:

[jira] [Created] (YARN-10944) AbstractCSQueue: Eliminate code duplication in overloaded versions of setMaxCapacity

2021-09-14 Thread Szilard Nemeth (Jira)
Szilard Nemeth created YARN-10944: - Summary: AbstractCSQueue: Eliminate code duplication in overloaded versions of setMaxCapacity Key: YARN-10944 URL: https://issues.apache.org/jira/browse/YARN-10944

[jira] [Created] (YARN-10945) Add javadoc to all methods of AbstractCSQueue

2021-09-14 Thread Szilard Nemeth (Jira)
Szilard Nemeth created YARN-10945: - Summary: Add javadoc to all methods of AbstractCSQueue Key: YARN-10945 URL: https://issues.apache.org/jira/browse/YARN-10945 Project: Hadoop YARN Issue

[jira] [Updated] (YARN-10943) AbstractCSQueue: Create separate class for encapsulating Min / Max Resource

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10943: -- Description: There are certain methods where min and max Resources are used in tandem. Some

[jira] [Updated] (YARN-10943) AbstractCSQueue: Create separate class for encapsulating Min / Max Resource

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10943: -- Description: There are certain methods where min and max Resources are used in tandem. Some

[jira] [Created] (YARN-10943) AbstractCSQueue: Create separate class for encapsulating Min / Max Resource

2021-09-14 Thread Szilard Nemeth (Jira)
Szilard Nemeth created YARN-10943: - Summary: AbstractCSQueue: Create separate class for encapsulating Min / Max Resource Key: YARN-10943 URL: https://issues.apache.org/jira/browse/YARN-10943 Project:

[jira] [Updated] (YARN-10920) Create a dedicated class for Node Labels

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10920: -- Summary: Create a dedicated class for Node Labels (was: Created a dedicated class for Node

[jira] [Commented] (YARN-10178) Global Scheduler async thread crash caused by 'Comparison method violates its general contract'

2021-09-14 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415053#comment-17415053 ] Wangda Tan commented on YARN-10178: --- [~pbacsko], [~MatthewSharp], I just took a quick look at the patch

[jira] [Commented] (YARN-10178) Global Scheduler async thread crash caused by 'Comparison method violates its general contract'

2021-09-14 Thread Matthew Sharp (Jira)
[ https://issues.apache.org/jira/browse/YARN-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415025#comment-17415025 ] Matthew Sharp commented on YARN-10178: -- We have been seeing this frequently in our 3.3.0 cluster as

[jira] [Updated] (YARN-10870) Missing user filtering check -> yarn.webapp.filter-entity-list-by-user for RM Scheduler page

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10870: -- Fix Version/s: 3.2.4 > Missing user filtering check -> yarn.webapp.filter-entity-list-by-user

[jira] [Commented] (YARN-10870) Missing user filtering check -> yarn.webapp.filter-entity-list-by-user for RM Scheduler page

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415024#comment-17415024 ] Szilard Nemeth commented on YARN-10870: --- [~shuzirra] Thanks for working on the backports. The

[jira] [Resolved] (YARN-10870) Missing user filtering check -> yarn.webapp.filter-entity-list-by-user for RM Scheduler page

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth resolved YARN-10870. --- Resolution: Fixed > Missing user filtering check -> yarn.webapp.filter-entity-list-by-user

[jira] [Updated] (YARN-10870) Missing user filtering check -> yarn.webapp.filter-entity-list-by-user for RM Scheduler page

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10870: -- Fix Version/s: 3.3.2 > Missing user filtering check -> yarn.webapp.filter-entity-list-by-user

[jira] [Updated] (YARN-10915) AbstractCSQueue: Simplify complex logic in methods: deriveCapacityFromAbsoluteConfigurations and updateEffectiveResources

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10915: -- Fix Version/s: 3.4.0 > AbstractCSQueue: Simplify complex logic in methods: >

[jira] [Updated] (YARN-10917) Investigate and simplify CapacitySchedulerConfigValidator#validateQueueHierarchy

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10917: -- Fix Version/s: 3.4.0 > Investigate and simplify >

[jira] [Updated] (YARN-10901) Permission checking error on an existing directory in LogAggregationFileController#verifyAndCreateRemoteLogDir

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10901: -- Fix Version/s: 3.3.2 3.2.3 > Permission checking error on an existing

[jira] [Resolved] (YARN-10912) AbstractCSQueue#updateConfigurableResourceRequirement: Separate validation logic from initialization logic

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth resolved YARN-10912. --- Hadoop Flags: Reviewed Resolution: Fixed >

[jira] [Updated] (YARN-10912) AbstractCSQueue#updateConfigurableResourceRequirement: Separate validation logic from initialization logic

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10912: -- Fix Version/s: 3.4.0 > AbstractCSQueue#updateConfigurableResourceRequirement: Separate

[jira] [Comment Edited] (YARN-10920) Created a dedicated class for Node Labels

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414979#comment-17414979 ] Szilard Nemeth edited comment on YARN-10920 at 9/14/21, 2:46 PM: - This is

[jira] [Comment Edited] (YARN-10920) Created a dedicated class for Node Labels

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414979#comment-17414979 ] Szilard Nemeth edited comment on YARN-10920 at 9/14/21, 2:46 PM: - This is

[jira] [Commented] (YARN-10920) Created a dedicated class for Node Labels

2021-09-14 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-10920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414979#comment-17414979 ] Szilard Nemeth commented on YARN-10920: --- This is one hell of an effort. 1. First, I need to

[jira] [Commented] (YARN-10940) Fix Documentation for AWS-Hadoop integration / yarn-site.xml

2021-09-14 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/YARN-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414876#comment-17414876 ] Steve Loughran commented on YARN-10940: --- locally I use Xinclude in core-site XML files to include a