[jira] [Commented] (YARN-6257) CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains duplicate keys

2018-11-05 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674916#comment-16674916 ] Tao Yang commented on YARN-6257: Thanks [~cheersyang] for your quick update! It looks good to me. >

[jira] [Updated] (YARN-8925) Updating distributed node attributes only when necessary

2018-11-05 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8925: --- Attachment: YARN-8925.003.patch > Updating distributed node attributes only when necessary >

[jira] [Commented] (YARN-8925) Updating distributed node attributes only when necessary

2018-11-05 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674825#comment-16674825 ] Tao Yang commented on YARN-8925: Attached V3 patch with modifications as follow: (1) Adding RM side test

[jira] [Commented] (YARN-8233) NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null

2018-11-04 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674625#comment-16674625 ] Tao Yang commented on YARN-8233: Attached v3 patch to fix checkstyle warnings through removing useless

[jira] [Updated] (YARN-8233) NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null

2018-11-04 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8233: --- Attachment: YARN-8233.003.patch > NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal

[jira] [Commented] (YARN-2599) Standby RM should also expose some jmx and metrics

2018-11-04 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674624#comment-16674624 ] Tao Yang commented on YARN-2599: Hi, [~Naganarasimha] & [~kasha] & [~rohithsharma], We have met problems

[jira] [Commented] (YARN-8233) NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null

2018-11-02 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673031#comment-16673031 ] Tao Yang commented on YARN-8233: Attached v2 patch for review. > NPE in CapacityScheduler#tryCommit when

[jira] [Updated] (YARN-8233) NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null

2018-11-02 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8233: --- Attachment: YARN-8233.002.patch > NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal

[jira] [Commented] (YARN-8958) Schedulable entities leak in fair ordering policy when recovering containers between remove app attempt and remove app

2018-11-02 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672875#comment-16672875 ] Tao Yang commented on YARN-8958: {quote} I am not sure about that... The cached usage is used by the

[jira] [Commented] (YARN-8233) NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null

2018-11-02 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672835#comment-16672835 ] Tao Yang commented on YARN-8233: Hi, [~surmountian], [~cheersyang] {quote}I also met this issue in Hadoop

[jira] [Comment Edited] (YARN-8925) Updating distributed node attributes only when necessary

2018-11-02 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672769#comment-16672769 ] Tao Yang edited comment on YARN-8925 at 11/2/18 8:41 AM: - Hi, [~cheersyang]. I

[jira] [Commented] (YARN-8925) Updating distributed node attributes only when necessary

2018-11-02 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672769#comment-16672769 ] Tao Yang commented on YARN-8925: Hi, [~cheersyang]. I have attached V2 patch for review, this patch is not

[jira] [Updated] (YARN-8925) Updating distributed node attributes only when necessary

2018-11-02 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8925: --- Attachment: YARN-8925.002.patch > Updating distributed node attributes only when necessary >

[jira] [Commented] (YARN-8958) Schedulable entities leak in fair ordering policy when recovering containers between remove app attempt and remove app

2018-11-02 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672606#comment-16672606 ] Tao Yang commented on YARN-8958: Hi, [~cheersyang], thanks for the review. {quote} Can we also only do the

[jira] [Commented] (YARN-8958) Schedulable entities leak in fair ordering policy when recovering containers between remove app attempt and remove app

2018-10-31 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671030#comment-16671030 ] Tao Yang commented on YARN-8958: There is no UT failure but still got -1 for unit by Hadoop QA.

[jira] [Comment Edited] (YARN-8958) Schedulable entities leak in fair ordering policy when recovering containers between remove app attempt and remove app

2018-10-31 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671024#comment-16671024 ] Tao Yang edited comment on YARN-8958 at 11/1/18 2:09 AM: - Thanks [~cheersyang] for

[jira] [Updated] (YARN-8958) Schedulable entities leak in fair ordering policy when recovering containers between remove app attempt and remove app

2018-10-31 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8958: --- Description: We found a NPE in ClientRMService#getApplications when querying apps with specified queue. The

[jira] [Commented] (YARN-8958) Schedulable entities leak in fair ordering policy when recovering containers between remove app attempt and remove app

2018-10-31 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671024#comment-16671024 ] Tao Yang commented on YARN-8958: Thanks [~cheersyang] for the review. {quote} In

[jira] [Commented] (YARN-8958) Schedulable entities leak in fair ordering policy when recovering containers between remove app attempt and remove app

2018-10-30 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669111#comment-16669111 ] Tao Yang commented on YARN-8958: Attached v2 patch to fix UT failures (1) Set

[jira] [Updated] (YARN-8958) Schedulable entities leak in fair ordering policy when recovering containers between remove app attempt and remove app

2018-10-30 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8958: --- Attachment: YARN-8958.002.patch > Schedulable entities leak in fair ordering policy when recovering

[jira] [Updated] (YARN-8958) Schedulable entities leak in fair ordering policy when recovering containers between remove app attempt and remove app

2018-10-30 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8958: --- Attachment: YARN-8958.001.patch > Schedulable entities leak in fair ordering policy when recovering

[jira] [Commented] (YARN-8958) Schedulable entities leak in fair ordering policy when recovering containers between remove app attempt and remove app

2018-10-30 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668305#comment-16668305 ] Tao Yang commented on YARN-8958: Attached v1 patch for review. > Schedulable entities leak in fair

[jira] [Created] (YARN-8958) Schedulable entities leak in fair ordering policy when recovering containers between remove app attempt and remove app

2018-10-30 Thread Tao Yang (JIRA)
Tao Yang created YARN-8958: -- Summary: Schedulable entities leak in fair ordering policy when recovering containers between remove app attempt and remove app Key: YARN-8958 URL:

[jira] [Commented] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource

2018-10-26 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16665841#comment-16665841 ] Tao Yang commented on YARN-8917: Thanks [~sunilg] for the review. I can't reproduce these UT failures in

[jira] [Commented] (YARN-8945) Calculation of maximum applications should respect specified and global maximum applications for absolute resource

2018-10-25 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16664584#comment-16664584 ] Tao Yang commented on YARN-8945: Attached v1 patch for review. > Calculation of maximum applications

[jira] [Updated] (YARN-8945) Calculation of maximum applications should respect specified and global maximum applications for absolute resource

2018-10-25 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8945: --- Attachment: YARN-8945.001.patch > Calculation of maximum applications should respect specified and global >

[jira] [Commented] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource

2018-10-25 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16664570#comment-16664570 ] Tao Yang commented on YARN-8917: Hi, [~sunilg] [~leftnoteasy], v2 patch updated UT to add assert

[jira] [Updated] (YARN-8945) Calculation of maximum applications should respect specified and global maximum applications for absolute resource

2018-10-25 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8945: --- Summary: Calculation of maximum applications should respect specified and global maximum applications for

[jira] [Created] (YARN-8945) Calculation of Maximum applications should respect specified and global maximum applications for absolute resource

2018-10-25 Thread Tao Yang (JIRA)
Tao Yang created YARN-8945: -- Summary: Calculation of Maximum applications should respect specified and global maximum applications for absolute resource Key: YARN-8945 URL:

[jira] [Commented] (YARN-8925) Updating distributed node attributes only when necessary

2018-10-23 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660592#comment-16660592 ] Tao Yang commented on YARN-8925: Thanks [~cheersyang] for your review and suggestion. Yes, that makes

[jira] [Commented] (YARN-8925) Updating distributed node attributes only when necessary

2018-10-22 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16658689#comment-16658689 ] Tao Yang commented on YARN-8925: Attached v1 patch for review. [~cheersyang], could you help to review

[jira] [Updated] (YARN-8925) Updating distributed node attributes only when necessary

2018-10-22 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8925: --- Attachment: YARN-8925.001.patch > Updating distributed node attributes only when necessary >

[jira] [Created] (YARN-8925) Updating distributed node attributes only when necessary

2018-10-22 Thread Tao Yang (JIRA)
Tao Yang created YARN-8925: -- Summary: Updating distributed node attributes only when necessary Key: YARN-8925 URL: https://issues.apache.org/jira/browse/YARN-8925 Project: Hadoop YARN Issue Type:

[jira] [Updated] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource

2018-10-19 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8917: --- Attachment: YARN-8917.002.patch > Absolute (maximum) capacity of level3+ queues is wrongly calculated for >

[jira] [Commented] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource

2018-10-19 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657637#comment-16657637 ] Tao Yang commented on YARN-8917: Thanks [~sunilg], [~leftnoteasy] for the review. {quote} cud u pls write

[jira] [Commented] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource

2018-10-19 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656524#comment-16656524 ] Tao Yang commented on YARN-8917: Attached v1 patch for review. > Absolute (maximum) capacity of level3+

[jira] [Updated] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource

2018-10-19 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8917: --- Attachment: YARN-8917.001.patch > Absolute (maximum) capacity of level3+ queues is wrongly calculated for >

[jira] [Created] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource

2018-10-19 Thread Tao Yang (JIRA)
Tao Yang created YARN-8917: -- Summary: Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource Key: YARN-8917 URL: https://issues.apache.org/jira/browse/YARN-8917 Project:

[jira] [Commented] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633185#comment-16633185 ] Tao Yang commented on YARN-8774: Thanks [~eepayne] for the review and commit. {quote} It would have been

[jira] [Commented] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-26 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629699#comment-16629699 ] Tao Yang commented on YARN-8804: Thanks [~jlowe] for the review and commit! > resourceLimits may be

[jira] [Comment Edited] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624367#comment-16624367 ] Tao Yang edited comment on YARN-8774 at 9/22/18 12:17 AM: -- Thanks [~eepayne] for

[jira] [Commented] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624367#comment-16624367 ] Tao Yang commented on YARN-8774: Thanks [~eepayne] for the review. Attached v2 patch to improve the UT

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: YARN-8774.branch-2.001.patch > Memory leak when CapacityScheduler allocates from reserved

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: YARN-8774.branch-2.8.001.patch > Memory leak when CapacityScheduler allocates from reserved

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: YARN-8774.002.patch > Memory leak when CapacityScheduler allocates from reserved container with

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: (was: YARN-8774.branch-2.8.001.patch) > Memory leak when CapacityScheduler allocates from

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: (was: YARN-8774.branch-2.001.patch) > Memory leak when CapacityScheduler allocates from

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: YARN-8774.branch-2.8.001.patch > Memory leak when CapacityScheduler allocates from reserved

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: YARN-8774.branch-2.001.patch > Memory leak when CapacityScheduler allocates from reserved

[jira] [Commented] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624281#comment-16624281 ] Tao Yang commented on YARN-8804: Thanks [~jlowe] for the review. Attached v3 patch to rebase it on latest

[jira] [Updated] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8804: --- Attachment: YARN-8804.003.patch > resourceLimits may be wrongly calculated when leaf-queue is blocked in >

[jira] [Commented] (YARN-2599) Standby RM should also expose some jmx and metrics

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623315#comment-16623315 ] Tao Yang commented on YARN-2599: Hi, [~Naganarasimha]. Some JVM metrics of standby RM can be evaluated

[jira] [Comment Edited] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623035#comment-16623035 ] Tao Yang edited comment on YARN-8804 at 9/21/18 4:17 AM: - Thanks

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

2018-09-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623040#comment-16623040 ] Tao Yang commented on YARN-8513: Hi, [~cyfdecyf] Dose your cluster have empty resource type? This problem

[jira] [Commented] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623037#comment-16623037 ] Tao Yang commented on YARN-8804: Attached v2 patch to remove violate keyword and improve the calculation.

[jira] [Updated] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8804: --- Attachment: YARN-8804.002.patch > resourceLimits may be wrongly calculated when leaf-queue is blocked in >

[jira] [Comment Edited] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623035#comment-16623035 ] Tao Yang edited comment on YARN-8804 at 9/21/18 4:05 AM: - Thanks

[jira] [Commented] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623035#comment-16623035 ] Tao Yang commented on YARN-8804: Thanks [~jlowe],[~leftnoteasy] for your review and reply.  For the 

[jira] [Updated] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8804: --- Attachment: YARN-8804.001.patch > resourceLimits may be wrongly calculated when leaf-queue is blocked in >

[jira] [Commented] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621631#comment-16621631 ] Tao Yang commented on YARN-8804: Attached v1 patch which will add a blockedHeadroom field in

[jira] [Updated] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8804: --- Description: This problem is due to YARN-4280, parent queue will deduct child queue's headroom when the

[jira] [Created] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Tao Yang (JIRA)
Tao Yang created YARN-8804: -- Summary: resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues Key: YARN-8804 URL: https://issues.apache.org/jira/browse/YARN-8804

[jira] [Commented] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-19 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621414#comment-16621414 ] Tao Yang commented on YARN-8771: Thanks [~cheersyang] and [~leftnoteasy] ! > CapacityScheduler fails to

[jira] [Commented] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-19 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620333#comment-16620333 ] Tao Yang commented on YARN-8771: Attached v4 patch to fix check-style error. UT failures seem unrelated

[jira] [Updated] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-19 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8771: --- Attachment: YARN-8771.004.patch > CapacityScheduler fails to unreserve when cluster resource contains empty

[jira] [Commented] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-18 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620042#comment-16620042 ] Tao Yang commented on YARN-8771: Attached v3 patch to improve unit test without adding new

[jira] [Updated] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-18 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8771: --- Attachment: YARN-8771.003.patch > CapacityScheduler fails to unreserve when cluster resource contains empty

[jira] [Commented] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-18 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620001#comment-16620001 ] Tao Yang commented on YARN-8771: Thanks [~leftnoteasy] for your review and suggestion. I will update the

[jira] [Commented] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-18 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619181#comment-16619181 ] Tao Yang commented on YARN-8771: Thanks [~cheersyang] for the review. {quote} Instead of adding a new

[jira] [Updated] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-16 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8771: --- Description: We found this problem when cluster is almost but not exhausted (93% used), scheduler kept

[jira] [Updated] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-16 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8771: --- Description: We found this problem when cluster is almost but not exhausted (93% used), scheduler kept

[jira] [Updated] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-16 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8771: --- Description: We found this problem when cluster is almost but not exhausted (93% used), scheduler kept

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-16 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Affects Version/s: 2.8.5 3.2.0 > Memory leak when CapacityScheduler allocates from

[jira] [Commented] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-14 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16614866#comment-16614866 ] Tao Yang commented on YARN-8774: Attached v1 patch for review. > Memory leak when CapacityScheduler

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-14 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: YARN-8774.001.patch > Memory leak when CapacityScheduler allocates from reserved container with

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-14 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Description: The cause is that the RMContainerImpl instance of reserved container lost its node label

[jira] [Created] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-14 Thread Tao Yang (JIRA)
Tao Yang created YARN-8774: -- Summary: Memory leak when CapacityScheduler allocates from reserved container with non-default label Key: YARN-8774 URL: https://issues.apache.org/jira/browse/YARN-8774 Project:

[jira] [Updated] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-13 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8771: --- Attachment: YARN-8771.002.patch > CapacityScheduler fails to unreserve when cluster resource contains empty

[jira] [Commented] (YARN-8729) Node status updater thread could be lost after it is restarted

2018-09-13 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16614289#comment-16614289 ] Tao Yang commented on YARN-8729: Thanks [~cheersyang], [~ebadger] ! > Node status updater thread could be

[jira] [Commented] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-13 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613299#comment-16613299 ] Tao Yang commented on YARN-8771: Attached v1 patch for review. [~cheersyang], can you help to review this

[jira] [Updated] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-13 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8771: --- Attachment: YARN-8771.001.patch > CapacityScheduler fails to unreserve when cluster resource contains empty

[jira] [Created] (YARN-8771) CapacityScheduler fails to unreserve when cluster resource contains empty resource type

2018-09-13 Thread Tao Yang (JIRA)
Tao Yang created YARN-8771: -- Summary: CapacityScheduler fails to unreserve when cluster resource contains empty resource type Key: YARN-8771 URL: https://issues.apache.org/jira/browse/YARN-8771 Project:

[jira] [Commented] (YARN-8764) Comparison method violates its general contract in Async Schedule

2018-09-11 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611604#comment-16611604 ] Tao Yang commented on YARN-8764: Just notice that this issue is about Hadoop2.8, I agree that this should

[jira] [Commented] (YARN-8764) Comparison method violates its general contract in Async Schedule

2018-09-11 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611463#comment-16611463 ] Tao Yang commented on YARN-8764: Hi, [~ziqian hu]. I have found this problem and suggested to add

[jira] [Commented] (YARN-8709) CS preemption monitor always fails since one under-served queue was deleted

2018-09-10 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16609997#comment-16609997 ] Tao Yang commented on YARN-8709: Thanks [~eepayne],[~cheersyang] and [~sunilg]. > CS preemption monitor

[jira] [Updated] (YARN-8709) intra-queue preemption checker always fail since one under-served queue was deleted

2018-09-07 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8709: --- Attachment: YARN-8709.002.patch > intra-queue preemption checker always fail since one under-served queue was

[jira] [Commented] (YARN-8709) intra-queue preemption checker always fail since one under-served queue was deleted

2018-09-07 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16607829#comment-16607829 ] Tao Yang commented on YARN-8709: Thanks [~eepayne] for the review! There are several similar problems in

[jira] [Updated] (YARN-8729) Node status updater thread could be lost after it restarted

2018-09-03 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8729: --- Attachment: YARN-8729.001.patch > Node status updater thread could be lost after it restarted >

[jira] [Commented] (YARN-8729) Node status updater thread could be lost after it restarted

2018-09-03 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601938#comment-16601938 ] Tao Yang commented on YARN-8729: Thanks [~ebadger], [~cheersyang] for your reply. {quote} Setting

[jira] [Updated] (YARN-8737) Race condition in ParentQueue when reinitializing and sorting child queues in the meanwhile

2018-08-30 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8737: --- Attachment: YARN-8737.001.patch > Race condition in ParentQueue when reinitializing and sorting child queues

[jira] [Created] (YARN-8737) Race condition in ParentQueue when reinitializing and sorting child queues in the meanwhile

2018-08-30 Thread Tao Yang (JIRA)
Tao Yang created YARN-8737: -- Summary: Race condition in ParentQueue when reinitializing and sorting child queues in the meanwhile Key: YARN-8737 URL: https://issues.apache.org/jira/browse/YARN-8737 Project:

[jira] [Commented] (YARN-8728) Wrong available resource in AllocateResponse when allocating containers to different partitions in the same queue

2018-08-30 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598114#comment-16598114 ] Tao Yang commented on YARN-8728: Thanks [~cheersyang] for your mention! Forgot to upload the patch and

[jira] [Updated] (YARN-8728) Wrong available resource in AllocateResponse when allocating containers to different partitions in the same queue

2018-08-30 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8728: --- Attachment: YARN-8728.001.patch > Wrong available resource in AllocateResponse when allocating containers to

[jira] [Updated] (YARN-8729) Node status updater thread could be lost after it restarted

2018-08-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8729: --- Attachment: YARN-8729.001.patch > Node status updater thread could be lost after it restarted >

[jira] [Commented] (YARN-8729) Node status updater thread could be lost after it restarted

2018-08-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596235#comment-16596235 ] Tao Yang commented on YARN-8729: Attached v1 patch for review. > Node status updater thread could be lost

[jira] [Created] (YARN-8729) Node status updater thread could be lost after it restarted

2018-08-29 Thread Tao Yang (JIRA)
Tao Yang created YARN-8729: -- Summary: Node status updater thread could be lost after it restarted Key: YARN-8729 URL: https://issues.apache.org/jira/browse/YARN-8729 Project: Hadoop YARN Issue

[jira] [Commented] (YARN-8709) intra-queue preemption checker always fail since one under-served queue was deleted

2018-08-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596219#comment-16596219 ] Tao Yang commented on YARN-8709: Attached v1 patch for review. [~eepayne], [~sunilg], please help to

[jira] [Updated] (YARN-8728) Wrong available resource in AllocateResponse when allocating containers to different partitions in the same queue

2018-08-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8728: --- Attachment: (was: YARN-8728.001.patch) > Wrong available resource in AllocateResponse when allocating

[jira] [Updated] (YARN-8709) intra-queue preemption checker always fail since one under-served queue was deleted

2018-08-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8709: --- Attachment: YARN-8709.001.patch > intra-queue preemption checker always fail since one under-served queue was

[jira] [Commented] (YARN-8728) Wrong available resource in AllocateResponse when allocating containers to different partitions in the same queue

2018-08-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596198#comment-16596198 ] Tao Yang commented on YARN-8728: Attached v1 patch for review. > Wrong available resource in

<    1   2   3   4   5   6   7   8   9   10   >