[jira] [Commented] (YARN-10458) Hive On Tez queries fails upon submission to dynamically created pools

2020-10-27 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221616#comment-17221616 ] Wangda Tan commented on YARN-10458: --- It looks good to me, the only question/ask is since this issue

[jira] [Commented] (YARN-10458) Hive On Tez queries fails upon submission to dynamically created pools

2020-10-26 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220808#comment-17220808 ] Wangda Tan commented on YARN-10458: --- [~pbacsko],  Thanks for working on the patch,  I'm trying to

[jira] [Comment Edited] (YARN-10178) Global Scheduler asycthread crash caused by 'Comparison method violates its general contract'

2020-10-19 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217285#comment-17217285 ] Wangda Tan edited comment on YARN-10178 at 10/20/20, 5:33 AM: -- Since

[jira] [Commented] (YARN-10178) Global Scheduler asycthread crash caused by 'Comparison method violates its general contract'

2020-10-19 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217285#comment-17217285 ] Wangda Tan commented on YARN-10178: --- Since recently we have a customer has the same issue, I spent some

[jira] [Commented] (YARN-8737) Race condition in ParentQueue when reinitializing and sorting child queues in the meanwhile

2020-10-19 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217279#comment-17217279 ] Wangda Tan commented on YARN-8737: -- Rekicked Jenkins, after reviewed the case, the fix looks good to me,

[jira] [Commented] (YARN-8737) Race condition in ParentQueue when reinitializing and sorting child queues in the meanwhile

2020-09-28 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203545#comment-17203545 ] Wangda Tan commented on YARN-8737: -- cc: [~snemeth], [~bteke] to help with patch reviews, test, and

[jira] [Commented] (YARN-8737) Race condition in ParentQueue when reinitializing and sorting child queues in the meanwhile

2020-09-28 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203544#comment-17203544 ] Wangda Tan commented on YARN-8737: -- [~Tao Yang], missed this ticket, we recently got a customer report

[jira] [Commented] (YARN-4971) RM fails to re-bind to wildcard IP after failover in multi homed clusters

2020-09-11 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194378#comment-17194378 ] Wangda Tan commented on YARN-4971: -- I think we should revisit the patch based on comment from Karthik: 

[jira] [Commented] (YARN-10380) Import logic of multi-node allocation in CapacityScheduler

2020-07-30 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168116#comment-17168116 ] Wangda Tan commented on YARN-10380: --- cc: [~prabhujoseph], I think we identified more issues during a

[jira] [Created] (YARN-10380) Import logic of multi-node allocation in CapacityScheduler

2020-07-30 Thread Wangda Tan (Jira)
Wangda Tan created YARN-10380: - Summary: Import logic of multi-node allocation in CapacityScheduler Key: YARN-10380 URL: https://issues.apache.org/jira/browse/YARN-10380 Project: Hadoop YARN

[jira] [Commented] (YARN-10352) Skip schedule on not heartbeated nodes in Multi Node Placement

2020-07-21 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17162139#comment-17162139 ] Wangda Tan commented on YARN-10352: --- Hi [~prabhujoseph], thanks for the update, unRegisterNM discussion

[jira] [Commented] (YARN-10352) Skip schedule on not heartbeated nodes in Multi Node Placement

2020-07-20 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17161699#comment-17161699 ] Wangda Tan commented on YARN-10352: --- Also, we need to systematically handle the node heartbeat interval

[jira] [Commented] (YARN-10352) Skip schedule on not heartbeated nodes in Multi Node Placement

2020-07-20 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17161696#comment-17161696 ] Wangda Tan commented on YARN-10352: --- Thanks [~prabhujoseph],  Then it makes sense, but the original

[jira] [Commented] (YARN-10352) Skip schedule on not heartbeated nodes in Multi Node Placement

2020-07-20 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17161615#comment-17161615 ] Wangda Tan commented on YARN-10352: --- [~prabhujoseph], I'm trying to understand this logic, why we have

[jira] [Commented] (YARN-10293) Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement (YARN-10259)

2020-06-12 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134338#comment-17134338 ] Wangda Tan commented on YARN-10293: --- Missed last comments, thanks [~prabhujoseph]/[~Tao Yang]!  >

[jira] [Commented] (YARN-9930) Support max running app logic for CapacityScheduler

2020-06-10 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132598#comment-17132598 ] Wangda Tan commented on YARN-9930: -- Thanks [~pbacsko], it will be make sense to create a one-pager doc

[jira] [Commented] (YARN-10293) Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement (YARN-10259)

2020-06-02 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124195#comment-17124195 ] Wangda Tan commented on YARN-10293: --- [~Tao Yang], the suggestion totally make sense to me. When we have

[jira] [Commented] (YARN-10296) Make ContainerPBImpl#getId/setId synchronized

2020-06-02 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124121#comment-17124121 ] Wangda Tan commented on YARN-10296: --- [~bteke],  It makes sense to covert other method which uses

[jira] [Commented] (YARN-10293) Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement (YARN-10259)

2020-06-01 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121419#comment-17121419 ] Wangda Tan commented on YARN-10293: --- [~prabhujoseph], I agree with you, I think the entire {{if}} check

[jira] [Commented] (YARN-10293) Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement (YARN-10259)

2020-05-29 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119866#comment-17119866 ] Wangda Tan commented on YARN-10293: --- [~prabhujoseph],   This looks like a valid bug, but I'm wondering

[jira] [Commented] (YARN-10259) Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement

2020-05-29 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119822#comment-17119822 ] Wangda Tan commented on YARN-10259: --- Thanks [~prabhujoseph], I think we should also put this to 3.3.1,

[jira] [Commented] (YARN-10259) Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement

2020-05-12 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105675#comment-17105675 ] Wangda Tan commented on YARN-10259: --- Reviewed the patch, it looks good to me, I think it may introduce

[jira] [Commented] (YARN-10154) CS Dynamic Queues cannot be configured with absolute resources

2020-04-16 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085078#comment-17085078 ] Wangda Tan commented on YARN-10154: --- [~maniraj...@gmail.com], there's an ASF license issue. [~sunilg],

[jira] [Commented] (YARN-10154) CS Dynamic Queues cannot be configured with absolute resources

2020-04-15 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17084389#comment-17084389 ] Wangda Tan commented on YARN-10154: --- [~maniraj...@gmail.com], thank you so much for the patch! It looks

[jira] [Commented] (YARN-10154) CS Dynamic Queues cannot be configured with absolute resources

2020-04-15 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17084390#comment-17084390 ] Wangda Tan commented on YARN-10154: --- cc: [~prabhujoseph] > CS Dynamic Queues cannot be configured

[jira] [Resolved] (YARN-10151) Disable Capacity Scheduler's move app between queue functionality

2020-04-06 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved YARN-10151. --- Resolution: Won't Fix Thanks folks for commenting about YARN-9838. I think we don't need this change

[jira] [Commented] (YARN-10219) YARN service placement constraints is broken

2020-04-03 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17074778#comment-17074778 ] Wangda Tan commented on YARN-10219: --- Thanks [~eyang] for creating the JIRA and upload fixes. cc:

[jira] [Commented] (YARN-10154) CS Dynamic Queues cannot be configured with absolute resources

2020-03-27 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069082#comment-17069082 ] Wangda Tan commented on YARN-10154: --- I thought I submitted the review comments:

[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS

2020-03-14 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059529#comment-17059529 ] Wangda Tan commented on YARN-9879: -- Thanks [~shuzirra] for the update. I only checked the updates of

[jira] [Commented] (YARN-10192) CapacityScheduler stuck in loop rejecting allocation proposals

2020-03-11 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057277#comment-17057277 ] Wangda Tan commented on YARN-10192: --- [~Tao Yang] did you remember to see this issue before? >

[jira] [Comment Edited] (YARN-9879) Allow multiple leaf queues with the same name in CS

2020-03-11 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057274#comment-17057274 ] Wangda Tan edited comment on YARN-9879 at 3/11/20, 5:55 PM: Thanks [~shuzirra] 

[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS

2020-03-11 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057274#comment-17057274 ] Wangda Tan commented on YARN-9879: -- Thanks [~shuzirra] for uploading another monster patch!  I didn't

[jira] [Commented] (YARN-10168) FS-CS Converter: tool doesn't handle min/max resource conversion correctly

2020-03-09 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055209#comment-17055209 ] Wangda Tan commented on YARN-10168: --- [~pbacsko], suggested change make sense to me. > FS-CS Converter:

[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS

2020-03-05 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17052616#comment-17052616 ] Wangda Tan commented on YARN-9879: -- Thanks [~shuzirra] for the monster patch!  Took a quick look at the

[jira] [Commented] (YARN-10180) TimelineV2ClientImpl$TimelineEntityDispatcher threads leak

2020-03-04 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17051499#comment-17051499 ] Wangda Tan commented on YARN-10180: --- Thanks [~prabhujoseph] for filing this!  I think we should think

[jira] [Commented] (YARN-10178) Global Scheduler asycthread crash caused by 'Comparison method violates its general contract'

2020-03-02 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049947#comment-17049947 ] Wangda Tan commented on YARN-10178: --- [~tuyu], can you add more details like error message, thread

[jira] [Commented] (YARN-10167) FS-CS Converter: Need validate c-s.xml after converting

2020-02-27 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046749#comment-17046749 ] Wangda Tan commented on YARN-10167: --- [~pbacsko], agree with this:  {quote}Note that the converter

[jira] [Commented] (YARN-10168) FS-CS Convert: Converter tool doesn't handle min/max resource conversion correct

2020-02-27 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046736#comment-17046736 ] Wangda Tan commented on YARN-10168: --- [~pbacsko], what you mentioned are all make sense to me. I think

[jira] [Commented] (YARN-10167) FS-CS Converter: Need validate c-s.xml after converting

2020-02-26 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046233#comment-17046233 ] Wangda Tan commented on YARN-10167: --- Kinga: we may not be able to use that since we can not assume

[jira] [Commented] (YARN-10170) Should revisit mix-usage of percentage-based and absolute-value-based min/max resource in CapacityScheduler

2020-02-26 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046076#comment-17046076 ] Wangda Tan commented on YARN-10170: --- cc: [~sunil.gov...@gmail.com] > Should revisit mix-usage of

[jira] [Created] (YARN-10170) Should revisit mix-usage of percentage-based and absolute-value-based min/max resource in CapacityScheduler

2020-02-26 Thread Wangda Tan (Jira)
Wangda Tan created YARN-10170: - Summary: Should revisit mix-usage of percentage-based and absolute-value-based min/max resource in CapacityScheduler Key: YARN-10170 URL:

[jira] [Commented] (YARN-10169) Mixed absolute resource value and percentage-based resource value in CapacityScheduler should fail

2020-02-26 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046017#comment-17046017 ] Wangda Tan commented on YARN-10169: --- cc: [~sunil.gov...@gmail.com] > Mixed absolute resource value and

[jira] [Created] (YARN-10169) Mixed absolute resource value and percentage-based resource value in CapacityScheduler should fail

2020-02-26 Thread Wangda Tan (Jira)
Wangda Tan created YARN-10169: - Summary: Mixed absolute resource value and percentage-based resource value in CapacityScheduler should fail Key: YARN-10169 URL: https://issues.apache.org/jira/browse/YARN-10169

[jira] [Created] (YARN-10168) FS-CS Convert: Converter tool doesn't handle min/max resource conversion correct

2020-02-26 Thread Wangda Tan (Jira)
Wangda Tan created YARN-10168: - Summary: FS-CS Convert: Converter tool doesn't handle min/max resource conversion correct Key: YARN-10168 URL: https://issues.apache.org/jira/browse/YARN-10168 Project:

[jira] [Updated] (YARN-10167) FS-CS Converter: Need validate c-s.xml after converting

2020-02-26 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-10167: -- Summary: FS-CS Converter: Need validate c-s.xml after converting (was: Need validate c-s.xml after

[jira] [Created] (YARN-10167) Need validate c-s.xml after converting

2020-02-26 Thread Wangda Tan (Jira)
Wangda Tan created YARN-10167: - Summary: Need validate c-s.xml after converting Key: YARN-10167 URL: https://issues.apache.org/jira/browse/YARN-10167 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Commented] (YARN-10151) Disable Capacity Scheduler's move app between queue functionality

2020-02-18 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039495#comment-17039495 ] Wangda Tan commented on YARN-10151: --- And this should apply to all branches. > Disable Capacity

[jira] [Created] (YARN-10151) Disable Capacity Scheduler's move app between queue functionality

2020-02-18 Thread Wangda Tan (Jira)
Wangda Tan created YARN-10151: - Summary: Disable Capacity Scheduler's move app between queue functionality Key: YARN-10151 URL: https://issues.apache.org/jira/browse/YARN-10151 Project: Hadoop YARN

[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS

2020-01-22 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021651#comment-17021651 ] Wangda Tan commented on YARN-9879: -- Thanks [~shuzirra], [~wilfreds] for sharing your thoughts! 1)

[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS

2020-01-21 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020522#comment-17020522 ] Wangda Tan commented on YARN-9879: -- [~shuzirra], I think we should not change semantics of GetQueueName

[jira] [Commented] (YARN-10049) FIFOOrderingPolicy Improvements

2020-01-17 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018193#comment-17018193 ] Wangda Tan commented on YARN-10049: --- [~sunilg], I agree that priority > FIFO, for both fair/fifo

[jira] [Commented] (YARN-10085) FS-CS converter: remove mixed ordering policy check

2020-01-16 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017331#comment-17017331 ] Wangda Tan commented on YARN-10085: --- [~pbacsko], I also posted a comment on YARN-10043, to me it is

[jira] [Commented] (YARN-10043) FairOrderingPolicy Improvements

2020-01-16 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017328#comment-17017328 ] Wangda Tan commented on YARN-10043: --- Thanks [~maniraj...@gmail.com]  for posting thoughts on this. In

[jira] [Commented] (YARN-9892) Capacity scheduler: support DRF ordering policy on queue level

2020-01-15 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016160#comment-17016160 ] Wangda Tan commented on YARN-9892: -- Spent a bit time to look at the code, not fully dig into all details,

[jira] [Commented] (YARN-9892) Capacity scheduler: support DRF ordering policy on queue level

2020-01-15 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016151#comment-17016151 ] Wangda Tan commented on YARN-9892: -- [~pbacsko], My concern is adding DRF only in application ordering

[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS

2020-01-15 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016121#comment-17016121 ] Wangda Tan commented on YARN-9879: -- [~wilfreds], I agree with, {quote}The behaviour inside the scheduler

[jira] [Commented] (YARN-9892) Capacity scheduler: support DRF ordering policy on queue level

2020-01-15 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016112#comment-17016112 ] Wangda Tan commented on YARN-9892: -- [~pbacsko], [~maniraj...@gmail.com] , thanks for working on this.

[jira] [Comment Edited] (YARN-9879) Allow multiple leaf queues with the same name in CS

2020-01-14 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015404#comment-17015404 ] Wangda Tan edited comment on YARN-9879 at 1/14/20 10:01 PM: [~snemeth], most

[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS

2020-01-14 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015404#comment-17015404 ] Wangda Tan commented on YARN-9879: -- [~snemeth], most of the explanation looks reasonable to me. Regarding

[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS

2020-01-09 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012282#comment-17012282 ] Wangda Tan commented on YARN-9879: -- Thanks [~shuzirra], I think adding a flag (suggestion from

[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS

2020-01-07 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009975#comment-17009975 ] Wangda Tan commented on YARN-9879: -- [~pbacsko], thanks for working on the design. In general, I agree

[jira] [Commented] (YARN-10009) In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined

2019-12-06 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990169#comment-16990169 ] Wangda Tan commented on YARN-10009: --- [~epayne], is the failure related? Thanks > In Capacity

[jira] [Commented] (YARN-10009) In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined

2019-12-04 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16988283#comment-16988283 ] Wangda Tan commented on YARN-10009: --- +1 from my side, except one comment:

[jira] [Updated] (YARN-10009) In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined

2019-12-04 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-10009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-10009: -- Priority: Critical (was: Major) > In Capacity Scheduler, DRC can treat minimum user limit percent as

[jira] [Commented] (YARN-8373) RM Received RMFatalEvent of type CRITICAL_THREAD_CRASH

2019-11-20 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-8373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16978955#comment-16978955 ] Wangda Tan commented on YARN-8373: -- Thanks [~wilfreds]  for the patch and everybody for the review!

[jira] [Commented] (YARN-9927) RM multi-thread event processing mechanism

2019-10-22 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16957378#comment-16957378 ] Wangda Tan commented on YARN-9927: -- Thanks [~hcarrot] for working on this. Tagging: [~prabhujoseph] ,

[jira] [Commented] (YARN-9887) Capacity scheduler: add support for limiting maxRunningApps per user

2019-10-21 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956297#comment-16956297 ] Wangda Tan commented on YARN-9887: -- [~pbacsko], [~epayne], IIRC the max app per user in FS is across

[jira] [Commented] (YARN-9886) Queue mapping based on userid passed through application tag

2019-10-21 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956284#comment-16956284 ] Wangda Tan commented on YARN-9886: -- [~kmarton], can we also make sure apps from privileged users can do

[jira] [Issue Comment Deleted] (YARN-9889) [UI] Add Application Tag column to RM All Applications table

2019-10-15 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-9889: - Comment: was deleted (was: Thanks [~kmarton] , thanks for working on this. To me there's no strong

[jira] [Commented] (YARN-9886) Queue mapping based on userid passed through application tag

2019-10-15 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952267#comment-16952267 ] Wangda Tan commented on YARN-9886: -- [~kmarton] , thanks for working on this. To me there's no strong

[jira] [Commented] (YARN-9889) [UI] Add Application Tag column to RM All Applications table

2019-10-15 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952265#comment-16952265 ] Wangda Tan commented on YARN-9889: -- Thanks [~kmarton] , thanks for working on this. To me there's no

[jira] [Commented] (YARN-9656) Plugin to avoid scheduling jobs on node which are not in "schedulable" state, but are healthy otherwise.

2019-10-14 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951399#comment-16951399 ] Wangda Tan commented on YARN-9656: -- [~pgolash], [~mayank_bansal], to me if a node cannot schedule new

[jira] [Deleted] (YARN-9878) the

2019-10-08 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan deleted YARN-9878: - > the > --- > > Key: YARN-9878 > URL:

[jira] [Commented] (YARN-9656) Plugin to avoid scheduling jobs on node which are not in "schedulable" state, but are healthy otherwise.

2019-09-29 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16940508#comment-16940508 ] Wangda Tan commented on YARN-9656: -- [~pgolash], how about just define these nodes in unhealthy state? The

[jira] [Commented] (YARN-4946) RM should not consider an application as COMPLETED when log aggregation is not in a terminal state

2019-09-20 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934797#comment-16934797 ] Wangda Tan commented on YARN-4946: -- I would still prefer to revert the patch. But due to my bandwidth, I

[jira] [Commented] (YARN-9813) RM does not start on JDK11 when UIv2 is enabled

2019-09-06 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924600#comment-16924600 ] Wangda Tan commented on YARN-9813: -- Thanks [~eyang] for updating the patch.  +1, pending Jenkins. > RM

[jira] [Commented] (YARN-9698) [Umbrella] Tools to help migration from Fair Scheduler to Capacity Scheduler

2019-09-05 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923763#comment-16923763 ] Wangda Tan commented on YARN-9698: -- Thanks [~shuzirra] , [~Prabhu Joseph] ,[~snemeth] , [~wilfreds] ,

[jira] [Commented] (YARN-9795) ClusterMetrics to include AM allocation delay

2019-09-04 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922915#comment-16922915 ] Wangda Tan commented on YARN-9795: -- [~fengnanli], thanks for working on the Jira. I just added you to

[jira] [Assigned] (YARN-9795) ClusterMetrics to include AM allocation delay

2019-09-04 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan reassigned YARN-9795: Assignee: Fengnan Li > ClusterMetrics to include AM allocation delay >

[jira] [Commented] (YARN-9785) Fix DominantResourceCalculator when one resource is zero

2019-09-02 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921081#comment-16921081 ] Wangda Tan commented on YARN-9785: -- +1 to the latest patch. [~sunilg] do you want to take another look?

[jira] [Commented] (YARN-9785) Fix DominantResourceCalculator when one resource is zero

2019-08-30 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919264#comment-16919264 ] Wangda Tan commented on YARN-9785: -- Can we add tests to make sure no regression after this patch? And

[jira] [Commented] (YARN-9785) Fix DominantResourceCalculator when one resource is zero

2019-08-29 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918357#comment-16918357 ] Wangda Tan commented on YARN-9785: -- Thanks [~BilwaST] for the patch and everybody for discussing.  I'm

[jira] [Commented] (YARN-9770) Create a queue ordering policy which picks child queues with equal probability

2019-08-28 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917528#comment-16917528 ] Wangda Tan commented on YARN-9770: -- [~jhung] , I understand the use case, however I think this will break

[jira] [Updated] (YARN-8657) User limit calculation should be read-lock-protected within LeafQueue

2019-08-26 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8657: - Priority: Major (was: Critical) > User limit calculation should be read-lock-protected within LeafQueue

[jira] [Commented] (YARN-8657) User limit calculation should be read-lock-protected within LeafQueue

2019-08-26 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915509#comment-16915509 ] Wangda Tan commented on YARN-8657: -- I'd prefer to move it to next releases and downgrade the priority.

[jira] [Updated] (YARN-8657) User limit calculation should be read-lock-protected within LeafQueue

2019-08-26 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8657: - Target Version/s: 3.2.2, 3.1.4 (was: 3.2.1, 3.1.3) > User limit calculation should be

[jira] [Commented] (YARN-9751) Separate queue and app ordering policy capacity scheduler configs

2019-08-20 Thread Wangda Tan (Jira)
[ https://issues.apache.org/jira/browse/YARN-9751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911891#comment-16911891 ] Wangda Tan commented on YARN-9751: -- Thanks [~jhung] for the patch. Are there any changes of behavior

[jira] [Updated] (YARN-9698) [Umbrella] Tools to help migration from Fair Scheduler to Capacity Scheduler

2019-08-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-9698: - Target Version/s: 3.3.0 > [Umbrella] Tools to help migration from Fair Scheduler to Capacity Scheduler >

[jira] [Commented] (YARN-9195) RM Queue's pending container number might get decreased unexpectedly or even become negative once RM failover

2019-02-11 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765169#comment-16765169 ] Wangda Tan commented on YARN-9195: -- Thanks [~ssy], [~sunilg], [~cheersyang] if you have bandwidth, could

[jira] [Commented] (YARN-8761) Service AM support for decommissioning component instances

2019-02-08 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763837#comment-16763837 ] Wangda Tan commented on YARN-8761: -- +1 to back port to branch-3.1, branch-3.2, thanks [~billie.rinaldi],

[jira] [Commented] (YARN-9209) When nodePartition is not set in Placement Constraints, containers are allocated only in default partition

2019-01-28 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754249#comment-16754249 ] Wangda Tan commented on YARN-9209: -- [~tarunparimi], [~cheersyang], Actually, this is by design.

[jira] [Commented] (YARN-9195) RM Queue's pending container number might get decreased unexpectedly or even become negative once RM failover

2019-01-24 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751887#comment-16751887 ] Wangda Tan commented on YARN-9195: -- Thanks [~ssy],   Could u rename the patch to YARN-9175.001.patch?

[jira] [Commented] (YARN-9195) RM Queue's pending container number might get decreased unexpectedly or even become negative once RM failover

2019-01-24 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751889#comment-16751889 ] Wangda Tan commented on YARN-9195: -- [~ssy] add you to contributor list so you can assign Jira to yourself

[jira] [Assigned] (YARN-9195) RM Queue's pending container number might get decreased unexpectedly or even become negative once RM failover

2019-01-24 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan reassigned YARN-9195: Assignee: Shengyang Sha > RM Queue's pending container number might get decreased unexpectedly or

[jira] [Commented] (YARN-9204) RM fails to start if absolute resource is specified for partition capacity in CS queues

2019-01-21 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748110#comment-16748110 ] Wangda Tan commented on YARN-9204: -- Cherry-picked to branch-3.1.2 as well, thanks [~yangjiandan]/

[jira] [Updated] (YARN-8747) [UI2] YARN UI2 page loading failed due to js error under some time zone configuration

2019-01-21 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8747: - Fix Version/s: (was: 3.1.3) 3.1.2 > [UI2] YARN UI2 page loading failed due to js

[jira] [Updated] (YARN-9204) RM fails to start if absolute resource is specified for partition capacity in CS queues

2019-01-21 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-9204: - Fix Version/s: (was: 3.1.3) 3.1.2 > RM fails to start if absolute resource is

[jira] [Commented] (YARN-9194) Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM

2019-01-21 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748109#comment-16748109 ] Wangda Tan commented on YARN-9194: -- Cherry-picked to branch-3.1.2 as well. > Invalid event: REGISTERED

[jira] [Updated] (YARN-9194) Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM

2019-01-21 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-9194: - Fix Version/s: (was: 3.1.3) 3.1.2 > Invalid event: REGISTERED and LAUNCH_FAILED at

[jira] [Commented] (YARN-8747) [UI2] YARN UI2 page loading failed due to js error under some time zone configuration

2019-01-21 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748108#comment-16748108 ] Wangda Tan commented on YARN-8747: -- Cherry-picked to branch-3.1.2 as well. Updated fix version > [UI2]

<    1   2   3   4   5   6   7   8   9   10   >