[jira] [Updated] (YARN-7619) Max AM Resource value in CS UI is different for every user

2017-12-12 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7619: - Attachment: YARN-7619.001.patch Uploading patch 001. This is not a perfect solution, but it's close. The

[jira] [Commented] (YARN-7619) Max AM Resource value in CS UI is different for every user

2017-12-07 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16282707#comment-16282707 ] Eric Payne commented on YARN-7619: -- All of the other solutions I could think of seem undesirable. One

[jira] [Updated] (YARN-7619) Max AM Resource value in CS UI is different for every user

2017-12-06 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7619: - Attachment: Max AM Resources is Different for Each User.png > Max AM Resource value in CS UI is different

[jira] [Created] (YARN-7619) Max AM Resource value in CS UI is different for every user

2017-12-06 Thread Eric Payne (JIRA)
Eric Payne created YARN-7619: Summary: Max AM Resource value in CS UI is different for every user Key: YARN-7619 URL: https://issues.apache.org/jira/browse/YARN-7619 Project: Hadoop YARN Issue

[jira] [Commented] (YARN-6124) Make SchedulingEditPolicy can be enabled / disabled / updated with RMAdmin -refreshQueues

2017-12-03 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276200#comment-16276200 ] Eric Payne commented on YARN-6124: -- Sorry for the delay. Belatedly, I am fine with the patch. > Make

[jira] [Commented] (YARN-7575) When using absolute capacity configuration with no max capacity, scheduler UI NPEs and can't grow queue

2017-11-28 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269455#comment-16269455 ] Eric Payne commented on YARN-7575: -- Sorry, my bad. My ULF is set to 2.0 on the default queue. After

[jira] [Commented] (YARN-7575) When using absolute capacity configuration with no max capacity, scheduler UI NPEs and can't grow queue

2017-11-28 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269175#comment-16269175 ] Eric Payne commented on YARN-7575: -- [~sunilg], the fix for the UI NPE looks good, but the other problem

[jira] [Updated] (YARN-7575) When using absolute capacity configuration with no max capacity, scheduler UI NPEs and can't grow queue

2017-11-28 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7575: - Issue Type: Sub-task (was: Bug) Parent: YARN-5881 > When using absolute capacity configuration

[jira] [Created] (YARN-7575) When using absolute capacity configuration with no max capacity, scheduler UI NPEs and can't grow queue

2017-11-28 Thread Eric Payne (JIRA)
Eric Payne created YARN-7575: Summary: When using absolute capacity configuration with no max capacity, scheduler UI NPEs and can't grow queue Key: YARN-7575 URL: https://issues.apache.org/jira/browse/YARN-7575

[jira] [Commented] (YARN-7496) CS Intra-queue preemption user-limit calculations are not in line with LeafQueue user-limit calculations

2017-11-25 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16265837#comment-16265837 ] Eric Payne commented on YARN-7496: -- Thank you very much, [~leftnoteasy] > CS Intra-queue preemption

[jira] [Commented] (YARN-7496) CS Intra-queue preemption user-limit calculations are not in line with LeafQueue user-limit calculations

2017-11-21 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261617#comment-16261617 ] Eric Payne commented on YARN-7496: -- {code} hadoop.yarn.server.resourcemanager.TestClientRMTokens

[jira] [Commented] (YARN-7533) Documentation for absolute resource support in CS

2017-11-21 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261565#comment-16261565 ] Eric Payne commented on YARN-7533: -- [~sunilg] this looks good to me. > Documentation for absolute

[jira] [Commented] (YARN-7496) CS Intra-queue preemption user-limit calculations are not in line with LeafQueue user-limit calculations

2017-11-21 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261242#comment-16261242 ] Eric Payne commented on YARN-7496: -- Thanks for your review, [~leftnoteasy]. What's the next step? > CS

[jira] [Commented] (YARN-7533) Documentation for absolute resource support in CS

2017-11-20 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260064#comment-16260064 ] Eric Payne commented on YARN-7533: -- Hi [~sunilg]. Thanks for the patch. I have just a couple of

[jira] [Commented] (YARN-6124) Make SchedulingEditPolicy can be enabled / disabled / updated with RMAdmin -refreshQueues

2017-11-20 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259865#comment-16259865 ] Eric Payne commented on YARN-6124: -- bq. AdminService#refreshQueues, conf.size(): I'm not sure why this is

[jira] [Commented] (YARN-6124) Make SchedulingEditPolicy can be enabled / disabled / updated with RMAdmin -refreshQueues

2017-11-20 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259540#comment-16259540 ] Eric Payne commented on YARN-6124: -- Thanks [~Zian Chen]. I appreciate the good work here. sorry for the

[jira] [Commented] (YARN-7496) CS Intra-queue preemption user-limit calculations are not in line with LeafQueue user-limit calculations

2017-11-20 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259412#comment-16259412 ] Eric Payne commented on YARN-7496: -- Thanks for looking at this [~leftnoteasy] > CS Intra-queue preemption

[jira] [Updated] (YARN-7496) CS Intra-queue preemption user-limit calculations are not in line with LeafQueue user-limit calculations

2017-11-16 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7496: - Attachment: YARN-7496.001.branch-2.8.patch Attaching a fix for branch-2.8. This change is in

[jira] [Commented] (YARN-7469) Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit

2017-11-16 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255891#comment-16255891 ] Eric Payne commented on YARN-7469: -- Thank you very much [~sunilg]. Just one concern is that this fix

[jira] [Commented] (YARN-7469) Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit

2017-11-15 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253828#comment-16253828 ] Eric Payne commented on YARN-7469: -- Thanks [~sunilg]! That would be great! > Capacity Scheduler

[jira] [Commented] (YARN-7469) Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit

2017-11-15 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253721#comment-16253721 ] Eric Payne commented on YARN-7469: -- bq. now min container is the dead zone here I filed YARN-7501 to

[jira] [Created] (YARN-7501) Capacity Scheduler Intra-queue preemption should have a "dead zone" around user limit

2017-11-15 Thread Eric Payne (JIRA)
Eric Payne created YARN-7501: Summary: Capacity Scheduler Intra-queue preemption should have a "dead zone" around user limit Key: YARN-7501 URL: https://issues.apache.org/jira/browse/YARN-7501 Project:

[jira] [Commented] (YARN-7496) CS Intra-queue preemption user-limit calculations are not in line with LeafQueue user-limit calculations

2017-11-14 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252643#comment-16252643 ] Eric Payne commented on YARN-7496: -- Cluster Configuration: - Cluster Memory: 20GB - Queue1 capacity and

[jira] [Created] (YARN-7496) CS Intra-queue preemption user-limit calculations are not in line with LeafQueue user-limit calculations

2017-11-14 Thread Eric Payne (JIRA)
Eric Payne created YARN-7496: Summary: CS Intra-queue preemption user-limit calculations are not in line with LeafQueue user-limit calculations Key: YARN-7496 URL: https://issues.apache.org/jira/browse/YARN-7496

[jira] [Commented] (YARN-7469) Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit

2017-11-13 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16250237#comment-16250237 ] Eric Payne commented on YARN-7469: -- bq. In broader perspective, i think we are lacking dead zone here. In

[jira] [Updated] (YARN-7469) Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit

2017-11-13 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7469: - Attachment: YARN-7469.001.patch Attaching a proposal for a patch to fix this problem. Proposed fix: In

[jira] [Commented] (YARN-7469) Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit

2017-11-10 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247790#comment-16247790 ] Eric Payne commented on YARN-7469: -- When a queue is in the state as described above,

[jira] [Updated] (YARN-7469) Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit

2017-11-09 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7469: - Attachment: UnitTestToShowStarvedUser.patch Uploading a unit test that demonstrates this. > Capacity

[jira] [Updated] (YARN-7469) Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit

2017-11-09 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7469: - Description: Queue Configuration: - Total Memory: 20GB - 2 Queues -- Queue1 --- Memory: 10GB --- MULP: 10%

[jira] [Created] (YARN-7469) Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit

2017-11-09 Thread Eric Payne (JIRA)
Eric Payne created YARN-7469: Summary: Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit Key: YARN-7469 URL: https://issues.apache.org/jira/browse/YARN-7469

[jira] [Commented] (YARN-7424) Capacity Scheduler Intra-queue preemption: add property to only preempt up to configured MULP

2017-11-03 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16237768#comment-16237768 ] Eric Payne commented on YARN-7424: -- Thanks for the review, [~sunilg]. bq.

[jira] [Commented] (YARN-7370) Preemption properties should be refreshable

2017-11-02 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16236151#comment-16236151 ] Eric Payne commented on YARN-7370: -- Findbugs warnings are for

[jira] [Commented] (YARN-7370) Preemption properties should be refreshable

2017-11-02 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16236089#comment-16236089 ] Eric Payne commented on YARN-7370: -- Waiting for precommit to complete:

[jira] [Commented] (YARN-7370) Preemption properties should be refreshable

2017-11-02 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235862#comment-16235862 ] Eric Payne commented on YARN-7370: -- [~sunilg], thanks for your comment. I see what you mean. If we take

[jira] [Commented] (YARN-7370) Preemption properties should be refreshable

2017-11-02 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235817#comment-16235817 ] Eric Payne commented on YARN-7370: -- [~GergelyNovak], Thanks for your effort on this feature. Patch LGTM +1

[jira] [Commented] (YARN-7370) Preemption properties should be refreshable

2017-11-02 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235747#comment-16235747 ] Eric Payne commented on YARN-7370: -- [~subru] and [~asuresh], I am aware that you have requested that

[jira] [Commented] (YARN-7370) Preemption properties should be refreshable

2017-11-02 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235740#comment-16235740 ] Eric Payne commented on YARN-7370: -- Thanks for the patch, [~GergelyNovak]. I will review today. bq. The

[jira] [Assigned] (YARN-7424) Capacity Scheduler Intra-queue preemption: add property to only preempt up to configured MULP

2017-10-31 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne reassigned YARN-7424: Assignee: Eric Payne > Capacity Scheduler Intra-queue preemption: add property to only preempt up

[jira] [Commented] (YARN-7424) Capacity Scheduler Intra-queue preemption: add property to only preempt up to configured MULP

2017-10-31 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227528#comment-16227528 ] Eric Payne commented on YARN-7424: -- In a large, multi-tenant queue with MULP of 1%, after instrumenting

[jira] [Updated] (YARN-7424) Capacity Scheduler Intra-queue preemption: add property to only preempt up to configured MULP

2017-10-31 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7424: - Description: If the queue's configured minimum user limit percent (MULP) is something small like 1%, all

[jira] [Created] (YARN-7424) Capacity Scheduler Intra-queue preemption: add property to only preempt up to configured MULP

2017-10-31 Thread Eric Payne (JIRA)
Eric Payne created YARN-7424: Summary: Capacity Scheduler Intra-queue preemption: add property to only preempt up to configured MULP Key: YARN-7424 URL: https://issues.apache.org/jira/browse/YARN-7424

[jira] [Commented] (YARN-7370) Preemption properties should be refreshable

2017-10-31 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227276#comment-16227276 ] Eric Payne commented on YARN-7370: -- [~GergelyNovak], Thanks for the updated patch. Just a couple of

[jira] [Commented] (YARN-7370) Preemption properties should be refreshable

2017-10-27 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222830#comment-16222830 ] Eric Payne commented on YARN-7370: -- Thanks [~GergelyNovak] for the work on this patch. I just have a

[jira] [Commented] (YARN-6124) Make SchedulingEditPolicy can be enabled / disabled / updated with RMAdmin -refreshQueues

2017-10-26 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16221064#comment-16221064 ] Eric Payne commented on YARN-6124: -- Thanks [~leftnoteasy]. I will document my findings and you can work on

[jira] [Commented] (YARN-6124) Make SchedulingEditPolicy can be enabled / disabled / updated with RMAdmin -refreshQueues

2017-10-26 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220844#comment-16220844 ] Eric Payne commented on YARN-6124: -- Thanks [~leftnoteasy]. The proof of concept looks good, but in this

[jira] [Commented] (YARN-7370) Preemption properties should be refreshable

2017-10-25 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218617#comment-16218617 ] Eric Payne commented on YARN-7370: -- Thanks [~leftnoteasy] for the further design specifications. bq.

[jira] [Commented] (YARN-6124) Make SchedulingEditPolicy can be enabled / disabled / updated with RMAdmin -refreshQueues

2017-10-24 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216867#comment-16216867 ] Eric Payne commented on YARN-6124: -- Yes, I agree that these should be part of the scheduler. That makes a

[jira] [Commented] (YARN-7370) Intra-queue preemption properties should be refreshable

2017-10-23 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16215525#comment-16215525 ] Eric Payne commented on YARN-7370: -- Thanks [~leftnoteasy], [~sunilg], and [~GergelyNovak]. So, just to be

[jira] [Commented] (YARN-4163) Audit getQueueInfo and getApplications calls

2017-10-23 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16215426#comment-16215426 ] Eric Payne commented on YARN-4163: -- Thanks [~jlowe] and [~lichangleo]. I will commit this to trunk,

[jira] [Updated] (YARN-4163) Audit getQueueInfo and getApplications calls

2017-10-23 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-4163: - Attachment: YARN-4163.007.branch-2.8.patch Attach branch-2.8 specific patch > Audit getQueueInfo and

[jira] [Commented] (YARN-7370) Intra-queue preemption properties should be refreshable

2017-10-20 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212646#comment-16212646 ] Eric Payne commented on YARN-7370: -- [~GergelyNovak], thank you for your interest. Please go ahead and take

[jira] [Created] (YARN-7370) Intra-queue preemption properties should be refreshable

2017-10-19 Thread Eric Payne (JIRA)
Eric Payne created YARN-7370: Summary: Intra-queue preemption properties should be refreshable Key: YARN-7370 URL: https://issues.apache.org/jira/browse/YARN-7370 Project: Hadoop YARN Issue

[jira] [Commented] (YARN-4163) Audit getQueueInfo and getApplications calls

2017-10-18 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209494#comment-16209494 ] Eric Payne commented on YARN-4163: -- The unit tests are passing for me in my local environment. > Audit

[jira] [Updated] (YARN-4163) Audit getQueueInfo and getApplications calls

2017-10-17 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-4163: - Attachment: YARN-4163.007.patch Thanks [~jlowe] for the review. I have made some "generic" APIs for

[jira] [Commented] (YARN-7245) In Cap Sched UI, Max AM Resource column in Active Users Info section should be per-user

2017-10-05 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193591#comment-16193591 ] Eric Payne commented on YARN-7245: -- Thanks [~sunilg]. Looking forward to your response. > In Cap Sched

[jira] [Updated] (YARN-4163) Audit getQueueInfo and getApplications calls

2017-10-05 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-4163: - Attachment: YARN-4163.006.branch-2.8.patch Attaching branch-2.8 patch > Audit getQueueInfo and

[jira] [Updated] (YARN-4163) Audit getQueueInfo and getApplications calls

2017-10-05 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-4163: - Attachment: YARN-4163.006.patch Attaching {{YARN-4163.006.patch}} to address checkstyle issues. > Audit

[jira] [Commented] (YARN-4163) Audit getQueueInfo and getApplications calls

2017-10-05 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193102#comment-16193102 ] Eric Payne commented on YARN-4163: -- OK, so there are still some valid checkstyle warnings. Interestingly,

[jira] [Updated] (YARN-4163) Audit getQueueInfo and getApplications calls

2017-10-05 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-4163: - Attachment: YARN-4163.005.patch [~jlowe], attaching patch {{YARN-4163.005.patch}}. This contains an

[jira] [Updated] (YARN-7245) In Cap Sched UI, Max AM Resource column in Active Users Info section should be per-user

2017-10-04 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7245: - Attachment: Max AM Resource Per User -- Fixed.png I attached {{YARN-7245.001.patch}} to address this. I

[jira] [Updated] (YARN-7245) In Cap Sched UI, Max AM Resource column in Active Users Info section should be per-user

2017-10-04 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7245: - Attachment: YARN-7245.001.patch > In Cap Sched UI, Max AM Resource column in Active Users Info section

[jira] [Commented] (YARN-7271) Add a yarn application cost calculation framework in TimelineService v2

2017-10-02 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188450#comment-16188450 ] Eric Payne commented on YARN-7271: -- [~vrushalic], The RM has a built-in calculation that keeps track of

[jira] [Commented] (YARN-7245) In Cap Sched UI, Max AM Resource column in Active Users Info section should be per-user

2017-10-02 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188018#comment-16188018 ] Eric Payne commented on YARN-7245: -- bq. This is bad. We ideally need user based max-am-limit. [~sunilg],

[jira] [Assigned] (YARN-7245) In Cap Sched UI, Max AM Resource column in Active Users Info section should be per-user

2017-10-02 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne reassigned YARN-7245: Assignee: Eric Payne > In Cap Sched UI, Max AM Resource column in Active Users Info section should

[jira] [Updated] (YARN-4163) Audit getQueueInfo and getApplications calls

2017-09-29 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-4163: - Attachment: YARN-4163.004.patch I'm uploading YARN-4163.004.patch to upmerge the patch to trunk. Also,

[jira] [Commented] (YARN-7084) TestSchedulingMonitor#testRMStarts fails sporadically

2017-09-29 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185991#comment-16185991 ] Eric Payne commented on YARN-7084: -- Thanks [~jlowe] for reporting the issue and the fix. The strategy and

[jira] [Updated] (YARN-7249) Fix CapacityScheduler NPE issue when a container preempted while the node is being removed

2017-09-26 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7249: - Fix Version/s: 2.8.2 OK. I cherry-picked this to 2.8.2. Thanks. > Fix CapacityScheduler NPE issue when a

[jira] [Commented] (YARN-7249) Fix CapacityScheduler NPE issue when a container preempted while the node is being removed

2017-09-26 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180806#comment-16180806 ] Eric Payne commented on YARN-7249: -- {quote} I think it should be fine: containers are properly released

[jira] [Commented] (YARN-7249) Fix CapacityScheduler NPE issue when a container preempted while the node is being removed

2017-09-25 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179653#comment-16179653 ] Eric Payne commented on YARN-7249: -- [~leftnoteasy], I recognize that calling {{queue.completedContainer}}

[jira] [Commented] (YARN-7249) Fix CapacityScheduler NPE issue when a container preempted while the node is being removed

2017-09-25 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179553#comment-16179553 ] Eric Payne commented on YARN-7249: -- [~leftnoteasy]: Sure. Looking now. > Fix CapacityScheduler NPE issue

[jira] [Commented] (YARN-4163) Audit getQueueInfo and getApplications calls

2017-09-25 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179253#comment-16179253 ] Eric Payne commented on YARN-4163: -- [~lichangleo], please let me know if you plan on up-merging the patch

[jira] [Updated] (YARN-7245) In Cap Sched UI, Max AM Resource column in Active Users Info section should be per-user

2017-09-22 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7245: - Attachment: CapSched UI Showing Inaccurate Per User Max AM Resource.png The value in the {{Max AM

[jira] [Created] (YARN-7245) In Cap Sched UI, Max AM Resource column in Active Users Info section should be per-user

2017-09-22 Thread Eric Payne (JIRA)
Eric Payne created YARN-7245: Summary: In Cap Sched UI, Max AM Resource column in Active Users Info section should be per-user Key: YARN-7245 URL: https://issues.apache.org/jira/browse/YARN-7245 Project:

[jira] [Updated] (YARN-7116) CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage.

2017-09-22 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7116: - Fix Version/s: 2.8.3 > CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM >

[jira] [Commented] (YARN-7116) CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage.

2017-09-22 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16176700#comment-16176700 ] Eric Payne commented on YARN-7116: -- [~leftnoteasy], [~sunilg] If there are no objections, I'll backport

[jira] [Updated] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-18 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7149: - Fix Version/s: 3.1.0 2.9.0 > Cross-queue preemption sometimes starves an underserved

[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-18 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170157#comment-16170157 ] Eric Payne commented on YARN-7149: -- Thanks a lot [~leftnoteasy]. Also, this needs to be pulled back into

[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-15 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168499#comment-16168499 ] Eric Payne commented on YARN-7149: -- The following unit tests are succeeding for me in my environment:

[jira] [Updated] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-15 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7149: - Attachment: YARN-7149.002.patch bq. Do you think does it make sense to merge the

[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-14 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166286#comment-16166286 ] Eric Payne commented on YARN-7149: -- Unit test failures are not related to this patch:

[jira] [Updated] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-13 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7149: - Attachment: YARN-7149.001.patch Rather than use this JIRA to revert the {{computeUserLimit}} behavior to

[jira] [Commented] (YARN-4727) Unable to override the $HADOOP_CONF_DIR env variable for container

2017-09-13 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165194#comment-16165194 ] Eric Payne commented on YARN-4727: -- +1 Thanks [~jlowe] > Unable to override the $HADOOP_CONF_DIR env

[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-11 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16161546#comment-16161546 ] Eric Payne commented on YARN-7149: -- bq. You could check the unit test code to see if that matches your

[jira] [Updated] (YARN-6248) user is not removed from UsersManager’s when app is killed with pending container requests.

2017-09-07 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-6248: - Fix Version/s: 2.9.0 > user is not removed from UsersManager’s when app is killed with pending >

[jira] [Commented] (YARN-6248) user is not removed from UsersManager’s when app is killed with pending container requests.

2017-09-07 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157522#comment-16157522 ] Eric Payne commented on YARN-6248: -- I'm seeing this in branch-2 (2.9.0) as well. I will backport. > user

[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-07 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157471#comment-16157471 ] Eric Payne commented on YARN-7149: -- bq. Yes you're correct, The max op is consistent with old behavior, we

[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-07 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157263#comment-16157263 ] Eric Payne commented on YARN-7149: -- Thanks very much for your insights [~leftnoteasy]. bq. When we have

[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-06 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155872#comment-16155872 ] Eric Payne commented on YARN-7149: -- bq. I would like to still pursue Jason Lowe's suggestion about

[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-06 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155865#comment-16155865 ] Eric Payne commented on YARN-7149: -- Thanks for your insights, [~sunilg]. bq. Here I think we have to use

[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-05 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153529#comment-16153529 ] Eric Payne commented on YARN-7149: -- [~sunilg], [~leftnoteasy], and [~jlowe], I would be interested in your

[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-01 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151071#comment-16151071 ] Eric Payne commented on YARN-7149: -- I believe that this is related to the refactoring done to the way User

[jira] [Created] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue

2017-09-01 Thread Eric Payne (JIRA)
Eric Payne created YARN-7149: Summary: Cross-queue preemption sometimes starves an underserved queue Key: YARN-7149 URL: https://issues.apache.org/jira/browse/YARN-7149 Project: Hadoop YARN

[jira] [Updated] (YARN-7120) CapacitySchedulerPage NPE in "Aggregate scheduler counts" section

2017-08-29 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7120: - Attachment: YARN-7120.001.patch > CapacitySchedulerPage NPE in "Aggregate scheduler counts" section >

[jira] [Updated] (YARN-7120) CapacitySchedulerPage NPE in "Aggregate scheduler counts" section

2017-08-29 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7120: - Attachment: Aggregate Scheduler Counts All Sections.png Aggregate Scheduler Counts Bottom

[jira] [Created] (YARN-7120) CapacitySchedulerPage NPE in "Aggregate scheduler counts" section

2017-08-29 Thread Eric Payne (JIRA)
Eric Payne created YARN-7120: Summary: CapacitySchedulerPage NPE in "Aggregate scheduler counts" section Key: YARN-7120 URL: https://issues.apache.org/jira/browse/YARN-7120 Project: Hadoop YARN

[jira] [Commented] (YARN-7051) Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin

2017-08-28 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16143830#comment-16143830 ] Eric Payne commented on YARN-7051: -- Thanks [~sunilg] for your help in resolving this problem. > Avoid

[jira] [Commented] (YARN-7087) NM failed to perform log aggregation due to absent container

2017-08-25 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142140#comment-16142140 ] Eric Payne commented on YARN-7087: -- [~jlowe], thanks for finding, reporting, and fixing this issue. +1.

[jira] [Commented] (YARN-7052) RM SchedulingMonitor gives no indication why the spawned thread crashed.

2017-08-25 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141746#comment-16141746 ] Eric Payne commented on YARN-7052: -- The following unit tests are all passing for me in my environment:

[jira] [Commented] (YARN-7051) FifoIntraQueuePreemptionPlugin can get concurrent modification exception

2017-08-25 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141684#comment-16141684 ] Eric Payne commented on YARN-7051: -- Unit test failures are not related to this patch. -

[jira] [Updated] (YARN-7051) FifoIntraQueuePreemptionPlugin can get concurrent modification exception

2017-08-24 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7051: - Attachment: YARN-7051.002.patch bq. so this won't be changing while createTempAppForResCalculation is

<    4   5   6   7   8   9   10   11   12   13   >