[jira] [Created] (YARN-3885) ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level

Ajith S created YARN-3885: - Summary: ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level Key: YARN-3885 URL: https://issues.apache.org/jira/browse/YARN-3885 Project: Hadoop

[jira] [Updated] (YARN-3885) ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level

[ https://issues.apache.org/jira/browse/YARN-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated YARN-3885: -- Attachment: YARN-3885.patch submitting patch ProportionalCapacityPreemptionPolicy doesn't preempt if queue is

[jira] [Updated] (YARN-3885) ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level

[ https://issues.apache.org/jira/browse/YARN-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated YARN-3885: -- Description: when preemption policy is {{ProportionalCapacityPreemptionPolicy.cloneQueues}} this piece of code,

[jira] [Commented] (YARN-3885) ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level

[ https://issues.apache.org/jira/browse/YARN-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614182#comment-14614182 ] Ajith S commented on YARN-3885: --- Hi all, I already have a patch for it, please assign this

[jira] [Commented] (YARN-3882) AggregatedLogFormat should close aclScanner and ownerScanner after create them.

[ https://issues.apache.org/jira/browse/YARN-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614170#comment-14614170 ] zhihai xu commented on YARN-3882: - thanks [~xgong] for the review and committing the patch!

[jira] [Commented] (YARN-3643) Provide a way to store only running applications in the state store

[ https://issues.apache.org/jira/browse/YARN-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614185#comment-14614185 ] Varun Saxena commented on YARN-3643: Yeah. But based on this, how do we go about

[jira] [Updated] (YARN-3885) ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level

[ https://issues.apache.org/jira/browse/YARN-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated YARN-3885: -- Description: when preemption policy is {{ProportionalCapacityPreemptionPolicy.cloneQueues}} this piece of code,

[jira] [Commented] (YARN-3882) AggregatedLogFormat should close aclScanner and ownerScanner after create them.

[ https://issues.apache.org/jira/browse/YARN-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614264#comment-14614264 ] Hudson commented on YARN-3882: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2174 (See

[jira] [Commented] (YARN-3882) AggregatedLogFormat should close aclScanner and ownerScanner after create them.

[ https://issues.apache.org/jira/browse/YARN-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614266#comment-14614266 ] Hudson commented on YARN-3882: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #235 (See

[jira] [Commented] (YARN-3885) ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level

[ https://issues.apache.org/jira/browse/YARN-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614194#comment-14614194 ] Xianyin Xin commented on YARN-3885: --- Hi Ajith, CHILDRENPREEMPTABLE may be over counted.

[jira] [Commented] (YARN-3882) AggregatedLogFormat should close aclScanner and ownerScanner after create them.

[ https://issues.apache.org/jira/browse/YARN-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614209#comment-14614209 ] Hudson commented on YARN-3882: -- FAILURE: Integrated in Hadoop-Yarn-trunk #977 (See

[jira] [Commented] (YARN-3882) AggregatedLogFormat should close aclScanner and ownerScanner after create them.

[ https://issues.apache.org/jira/browse/YARN-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614222#comment-14614222 ] Hudson commented on YARN-3882: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #247 (See

[jira] [Commented] (YARN-3882) AggregatedLogFormat should close aclScanner and ownerScanner after create them.

[ https://issues.apache.org/jira/browse/YARN-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614296#comment-14614296 ] Hudson commented on YARN-3882: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2193 (See

[jira] [Commented] (YARN-3882) AggregatedLogFormat should close aclScanner and ownerScanner after create them.

[ https://issues.apache.org/jira/browse/YARN-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614285#comment-14614285 ] Hudson commented on YARN-3882: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #245 (See

[jira] [Commented] (YARN-3045) [Event producers] Implement NM writing container lifecycle events to ATS

[ https://issues.apache.org/jira/browse/YARN-3045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614346#comment-14614346 ] Naganarasimha G R commented on YARN-3045: - Hi [~djp], [~sjlee0] [~zjshen] The

[jira] [Commented] (YARN-3643) Provide a way to store only running applications in the state store

[ https://issues.apache.org/jira/browse/YARN-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614344#comment-14614344 ] Karthik Kambatla commented on YARN-3643: Since setting max-completed-applications

[jira] [Commented] (YARN-3643) Provide a way to store only running applications in the state store

[ https://issues.apache.org/jira/browse/YARN-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614350#comment-14614350 ] Varun Saxena commented on YARN-3643: Yes, YARN-2962 can go in only in trunk. But I

[jira] [Updated] (YARN-3878) AsyncDispatcher can hang while stopping if it is configured for draining events on stop

[ https://issues.apache.org/jira/browse/YARN-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-3878: --- Attachment: YARN-3878.05.patch AsyncDispatcher can hang while stopping if it is configured for

[jira] [Commented] (YARN-3878) AsyncDispatcher can hang while stopping if it is configured for draining events on stop

[ https://issues.apache.org/jira/browse/YARN-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614375#comment-14614375 ] Hadoop QA commented on YARN-3878: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Updated] (YARN-3878) AsyncDispatcher can hang while stopping if it is configured for draining events on stop

[ https://issues.apache.org/jira/browse/YARN-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-3878: --- Attachment: YARN-3878.06.patch AsyncDispatcher can hang while stopping if it is configured for

[jira] [Commented] (YARN-3878) AsyncDispatcher can hang while stopping if it is configured for draining events on stop

[ https://issues.apache.org/jira/browse/YARN-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614398#comment-14614398 ] Hadoop QA commented on YARN-3878: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (YARN-3878) AsyncDispatcher can hang while stopping if it is configured for draining events on stop

[ https://issues.apache.org/jira/browse/YARN-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614355#comment-14614355 ] Varun Saxena commented on YARN-3878: Thanks for the review [~kasha] Will add a method

[jira] [Commented] (YARN-3878) AsyncDispatcher can hang while stopping if it is configured for draining events on stop

[ https://issues.apache.org/jira/browse/YARN-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614431#comment-14614431 ] Karthik Kambatla commented on YARN-3878: {{hasNoPendingEvents}} seems a little

[jira] [Commented] (YARN-1012) Report NM aggregated container resource utilization in heartbeat

[ https://issues.apache.org/jira/browse/YARN-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614463#comment-14614463 ] Inigo Goiri commented on YARN-1012: --- Thanks for the feedback, it makes sense. I have one

[jira] [Commented] (YARN-2962) ZKRMStateStore: Limit the number of znodes under a znode

[ https://issues.apache.org/jira/browse/YARN-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614432#comment-14614432 ] Karthik Kambatla commented on YARN-2962: For a trunk only patch, do we really need

[jira] [Updated] (YARN-2962) ZKRMStateStore: Limit the number of znodes under a znode

[ https://issues.apache.org/jira/browse/YARN-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-2962: --- Target Version/s: 3.0.0 (was: 2.8.0) ZKRMStateStore: Limit the number of znodes under a

[jira] [Commented] (YARN-1012) Report NM aggregated container resource utilization in heartbeat

[ https://issues.apache.org/jira/browse/YARN-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614442#comment-14614442 ] Karthik Kambatla commented on YARN-1012: Thanks for updating the patch, Inigo.

[jira] [Commented] (YARN-1012) Report NM aggregated container resource utilization in heartbeat

[ https://issues.apache.org/jira/browse/YARN-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614469#comment-14614469 ] Karthik Kambatla commented on YARN-1012: Actually, looks like we always monitor the

[jira] [Updated] (YARN-3885) ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level

[ https://issues.apache.org/jira/browse/YARN-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyank Rastogi updated YARN-3885: -- Description: when preemption policy is {{ProportionalCapacityPreemptionPolicy.cloneQueues}} this

[jira] [Updated] (YARN-1012) Report NM aggregated container resource utilization in heartbeat

[ https://issues.apache.org/jira/browse/YARN-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-1012: -- Attachment: YARN-1012-10.patch Improved patch based on feedback. Report NM aggregated container

[jira] [Commented] (YARN-3849) Too much of preemption activity causing continuos killing of containers across queues

[ https://issues.apache.org/jira/browse/YARN-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614571#comment-14614571 ] Hadoop QA commented on YARN-3849: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Created] (YARN-3886) Add cumulative wait times of apps at Queue level

Raju Bairishetti created YARN-3886: -- Summary: Add cumulative wait times of apps at Queue level Key: YARN-3886 URL: https://issues.apache.org/jira/browse/YARN-3886 Project: Hadoop YARN Issue

[jira] [Updated] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]

[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-2003: -- Attachment: 0016-YARN-2003.patch Thank you [~jianhe]. As per the discussion in

[jira] [Updated] (YARN-2003) Support for Application priority : Changes in RM and Capacity Scheduler

[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-2003: -- Summary: Support for Application priority : Changes in RM and Capacity Scheduler (was: Support to process Job

[jira] [Resolved] (YARN-2004) Priority scheduling support in Capacity scheduler

[ https://issues.apache.org/jira/browse/YARN-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G resolved YARN-2004. --- Resolution: Implemented Priority scheduling support in Capacity scheduler

[jira] [Updated] (YARN-3878) AsyncDispatcher can hang while stopping if it is configured for draining events on stop

[ https://issues.apache.org/jira/browse/YARN-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-3878: --- Attachment: YARN-3878.07.patch AsyncDispatcher can hang while stopping if it is configured for

[jira] [Commented] (YARN-3878) AsyncDispatcher can hang while stopping if it is configured for draining events on stop

[ https://issues.apache.org/jira/browse/YARN-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614538#comment-14614538 ] Devaraj K commented on YARN-3878: - Adding to [~kasha] comment, 1. Can you add

[jira] [Commented] (YARN-3878) AsyncDispatcher can hang while stopping if it is configured for draining events on stop

[ https://issues.apache.org/jira/browse/YARN-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614560#comment-14614560 ] Hadoop QA commented on YARN-3878: - \\ \\ | (/) *{color:green}+1 overall{color}* | \\ \\ ||

[jira] [Updated] (YARN-3885) ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level

[ https://issues.apache.org/jira/browse/YARN-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated YARN-3885: -- Attachment: YARN-3885.03.patch refactored according to [~xinxianyin] comments

[jira] [Commented] (YARN-3885) ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level

[ https://issues.apache.org/jira/browse/YARN-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614505#comment-14614505 ] Ajith S commented on YARN-3885: --- Hello xin, Thanks for the input, yes you are right, i will

[jira] [Updated] (YARN-3849) Too much of preemption activity causing continuos killing of containers across queues

[ https://issues.apache.org/jira/browse/YARN-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-3849: -- Attachment: 0004-YARN-3849.patch Kicking jenkins again. Too much of preemption activity causing continuos

[jira] [Commented] (YARN-1012) Report NM aggregated container resource utilization in heartbeat

[ https://issues.apache.org/jira/browse/YARN-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614489#comment-14614489 ] Hadoop QA commented on YARN-1012: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Updated] (YARN-3885) ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level

[ https://issues.apache.org/jira/browse/YARN-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated YARN-3885: -- Attachment: YARN-3885.02.patch ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2

[jira] [Updated] (YARN-3849) Too much of preemption activity causing continuos killing of containers across queues

[ https://issues.apache.org/jira/browse/YARN-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-3849: -- Attachment: (was: 0004-YARN-3849.patch) Too much of preemption activity causing continuos killing of