[jira] [Commented] (YARN-9289) Backport YARN-7330 for GPU in UI to branch-2

[ https://issues.apache.org/jira/browse/YARN-9289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763836#comment-16763836 ] Arun Suresh commented on YARN-9289: --- +1, Thanks [~jhung] > Backport YARN-7330 for GPU in UI to branch-2

[jira] [Commented] (YARN-9280) Backport YARN-6620 to YARN-8200/branch-2

[ https://issues.apache.org/jira/browse/YARN-9280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762144#comment-16762144 ] Arun Suresh commented on YARN-9280: --- Thanks for the patch [~jhung] Maybe we should kick jenkins once

[jira] [Commented] (YARN-9174) Backport YARN-7224 for refactoring of GpuDevice class

[ https://issues.apache.org/jira/browse/YARN-9174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762145#comment-16762145 ] Arun Suresh commented on YARN-9174: --- Thanks [~jhung]. +1 pending jenkins on the latest patch > Backport

[jira] [Commented] (YARN-9180) Port YARN-7033 NM recovery of assigned resources to branch-3.0/branch-2

[ https://issues.apache.org/jira/browse/YARN-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758701#comment-16758701 ] Arun Suresh commented on YARN-9180: --- Thanks for the backport [~jhung] The checkstyles look fixable

[jira] [Commented] (YARN-9222) Print launchTime in ApplicationSummary

[ https://issues.apache.org/jira/browse/YARN-9222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752609#comment-16752609 ] Arun Suresh commented on YARN-9222: --- Thanks for the patch [~jhung] and the review

[jira] [Commented] (YARN-7088) Add application launch time to Resource Manager REST API

[ https://issues.apache.org/jira/browse/YARN-7088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751482#comment-16751482 ] Arun Suresh commented on YARN-7088: --- +1 in that case > Add application launch time to Resource Manager

[jira] [Commented] (YARN-7088) Add application launch time to Resource Manager REST API

[ https://issues.apache.org/jira/browse/YARN-7088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751444#comment-16751444 ] Arun Suresh commented on YARN-7088: --- [~jhung] the backport for branch-2 look pretty straightforward.

[jira] [Commented] (YARN-7746) Fix PlacementProcessor to support app priority

[ https://issues.apache.org/jira/browse/YARN-7746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749563#comment-16749563 ] Arun Suresh commented on YARN-7746: --- Apologize for the long delay.. The patch looks good

[jira] [Commented] (YARN-9187) Backport YARN-6852 for GPU-specific native changes to branch-2

[ https://issues.apache.org/jira/browse/YARN-9187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740626#comment-16740626 ] Arun Suresh commented on YARN-9187: --- Thanks [~jhung] +1 > Backport YARN-6852 for GPU-specific native

[jira] [Commented] (YARN-9188) Port YARN-7136 to branch-2

[ https://issues.apache.org/jira/browse/YARN-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739862#comment-16739862 ] Arun Suresh commented on YARN-9188: --- Thanks for the patch [~jhung]. Can we get rid of the deprecation

[jira] [Commented] (YARN-9182) Backport YARN-6445 resource profile performance improvements to branch-2

[ https://issues.apache.org/jira/browse/YARN-9182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737433#comment-16737433 ] Arun Suresh commented on YARN-9182: --- Looks good +1 Thanks [~jhung] > Backport YARN-6445 resource

[jira] [Commented] (YARN-9181) Backport YARN-6232 for generic resource type usage to branch-2

[ https://issues.apache.org/jira/browse/YARN-9181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737434#comment-16737434 ] Arun Suresh commented on YARN-9181: --- +1 pending jenkins after youve committed YARN-9177 > Backport

[jira] [Commented] (YARN-9177) Use resource map for app metrics in TestCombinedSystemMetricsPublisher for branch-2

[ https://issues.apache.org/jira/browse/YARN-9177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737393#comment-16737393 ] Arun Suresh commented on YARN-9177: --- Looks good +1 Thanks [~jhung] > Use resource map for app metrics

[jira] [Commented] (YARN-9180) Port YARN-7033 NM recovery of assigned resources to branch-3.0/branch-2

[ https://issues.apache.org/jira/browse/YARN-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736376#comment-16736376 ] Arun Suresh commented on YARN-9180: --- Thanks for the patch [~jhung] Looks strightforward.. am guessing

[jira] [Commented] (YARN-9174) branch-3.0/branch-2 refactoring of GpuDevice class

[ https://issues.apache.org/jira/browse/YARN-9174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16735048#comment-16735048 ] Arun Suresh commented on YARN-9174: --- Thanks for the patch [~jhung] To be honest, I am not very happy

[jira] [Comment Edited] (YARN-9174) branch-3.0/branch-2 refactoring of GpuDevice class

[ https://issues.apache.org/jira/browse/YARN-9174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16735048#comment-16735048 ] Arun Suresh edited comment on YARN-9174 at 1/5/19 11:55 PM: Thanks for the

[jira] [Updated] (YARN-8827) Plumb aggregated application resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8827: -- Fix Version/s: YARN-1011 > Plumb aggregated application resource utilization from the NM to RM >

[jira] [Updated] (YARN-8827) Plumb aggregated application resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8827: -- Affects Version/s: YARN-1011 > Plumb aggregated application resource utilization from the NM to RM >

[jira] [Updated] (YARN-8827) Plumb aggregated application resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8827: -- Summary: Plumb aggregated application resource utilization from the NM to RM (was: Plumb per

[jira] [Commented] (YARN-8827) Plumb per application resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644410#comment-16644410 ] Arun Suresh commented on YARN-8827: --- Thanks for the review [~elgoiri]. Committing this.. > Plumb per

[jira] [Updated] (YARN-8827) Plumb per application resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8827: -- Summary: Plumb per application resource utilization from the NM to RM (was: Plumb per app, per user

[jira] [Updated] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8827: -- Attachment: YARN-8827-YARN-1011.07.patch > Plumb per app, per user and per queue resource utilization

[jira] [Commented] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643991#comment-16643991 ] Arun Suresh commented on YARN-8827: --- The {{TestCapacityOverTimePolicy}} failure is unrelated and

[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate

[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643785#comment-16643785 ] Arun Suresh commented on YARN-1013: --- Linking to YARN-8827, since we plan to get the CS to use the

[jira] [Commented] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643783#comment-16643783 ] Arun Suresh commented on YARN-8827: --- Thanks [~elgoiri] .. Attached updated patch. bq. .. I think we

[jira] [Updated] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8827: -- Attachment: YARN-8827-YARN-1011.06.patch > Plumb per app, per user and per queue resource utilization

[jira] [Updated] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8827: -- Attachment: YARN-8827-YARN-1011.05.patch > Plumb per app, per user and per queue resource utilization

[jira] [Commented] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642842#comment-16642842 ] Arun Suresh commented on YARN-8827: --- Updating patch with checkstyle fixes. The test case failure should

[jira] [Updated] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8827: -- Attachment: YARN-8827-YARN-1011.04.patch > Plumb per app, per user and per queue resource utilization

[jira] [Comment Edited] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642596#comment-16642596 ] Arun Suresh edited comment on YARN-8827 at 10/8/18 11:29 PM: - Updating patch

[jira] [Commented] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642596#comment-16642596 ] Arun Suresh commented on YARN-8827: --- Updating patch based on your suggestions.. bq. I was talking about

[jira] [Updated] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8827: -- Attachment: YARN-8827-YARN-1011.03.patch > Plumb per app, per user and per queue resource utilization

[jira] [Comment Edited] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642496#comment-16642496 ] Arun Suresh edited comment on YARN-8827 at 10/8/18 9:13 PM: Thanks for the

[jira] [Commented] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642496#comment-16642496 ] Arun Suresh commented on YARN-8827: --- Thanks for the review [~elgoiri] Attached patch addressing most of

[jira] [Updated] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8827: -- Attachment: YARN-8827-YARN-1011.02.patch > Plumb per app, per user and per queue resource utilization

[jira] [Commented] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641160#comment-16641160 ] Arun Suresh commented on YARN-8827: --- Attaching initial patch. Do take a look (cc: [~elgoiri] /

[jira] [Updated] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8827: -- Attachment: YARN-8827-YARN-1011.01.patch > Plumb per app, per user and per queue resource utilization

[jira] [Commented] (YARN-8849) DynoYARN: A simulation and testing infrastructure for YARN clusters

[ https://issues.apache.org/jira/browse/YARN-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640339#comment-16640339 ] Arun Suresh commented on YARN-8849: --- Thanks for the comments [~leftnoteasy] bq. If no container

[jira] [Updated] (YARN-8849) DynoYARN: A simulation and testing infrastructure for YARN clusters

[ https://issues.apache.org/jira/browse/YARN-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8849: -- Description: Traditionally, YARN workload simulation is performed using SLS (Scheduler Load Simulator)

[jira] [Updated] (YARN-8849) DynoYARN: A simulation and testing infrastructure for YARN clusters

[ https://issues.apache.org/jira/browse/YARN-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8849: -- Description: Traditionally, YARN workload simulation is performed using SLS (Scheduler Load Simulator)

[jira] [Created] (YARN-8850) Make certain aspects of the NM pluggable to support a DynoYARN cluster

Arun Suresh created YARN-8850: - Summary: Make certain aspects of the NM pluggable to support a DynoYARN cluster Key: YARN-8850 URL: https://issues.apache.org/jira/browse/YARN-8850 Project: Hadoop YARN

[jira] [Created] (YARN-8849) DynoYARN: A simulation and testing infrastructure for YARN clusters

Arun Suresh created YARN-8849: - Summary: DynoYARN: A simulation and testing infrastructure for YARN clusters Key: YARN-8849 URL: https://issues.apache.org/jira/browse/YARN-8849 Project: Hadoop YARN

[jira] [Created] (YARN-8848) Improvements to YARN over-allocation (YARN-1011)

Arun Suresh created YARN-8848: - Summary: Improvements to YARN over-allocation (YARN-1011) Key: YARN-8848 URL: https://issues.apache.org/jira/browse/YARN-8848 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-6723) NM overallocation based on over-time rather than snapshot utilization

[ https://issues.apache.org/jira/browse/YARN-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6723: -- Parent Issue: YARN-8848 (was: YARN-1011) > NM overallocation based on over-time rather than snapshot

[jira] [Updated] (YARN-6690) Consolidate NM overallocation thresholds with ResourceTypes

[ https://issues.apache.org/jira/browse/YARN-6690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6690: -- Parent Issue: YARN-8848 (was: YARN-1011) > Consolidate NM overallocation thresholds with

[jira] [Created] (YARN-8846) Allow Applications to demand Guaranteed Containers

Arun Suresh created YARN-8846: - Summary: Allow Applications to demand Guaranteed Containers Key: YARN-8846 URL: https://issues.apache.org/jira/browse/YARN-8846 Project: Hadoop YARN Issue Type:

[jira] [Assigned] (YARN-8846) Allow Applications to demand Guaranteed Containers

[ https://issues.apache.org/jira/browse/YARN-8846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reassigned YARN-8846: - Assignee: Arun Suresh > Allow Applications to demand Guaranteed Containers >

[jira] [Updated] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8250: -- Issue Type: Improvement (was: Sub-task) Parent: (was: YARN-1011) > Create another

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6668: -- Issue Type: Improvement (was: Sub-task) Parent: (was: YARN-1011) > Use cgroup to get

[jira] [Updated] (YARN-6843) Add unit test for NM to preempt OPPORTUNISTIC containers under high utilization

[ https://issues.apache.org/jira/browse/YARN-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6843: -- Issue Type: Bug (was: Sub-task) Parent: (was: YARN-1011) > Add unit test for NM to preempt

[jira] [Updated] (YARN-6806) Scheduler-agnostic RM changes support oversubscription

[ https://issues.apache.org/jira/browse/YARN-6806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6806: -- Issue Type: Bug (was: Sub-task) Parent: (was: YARN-1011) > Scheduler-agnostic RM changes

[jira] [Updated] (YARN-6800) Add opportunity to start containers while periodically checking for preemption

[ https://issues.apache.org/jira/browse/YARN-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6800: -- Issue Type: Bug (was: Sub-task) Parent: (was: YARN-1011) > Add opportunity to start

[jira] [Updated] (YARN-6796) Add unit test for NM to launch OPPORTUNISTIC container for overallocation

[ https://issues.apache.org/jira/browse/YARN-6796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6796: -- Issue Type: Bug (was: Sub-task) Parent: (was: YARN-1011) > Add unit test for NM to launch

[jira] [Updated] (YARN-7015) Handle Container ExecType update (Promotion/Demotion) in cgroups resource handlers

[ https://issues.apache.org/jira/browse/YARN-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7015: -- Issue Type: Improvement (was: Sub-task) Parent: (was: YARN-1011) > Handle Container

[jira] [Updated] (YARN-6673) Add cpu cgroup configurations for opportunistic containers

[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6673: -- Issue Type: Bug (was: Sub-task) Parent: (was: YARN-1011) > Add cpu cgroup configurations

[jira] [Updated] (YARN-6674) Add memory cgroup settings for opportunistic containers

[ https://issues.apache.org/jira/browse/YARN-6674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6674: -- Issue Type: Improvement (was: Bug) > Add memory cgroup settings for opportunistic containers >

[jira] [Updated] (YARN-6674) Add memory cgroup settings for opportunistic containers

[ https://issues.apache.org/jira/browse/YARN-6674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6674: -- Issue Type: Bug (was: Sub-task) Parent: (was: YARN-1011) > Add memory cgroup settings for

[jira] [Updated] (YARN-6671) Add container type awareness in ResourceHandlers.

[ https://issues.apache.org/jira/browse/YARN-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6671: -- Issue Type: Improvement (was: Sub-task) Parent: (was: YARN-1011) > Add container type

[jira] [Updated] (YARN-4293) ResourceUtilization should be a part of yarn node CLI

[ https://issues.apache.org/jira/browse/YARN-4293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4293: -- Issue Type: Improvement (was: Sub-task) Parent: (was: YARN-1011) > ResourceUtilization

[jira] [Updated] (YARN-1012) Report NM aggregated container resource utilization in heartbeat

[ https://issues.apache.org/jira/browse/YARN-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-1012: -- Issue Type: Bug (was: Sub-task) Parent: (was: YARN-1011) > Report NM aggregated container

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3980: -- Issue Type: Improvement (was: Sub-task) Parent: (was: YARN-1011) > Plumb

[jira] [Updated] (YARN-4292) ResourceUtilization should be a part of NodeInfo REST API

[ https://issues.apache.org/jira/browse/YARN-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4292: -- Issue Type: Improvement (was: Sub-task) Parent: (was: YARN-1011) > ResourceUtilization

[jira] [Updated] (YARN-4291) ResourceUtilization should be a part of NodeReport API.

[ https://issues.apache.org/jira/browse/YARN-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4291: -- Issue Type: Improvement (was: Sub-task) Parent: (was: YARN-1011) > ResourceUtilization

[jira] [Updated] (YARN-3534) Collect memory/cpu usage on the node

[ https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3534: -- Issue Type: Improvement (was: Sub-task) Parent: (was: YARN-1011) > Collect memory/cpu

[jira] [Updated] (YARN-1012) Report NM aggregated container resource utilization in heartbeat

[ https://issues.apache.org/jira/browse/YARN-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-1012: -- Issue Type: Improvement (was: Bug) > Report NM aggregated container resource utilization in heartbeat

[jira] [Commented] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637342#comment-16637342 ] Arun Suresh commented on YARN-8808: --- Thanks [~haibochen]. Committed to YARN-1011 > Use aggregate

[jira] [Commented] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637335#comment-16637335 ] Arun Suresh commented on YARN-8808: --- Nope, +1 Will commit shortly.. > Use aggregate container

[jira] [Comment Edited] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631359#comment-16631359 ] Arun Suresh edited comment on YARN-8808 at 9/28/18 5:15 AM: bq.

[jira] [Commented] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631359#comment-16631359 ] Arun Suresh commented on YARN-8808: --- bq. containersUtilization and nodeUtilization in SchedulerNode are

[jira] [Commented] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631358#comment-16631358 ] Arun Suresh commented on YARN-8808: --- bq. The scheduler would see this 'Node' has 10GBs to allocate

[jira] [Commented] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631145#comment-16631145 ] Arun Suresh commented on YARN-8808: --- bq. Node has capacity for 4 1GB containers, but is currently

[jira] [Commented] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16630610#comment-16630610 ] Arun Suresh commented on YARN-8827: --- I guess we just need per app utilization.. since the queue and user

[jira] [Commented] (YARN-7746) Fix PlacementProcessor to support app priority

[ https://issues.apache.org/jira/browse/YARN-7746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629774#comment-16629774 ] Arun Suresh commented on YARN-7746: --- Thanks for the patch [~maniraj...@gmail.com] Instead of having a

[jira] [Comment Edited] (YARN-7746) Fix PlacementProcessor to support app priority

[ https://issues.apache.org/jira/browse/YARN-7746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629774#comment-16629774 ] Arun Suresh edited comment on YARN-7746 at 9/27/18 5:28 AM: Thanks for the

[jira] [Assigned] (YARN-7746) Fix PlacementProcessor to support app priority

[ https://issues.apache.org/jira/browse/YARN-7746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reassigned YARN-7746: - Assignee: Manikandan R (was: Arun Suresh) > Fix PlacementProcessor to support app priority >

[jira] [Updated] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

[ https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8827: -- Description: Opportunistic Containers for OverAllocation need to be allocated to pending applications

[jira] [Created] (YARN-8827) Plumb per app, per user and per queue resource utilization from the NM to RM

Arun Suresh created YARN-8827: - Summary: Plumb per app, per user and per queue resource utilization from the NM to RM Key: YARN-8827 URL: https://issues.apache.org/jira/browse/YARN-8827 Project: Hadoop

[jira] [Commented] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626864#comment-16626864 ] Arun Suresh commented on YARN-8808: --- {quote}Not sure if I follow you correctly. Sounds like you are

[jira] [Commented] (YARN-7974) Allow updating application tracking url after registration

[ https://issues.apache.org/jira/browse/YARN-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626817#comment-16626817 ] Arun Suresh commented on YARN-7974: --- +1 for the branch-2 patch. I just kicked jenkins again - to see if

[jira] [Commented] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624831#comment-16624831 ] Arun Suresh commented on YARN-8808: --- Also, it looks like you need to null check getNodeUtilization() /

[jira] [Comment Edited] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624224#comment-16624224 ] Arun Suresh edited comment on YARN-8808 at 9/21/18 9:50 PM: While working on

[jira] [Commented] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624224#comment-16624224 ] Arun Suresh commented on YARN-8808: --- While working on YARN-1013, looks like we should also check for

[jira] [Updated] (YARN-8809) Refactor AbstractYarnScheduler and CapacityScheduler OPPORTUNISTIC container completion codepaths

[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8809: -- Summary: Refactor AbstractYarnScheduler and CapacityScheduler OPPORTUNISTIC container completion

[jira] [Commented] (YARN-1011) [Umbrella] Schedule containers based on utilization of currently allocated containers

[ https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624032#comment-16624032 ] Arun Suresh commented on YARN-1011: --- I was trying to rebase the branch with trunk.. Got a couple of

[jira] [Commented] (YARN-8809) Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released.

[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624029#comment-16624029 ] Arun Suresh commented on YARN-8809: --- Thanks [~haibochen] +1, will commit shortly > Fair Scheduler does

[jira] [Commented] (YARN-8809) Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released.

[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622822#comment-16622822 ] Arun Suresh commented on YARN-8809: --- I believe {{TestOpportunisticContainerAllocatorAMService}} does

[jira] [Comment Edited] (YARN-8809) Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released.

[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622822#comment-16622822 ] Arun Suresh edited comment on YARN-8809 at 9/20/18 10:42 PM: - I believe

[jira] [Commented] (YARN-8809) Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released.

[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622784#comment-16622784 ] Arun Suresh commented on YARN-8809: --- Got it... So, in essence, FS has a unified code path for completing

[jira] [Comment Edited] (YARN-1011) [Umbrella] Schedule containers based on utilization of currently allocated containers

[ https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622450#comment-16622450 ] Arun Suresh edited comment on YARN-1011 at 9/20/18 6:11 PM: Planning on

[jira] [Commented] (YARN-1011) [Umbrella] Schedule containers based on utilization of currently allocated containers

[ https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622450#comment-16622450 ] Arun Suresh commented on YARN-1011: --- Planning on spending more cycles on this now. Looking at the

[jira] [Commented] (YARN-8809) Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released.

[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622445#comment-16622445 ] Arun Suresh commented on YARN-8809: --- Thanks for the patch [~haibochen].. >From the patch, it is not

[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate

[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619809#comment-16619809 ] Arun Suresh commented on YARN-1013: --- Thanks for taking a quick look [~elgoiri] So, the patch was more

[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate

[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619709#comment-16619709 ] Arun Suresh commented on YARN-1013: --- Attached an initial version of the patch for branch-2. Kindly

[jira] [Updated] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate

[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-1013: -- Attachment: YARN-1013-001.branch-2.patch > CS should watch resource utilization of containers and

[jira] [Assigned] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate

[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reassigned YARN-1013: - Assignee: Arun Suresh (was: Weiwei Yang) > CS should watch resource utilization of containers

[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate

[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593495#comment-16593495 ] Arun Suresh commented on YARN-1013: --- [~cheersyang], If you havn't started with this, wondering if I

[jira] [Commented] (YARN-7086) Release all containers aynchronously

[ https://issues.apache.org/jira/browse/YARN-7086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582803#comment-16582803 ] Arun Suresh commented on YARN-7086: --- Assigning to [~maniraj...@gmail.com], since he's kindly agreed to

[jira] [Assigned] (YARN-7086) Release all containers aynchronously

[ https://issues.apache.org/jira/browse/YARN-7086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reassigned YARN-7086: - Assignee: Manikandan R (was: Arun Suresh) > Release all containers aynchronously >

[jira] [Commented] (YARN-4175) Example of use YARN-1197

[ https://issues.apache.org/jira/browse/YARN-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560087#comment-16560087 ] Arun Suresh commented on YARN-4175: --- Thanks for taking this up.. yup makes sense . Do post your patch..

[jira] [Commented] (YARN-8544) [DS] AM registration fails when hadoop authorization is enabled

[ https://issues.apache.org/jira/browse/YARN-8544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552264#comment-16552264 ] Arun Suresh commented on YARN-8544: --- Got it. +1 > [DS] AM registration fails when hadoop authorization

  1   2   3   4   5   6   7   8   9   10   >