[jira] [Updated] (YARN-6920) fix TestNMClient failure due to YARN-6706

2017-08-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6920: -- Fix Version/s: 3.0.0-beta1 2.9.0 > fix TestNMClient failure due to YARN-6706 >

[jira] [Updated] (YARN-6920) fix TestNMClient failure due to YARN-6706

2017-08-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6920: -- Fix Version/s: (was: 3.0.0-beta1) (was: 2.9.0) > fix TestNMClient failure due

[jira] [Updated] (YARN-6920) fix TestNMClient failure due to YARN-6706

2017-08-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6920: -- Attachment: YARN-6920.001.patch > fix TestNMClient failure due to YARN-6706 >

[jira] [Commented] (YARN-6920) fix TestNMClient failure due to YARN-6706

2017-08-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110009#comment-16110009 ] Arun Suresh commented on YARN-6920: --- Actually, let me take over this [~haibochen], if you don't mind. On

[jira] [Comment Edited] (YARN-6593) [API] Introduce Placement Constraint object

2017-08-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109921#comment-16109921 ] Arun Suresh edited comment on YARN-6593 at 8/1/17 10:38 PM: bq. Actually, my

[jira] [Commented] (YARN-6593) [API] Introduce Placement Constraint object

2017-08-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109921#comment-16109921 ] Arun Suresh commented on YARN-6593: --- bq. Actually, my comments should apply to YARN-6594 rather this

[jira] [Comment Edited] (YARN-6920) fix TestNMClient failure due to YARN-6706

2017-08-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109197#comment-16109197 ] Arun Suresh edited comment on YARN-6920 at 8/1/17 4:31 PM: --- It looks like its

[jira] [Updated] (YARN-6920) fix TestNMClient failure due to YARN-6706

2017-08-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6920: -- Summary: fix TestNMClient failure due to YARN-6706 (was: fix TestNMClient failure) > fix TestNMClient

[jira] [Commented] (YARN-6920) fix TestNMClient failure

2017-08-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109197#comment-16109197 ] Arun Suresh commented on YARN-6920: --- It looks like its been failing after YARN-6706. [~haibo.chen] Can

[jira] [Created] (YARN-6920) fix TestNMClient failure

2017-08-01 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-6920: - Summary: fix TestNMClient failure Key: YARN-6920 URL: https://issues.apache.org/jira/browse/YARN-6920 Project: Hadoop YARN Issue Type: Bug Reporter:

[jira] [Updated] (YARN-5977) ContainerManagementProtocol changes to support change of container ExecutionType

2017-07-30 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5977: -- Description: JIRA to track the following changes: * Changes in the ContainerManagementProtocol - add an

[jira] [Assigned] (YARN-5978) ContainerScheduler and Container state machine changes to support ExecType update

2017-07-29 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reassigned YARN-5978: - Assignee: kartheek muthyala (was: Arun Suresh) > ContainerScheduler and Container state machine

[jira] [Assigned] (YARN-5977) ContainerManagementProtocol changes to support change of container ExecutionType

2017-07-29 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reassigned YARN-5977: - Assignee: Arun Suresh > ContainerManagementProtocol changes to support change of container >

[jira] [Assigned] (YARN-5977) ContainerManagementProtocol changes to support change of container ExecutionType

2017-07-29 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reassigned YARN-5977: - Assignee: kartheek muthyala (was: Arun Suresh) > ContainerManagementProtocol changes to support

[jira] [Updated] (YARN-6870) Fix floating point inaccuracies in resource availability check in AllocationBasedResourceUtilizationTracker

2017-07-28 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6870: -- Fix Version/s: 3.0.0-beta1 2.9.0 > Fix floating point inaccuracies in resource

[jira] [Updated] (YARN-6870) Fix floating point inaccuracies in resource availability check in AllocationBasedResourceUtilizationTracker

2017-07-28 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6870: -- Summary: Fix floating point inaccuracies in resource availability check in

[jira] [Updated] (YARN-6870) Fix resource availability check in

2017-07-28 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6870: -- Summary: Fix resource availability check in (was: Fix

[jira] [Updated] (YARN-6870) Fix AllocationBasedResourceUtilizationTracker::ResourceUtilization/ContainersMonitorImpl is calculating CPU utilization as a float, which is imprecise

2017-07-28 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6870: -- Summary: Fix AllocationBasedResourceUtilizationTracker::ResourceUtilization/ContainersMonitorImpl is

[jira] [Updated] (YARN-6870) Fix ResourceUtilization/ContainersMonitorImpl is calculating CPU utilization as a float, which is imprecise

2017-07-28 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6870: -- Summary: Fix ResourceUtilization/ContainersMonitorImpl is calculating CPU utilization as a float, which

[jira] [Commented] (YARN-6870) ResourceUtilization/ContainersMonitorImpl is calculating CPU utilization as a float, which is imprecise

2017-07-28 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105850#comment-16105850 ] Arun Suresh commented on YARN-6870: --- Thanks [~brookz]. will commit this shortly >

[jira] [Commented] (YARN-6870) ResourceUtilization/ContainersMonitorImpl is calculating CPU utilization as a float, which is imprecise

2017-07-28 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105398#comment-16105398 ] Arun Suresh commented on YARN-6870: --- +1 pending the javadoc fix >

[jira] [Commented] (YARN-6870) ResourceUtilization/ContainersMonitorImpl is calculating CPU utilization as a float, which is imprecise

2017-07-28 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105396#comment-16105396 ] Arun Suresh commented on YARN-6870: --- Thanks for the patch Brook, Looks like the javadoc issue is due to

[jira] [Comment Edited] (YARN-6594) [API] Introduce SchedulingRequest object

2017-07-27 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103986#comment-16103986 ] Arun Suresh edited comment on YARN-6594 at 7/27/17 10:01 PM: - Given that

[jira] [Commented] (YARN-6594) [API] Introduce SchedulingRequest object

2017-07-27 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103986#comment-16103986 ] Arun Suresh commented on YARN-6594: --- Given that YARN-6593 looks close to committing, Think we should move

[jira] [Comment Edited] (YARN-6870) ResourceUtilization/ContainersMonitorImpl is calculating CPU utilization as a float, which is imprecise

2017-07-26 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102371#comment-16102371 ] Arun Suresh edited comment on YARN-6870 at 7/27/17 2:22 AM: [~brookz], I think

[jira] [Commented] (YARN-6870) ResourceUtilization/ContainersMonitorImpl is calculating CPU utilization as a float, which is imprecise

2017-07-26 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102371#comment-16102371 ] Arun Suresh commented on YARN-6870: --- @Brook Zhou, I think we should start with addressing the bug in this

[jira] [Updated] (YARN-6870) ResourceUtilization/ContainersMonitorImpl is calculating CPU utilization as a float, which is imprecise

2017-07-25 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6870: -- Target Version/s: 2.9.0, 3.0.0-beta1 > ResourceUtilization/ContainersMonitorImpl is calculating CPU

[jira] [Commented] (YARN-6870) ResourceUtilization/ContainersMonitorImpl is calculating CPU utilization as a float, which is imprecise

2017-07-25 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100791#comment-16100791 ] Arun Suresh commented on YARN-6870: --- Thanks for raising this [~brookz]. Agreed, this can be a problem..

[jira] [Commented] (YARN-6858) Attribute Manager to store and provide the attributes in RM

2017-07-24 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098541#comment-16098541 ] Arun Suresh commented on YARN-6858: --- Thanks for raising these [~Naganarasimha], I was wondering if,

[jira] [Created] (YARN-6849) NMContainerStatus should have the Container ExecutionType.

2017-07-19 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-6849: - Summary: NMContainerStatus should have the Container ExecutionType. Key: YARN-6849 URL: https://issues.apache.org/jira/browse/YARN-6849 Project: Hadoop YARN Issue

[jira] [Created] (YARN-6848) Move Router ClientRMServices Interceptor and chain into yarn api and common package

2017-07-19 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-6848: - Summary: Move Router ClientRMServices Interceptor and chain into yarn api and common package Key: YARN-6848 URL: https://issues.apache.org/jira/browse/YARN-6848 Project:

[jira] [Commented] (YARN-6777) Support for ApplicationMasterService processing chain of interceptors

2017-07-19 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093556#comment-16093556 ] Arun Suresh commented on YARN-6777: --- The testcase failure is unrelated. > Support for

[jira] [Updated] (YARN-6777) Support for ApplicationMasterService processing chain of interceptors

2017-07-18 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6777: -- Attachment: YARN-6777.006.patch Updating javadocs. > Support for ApplicationMasterService processing

[jira] [Commented] (YARN-6777) Support for ApplicationMasterService processing chain of interceptors

2017-07-18 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092309#comment-16092309 ] Arun Suresh commented on YARN-6777: --- bq. is it OK to propagate the exception in getProcessorList.. Think

[jira] [Commented] (YARN-6831) Miscellaneous refactoring changes of ContainScheduler

2017-07-18 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091823#comment-16091823 ] Arun Suresh commented on YARN-6831: --- I was thinking about removing *maxOppQueueLength* which led me to

[jira] [Updated] (YARN-6777) Support for ApplicationMasterService processing chain of interceptors

2017-07-18 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6777: -- Attachment: YARN-6777.005.patch Updating patch with: # some javadoc fixes. # checkstyle fixes. # Also

[jira] [Updated] (YARN-6838) Add support to LinuxContainerExecutor to support container PAUSE

2017-07-18 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6838: -- Description: This JIRA tracks the changes needed to the {{LinuxContainerExecutor}},

[jira] [Updated] (YARN-6838) Add support to LinuxContainerExecutor to support container PAUSE

2017-07-18 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6838: -- Summary: Add support to LinuxContainerExecutor to support container PAUSE (was: Add support to linux

[jira] [Updated] (YARN-6838) Add support to linux container-executor to support container PAUSE

2017-07-18 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6838: -- Summary: Add support to linux container-executor to support container PAUSE (was: Add support to linux

[jira] [Updated] (YARN-6838) Add support to linux container-executor to support container freezing

2017-07-18 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6838: -- Description: This JIRA tracks the changes needed in the container-executor linux binary to support

[jira] [Created] (YARN-6838) Add support to linux container-executor to support container freezing

2017-07-18 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-6838: - Summary: Add support to linux container-executor to support container freezing Key: YARN-6838 URL: https://issues.apache.org/jira/browse/YARN-6838 Project: Hadoop YARN

[jira] [Updated] (YARN-6777) Support for ApplicationMasterService processing chain of interceptors

2017-07-17 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6777: -- Attachment: YARN-6777.004.patch Thanks for the rev [~subru]. Updated the patch based on your

[jira] [Commented] (YARN-6831) Miscellaneous refactoring changes of ContainScheduler

2017-07-17 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090832#comment-16090832 ] Arun Suresh commented on YARN-6831: --- Do take a look at YARN-6835 where i've posted an initial patch

[jira] [Updated] (YARN-6835) Remove runningContainers from ContainerScheduler

2017-07-17 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6835: -- Attachment: YARN-6835.001.patch Attaching initial patch > Remove runningContainers from

[jira] [Updated] (YARN-6835) Remove runningContainers from ContainerScheduler

2017-07-17 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6835: -- Priority: Minor (was: Major) > Remove runningContainers from ContainerScheduler >

[jira] [Created] (YARN-6835) Remove runningContainers from ContainerScheduler

2017-07-17 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-6835: - Summary: Remove runningContainers from ContainerScheduler Key: YARN-6835 URL: https://issues.apache.org/jira/browse/YARN-6835 Project: Hadoop YARN Issue Type:

[jira] [Comment Edited] (YARN-6831) Miscellaneous refactoring changes of ContainScheduler

2017-07-17 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090609#comment-16090609 ] Arun Suresh edited comment on YARN-6831 at 7/17/17 10:33 PM: - Thanks for

[jira] [Commented] (YARN-6831) Miscellaneous refactoring changes of ContainScheduler

2017-07-17 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090609#comment-16090609 ] Arun Suresh commented on YARN-6831: --- Thanks for raising this [~haibochen] / [~kasha]. Some thoughts: bq.

[jira] [Commented] (YARN-6706) Refactor ContainerScheduler to make oversubscription change easier

2017-07-17 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090580#comment-16090580 ] Arun Suresh commented on YARN-6706: --- [~kkaranasos].. Apologies, but think you send the message just as I

[jira] [Commented] (YARN-6706) Refactor ContainerScheduler to make oversubscription change easier

2017-07-17 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090544#comment-16090544 ] Arun Suresh commented on YARN-6706: --- Thanks Haibo.. +1 for the latest patch.. will check it in shortly >

[jira] [Updated] (YARN-6826) SLS NMSimulator support for Opportunistic Container Queuing

2017-07-17 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6826: -- Attachment: YARN-6826.001.patch Attaching initial patch > SLS NMSimulator support for Opportunistic

[jira] [Updated] (YARN-6826) SLS NMSimulator support for Opportunistic Container Queuing

2017-07-17 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6826: -- Description: Allow the NMSimulator to simulate Opportunistic containers. This Essentially means: #

[jira] [Updated] (YARN-6826) SLS NMSimulator support for Opportunistic Container Queuing

2017-07-15 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6826: -- Issue Type: Sub-task (was: Improvement) Parent: YARN-6828 > SLS NMSimulator support for

[jira] [Commented] (YARN-6828) [Umbrella] Container preemption using OPPORTUNISTIC containers

2017-07-15 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088662#comment-16088662 ] Arun Suresh commented on YARN-6828: --- Opening this as an umbrella JIRA as per [~leftnoteasy]'s request.

[jira] [Updated] (YARN-6828) [Umbrella] Container preemption using OPPORTUNISTIC containers

2017-07-15 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6828: -- Description: This is based on discussions with [~kasha] and [~kkaranasos]. Currently, the YARN

[jira] [Created] (YARN-6829) Promote Opportunistic Containers to Guaranteed containers when Guaranteed containers complete

2017-07-15 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-6829: - Summary: Promote Opportunistic Containers to Guaranteed containers when Guaranteed containers complete Key: YARN-6829 URL: https://issues.apache.org/jira/browse/YARN-6829

[jira] [Updated] (YARN-6828) [Umbrella] Container preemption using OPPORTUNISTIC containers

2017-07-15 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6828: -- Description: Currently, the YARN schedulers selects containers for preemption only in response to a

[jira] [Created] (YARN-6828) [Umbrella] Container preemption using OPPORTUNISTIC containers

2017-07-15 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-6828: - Summary: [Umbrella] Container preemption using OPPORTUNISTIC containers Key: YARN-6828 URL: https://issues.apache.org/jira/browse/YARN-6828 Project: Hadoop YARN

[jira] [Updated] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-15 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6808: -- Issue Type: Sub-task (was: New Feature) Parent: YARN-6828 > Allow Schedulers to return

[jira] [Commented] (YARN-6798) NM startup failure with old state store due to version mismatch

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088058#comment-16088058 ] Arun Suresh commented on YARN-6798: --- [~rchiang], I've committed YARN-5049 to branch-2 (cherry-picked and

[jira] [Updated] (YARN-5049) Extend NMStateStore to save queued container information

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5049: -- Fix Version/s: 2.9.0 > Extend NMStateStore to save queued container information >

[jira] [Resolved] (YARN-5049) Extend NMStateStore to save queued container information

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh resolved YARN-5049. --- Resolution: Fixed Committed to branch-2 as well > Extend NMStateStore to save queued container

[jira] [Assigned] (YARN-5049) Extend NMStateStore to save queued container information

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reassigned YARN-5049: - Assignee: Konstantinos Karanasos (was: Arun Suresh) > Extend NMStateStore to save queued

[jira] [Reopened] (YARN-5049) Extend NMStateStore to save queued container information

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reopened YARN-5049: --- Assignee: Arun Suresh (was: Konstantinos Karanasos) > Extend NMStateStore to save queued container

[jira] [Commented] (YARN-5049) Extend NMStateStore to save queued container information

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087964#comment-16087964 ] Arun Suresh commented on YARN-5049: --- Re-opening for branch-2 commit > Extend NMStateStore to save queued

[jira] [Commented] (YARN-6798) NM startup failure with old state store due to version mismatch

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087965#comment-16087965 ] Arun Suresh commented on YARN-6798: --- I had rolled back YARN-5049 from branch-2 precisely because of the

[jira] [Commented] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087835#comment-16087835 ] Arun Suresh commented on YARN-6808: --- [~leftnoteasy], bq. Currently, opportunistic container allocation

[jira] [Updated] (YARN-6826) SLS NMSimulator support for Opportunistic Container Queuing

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6826: -- Issue Type: Improvement (was: Bug) > SLS NMSimulator support for Opportunistic Container Queuing >

[jira] [Updated] (YARN-6826) SLS NMSimulator support for Opportunistic Container Queuing

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6826: -- Summary: SLS NMSimulator support for Opportunistic Container Queuing (was: SLS NMSimulator support for

[jira] [Created] (YARN-6826) SLS NMSimulator support for Opportunistic Container Queuing.

2017-07-14 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-6826: - Summary: SLS NMSimulator support for Opportunistic Container Queuing. Key: YARN-6826 URL: https://issues.apache.org/jira/browse/YARN-6826 Project: Hadoop YARN

[jira] [Comment Edited] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087500#comment-16087500 ] Arun Suresh edited comment on YARN-6808 at 7/14/17 3:55 PM: [~leftnoteasy],

[jira] [Comment Edited] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087500#comment-16087500 ] Arun Suresh edited comment on YARN-6808 at 7/14/17 3:50 PM: [~leftnoteasy],

[jira] [Comment Edited] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087500#comment-16087500 ] Arun Suresh edited comment on YARN-6808 at 7/14/17 3:49 PM: [~leftnoteasy],

[jira] [Updated] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6808: -- Attachment: YARN-6808.002.patch Attaching v002. > Allow Schedulers to return OPPORTUNISTIC containers

[jira] [Commented] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087500#comment-16087500 ] Arun Suresh commented on YARN-6808: --- [~leftnoteasy], good questions.. bq. Use opportunistic container to

[jira] [Comment Edited] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-12 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085184#comment-16085184 ] Arun Suresh edited comment on YARN-6808 at 7/13/17 5:03 AM: [~haibochen], the

[jira] [Commented] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-12 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085184#comment-16085184 ] Arun Suresh commented on YARN-6808: --- [~haibochen], the plan to use it to provide stricter capacity

[jira] [Comment Edited] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-12 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085035#comment-16085035 ] Arun Suresh edited comment on YARN-6808 at 7/13/17 1:52 AM: [~leftnoteasy],

[jira] [Commented] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-12 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085035#comment-16085035 ] Arun Suresh commented on YARN-6808: --- [~leftnoteasy], Thanks for taking a look. This patch basically

[jira] [Comment Edited] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-12 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084955#comment-16084955 ] Arun Suresh edited comment on YARN-6808 at 7/13/17 1:20 AM: Attaching initial

[jira] [Assigned] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-12 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reassigned YARN-6808: - Assignee: Arun Suresh > Allow Schedulers to return OPPORTUNISTIC containers when queues go over

[jira] [Updated] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-12 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6808: -- Attachment: YARN-6808.001.patch Attaching initial patch to vet the idea. * Currently, only the

[jira] [Commented] (YARN-6706) Refactor ContainerScheduler to make oversubscription change easier

2017-07-12 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083931#comment-16083931 ] Arun Suresh commented on YARN-6706: --- Agreed... lets discuss this in a follow up JIRA. > Refactor

[jira] [Updated] (YARN-5292) NM Container lifecycle and state transitions to support for PAUSED container state.

2017-07-11 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5292: -- Attachment: YARN-5292.006.patch > NM Container lifecycle and state transitions to support for PAUSED

[jira] [Reopened] (YARN-5292) NM Container lifecycle and state transitions to support for PAUSED container state.

2017-07-11 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reopened YARN-5292: --- Re-opening to cherry-pick and run Jenkins on trunk > NM Container lifecycle and state transitions to

[jira] [Created] (YARN-6808) Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity

2017-07-11 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-6808: - Summary: Allow Schedulers to return OPPORTUNISTIC containers when queues go over configured capacity Key: YARN-6808 URL: https://issues.apache.org/jira/browse/YARN-6808

[jira] [Commented] (YARN-6706) Refactor ContainerScheduler to make oversubscription change easier

2017-07-11 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083261#comment-16083261 ] Arun Suresh commented on YARN-6706: --- bq. You are referring to the else statement in

[jira] [Commented] (YARN-6798) NM startup failure with old state store due to version mismatch

2017-07-11 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083034#comment-16083034 ] Arun Suresh commented on YARN-6798: --- I agree with Jason. To move forward, I suggest we follow YARN-6703

[jira] [Commented] (YARN-6594) [API] Introduce SchedulingRequest object

2017-07-11 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081745#comment-16081745 ] Arun Suresh commented on YARN-6594: --- So the reason I feel retaining the {{ContainerRequest}} is a good

[jira] [Commented] (YARN-6594) [API] Introduce SchedulingRequest object

2017-07-10 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081225#comment-16081225 ] Arun Suresh commented on YARN-6594: --- I agree.. AMRMClient is starting to get very heavy. But

[jira] [Updated] (YARN-6776) Refactor ApplicaitonMasterService to move actual processing logic to a separate class

2017-07-10 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6776: -- Fix Version/s: 3.0.0-beta1 > Refactor ApplicaitonMasterService to move actual processing logic to a >

[jira] [Commented] (YARN-6776) Refactor ApplicaitonMasterService to move actual processing logic to a separate class

2017-07-10 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081194#comment-16081194 ] Arun Suresh commented on YARN-6776: --- Thanks for the rev Subru.. Committing this shortly > Refactor

[jira] [Commented] (YARN-6706) Refactor ContainerScheduler to make oversubscription change easier

2017-07-10 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081132#comment-16081132 ] Arun Suresh commented on YARN-6706: --- Yup. Agree this can go into trunk. The patch generally looks good.

[jira] [Commented] (YARN-6794) promote OPPORTUNISITC containers locally at the node where they're running

2017-07-10 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081112#comment-16081112 ] Arun Suresh commented on YARN-6794: --- [~haibo.chen] / [~kasha] I expect some overlap between this and

[jira] [Updated] (YARN-6776) Refactor ApplicaitonMasterService to move actual processing logic to a separate class

2017-07-10 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6776: -- Attachment: YARN-6776.004.patch Updating patch to fix unused import. The testcase failures are

[jira] [Updated] (YARN-6355) Preprocessor framework for AM and Client interactions with the RM

2017-07-09 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6355: -- Attachment: YARN-6355.007.patch > Preprocessor framework for AM and Client interactions with the RM >

[jira] [Updated] (YARN-6777) Support for ApplicationMasterService processing chain of interceptors

2017-07-09 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6777: -- Attachment: YARN-6777.003.patch * Fixed some javadoc comments * Updated the API to pass the response in

[jira] [Updated] (YARN-6777) Support for ApplicationMasterService processing chain of interceptors

2017-07-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6777: -- Attachment: YARN-6777.002.patch Updating patch fix checkstyle and findbugs issues uncovered in YARN-6355

[jira] [Updated] (YARN-6355) Preprocessor framework for AM and Client interactions with the RM

2017-07-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6355: -- Attachment: YARN-6355.006.patch > Preprocessor framework for AM and Client interactions with the RM >

[jira] [Updated] (YARN-6776) Refactor ApplicaitonMasterService to move actual processing logic to a separate class

2017-07-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6776: -- Attachment: YARN-6776.003.patch Fixing testcase and remaining checkstyle. > Refactor

<    5   6   7   8   9   10   11   12   13   14   >