[jira] [Updated] (YARN-4270) Limit application resource reservation on nodes for non-node/rack specific requests

2015-10-19 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4270: -- Issue Type: Bug (was: Improvement) > Limit application resource reservation on nodes for non-node/rack s

[jira] [Updated] (YARN-4270) Limit application resource reservation on nodes for non-node/rack specific requests

2015-10-19 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4270: -- Issue Type: Improvement (was: Bug) > Limit application resource reservation on nodes for non-node/rack s

[jira] [Updated] (YARN-4270) Limit application resource reservation on nodes for non-node/rack specific requests

2015-10-19 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4270: -- Fix Version/s: 2.8.0 > Limit application resource reservation on nodes for non-node/rack specific > requ

[jira] [Commented] (YARN-3738) Add support for recovery of reserved apps (running under dynamic queues) to Capacity Scheduler

2015-10-24 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973046#comment-14973046 ] Arun Suresh commented on YARN-3738: --- I agree that not much can be done about the checksty

[jira] [Updated] (YARN-3738) Add support for recovery of reserved apps running under dynamic queues

2015-10-24 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3738: -- Summary: Add support for recovery of reserved apps running under dynamic queues (was: Add support for re

[jira] [Commented] (YARN-3738) Add support for recovery of reserved apps running under dynamic queues

2015-10-24 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973052#comment-14973052 ] Arun Suresh commented on YARN-3738: --- Committed this to trunk and branch-2, Thanks for the

[jira] [Resolved] (YARN-3737) Add support for recovery of reserved apps (running under dynamic queues) to Fair Scheduler

2015-10-24 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh resolved YARN-3737. --- Resolution: Done Fix Version/s: 2.8.0 > Add support for recovery of reserved apps (running under

[jira] [Created] (YARN-4310) Reduce log level for certain messages to prevent overrunning log file

2015-10-28 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-4310: - Summary: Reduce log level for certain messages to prevent overrunning log file Key: YARN-4310 URL: https://issues.apache.org/jira/browse/YARN-4310 Project: Hadoop YARN

[jira] [Updated] (YARN-4310) Reduce log level for certain messages to prevent overrunning log file

2015-10-28 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4310: -- Attachment: YARN-4310.1.patch Attaching trivial patch > Reduce log level for certain messages to prevent

[jira] [Updated] (YARN-4310) FairScheduler: Log skipping reservation messages at DEBUG level

2015-10-28 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4310: -- Attachment: YARN-4310.2.patch [~kasha] thanks for the review... updating patch > FairScheduler: Log skip

[jira] [Updated] (YARN-4310) FairScheduler: Log skipping reservation messages at DEBUG level

2015-10-29 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4310: -- Fix Version/s: 2.8.0 > FairScheduler: Log skipping reservation messages at DEBUG level >

[jira] [Commented] (YARN-2882) Introducing container types

2015-11-04 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14990881#comment-14990881 ] Arun Suresh commented on YARN-2882: --- Thanks for the patch [~kkaranasos]. The patch looks

[jira] [Assigned] (YARN-2885) LocalRM: distributed scheduling decisions for queueable containers

2015-11-04 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reassigned YARN-2885: - Assignee: Arun Suresh > LocalRM: distributed scheduling decisions for queueable containers > -

[jira] [Commented] (YARN-4184) Remove update reservation state api from state store as its not used by ReservationSystem

2015-11-12 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15002881#comment-15002881 ] Arun Suresh commented on YARN-4184: --- Thanks for the patch [~seanpo03]. it looks fine.. +1

[jira] [Commented] (YARN-3454) RLESparseResourceAllocation does not handle removal of partial intervals (+ introducing support for efficient "merge" operations)

2015-11-17 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009846#comment-15009846 ] Arun Suresh commented on YARN-3454: --- Thanks for the patch [~curino]. Took a pass at the

[jira] [Commented] (YARN-3454) RLESparseResourceAllocation does not handle removal of partial intervals (+ introducing support for efficient "merge" operations)

2015-11-17 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009886#comment-15009886 ] Arun Suresh commented on YARN-3454: --- Other minor nits: # You should probably use {{Resour

[jira] [Commented] (YARN-3454) RLESparseResourceAllocation does not handle removal of partial intervals (+ introducing support for efficient "merge" operations)

2015-11-18 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15012511#comment-15012511 ] Arun Suresh commented on YARN-3454: --- Thanks for the rev [~curino]. +1 pending jenkins >

[jira] [Updated] (YARN-3454) Add efficient merge operation to RLESparseResourceAllocation

2015-11-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3454: -- Summary: Add efficient merge operation to RLESparseResourceAllocation (was: RLESparseResourceAllocation

[jira] [Updated] (YARN-3454) Add efficient merge operation to RLESparseResourceAllocation

2015-11-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3454: -- Fix Version/s: 2.8.0 > Add efficient merge operation to RLESparseResourceAllocation > ---

[jira] [Commented] (YARN-4358) Improve relationship between SharingPolicy and ReservationAgent

2015-11-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15020611#comment-15020611 ] Arun Suresh commented on YARN-4358: --- Thanks for the patch [~curino]. The patch looks good

[jira] [Commented] (YARN-4378) FairScheduler could not support DRF policy in lead queue when parent queue is fair policy

2015-11-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15020657#comment-15020657 ] Arun Suresh commented on YARN-4378: --- This case is handled by YARN-4212 > FairScheduler c

[jira] [Resolved] (YARN-4378) FairScheduler could not support DRF policy in lead queue when parent queue is fair policy

2015-11-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh resolved YARN-4378. --- Resolution: Duplicate > FairScheduler could not support DRF policy in lead queue when parent queue is

[jira] [Commented] (YARN-4358) Improve relationship between SharingPolicy and ReservationAgent

2015-11-26 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15029125#comment-15029125 ] Arun Suresh commented on YARN-4358: --- Thanks for the rev [~curino]. The changes look fine.

[jira] [Resolved] (YARN-4335) Introducing resource request types

2015-11-26 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh resolved YARN-4335. --- Resolution: Duplicate > Introducing resource request types > -- > >

[jira] [Commented] (YARN-4335) Introducing resource request types

2015-11-26 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15029214#comment-15029214 ] Arun Suresh commented on YARN-4335: --- Marking this as Duplicate since the YARN-2882 patch

[jira] [Updated] (YARN-2882) Add ExecutionType to denote if a container execution is GUARANTEED or QUEUEABLE

2015-11-26 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-2882: -- Summary: Add ExecutionType to denote if a container execution is GUARANTEED or QUEUEABLE (was: Introduci

[jira] [Updated] (YARN-4335) Allow ResourceRequests to specify ExecutionType of a request ask

2015-11-26 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4335: -- Summary: Allow ResourceRequests to specify ExecutionType of a request ask (was: Introducing resource req

[jira] [Updated] (YARN-2882) Add ExecutionType to denote if a container execution is GUARANTEED or QUEUEABLE

2015-11-26 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-2882: -- Attachment: YARN-2882-yarn-2877.002.patch Updating patch to fix test cases. [~ka...@cloudera.com], [~left

[jira] [Updated] (YARN-2885) Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers

2015-11-30 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-2885: -- Summary: Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containe

[jira] [Updated] (YARN-2885) Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers

2015-11-30 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-2885: -- Attachment: YARN-2885-yarn-2877.001.patch Uploading a preliminary version of the patch.. # This depends

[jira] [Created] (YARN-4412) Create ClusterManager to compute ordered list of preferred NMs for QUEUEABLE containers

2015-12-02 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-4412: - Summary: Create ClusterManager to compute ordered list of preferred NMs for QUEUEABLE containers Key: YARN-4412 URL: https://issues.apache.org/jira/browse/YARN-4412 Project

[jira] [Commented] (YARN-2885) Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers

2015-12-02 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037386#comment-15037386 ] Arun Suresh commented on YARN-2885: --- Thank you for the review [~leftnoteasy] !! let me tr

[jira] [Updated] (YARN-2885) Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers

2015-12-05 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-2885: -- Attachment: YARN-2885_api_changes.patch Attaching the API changes as a separate patch. As suggested by [

[jira] [Updated] (YARN-2885) Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers

2015-12-05 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-2885: -- Attachment: YARN-2885-yarn-2877.002.patch Attaching v002 of the complete patch (This also includes the AP

[jira] [Commented] (YARN-2882) Add ExecutionType to denote if a container execution is GUARANTEED or QUEUEABLE

2015-12-05 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15043621#comment-15043621 ] Arun Suresh commented on YARN-2882: --- [~leftnoteasy], is it possible to give this one a re

[jira] [Updated] (YARN-2885) Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers

2015-12-05 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-2885: -- Attachment: YARN-2885-yarn-2877.full.patch Uploading a combined patch (YARN-2882 + YARN-2885) to kick off

[jira] [Updated] (YARN-2885) Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers

2015-12-06 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-2885: -- Attachment: YARN-2885-yarn-2877.full-2.patch Updating patch: # The earlier patch had changed the contain

[jira] [Commented] (YARN-2885) Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers

2015-12-07 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046199#comment-15046199 ] Arun Suresh commented on YARN-2885: --- Thanks for the review [~leftnoteasy].. I understand

[jira] [Commented] (YARN-110) AM releases too many containers due to the protocol

2015-12-09 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15049301#comment-15049301 ] Arun Suresh commented on YARN-110: -- [~ka...@cloudera.com], [~vinodkv], I understand from MA

[jira] [Updated] (YARN-4358) Improve relationship between SharingPolicy and ReservationAgent

2015-12-09 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4358: -- Attachment: YARN-4358.addendum.patch > Improve relationship between SharingPolicy and ReservationAgent >

[jira] [Updated] (YARN-4358) Improve relationship between SharingPolicy and ReservationAgent

2015-12-10 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4358: -- Attachment: YARN-4358.addendum-2.patch Updated based on [~subru]'s suggestion > Improve relationship bet

[jira] [Updated] (YARN-4358) Improve relationship between SharingPolicy and ReservationAgent

2015-12-12 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4358: -- Attachment: mvn_package_dist.txt Thanks for the review [~ajisakaa]. Unfortunately, I missed the *pending

[jira] [Updated] (YARN-2885) Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers

2015-12-13 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-2885: -- Attachment: YARN-2885-yarn-2877.full-3.patch As per [~leftnoteasy]'s suggestion, updating patch to move t

[jira] [Updated] (YARN-3547) FairScheduler: Apps that have no resource demand should not participate scheduling

2015-05-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3547: -- Labels: (was: BB2015-05-TBR) > FairScheduler: Apps that have no resource demand should not participate

[jira] [Updated] (YARN-3547) FairScheduler: Apps that have no resource demand should not participate scheduling

2015-05-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3547: -- Labels: BB2015-05-TBR (was: ) > FairScheduler: Apps that have no resource demand should not participate

[jira] [Updated] (YARN-3395) [Fair Scheduler] Handle the user name correctly when user name is used as default queue name.

2015-05-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3395: -- Labels: (was: BB2015-05-TBR) > [Fair Scheduler] Handle the user name correctly when user name is used a

[jira] [Commented] (YARN-3395) [Fair Scheduler] Handle the user name correctly when user name is used as default queue name.

2015-05-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535070#comment-14535070 ] Arun Suresh commented on YARN-3395: --- Thanks for the patch [~zxu], I have just a minor nit

[jira] [Updated] (YARN-2684) FairScheduler should tolerate queue configuration changes across RM restarts

2015-05-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-2684: -- Labels: (was: BB2015-05-TBR) > FairScheduler should tolerate queue configuration changes across RM rest

[jira] [Updated] (YARN-2684) FairScheduler should tolerate queue configuration changes across RM restarts

2015-05-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-2684: -- Labels: BB2015-05-TBR (was: ) > FairScheduler should tolerate queue configuration changes across RM rest

[jira] [Updated] (YARN-2684) FairScheduler should if queue is removed during app recovery

2015-05-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-2684: -- Summary: FairScheduler should if queue is removed during app recovery (was: FairScheduler should tolerat

[jira] [Updated] (YARN-2684) FairScheduler should fast-fail if queue is removed during app recovery

2015-05-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-2684: -- Summary: FairScheduler should fast-fail if queue is removed during app recovery (was: FairScheduler shou

[jira] [Updated] (YARN-1297) Miscellaneous Fair Scheduler speedups

2015-05-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-1297: -- Attachment: YARN-1297.3.patch Updating patch.. fixing it up to so it rebases to trunk > Miscellaneous Fa

[jira] [Commented] (YARN-3395) [Fair Scheduler] Handle the user name correctly when user name is used as default queue name.

2015-05-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535653#comment-14535653 ] Arun Suresh commented on YARN-3395: --- +1, Will commit after Jenkins > [Fair Scheduler] Ha

[jira] [Updated] (YARN-1297) Miscellaneous Fair Scheduler speedups

2015-05-11 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-1297: -- Attachment: YARN-1297.4.patch Updating patch to fix the test failure * Had missed accounting for app con

[jira] [Updated] (YARN-1297) Miscellaneous Fair Scheduler speedups

2015-05-12 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-1297: -- Labels: (was: BB2015-05-TBR) > Miscellaneous Fair Scheduler speedups >

[jira] [Updated] (YARN-1297) Miscellaneous Fair Scheduler speedups

2015-05-12 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-1297: -- Attachment: YARN-1297.4.patch attaching again to kick off jenkins > Miscellaneous Fair Scheduler speedup

[jira] [Commented] (YARN-3655) FairScheduler: potential livelock due to maxAMShare limitation and container reservation

2015-05-18 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549547#comment-14549547 ] Arun Suresh commented on YARN-3655: --- Thanks for the patch [~zxu], I was just wondering t

[jira] [Created] (YARN-3676) Disregard 'assignMultiple' directive while scheduling apps with NODE_LOCAL resource requests

2015-05-18 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-3676: - Summary: Disregard 'assignMultiple' directive while scheduling apps with NODE_LOCAL resource requests Key: YARN-3676 URL: https://issues.apache.org/jira/browse/YARN-3676 Pr

[jira] [Commented] (YARN-3633) With Fair Scheduler, cluster can logjam when there are too many queues

2015-05-19 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549949#comment-14549949 ] Arun Suresh commented on YARN-3633: --- Thanks for the patch [~ragarwal], Assuming we allow,

[jira] [Assigned] (YARN-3691) Limit number of reservations for an app

2015-05-20 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reassigned YARN-3691: - Assignee: Arun Suresh > Limit number of reservations for an app >

[jira] [Created] (YARN-3691) Limit number of reservations for an app

2015-05-20 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-3691: - Summary: Limit number of reservations for an app Key: YARN-3691 URL: https://issues.apache.org/jira/browse/YARN-3691 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-3676) Disregard 'assignMultiple' directive while scheduling apps with NODE_LOCAL resource requests

2015-05-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3676: -- Attachment: YARN-3676.1.patch Attaching initial patch to validate approach.. will include test-case short

[jira] [Updated] (YARN-3676) Disregard 'assignMultiple' directive while scheduling apps with NODE_LOCAL resource requests

2015-05-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3676: -- Attachment: YARN-3676.2.patch Reattaching patch. > Disregard 'assignMultiple' directive while scheduling

[jira] [Commented] (YARN-3655) FairScheduler: potential livelock due to maxAMShare limitation and container reservation

2015-05-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14554745#comment-14554745 ] Arun Suresh commented on YARN-3655: --- Thanks for the update [~zxu]. I have only one nit :

[jira] [Commented] (YARN-3675) FairScheduler: RM quits when node removal races with continousscheduling on the same node

2015-05-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14554814#comment-14554814 ] Arun Suresh commented on YARN-3675: --- Thanks for the patch [~adhoot], +1, LGTM > FairS

[jira] [Updated] (YARN-3676) Disregard 'assignMultiple' directive while scheduling apps with NODE_LOCAL resource requests

2015-05-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3676: -- Attachment: YARN-3676.3.patch Updating patch with testcase > Disregard 'assignMultiple' directive while

[jira] [Commented] (YARN-3655) FairScheduler: potential livelock due to maxAMShare limitation and container reservation

2015-05-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1466#comment-1466 ] Arun Suresh commented on YARN-3655: --- makes sense... +1 from me.. will commit, unless [~ka

[jira] [Updated] (YARN-3676) Disregard 'assignMultiple' directive while scheduling apps with NODE_LOCAL resource requests

2015-05-22 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3676: -- Attachment: YARN-3676.4.patch Updating patch * Changed the accounting logic a bit * Improved test case *

[jira] [Updated] (YARN-3676) Disregard 'assignMultiple' directive while scheduling apps with NODE_LOCAL resource requests

2015-05-22 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3676: -- Attachment: YARN-3676.5.patch > Disregard 'assignMultiple' directive while scheduling apps with NODE_LOCA

[jira] [Assigned] (YARN-3453) Fair Scheduler : Parts of preemption logic uses DefaultResourceCalculator even in DRF mode causing thrashing

2015-06-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh reassigned YARN-3453: - Assignee: Arun Suresh > Fair Scheduler : Parts of preemption logic uses DefaultResourceCalculator

[jira] [Updated] (YARN-3453) Fair Scheduler : Parts of preemption logic uses DefaultResourceCalculator even in DRF mode causing thrashing

2015-06-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3453: -- Attachment: YARN-3453.1.patch [~peng.zhang], [~ashwinshankar77], Thank you reporting this.. and the assoc

[jira] [Commented] (YARN-3762) FairScheduler: CME on FSParentQueue#getQueueUserAclInfo

2015-06-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571409#comment-14571409 ] Arun Suresh commented on YARN-3762: --- [~kasha], Thanks for the patch * In the {{assignCon

[jira] [Commented] (YARN-3762) FairScheduler: CME on FSParentQueue#getQueueUserAclInfo

2015-06-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571512#comment-14571512 ] Arun Suresh commented on YARN-3762: --- Makes sense +1, LGTM > FairScheduler: CME on FSPare

[jira] [Updated] (YARN-3453) Fair Scheduler : Parts of preemption logic uses DefaultResourceCalculator even in DRF mode causing thrashing

2015-06-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3453: -- Attachment: YARN-3453.2.patch Agreed.. Updating patch with your suggestion.. > Fair Scheduler : Parts o

[jira] [Updated] (YARN-5180) Add ensureExecutionType boolean flag in ResourceRequest

2016-05-31 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5180: -- Attachment: YARN-5180.002.patch Updating patch. Based on discussions with [~kasha], Replaced the {{Execu

[jira] [Updated] (YARN-5180) Add ensureExecutionType boolean flag in ResourceRequest

2016-05-31 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5180: -- Attachment: YARN-5180.003.patch Updating patch addressing review comments suggestions. Thanks [~kkaranaso

[jira] [Comment Edited] (YARN-5180) Add ensureExecutionType boolean flag in ResourceRequest

2016-05-31 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15309325#comment-15309325 ] Arun Suresh edited comment on YARN-5180 at 6/1/16 5:51 AM: --- Updat

[jira] [Commented] (YARN-5165) NoOvercommitPolicy does not take advantage of RLE representation of plan

2016-06-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310272#comment-15310272 ] Arun Suresh commented on YARN-5165: --- [~curino], thanks for the patch.. the fix looks pret

[jira] [Commented] (YARN-5185) StageAllocaterGreedyRLE: NPE in corner case

2016-06-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310278#comment-15310278 ] Arun Suresh commented on YARN-5185: --- [~curino], The patch looks pretty straightforward. I

[jira] [Updated] (YARN-5124) Modify AMRMClient to set the ExecutionType in the ResourceRequest

2016-06-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5124: -- Attachment: YARN-5124_YARN-5180_combined.007.patch Uploading new patch rebased against YARN-5180. This is

[jira] [Updated] (YARN-5180) Add ensureExecutionType boolean flag in ResourceRequest

2016-06-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5180: -- Attachment: YARN-5180.004.patch Uploading a patch to fix the {{TestPBImplRecords}} failure. The rest as

[jira] [Comment Edited] (YARN-5180) Add ensureExecutionType boolean flag in ResourceRequest

2016-06-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310567#comment-15310567 ] Arun Suresh edited comment on YARN-5180 at 6/1/16 4:22 PM: --- Uploa

[jira] [Updated] (YARN-5180) Add ensureExecutionType boolean flag in ResourceRequest

2016-06-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5180: -- Attachment: YARN-5180.005.patch Updating patch.. thanks for the rev [~kasha].. bq. Can we get #2 of my p

[jira] [Commented] (YARN-5180) Add ensureExecutionType boolean flag in ResourceRequest

2016-06-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311392#comment-15311392 ] Arun Suresh commented on YARN-5180: --- [~kkaranasos], bq. Thinking more about it, do we wan

[jira] [Commented] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311515#comment-15311515 ] Arun Suresh commented on YARN-5171: --- YARN-5127 exposes the ExecutionType in the Container

[jira] [Commented] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311530#comment-15311530 ] Arun Suresh commented on YARN-5171: --- Thanks for the patch [~elgoiri].. The general appro

[jira] [Comment Edited] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311530#comment-15311530 ] Arun Suresh edited comment on YARN-5171 at 6/2/16 1:10 AM: --- Thank

[jira] [Commented] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311546#comment-15311546 ] Arun Suresh commented on YARN-5171: --- Actually... regarding the last part of my point #2 a

[jira] [Updated] (YARN-5180) Add ensureExecutionType boolean flag in ResourceRequest

2016-06-01 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5180: -- Attachment: YARN-5180.006.patch Updating patch with javadoc changes Konstantinos suggested. Like I menti

[jira] [Updated] (YARN-5180) Add ensureExecutionType boolean flag in ResourceRequest

2016-06-02 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5180: -- Attachment: YARN-5180.007.patch Thanks [~kasha] Oh well.. as per popular demand, changing the name to 'e

[jira] [Updated] (YARN-5124) Modify AMRMClient to set the ExecutionType in the ResourceRequest

2016-06-02 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5124: -- Attachment: YARN-5124_YARN-5180_combined.008.patch Rebasing against latest YARN-5180 patch and kicking of

[jira] [Updated] (YARN-5124) Modify AMRMClient to set the ExecutionType in the ResourceRequest

2016-06-02 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5124: -- Attachment: YARN-5124.008.patch > Modify AMRMClient to set the ExecutionType in the ResourceRequest > ---

[jira] [Updated] (YARN-5180) Allow ResourceRequest to specify enforceExecutionType flag

2016-06-02 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5180: -- Summary: Allow ResourceRequest to specify enforceExecutionType flag (was: Add ensureExecutionType boolea

[jira] [Updated] (YARN-5180) Allow ResourceRequest to specify an enforceExecutionType flag

2016-06-02 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5180: -- Summary: Allow ResourceRequest to specify an enforceExecutionType flag (was: Allow ResourceRequest to sp

[jira] [Updated] (YARN-5180) Allow ResourceRequest to specify an enforceExecutionType flag

2016-06-02 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5180: -- Issue Type: Sub-task (was: Improvement) Parent: YARN-4742 > Allow ResourceRequest to specify an

[jira] [Commented] (YARN-5180) Allow ResourceRequest to specify an enforceExecutionType flag

2016-06-02 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312559#comment-15312559 ] Arun Suresh commented on YARN-5180: --- Committed this to trunk and branch-2 Thanks [~kkara

[jira] [Commented] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-02 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312630#comment-15312630 ] Arun Suresh commented on YARN-5171: --- [~elgoiri], I don't think you are decrementing {{at

[jira] [Commented] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-02 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313313#comment-15313313 ] Arun Suresh commented on YARN-5171: --- bq. why not call the existing allocate method in the

[jira] [Commented] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313859#comment-15313859 ] Arun Suresh commented on YARN-5171: --- [~elgoiri], Couple of comments, # I don't think the

[jira] [Commented] (YARN-5165) NoOvercommitPolicy does not take advantage of RLE representation of plan

2016-06-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314876#comment-15314876 ] Arun Suresh commented on YARN-5165: --- +1 for the latest patch. Thanks [~curino] and [~subr

<    1   2   3   4   5   6   7   8   9   10   >