[jira] [Created] (YARN-4420) Add REST API for List Reservations
Sean Po created YARN-4420: - Summary: Add REST API for List Reservations Key: YARN-4420 URL: https://issues.apache.org/jira/browse/YARN-4420 Project: Hadoop YARN Issue Type: Bug Components: capacity scheduler, fairscheduler, resourcemanager Reporter: Sean Po Assignee: Sean Po Priority: Minor This JIRA tracks changes to the REST APIs of the reservation system and enables querying the reservation on which reservations exists by "time-range, and reservation-id". This task has a dependency on YARN-4340. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-4683) Document the List Reservations REST API
Sean Po created YARN-4683: - Summary: Document the List Reservations REST API Key: YARN-4683 URL: https://issues.apache.org/jira/browse/YARN-4683 Project: Hadoop YARN Issue Type: Task Reporter: Sean Po -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-4687) Document Reservation ACLs
Sean Po created YARN-4687: - Summary: Document Reservation ACLs Key: YARN-4687 URL: https://issues.apache.org/jira/browse/YARN-4687 Project: Hadoop YARN Issue Type: Task Reporter: Sean Po Assignee: Sean Po Priority: Minor Related to YARN-2575 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-5435) User AM resource limit for does not get updated for ReservationQueue after execution of plan-follower.
Sean Po created YARN-5435: - Summary: User AM resource limit for does not get updated for ReservationQueue after execution of plan-follower. Key: YARN-5435 URL: https://issues.apache.org/jira/browse/YARN-5435 Project: Hadoop YARN Issue Type: Bug Components: capacity scheduler, resourcemanager Affects Versions: 2.8.0 Reporter: Sean Po Assignee: Sean Po After a reservation queue is allocated, and the plan is synchronized by the plan follower, we expect the user am resource limit to reflect this change if the appropriate configuration is set. Instead, the user am resource limit is always the same. As a result, multiple AM cannot run in parallel for small reservations. To reproduce this issue, create a reservation, and submit multiple applications against the new reservation. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-5384) Expose priority in ReservationSystem submission APIs
Sean Po created YARN-5384: - Summary: Expose priority in ReservationSystem submission APIs Key: YARN-5384 URL: https://issues.apache.org/jira/browse/YARN-5384 Project: Hadoop YARN Issue Type: Task Components: capacity scheduler, fairscheduler, resourcemanager Reporter: Sean Po Assignee: Sean Po -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-5385) Add a PriorityAgent in ReservationSystem
Sean Po created YARN-5385: - Summary: Add a PriorityAgent in ReservationSystem Key: YARN-5385 URL: https://issues.apache.org/jira/browse/YARN-5385 Project: Hadoop YARN Issue Type: Sub-task Components: capacity scheduler, fairscheduler, resourcemanager Reporter: Sean Po Assignee: Sean Po -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-5386) Add a priority-aware Replanner
Sean Po created YARN-5386: - Summary: Add a priority-aware Replanner Key: YARN-5386 URL: https://issues.apache.org/jira/browse/YARN-5386 Project: Hadoop YARN Issue Type: Sub-task Components: capacity scheduler, fairscheduler, resourcemanager Reporter: Sean Po Assignee: Sean Po -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6226) Improve concurrency model in PriorityReservationAgent
Sean Po created YARN-6226: - Summary: Improve concurrency model in PriorityReservationAgent Key: YARN-6226 URL: https://issues.apache.org/jira/browse/YARN-6226 Project: Hadoop YARN Issue Type: Task Components: resourcemanager Reporter: Sean Po Assignee: Sean Po YARN-5385 introduces a PriorityReservationAgent that can be used to yield lower priority reservations so that higher priority reservations may fit if it initially could not be created in the plan. In YARN-5385, a write lock was used in the createReservation, updateReservation and deleteReservation methods to ensure that multi-threaded access will not be a problem for the new agent. This solution may be problematic if many multi-threaded reservation operations are made in a short period of time due to the heaviness of the createReservation and updateReservation methods. This JIRA aims to improve the concurrency model of the PriorityReservationAgent by replacing the write lock with a more suitable solution. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6227) Information about reservations that are yielded by higher priority reservations need to be surfaced to the client
Sean Po created YARN-6227: - Summary: Information about reservations that are yielded by higher priority reservations need to be surfaced to the client Key: YARN-6227 URL: https://issues.apache.org/jira/browse/YARN-6227 Project: Hadoop YARN Issue Type: Task Reporter: Sean Po YARN-5385 introduces a PriorityReservationAgent that can be used to yield lower priority reservations so that higher priority reservations may fit if it initially could not be created in the plan. In the event that a low priority reservation is yielded, the client does not have a way to know prior to using the reservation that it has been removed. This JIRA aims to surface such information to the client, so that it may be able to act appropriately to its lost reservation. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6249) TestFairSchedulerPreemption is inconsistently failing on trunk
Sean Po created YARN-6249: - Summary: TestFairSchedulerPreemption is inconsistently failing on trunk Key: YARN-6249 URL: https://issues.apache.org/jira/browse/YARN-6249 Project: Hadoop YARN Issue Type: Bug Components: fairscheduler, resourcemanager Reporter: Sean Po Tests in TestFairSchedulerPreemption.java will inconsistently fail on trunk. An example stack trace: Tests run: 24, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 24.879 sec <<< FAILURE! - in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption testPreemptionSelectNonAMContainer[MinSharePreemptionWithDRF](org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption) Time elapsed: 10.475 sec <<< FAILURE! java.lang.AssertionError: Incorrect number of containers on the greedy app expected:<4> but was:<8> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption.verifyPreemption(TestFairSchedulerPreemption.java:288) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption.testPreemptionSelectNonAMContainer(TestFairSchedulerPreemption.java:363) -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-5560) Clean up bad exception catching practices in TestYarnClient
Sean Po created YARN-5560:
-
Summary: Clean up bad exception catching practices in
TestYarnClient
Key: YARN-5560
URL: https://issues.apache.org/jira/browse/YARN-5560
Project: Hadoop YARN
Issue Type: Test
Reporter: Sean Po
Assignee: Sean Po
In TestYarnClient, tests commonly wrap methods that throw exceptions in a try
catch statement similar to the following:
{{try {}}
{{client.submitApplication(context);}}
{{} catch (Exception e) {}}
{{Assert.fail("Exception is not expected.");}}
{{}}}
This hides useful error messages, and surfaces less helpful ones.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-5656) ReservationACLsTestBase fails on trunk
Sean Po created YARN-5656: - Summary: ReservationACLsTestBase fails on trunk Key: YARN-5656 URL: https://issues.apache.org/jira/browse/YARN-5656 Project: Hadoop YARN Issue Type: Test Reporter: Sean Po ReservationACLsTestBase fails when verifying that a reservation can be successfully updated by a user who did not submit the reservation who also has an admin ACL. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-5703) ReservationAgents are not correctly configured
Sean Po created YARN-5703: - Summary: ReservationAgents are not correctly configured Key: YARN-5703 URL: https://issues.apache.org/jira/browse/YARN-5703 Project: Hadoop YARN Issue Type: Bug Components: capacity scheduler, resourcemanager Affects Versions: 3.0.0-alpha1 Reporter: Sean Po Assignee: Sean Po In AbstractReservationSystem, the method that instantiates a ReservationAgent does not properly initialize it with the appropriate configuration because it expects the ReservationAgent to implement Configurable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-5839) ClusterApplication API does not include a ReservationID
Sean Po created YARN-5839: - Summary: ClusterApplication API does not include a ReservationID Key: YARN-5839 URL: https://issues.apache.org/jira/browse/YARN-5839 Project: Hadoop YARN Issue Type: Improvement Reporter: Sean Po Assignee: Sean Po Currently, the ClusterApplication, and ClusterApplications API does not allow users to find the reservation queue that an application is running in. YARN-5435 proposes to add ReservationId to the ClusterApplication and ClusterApplications API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-5763) HttpListener takes upwards of 3 minutes to start on failover
Sean Po created YARN-5763: - Summary: HttpListener takes upwards of 3 minutes to start on failover Key: YARN-5763 URL: https://issues.apache.org/jira/browse/YARN-5763 Project: Hadoop YARN Issue Type: Bug Reporter: Sean Po Assignee: Sean Po When Yarn RM fails over to another instance, it takes multiple minutes before the new master Yarn RM can begin accepting requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-5881) Enable configuration of queue capacity in terms of absolute resources
Sean Po created YARN-5881: - Summary: Enable configuration of queue capacity in terms of absolute resources Key: YARN-5881 URL: https://issues.apache.org/jira/browse/YARN-5881 Project: Hadoop YARN Issue Type: Improvement Reporter: Sean Po Assignee: Sean Po Currently, Yarn RM supports the configuration of queue capacity in terms of a proportion to cluster capacity. In the context of Yarn being used as a public cloud service, it makes more sense if queues can be configured absolutely. This will allow administrators to set usage limits more concretely and simplify customer expectations for cluster allocation. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-6529) Add JMX metrics for Priority Reservation Agent
[ https://issues.apache.org/jira/browse/YARN-6529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po resolved YARN-6529. --- Resolution: Duplicate Duplicate of YARN-6530. > Add JMX metrics for Priority Reservation Agent > -- > > Key: YARN-6529 > URL: https://issues.apache.org/jira/browse/YARN-6529 > Project: Hadoop YARN > Issue Type: Task >Reporter: Sean Po >Assignee: Sean Po > > YARN-5211 proposes adding support for generalized priorities for reservations > in the YARN ReservationSystem. This JIRA is a sub-task to track the changes > needed to gauge the performance of the priority reservation agent. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6530) Add JMX metrics for Priority Reservation Agent
Sean Po created YARN-6530: - Summary: Add JMX metrics for Priority Reservation Agent Key: YARN-6530 URL: https://issues.apache.org/jira/browse/YARN-6530 Project: Hadoop YARN Issue Type: Sub-task Reporter: Sean Po YARN-5211 proposes adding support for generalized priorities for reservations in the YARN ReservationSystem. This JIRA is a sub-task to track the changes needed to gauge the performance of the priority reservation agent. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6528) Add JMX metrics for Plan Follower and Agent Placement and Plan Operations
Sean Po created YARN-6528: - Summary: Add JMX metrics for Plan Follower and Agent Placement and Plan Operations Key: YARN-6528 URL: https://issues.apache.org/jira/browse/YARN-6528 Project: Hadoop YARN Issue Type: Task Reporter: Sean Po Assignee: Sean Po -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6529) Add JMX metrics for Priority Reservation Agent
Sean Po created YARN-6529: - Summary: Add JMX metrics for Priority Reservation Agent Key: YARN-6529 URL: https://issues.apache.org/jira/browse/YARN-6529 Project: Hadoop YARN Issue Type: Task Reporter: Sean Po Assignee: Sean Po -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
