[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[ https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14159134#comment-14159134 ] Hudson commented on YARN-1708: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1891 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1891/]) YARN-1708. Public YARN APIs for creating/updating/deleting reservations. (cdouglas: rev c9266df40434fdde05bd3a4c75f7a6bc29db760f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ReservationIdPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationSubmissionResponsePBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationSubmissionContextPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationUpdateResponse.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationRequestInterpreter.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationClientProtocol.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ReservationRequestPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ReservationRequestsPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/applicationclient_protocol.proto * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationId.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationDeleteResponse.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationRequests.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationDefinition.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationSubmissionContext.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationDeleteRequestPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/UTCClock.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationRequest.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationSubmissionResponse.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ApplicationClientProtocolPBServiceImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationUpdateRequestPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationUpdateRequest.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ReservationDefinitionPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationSubmissionRequestPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * YARN-1051-CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ProtoUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationDeleteResponsePBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationSubmissionRequest.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationUpdateResponsePBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ApplicationClientProtocolPBClientImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationDeleteRequest.java Add a public API to reserve resources (part of YARN-1051) - Key: YARN-1708 URL: https://issues.apache.org/jira/browse/YARN-1708 Project: Hadoop YARN Issue Type: Sub-task
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[ https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14159158#comment-14159158 ] Hudson commented on YARN-1708: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1916 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1916/]) YARN-1708. Public YARN APIs for creating/updating/deleting reservations. (cdouglas: rev c9266df40434fdde05bd3a4c75f7a6bc29db760f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/UTCClock.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ReservationIdPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationUpdateResponse.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationRequest.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationRequestInterpreter.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationUpdateRequestPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationUpdateResponsePBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationDeleteRequest.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ProtoUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ApplicationClientProtocolPBServiceImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationClientProtocol.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ReservationRequestsPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationSubmissionContextPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationSubmissionRequestPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/applicationclient_protocol.proto * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationDeleteResponsePBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationUpdateRequest.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationSubmissionRequest.java * YARN-1051-CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ReservationRequestPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationId.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ApplicationClientProtocolPBClientImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationSubmissionContext.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationDeleteRequestPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationDeleteResponse.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationSubmissionResponsePBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationDefinition.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ReservationDefinitionPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationSubmissionResponse.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationRequests.java Add a public API to reserve resources (part of YARN-1051) - Key: YARN-1708 URL: https://issues.apache.org/jira/browse/YARN-1708 Project: Hadoop YARN Issue Type:
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[ https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14158770#comment-14158770 ] Hudson commented on YARN-1708: -- FAILURE: Integrated in Hadoop-trunk-Commit #6189 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6189/]) YARN-1708. Public YARN APIs for creating/updating/deleting reservations. (cdouglas: rev c9266df40434fdde05bd3a4c75f7a6bc29db760f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationUpdateResponsePBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationRequestInterpreter.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationUpdateRequestPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ReservationRequestsPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationSubmissionContextPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ReservationIdPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ApplicationClientProtocolPBServiceImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ReservationRequestPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationUpdateResponse.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationId.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationDeleteRequest.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ReservationDefinitionPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationRequests.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationSubmissionResponse.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/applicationclient_protocol.proto * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationRequest.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/UTCClock.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ReservationDefinition.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationSubmissionRequest.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationClientProtocol.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationSubmissionResponsePBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ApplicationClientProtocolPBClientImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationDeleteRequestPBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationUpdateRequest.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReservationDeleteResponse.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationDeleteResponsePBImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationSubmissionContext.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ProtoUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReservationSubmissionRequestPBImpl.java * YARN-1051-CHANGES.txt Add a public API to reserve resources (part of YARN-1051) - Key: YARN-1708 URL: https://issues.apache.org/jira/browse/YARN-1708 Project: Hadoop YARN Issue Type: Sub-task
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[
https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130923#comment-14130923
]
Hadoop QA commented on YARN-1708:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12668202/YARN-1708.patch
against trunk revision 6c08339.
{color:red}-1 patch{color}. Trunk compilation may be broken.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4900//console
This message is automatically generated.
Add a public API to reserve resources (part of YARN-1051)
-
Key: YARN-1708
URL: https://issues.apache.org/jira/browse/YARN-1708
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Carlo Curino
Assignee: Subramaniam Krishnan
Attachments: YARN-1708.patch, YARN-1708.patch, YARN-1708.patch,
YARN-1708.patch
This JIRA tracks the definition of a new public API for YARN, which allows
users to reserve resources (think of time-bounded queues). This is part of
the admission control enhancement proposed in YARN-1051.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[ https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123134#comment-14123134 ] Vinod Kumar Vavilapalli commented on YARN-1708: --- ReservationUpdateResponse.setReservationName() is still public, it needs to be private. Add a public API to reserve resources (part of YARN-1051) - Key: YARN-1708 URL: https://issues.apache.org/jira/browse/YARN-1708 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Carlo Curino Assignee: Subramaniam Krishnan Attachments: YARN-1708.patch, YARN-1708.patch, YARN-1708.patch This JIRA tracks the definition of a new public API for YARN, which allows users to reserve resources (think of time-bounded queues). This is part of the admission control enhancement proposed in YARN-1051. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[ https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123687#comment-14123687 ] Vinod Kumar Vavilapalli commented on YARN-1708: --- Sorry, my bad. I misread it. Patch looks good. +1. Add a public API to reserve resources (part of YARN-1051) - Key: YARN-1708 URL: https://issues.apache.org/jira/browse/YARN-1708 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Carlo Curino Assignee: Subramaniam Krishnan Attachments: YARN-1708.patch, YARN-1708.patch, YARN-1708.patch This JIRA tracks the definition of a new public API for YARN, which allows users to reserve resources (think of time-bounded queues). This is part of the admission control enhancement proposed in YARN-1051. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[
https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14121737#comment-14121737
]
Vinod Kumar Vavilapalli commented on YARN-1708:
---
Tx for the updated patch, [~subru]! This looks so much better!
A few minor comments
- All the newInstance methods and setters in the response objects should be
marked as private, for e.g in ReservationSubmissionResponse. Similarly in other
objects too. We don't expect users to call them because responses are generated
only by the platform.
- ReservationId: It's likely that IDEs generate a better hashCode instead of
us doing the long-to-int conversions?
- ReservationRequests.{set|get}Type - {set|get}Interpretor? Similarly
ReservationRequestsProto.type.
- Rename ReservationRequest.leaseDuration to be simply duration inline with
ReservationRequestProto.duration
Add a public API to reserve resources (part of YARN-1051)
-
Key: YARN-1708
URL: https://issues.apache.org/jira/browse/YARN-1708
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Carlo Curino
Assignee: Subramaniam Krishnan
Attachments: YARN-1708.patch, YARN-1708.patch
This JIRA tracks the definition of a new public API for YARN, which allows
users to reserve resources (think of time-bounded queues). This is part of
the admission control enhancement proposed in YARN-1051.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[
https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122128#comment-14122128
]
Hadoop QA commented on YARN-1708:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/1201/YARN-1708.patch
against trunk revision 51a4faf.
{color:red}-1 patch{color}. Trunk compilation may be broken.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4830//console
This message is automatically generated.
Add a public API to reserve resources (part of YARN-1051)
-
Key: YARN-1708
URL: https://issues.apache.org/jira/browse/YARN-1708
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Carlo Curino
Assignee: Subramaniam Krishnan
Attachments: YARN-1708.patch, YARN-1708.patch, YARN-1708.patch
This JIRA tracks the definition of a new public API for YARN, which allows
users to reserve resources (think of time-bounded queues). This is part of
the admission control enhancement proposed in YARN-1051.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[
https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112694#comment-14112694
]
Hadoop QA commented on YARN-1708:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12664714/YARN-1708.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4743//console
This message is automatically generated.
Add a public API to reserve resources (part of YARN-1051)
-
Key: YARN-1708
URL: https://issues.apache.org/jira/browse/YARN-1708
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Carlo Curino
Assignee: Subramaniam Krishnan
Attachments: YARN-1708.patch, YARN-1708.patch
This JIRA tracks the definition of a new public API for YARN, which allows
users to reserve resources (think of time-bounded queues). This is part of
the admission control enhancement proposed in YARN-1051.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[
https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059595#comment-14059595
]
Hadoop QA commented on YARN-1708:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12643481/YARN-1708.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4281//console
This message is automatically generated.
Add a public API to reserve resources (part of YARN-1051)
-
Key: YARN-1708
URL: https://issues.apache.org/jira/browse/YARN-1708
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Carlo Curino
Assignee: Subramaniam Venkatraman Krishnan
Attachments: YARN-1708.patch
This JIRA tracks the definition of a new public API for YARN, which allows
users to reserve resources (think of time-bounded queues). This is part of
the admission control enhancement proposed in YARN-1051.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[
https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042917#comment-14042917
]
Vinod Kumar Vavilapalli commented on YARN-1708:
---
I discussed a bit about this with [~subru] and [~curino] offline. Here's what
we found in addition to the above comments that I had
- It seems like ResourceRequest is getting overloaded with a bunch of things
like concurrency and leaseDuration that today don't make sense when AM
explicitly asks for containers. Similarly, priority today even after the
reservation patches won't have much significance when submitting reservations.
I propose we fork them so the ResourceRequest stays where it is today and then
we have a new entity called ReservationResourceRequest. Long term, they may
converge a little if not completely.
- ReservationDefinition.setResources() and getType() are really linked
together. We should try to collapse them into a single method/API.
- May be even rename ReservationDefinitionType to say something like
ResourceRequest{Structure|Interpretation} or something of that sort.
Add a public API to reserve resources (part of YARN-1051)
-
Key: YARN-1708
URL: https://issues.apache.org/jira/browse/YARN-1708
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Carlo Curino
Assignee: Subramaniam Venkatraman Krishnan
Attachments: YARN-1708.patch
This JIRA tracks the definition of a new public API for YARN, which allows
users to reserve resources (think of time-bounded queues). This is part of
the admission control enhancement proposed in YARN-1051.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[ https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011820#comment-14011820 ] Vinod Kumar Vavilapalli commented on YARN-1708: --- Thanks for the patch [~subru]! I started looking at this. Few comments: h4. Misc - I think we should create a ReservationID or ReservationHandle and use it instead of strings - ReservationResponse.message - errorMesage? Or Errors? h4. ApplicationClientProtocol - createReservation - submitReservation? - Let's have separate request/response records for submission, update and deletion of reservations. Deletion of reservations, for e.g only needs to supplied a reservationID. See submit/kill app for analogy. Similarly, ReservationRequest.reservationID doesn't need to be part of the request for the reservation-submission. h4. ReservationDefinition - Seems like there is a notion of absolute time. We should make it clear what the arrival/deadline long's really represent. Particularly given the possibility of different timezones between the RM and the client. - It may be also very useful to let users specify time in relative terms - 6hrs from now, etc. - It let's you specify a list of ResourceRequests. Not sure how we can specify things like RR1 for the first 5 mins, RR2 for the next 15 etc. h4. ReservationDefinitionType - It seems like if we instead have a list of records of type (arrival, ResourceRequest, deadline), we will cover all the cases in the definition-type and then some more? Thoughts? - Also any examples of where R_ANY is useful? Similarly as to how R_ORDER is not enough and instead we have a need for R_ORDER_NO_GAP? Focusing mainly on use-cases here. h4. ResourceRequest - concurrency is really a request for a gang of containers? - Meaning of leaseDuration? Is it indicating the scheduler as to how long the container will run for? I have suggestions for configuration props renames follow. We follow a component.sub-component.sub-component.property-name convention. (OT: I wish I looked at preemption related config names :) ) IAC, I need to see the bigger picture with the rest of the patches before I can suggest correct naming, let's drop the YarnConfiguration changes from this patch. Will look more carefully at the PB impls in the next cycle. bq. The patch posted here is not submitted, since it depends on many other patches part of the umbrella JIRA, the separation is designed only for ease of reviewing. I see this patch to be fairly independent and committable in isolation. Though we should wait till we have the entire set to make sure the changes here are all sufficient and necessary. Add a public API to reserve resources (part of YARN-1051) - Key: YARN-1708 URL: https://issues.apache.org/jira/browse/YARN-1708 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Carlo Curino Assignee: Subramaniam Krishnan Attachments: YARN-1708.patch This JIRA tracks the definition of a new public API for YARN, which allows users to reserve resources (think of time-bounded queues). This is part of the admission control enhancement proposed in YARN-1051. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[ https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990240#comment-13990240 ] Carlo Curino commented on YARN-1708: The attached patch represents a proposal for an extension of YARN's APIs. This is the externally visible portion of the the umbrella JIRA YARN-1051, and provides users with the opportunity to create/update/deleted time varying resource reservations within a queue (if the queue allows it). Reservations are expressed leveraging existing ResourceRequest objects and extending them with temporal semantics (e.g., I need 1h of 20 containers of size 2GB,1vcore some time between 2pm and 6pm). We allow also to express minimum concurrency constraints, and dependencies among different stages of a pipeline. The reservationID token obtained by the user during the reservation process is passed during application submission, and instructs the RM to use the reserved resources to satisfy this application needs. The patch posted here is not submitted, since it depends on many other patches part of the umbrella JIRA, the separation is designed only for ease of reviewing. A more broad discussion of this idea, and some experimental results are provided in the tech-report attached to the umbrella JIRA YARN-1051. We have a complete solution backing this API, which we are testing/hardening and will be posting the rest of it in the upcoming days/weeks. Add a public API to reserve resources (part of YARN-1051) - Key: YARN-1708 URL: https://issues.apache.org/jira/browse/YARN-1708 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Carlo Curino Assignee: Subramaniam Krishnan Attachments: YARN-1708.patch This JIRA tracks the definition of a new public API for YARN, which allows users to reserve resources (think of time-bounded queues). This is part of the admission control enhancement proposed in YARN-1051. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[ https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990239#comment-13990239 ] Subramaniam Krishnan commented on YARN-1708: Attaching the patch Add a public API to reserve resources (part of YARN-1051) - Key: YARN-1708 URL: https://issues.apache.org/jira/browse/YARN-1708 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Carlo Curino Assignee: Subramaniam Krishnan Attachments: YARN-1708.patch This JIRA tracks the definition of a new public API for YARN, which allows users to reserve resources (think of time-bounded queues). This is part of the admission control enhancement proposed in YARN-1051. -- This message was sent by Atlassian JIRA (v6.2#6252)
