[jira] [Commented] (YARN-2408) Resource Request REST API for YARN
[
https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14552849#comment-14552849
]
Renan DelValle commented on YARN-2408:
--
[~leftnoteasy], thanks for taking a look at the patch, really appreciate it.
1) I agree, the original patch I had was very verbose so I shrunk down the
amount of data being transferred by clustering resource requests together.
Seems to be the best alternative to keeping original ResourceRequest structures.
2) I will take a look at that and implement it that way. (Thank you for
pointing me in the right direction). On the resource-by-label inclusion, do you
think it would be better to wait until it is patched into the trunk in order to
make the process easier?
Resource Request REST API for YARN
--
Key: YARN-2408
URL: https://issues.apache.org/jira/browse/YARN-2408
Project: Hadoop YARN
Issue Type: New Feature
Components: webapp
Reporter: Renan DelValle
Labels: features
Attachments: YARN-2408-6.patch
I’m proposing a new REST API for YARN which exposes a snapshot of the
Resource Requests that exist inside of the Scheduler. My motivation behind
this new feature is to allow external software to monitor the amount of
resources being requested to gain more insightful information into cluster
usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys
admin so that the problem may be remedied.
Here is the proposed API (a JSON counterpart is also available):
{code:xml}
resourceRequests
MB7680/MB
VCores7/VCores
appMaster
applicationIdapplication_1412191664217_0001/applicationId
applicationAttemptIdappattempt_1412191664217_0001_01/applicationAttemptId
queueNamedefault/queueName
totalMB6144/totalMB
totalVCores6/totalVCores
numResourceRequests3/numResourceRequests
requests
request
MB1024/MB
VCores1/VCores
numContainers6/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
resourceNames
resourceNamelocalMachine/resourceName
resourceName/default-rack/resourceName
resourceName*/resourceName
/resourceNames
/request
/requests
/appMaster
appMaster
...
/appMaster
/resourceRequests
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2408) Resource Request REST API for YARN
[
https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542804#comment-14542804
]
Wangda Tan commented on YARN-2408:
--
[~rdelvalle],
Thanks for updating, I took at look at your latest patch, some overall comments:
1) About what the APIs should look like:
Basically we have two choices, one is to keep the original ResourceRequests
structure in app, which is:
- Priorities under an app
- ResourceRequests grouped by resourceName
And we should support return applications satisfy given criteria only, such as
a given queue, app-id, user, etc. We can support some of them with this patch,
but return all apps in the cluster is not useful/efficiency enough.
The other choice is aggregate ResourceRequests, like what you have done in your
patch. Keep less information and also less data needs to be tranfered from RM.
I prefer the latter one.
2) How to get aggregate ResourceRequests, I suggest you can take a look at
SchedulerApplicationAttempt.getAppAttemptResourceUsage, which should contains
what you needed. It is updated when you updates application's pending resource.
Also, I suggest to add pending resource-by-label as well since we already have
YARN-796.
Thoughts?
Resource Request REST API for YARN
--
Key: YARN-2408
URL: https://issues.apache.org/jira/browse/YARN-2408
Project: Hadoop YARN
Issue Type: New Feature
Components: webapp
Reporter: Renan DelValle
Labels: features
Attachments: YARN-2408-6.patch
I’m proposing a new REST API for YARN which exposes a snapshot of the
Resource Requests that exist inside of the Scheduler. My motivation behind
this new feature is to allow external software to monitor the amount of
resources being requested to gain more insightful information into cluster
usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys
admin so that the problem may be remedied.
Here is the proposed API (a JSON counterpart is also available):
{code:xml}
resourceRequests
MB7680/MB
VCores7/VCores
appMaster
applicationIdapplication_1412191664217_0001/applicationId
applicationAttemptIdappattempt_1412191664217_0001_01/applicationAttemptId
queueNamedefault/queueName
totalMB6144/totalMB
totalVCores6/totalVCores
numResourceRequests3/numResourceRequests
requests
request
MB1024/MB
VCores1/VCores
numContainers6/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
resourceNames
resourceNamelocalMachine/resourceName
resourceName/default-rack/resourceName
resourceName*/resourceName
/resourceNames
/request
/requests
/appMaster
appMaster
...
/appMaster
/resourceRequests
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2408) Resource Request REST API for YARN
[
https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541251#comment-14541251
]
Renan DelValle commented on YARN-2408:
--
[~leftnoteasy],
Just uploaded a new patch. Depending on the Hadoop QA results, it'd be great if
you could take a look at it and let me know what you think.
Thanks,
-Renan
Resource Request REST API for YARN
--
Key: YARN-2408
URL: https://issues.apache.org/jira/browse/YARN-2408
Project: Hadoop YARN
Issue Type: New Feature
Components: webapp
Reporter: Renan DelValle
Labels: features
Attachments: YARN-2408-5.patch
I’m proposing a new REST API for YARN which exposes a snapshot of the
Resource Requests that exist inside of the Scheduler. My motivation behind
this new feature is to allow external software to monitor the amount of
resources being requested to gain more insightful information into cluster
usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys
admin so that the problem may be remedied.
Here is the proposed API (a JSON counterpart is also available):
{code:xml}
resourceRequests
MB7680/MB
VCores7/VCores
appMaster
applicationIdapplication_1412191664217_0001/applicationId
applicationAttemptIdappattempt_1412191664217_0001_01/applicationAttemptId
queueNamedefault/queueName
totalMB6144/totalMB
totalVCores6/totalVCores
numResourceRequests3/numResourceRequests
requests
request
MB1024/MB
VCores1/VCores
numContainers6/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
resourceNames
resourceNamelocalMachine/resourceName
resourceName/default-rack/resourceName
resourceName*/resourceName
/resourceNames
/request
/requests
/appMaster
appMaster
...
/appMaster
/resourceRequests
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2408) Resource Request REST API for YARN
[
https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541275#comment-14541275
]
Hadoop QA commented on YARN-2408:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 4s | The patch command could not apply
the patch. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12732466/YARN-2408-5.patch |
| Optional Tests | shellcheck site javadoc javac unit findbugs checkstyle |
| git revision | trunk / 2463666 |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7909/console |
This message was automatically generated.
Resource Request REST API for YARN
--
Key: YARN-2408
URL: https://issues.apache.org/jira/browse/YARN-2408
Project: Hadoop YARN
Issue Type: New Feature
Components: webapp
Reporter: Renan DelValle
Labels: features
Attachments: YARN-2408-5.patch
I’m proposing a new REST API for YARN which exposes a snapshot of the
Resource Requests that exist inside of the Scheduler. My motivation behind
this new feature is to allow external software to monitor the amount of
resources being requested to gain more insightful information into cluster
usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys
admin so that the problem may be remedied.
Here is the proposed API (a JSON counterpart is also available):
{code:xml}
resourceRequests
MB7680/MB
VCores7/VCores
appMaster
applicationIdapplication_1412191664217_0001/applicationId
applicationAttemptIdappattempt_1412191664217_0001_01/applicationAttemptId
queueNamedefault/queueName
totalMB6144/totalMB
totalVCores6/totalVCores
numResourceRequests3/numResourceRequests
requests
request
MB1024/MB
VCores1/VCores
numContainers6/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
resourceNames
resourceNamelocalMachine/resourceName
resourceName/default-rack/resourceName
resourceName*/resourceName
/resourceNames
/request
/requests
/appMaster
appMaster
...
/appMaster
/resourceRequests
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2408) Resource Request REST API for YARN
[
https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514701#comment-14514701
]
Wangda Tan commented on YARN-2408:
--
[~rdelvalle],
Just went through this JIRA as well as discussion. I found you removed patches
attached in the JIRA, is there any specific reason to do so? I think this may
be an useful API, recently we have some patch in web UI to show requests and
relationship between requests/allocations. So I think REST API changes should
be added as well, getting app's ResourceRequests can be the first step.
Resource Request REST API for YARN
--
Key: YARN-2408
URL: https://issues.apache.org/jira/browse/YARN-2408
Project: Hadoop YARN
Issue Type: New Feature
Components: webapp
Reporter: Renan DelValle
Labels: features
I’m proposing a new REST API for YARN which exposes a snapshot of the
Resource Requests that exist inside of the Scheduler. My motivation behind
this new feature is to allow external software to monitor the amount of
resources being requested to gain more insightful information into cluster
usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys
admin so that the problem may be remedied.
Here is the proposed API (a JSON counterpart is also available):
{code:xml}
resourceRequests
MB7680/MB
VCores7/VCores
appMaster
applicationIdapplication_1412191664217_0001/applicationId
applicationAttemptIdappattempt_1412191664217_0001_01/applicationAttemptId
queueNamedefault/queueName
totalMB6144/totalMB
totalVCores6/totalVCores
numResourceRequests3/numResourceRequests
requests
request
MB1024/MB
VCores1/VCores
numContainers6/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
resourceNames
resourceNamelocalMachine/resourceName
resourceName/default-rack/resourceName
resourceName*/resourceName
/resourceNames
/request
/requests
/appMaster
appMaster
...
/appMaster
/resourceRequests
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2408) Resource Request REST API for YARN
[
https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514734#comment-14514734
]
Renan DelValle commented on YARN-2408:
--
[~leftnoteasy],
I removed the patches because because the newest version that I submitted did
not compile successfully for Hadoop QA. The code base seems to also have
changed significantly since I wrote this patch. I'll take a look at it tonight
and see what needs to be done to fix it since there does seem to be some
interest for it after all.
-Renan
Resource Request REST API for YARN
--
Key: YARN-2408
URL: https://issues.apache.org/jira/browse/YARN-2408
Project: Hadoop YARN
Issue Type: New Feature
Components: webapp
Reporter: Renan DelValle
Labels: features
I’m proposing a new REST API for YARN which exposes a snapshot of the
Resource Requests that exist inside of the Scheduler. My motivation behind
this new feature is to allow external software to monitor the amount of
resources being requested to gain more insightful information into cluster
usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys
admin so that the problem may be remedied.
Here is the proposed API (a JSON counterpart is also available):
{code:xml}
resourceRequests
MB7680/MB
VCores7/VCores
appMaster
applicationIdapplication_1412191664217_0001/applicationId
applicationAttemptIdappattempt_1412191664217_0001_01/applicationAttemptId
queueNamedefault/queueName
totalMB6144/totalMB
totalVCores6/totalVCores
numResourceRequests3/numResourceRequests
requests
request
MB1024/MB
VCores1/VCores
numContainers6/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
resourceNames
resourceNamelocalMachine/resourceName
resourceName/default-rack/resourceName
resourceName*/resourceName
/resourceNames
/request
/requests
/appMaster
appMaster
...
/appMaster
/resourceRequests
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2408) Resource Request REST API for YARN
[
https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14509106#comment-14509106
]
Renan DelValle commented on YARN-2408:
--
Hi Nikhil,
While I would be glad to finish the development of this feature, the fact is
that since being proposed on August 12, 2014 (more than 8 months ago), no
member of the Hadoop team has shown an interest in including this feature as
part the main software. Thus, to use this feature would mean always having to
patch the Hadoop source intended for use and hoping that nothing is broken in
future versions.
As Adam pointed out, alternative solutions exist which may allow you to achieve
this feature with a much more future-proof and painless solution, such as the
approach Myriad takes (https://github.com/mesos/myriad).
That having been said, I'd gladly release the source code for what I have
working. As for me, unfortunately, at this time, I don't feel like it is within
my best interests to put forth the time necessary to complete this feature.
-Renan
Resource Request REST API for YARN
--
Key: YARN-2408
URL: https://issues.apache.org/jira/browse/YARN-2408
Project: Hadoop YARN
Issue Type: New Feature
Components: webapp
Reporter: Renan DelValle
Labels: features
I’m proposing a new REST API for YARN which exposes a snapshot of the
Resource Requests that exist inside of the Scheduler. My motivation behind
this new feature is to allow external software to monitor the amount of
resources being requested to gain more insightful information into cluster
usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys
admin so that the problem may be remedied.
Here is the proposed API (a JSON counterpart is also available):
{code:xml}
resourceRequests
MB7680/MB
VCores7/VCores
appMaster
applicationIdapplication_1412191664217_0001/applicationId
applicationAttemptIdappattempt_1412191664217_0001_01/applicationAttemptId
queueNamedefault/queueName
totalMB6144/totalMB
totalVCores6/totalVCores
numResourceRequests3/numResourceRequests
requests
request
MB1024/MB
VCores1/VCores
numContainers6/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
resourceNames
resourceNamelocalMachine/resourceName
resourceName/default-rack/resourceName
resourceName*/resourceName
/resourceNames
/request
/requests
/appMaster
appMaster
...
/appMaster
/resourceRequests
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2408) Resource Request REST API for YARN
[
https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14508586#comment-14508586
]
Nikhil Mulley commented on YARN-2408:
-
Hi [~rdelvalle]
There are 8 people voting for it and 15 people watching this issue. I am not
sure what is the requirement in the community for having a general interest
though but I would be happy to help this move forward in terms of having the
patch deployed on my test cluster and give it a whirl and see where it goes.
I am as well interested in the rest api to provide means to monitor the cluster
resources, in general, to have a means to monitor the slow/starving jobs and
the resources requested/consumed per app/job via rest api.
Nikhil
Resource Request REST API for YARN
--
Key: YARN-2408
URL: https://issues.apache.org/jira/browse/YARN-2408
Project: Hadoop YARN
Issue Type: New Feature
Components: webapp
Reporter: Renan DelValle
Labels: features
I’m proposing a new REST API for YARN which exposes a snapshot of the
Resource Requests that exist inside of the Scheduler. My motivation behind
this new feature is to allow external software to monitor the amount of
resources being requested to gain more insightful information into cluster
usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys
admin so that the problem may be remedied.
Here is the proposed API (a JSON counterpart is also available):
{code:xml}
resourceRequests
MB7680/MB
VCores7/VCores
appMaster
applicationIdapplication_1412191664217_0001/applicationId
applicationAttemptIdappattempt_1412191664217_0001_01/applicationAttemptId
queueNamedefault/queueName
totalMB6144/totalMB
totalVCores6/totalVCores
numResourceRequests3/numResourceRequests
requests
request
MB1024/MB
VCores1/VCores
numContainers6/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
resourceNames
resourceNamelocalMachine/resourceName
resourceName/default-rack/resourceName
resourceName*/resourceName
/resourceNames
/request
/requests
/appMaster
appMaster
...
/appMaster
/resourceRequests
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2408) Resource Request REST API for YARN
[
https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14508593#comment-14508593
]
Adam B commented on YARN-2408:
--
FYI, one of the original use cases (Myriad, to run YARN on Mesos) now just
implements the YARN scheduler API directly, so it no longer needs a REST API
for resource requests. Other tools may be able to take a similar approach of
wrapping a traditional YARN scheduler, but that means that the tool is forced
to live on the RM node, in-process. Some tools (especially non-Java tools) will
not be able to take this approach.
Resource Request REST API for YARN
--
Key: YARN-2408
URL: https://issues.apache.org/jira/browse/YARN-2408
Project: Hadoop YARN
Issue Type: New Feature
Components: webapp
Reporter: Renan DelValle
Labels: features
I’m proposing a new REST API for YARN which exposes a snapshot of the
Resource Requests that exist inside of the Scheduler. My motivation behind
this new feature is to allow external software to monitor the amount of
resources being requested to gain more insightful information into cluster
usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys
admin so that the problem may be remedied.
Here is the proposed API (a JSON counterpart is also available):
{code:xml}
resourceRequests
MB7680/MB
VCores7/VCores
appMaster
applicationIdapplication_1412191664217_0001/applicationId
applicationAttemptIdappattempt_1412191664217_0001_01/applicationAttemptId
queueNamedefault/queueName
totalMB6144/totalMB
totalVCores6/totalVCores
numResourceRequests3/numResourceRequests
requests
request
MB1024/MB
VCores1/VCores
numContainers6/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
resourceNames
resourceNamelocalMachine/resourceName
resourceName/default-rack/resourceName
resourceName*/resourceName
/resourceNames
/request
/requests
/appMaster
appMaster
...
/appMaster
/resourceRequests
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2408) Resource Request REST API for YARN
[
https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14155372#comment-14155372
]
Hadoop QA commented on YARN-2408:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672388/YARN-2408.4.patch
against trunk revision 1f5b42a.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5203//console
This message is automatically generated.
Resource Request REST API for YARN
--
Key: YARN-2408
URL: https://issues.apache.org/jira/browse/YARN-2408
Project: Hadoop YARN
Issue Type: New Feature
Components: webapp
Reporter: Renan DelValle
Labels: features
Attachments: YARN-2408.4.patch
I’m proposing a new REST API for YARN which exposes a snapshot of the
Resource Requests that exist inside of the Scheduler. My motivation behind
this new feature is to allow external software to monitor the amount of
resources being requested to gain more insightful information into cluster
usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys
admin so that the problem may be remedied.
Here is the proposed API (a JSON counterpart is also available):
{code:xml}
resourceRequests
MB7680/MB
VCores7/VCores
appMaster
applicationIdapplication_1412191664217_0001/applicationId
applicationAttemptIdappattempt_1412191664217_0001_01/applicationAttemptId
queueNamedefault/queueName
totalMB6144/totalMB
totalVCores6/totalVCores
numResourceRequests3/numResourceRequests
requests
request
MB1024/MB
VCores1/VCores
numContainers6/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
resourceNames
resourceNamelocalMachine/resourceName
resourceName/default-rack/resourceName
resourceName*/resourceName
/resourceNames
/request
/requests
/appMaster
appMaster
...
/appMaster
/resourceRequests
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2408) Resource Request REST API for YARN
[
https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14107563#comment-14107563
]
Hadoop QA commented on YARN-2408:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12663726/YARN-2408-3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4697//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4697//console
This message is automatically generated.
Resource Request REST API for YARN
--
Key: YARN-2408
URL: https://issues.apache.org/jira/browse/YARN-2408
Project: Hadoop YARN
Issue Type: New Feature
Components: webapp
Reporter: Renan DelValle
Labels: features
Attachments: YARN-2408-3.patch
I’m proposing a new REST API for YARN which exposes a snapshot of the
Resource Requests that exist inside of the Scheduler. My motivation behind
this new feature is to allow external software to monitor the amount of
resources being requested to gain more insightful information into cluster
usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys
admin so that the problem may be remedied.
Here is the proposed API:
{code:xml}
resourceRequests
MB96256/MB
VCores94/VCores
appMaster
applicationIdapplication_/applicationId
applicationAttemptIdappattempt_/applicationAttemptId
queueNamedefault/queueName
totalPendingMB96256/totalPendingMB
totalPendingVCores94/totalPendingVCores
numResourceRequests3/numResourceRequests
resourceRequests
request
MB1024/MB
VCores1/VCores
resourceName/default-rack/resourceName
numContainers94/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
/request
request
MB1024/MB
VCores1/VCores
resourceName*/resourceName
numContainers94/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
/request
request
MB1024/MB
VCores1/VCores
resourceNamemaster/resourceName
numContainers94/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
/request
/resourceRequests
/appMaster
/resourceRequests
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2408) Resource Request REST API for YARN
[
https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14106259#comment-14106259
]
Hadoop QA commented on YARN-2408:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12663442/YARN-2408-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1261 javac
compiler warnings (more than the trunk's current 1260 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 3
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4686//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/4686//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Javac warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/4686//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4686//console
This message is automatically generated.
Resource Request REST API for YARN
--
Key: YARN-2408
URL: https://issues.apache.org/jira/browse/YARN-2408
Project: Hadoop YARN
Issue Type: New Feature
Components: webapp
Reporter: Renan DelValle
Labels: features
Attachments: YARN-2408-2.patch
I’m proposing a new REST API for YARN which exposes a snapshot of the
Resource Requests that exist inside of the Scheduler. My motivation behind
this new feature is to allow external software to monitor the amount of
resources being requested to gain more insightful information into cluster
usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys
admin so that the problem may be remedied.
Here is the proposed API:
{code:xml}
resourceRequests
MB96256/MB
VCores94/VCores
appMaster
applicationIdapplication_/applicationId
applicationAttemptIdappattempt_/applicationAttemptId
queueNamedefault/queueName
totalPendingMB96256/totalPendingMB
totalPendingVCores94/totalPendingVCores
numResourceRequests3/numResourceRequests
resourceRequests
request
MB1024/MB
VCores1/VCores
resourceName/default-rack/resourceName
numContainers94/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
/request
request
MB1024/MB
VCores1/VCores
resourceName*/resourceName
numContainers94/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
/request
request
MB1024/MB
VCores1/VCores
resourceNamemaster/resourceName
numContainers94/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
/request
/resourceRequests
/appMaster
/resourceRequests
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2408) Resource Request REST API for YARN
[
https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093781#comment-14093781
]
Hadoop QA commented on YARN-2408:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661139/YARN-2408.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4600//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4600//console
This message is automatically generated.
Resource Request REST API for YARN
--
Key: YARN-2408
URL: https://issues.apache.org/jira/browse/YARN-2408
Project: Hadoop YARN
Issue Type: New Feature
Components: webapp
Reporter: Renan DelValle
Priority: Minor
Labels: features
Attachments: YARN-2408.patch
I’m proposing a new REST API for YARN which exposes a snapshot of the
Resource Requests that exist inside of the Scheduler. My motivation behind
this new feature is to allow external software to monitor the amount of
resources being requested to gain more insightful information into cluster
usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys
admin so that the problem may be remedied.
Here is the proposed API:
{code:xml}
resourceRequests
MB96256/MB
VCores94/VCores
appMaster
applicationIdapplication_/applicationId
applicationAttemptIdappattempt_/applicationAttemptId
queueNamedefault/queueName
totalPendingMB96256/totalPendingMB
totalPendingVCores94/totalPendingVCores
numResourceRequests3/numResourceRequests
resourceRequests
request
MB1024/MB
VCores1/VCores
resourceName/default-rack/resourceName
numContainers94/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
/request
request
MB1024/MB
VCores1/VCores
resourceName*/resourceName
numContainers94/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
/request
request
MB1024/MB
VCores1/VCores
resourceNamemaster/resourceName
numContainers94/numContainers
relaxLocalitytrue/relaxLocality
priority20/priority
/request
/resourceRequests
/appMaster
/resourceRequests
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
