[
https://issues.apache.org/jira/browse/MAPREDUCE-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225231#comment-13225231
]
Robert Joseph Evans commented on MAPREDUCE-3971:
------------------------------------------------
Yes you are correct about the implementation of the /mapreduce/jobs API up
until MAPREDUCE-3944. MAPREDUCE-3944 changed it so that /mapreduce/jobs will
only return the equivalent of a PartialJob, not a CompletedJob. This reduces
the load a lot, and is hopefully a temporary measure until more issues can be
addressed as part of MAPREDUCE-3973.
I am not sure of the best way to implement this, because it is something that
we want to be able to do consistently across APIs if need be. But, it will
probably be something like keeping the limit that is already in place and
adding in a start parameter as well. We will probably also need to add in
fields to the returned list indicating the start index and end index of the
items returned, and possibly something else indicating the total number of
objects that can be returned.
The maximum limit would be in place regardless of what the limit is set in the
URL.
The real difficultly with this is that we now need to be able to maintain a
guaranteed consistent ordering of the values returned or the start will be
useless.
> Job History web services need to have limits on the number of itmes they can
> return.
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3971
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3971
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2
> Affects Versions: 0.23.2
> Reporter: Robert Joseph Evans
>
> The Job History web services canput a very large load on the job history
> server. We should put in a limit on the number of entries that can be
> returned by the web service, and also add in the ability to modify the
> starting location in the list, so that all entries can still be downlaoded.
> Just not all at once.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
