[
https://issues.apache.org/jira/browse/HADOOP-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12594790#action_12594790
]
amar_kamat edited comment on HADOOP-3116 at 5/6/08 11:21 PM:
-------------------------------------------------------------
One simple thing we can do is to use a {{MAP}} from _id-within-job_ to _TIP_
instead of an array.
was (Author: amar_kamat):
One simple thing we can do is to use a {{MAP}} from _attempt-id_ to _TIP_
instead of an array.
> Investigate whether the array in the JobInProgress that holds TIP references
> can be removed
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-3116
> URL: https://issues.apache.org/jira/browse/HADOOP-3116
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Devaraj Das
> Fix For: 0.18.0
>
>
> The array, in JobInProgress, that holds the references to TIPs is required
> now only to serve the clients (web UI via JSPs and JobClient APIs, that
> traverse the array and create the output). The array can now be removed since
> HADOOP-2119 introduces some datastructures for running/non-running TIPs and
> we could probably add some more if required (e.g., for completed TIPs). That
> way, we will gain when we have large jobs (many tasks) in the sense that we
> don't have to go through the big array every time a client request is made.
> We could instead iterate over the datastructures. Also, we should do these
> traversals without locking the JobTracker to avoid cases where long traversal
> leads to loss of heartbeats, etc. Some staleness at the client side (in favor
> of improving the JobTracker's performance) is probably okay.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
