[
https://issues.apache.org/jira/browse/HADOOP-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12595138#action_12595138
]
Amar Kamat commented on HADOOP-3296:
------------------------------------
bq. Minimally, calling the total number of maps scheduled "Local map tasks" is
confusing
Note that the counter will be incremented only if there is a locality at any
level. Its different from calling all the maps as LOCAL. I agree that we should
change the counter name. The intention is to show how many tasks got picked up
from the cache. So it includes node-local, rack-local, switch-local etc. I
think _Aggregate Locality_ sounds better, comments?
bq. For which users would this be valuable information?
In general when the users have cache topology more than 2 levels.
bq. If there are more than two levels in the task cache and the distinction is
significant, how is an aggregate counter resolving the ambiguity?
Consider a case where there is no locality at the node level and also at the
rack level. As per the trunk there is no way to infer whether the scheduling
went correctly or not. With this aggregate counter one can check if the maps
were from the cache or not. So the counter is just a count of how may maps got
picked up from the task cache.
> Some levels are skipped while creating the task cache in JobInProgress
> ----------------------------------------------------------------------
>
> Key: HADOOP-3296
> URL: https://issues.apache.org/jira/browse/HADOOP-3296
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Attachments: HADOOP-3296-v2.patch, HADOOP-3296.patch
>
>
> Consider the following piece of code
> {code:title=JobInProgress.createCache()|borderStyle=solid}
> Node node = jobtracker.resolveAndAddToTopology(host);
> for (int j = 0; j < maxLevel; j++) {
> node = JobTracker.getParentNode(node, j);
> .....
> {code}
> With {{maxLevel > 2}} the caches will be created in the following order
> ||j||node-level||
> |0|0|
> |1|1|
> |2|3|
> |3|6|
> which is not as desired.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
