[
https://issues.apache.org/jira/browse/HADOOP-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12595040#action_12595040
]
Chris Douglas commented on HADOOP-3296:
---------------------------------------
bq. The reason is that if there are more than 2 levels in the task cache (e.g
local/rack/gateway/...) then there is no way to know if the maps were scheduled
locally (at a level above rack) or not.
Minimally, calling the total number of maps scheduled "Local map tasks" is
confusing. I'll grant without hesitation that hard-coding two levels doesn't
cover all topologies, but this counter provides only slightly more information
than the existing counters, diluted. For which users would this be valuable
information? If there are more than two levels in the task cache and the
distinction is significant, how is an aggregate counter resolving the
ambiguity? Please correct this impression if it's mistaken, but this counter is
a partial fix to an issue orthogonal to the one described in this JIRA; it
should be left out.
> Some levels are skipped while creating the task cache in JobInProgress
> ----------------------------------------------------------------------
>
> Key: HADOOP-3296
> URL: https://issues.apache.org/jira/browse/HADOOP-3296
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Attachments: HADOOP-3296-v2.patch, HADOOP-3296.patch
>
>
> Consider the following piece of code
> {code:title=JobInProgress.createCache()|borderStyle=solid}
> Node node = jobtracker.resolveAndAddToTopology(host);
> for (int j = 0; j < maxLevel; j++) {
> node = JobTracker.getParentNode(node, j);
> .....
> {code}
> With {{maxLevel > 2}} the caches will be created in the following order
> ||j||node-level||
> |0|0|
> |1|1|
> |2|3|
> |3|6|
> which is not as desired.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
