[
https://issues.apache.org/jira/browse/HADOOP-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12596165#action_12596165
]
Chris Douglas commented on HADOOP-3296:
---------------------------------------
bq. This should include all tasks that could be scheduled on a cached location
(excluding the data_local and rack_local), and it could be disabled for cache
level <= 2. Would this work? I believe Amar's testcase banks on this counter
being present.
That sounds easier to understand and parse than the current counter, which gets
incremented for all cache hits.
bq. As per the trunk there is no way to infer whether the scheduling went
correctly or not. With this aggregate counter one can check if the maps were
from the cache or not. So the counter is just a count of how may maps got
picked up from the task cache.
Though if we apply the same reasoning used to justify a third counter- the
distinction between a task scheduled between level 3 and a cache miss is
significant- isn't the distinction between level 3 and level 4 significant?
Without this counter, writing a good testcase for this fix would be very
difficult, but other particular uses for it are more difficult to summon
> Some levels are skipped while creating the task cache in JobInProgress
> ----------------------------------------------------------------------
>
> Key: HADOOP-3296
> URL: https://issues.apache.org/jira/browse/HADOOP-3296
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Attachments: HADOOP-3296-v2.patch, HADOOP-3296.patch
>
>
> Consider the following piece of code
> {code:title=JobInProgress.createCache()|borderStyle=solid}
> Node node = jobtracker.resolveAndAddToTopology(host);
> for (int j = 0; j < maxLevel; j++) {
> node = JobTracker.getParentNode(node, j);
> .....
> {code}
> With {{maxLevel > 2}} the caches will be created in the following order
> ||j||node-level||
> |0|0|
> |1|1|
> |2|3|
> |3|6|
> which is not as desired.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
