[
https://issues.apache.org/jira/browse/HADOOP-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631744#action_12631744
]
Jothi Padmanabhan commented on HADOOP-3638:
-------------------------------------------
During the offline discussions, Chris expressed his reservations on the
dataStructure used in IndexCache.java for synchronizing access to the
IndexCache in the case where several threads try to insert the information to
the cache at the same time. The current logic uses a separate HashMap <MapID,
Lock> to synchronize the access. Any suggestions on how to do this better? The
current code is
{code}
IndexRecord getIndexInformation () {
indexInfo = cache.get(mapId);
if (indexInfo == null) {
indexInfo = readIndexFileToCache(fileName,mapId);
}
// do some book keeping for lru
return IndexInfo;
}
IndexInformation readIndexFileToCache {
synchronized (this) {
lock = lockMap.get(mapId);
if (lock == null) {
lock = new Object();
lockMap.put (mapId, lock);
}
}
synchronized (lock) {
info = cache.get(mapId);
if (info != null) return info;
//Code to Read Index Information from file
cache.put(info.mapId, info);
}
}
return info;
}
{code}
> Cache the iFile index files in memory to reduce seeks during map output
> serving
> -------------------------------------------------------------------------------
>
> Key: HADOOP-3638
> URL: https://issues.apache.org/jira/browse/HADOOP-3638
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Devaraj Das
> Assignee: Jothi Padmanabhan
> Fix For: 0.19.0
>
> Attachments: hadoop-3638-v1.patch, hadoop-3638-v2.patch,
> hadoop-3638-v3.patch, hadoop-3638-v4.patch, hadoop-3638-v5.patch
>
>
> The iFile index files can be cached in memory to reduce seeks during map
> output serving.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
