It should be possible to use the hash of a key to work-out which shard it is present in; you would then search over all entries in the relevant shard.
Miles On 03/03/2008, Xavier Stevens <[EMAIL PROTECTED]> wrote: > > I am curious how others might be solving this problem. I want to > retrieve a record from HDFS based on its key. Are there any methods > that can shortcut this type of search to avoid parsing all data until > you find it? Obviously Hbase would do this as well, but I wanted to > know if there is a way to do it using just Map/Reduce and HDFS. > > Thanks, > > > -Xavier > > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
