[ http://issues.apache.org/jira/browse/HADOOP-603?page=comments#action_12456202 ] Doug Cutting commented on HADOOP-603: -------------------------------------
This sounds reasonable to me. Are you working on a patch? > Extend SequenceFile to provide MapFile function by storing index at the end > of the file > --------------------------------------------------------------------------------------- > > Key: HADOOP-603 > URL: http://issues.apache.org/jira/browse/HADOOP-603 > Project: Hadoop > Issue Type: Improvement > Components: dfs > Reporter: Jim Kellerman > > MapFile increases the load on the name node as two files are created to > provide a index file format. If SequenceFile were extended by storing the > index at the end of the file, 1/2 of the files currently created for a > map/reduce operation would be needed, reducing the load on the name node. > Perhaps this is why Google implemented SSTable files in this manner. (SSTable > files are functionally identical to Hadoop MapFiles) (see the paper on > BigTable - section 4 "Building Blocks" > http://labs.google.com/papers/bigtable.html) -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
