Rong-en Fan wrote:
I have two questions regarding the mapfile in hadoop/hdfs. First, when using
MapFileOutputFormat as reducer's output, is there any way to change
the index interval (i.e., able to call setIndexInterval() on the
output MapFile)?
Not at present. It would probably be good to change MapFile to get this
value from the Configuration. A static method could be added,
MapFile#setIndexInterval(Configuration conf, int interval), that sets
io.mapfile.index.interval, and the MapFile constructor could read this
property from the Configuration. One could then use the static method
to set this on jobs.
If you need this, please file an issue in Jira. If possible, include a
patch too.
http://wiki.apache.org/hadoop/HowToContribute
Second, is it possible to tell what is the position in data file for a given
key, assuming index interval is 1 and # of keys are small?
One could read the index file explicitly. It's just a SequenceFile,
listing keys and positions in the data file. But why would you set
the index interval to 1? And why do you need to know the position?
Doug
