Generalize the SequenceFileInputFilter to apply to any InputFormat
------------------------------------------------------------------
Key: HADOOP-449
URL: http://issues.apache.org/jira/browse/HADOOP-449
Project: Hadoop
Issue Type: Improvement
Components: mapred
Affects Versions: 0.5.0
Reporter: Owen O'Malley
Assigned To: Owen O'Malley
Fix For: 0.6.0
I'd like to generalize the SequenceFileInputFormat that was introduced in
HADOOP-412 so that it can be applied to any InputFormat. To do this, I propose:
interface WritableFilter {
boolean accept(Writable item);
}
class FilterInputFormat implements InputFormat {
...
}
FilterInputFormat would look in the JobConf for:
mapred.input.filter.source = the underlying input format
mapred.input.filter.filters = a list of class names that implement
WritableFilter
The FilterInputFormat will work like the current SequenceFilter, but use an
internal RecordReader rather than the SequenceFile. This will require adding a
next(key) and getCurrentValue(value) to the RecordReader interface, but that
will be addressed in a different issue.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
