Andrew McNabb wrote:
On Mon, Jan 09, 2006 at 05:00:00PM -0800, Doug Cutting wrote:To read sequence files directly outside of MapReduce, just use SequenceFile directly, e.g., something like: MyKey key = new MyKey(); MyValue value = new MyValue(); SequenceFile.Reader reader = new SequenceFile.reader(NutchFileSystem.get("local"), "file"); while (reader.next(key, value)) { ... process key/value pair ... } Wouldn't that be simpler?Who knows? Maybe it would be. :) With the approach that you just described, what's the easiest way to get all of the files in a directory (the full output of a reduce)?
NutchFileSystem fs = NutchFileSystem.get(); File[] files = fs.listFiles(directory); Doug
