Andrew McNabb wrote:
On Mon, Jan 09, 2006 at 05:00:00PM -0800, Doug Cutting wrote:To read sequence files directly outside of MapReduce, just use SequenceFile directly, e.g., something like: MyKey key = new MyKey(); MyValue value = new MyValue(); SequenceFile.Reader reader = new SequenceFile.reader(NutchFileSystem.get("local"), "file"); while (reader.next(key, value)) { ... process key/value pair ... } Wouldn't that be simpler?Who knows? Maybe it would be. :) With the approach that you just described, what's the easiest way to get all of the files in a directory (the full output of a reduce)?
NutchFileSystem fs = NutchFileSystem.get(); File[] files = fs.listFiles(directory); Doug ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click _______________________________________________ Nutch-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-developers
