How would someone go about reading Map and Sequence file contents in
Mappers and Reducers? Is it best to only use the addInputDirectory
method and find a way to get all of the data one needs by key or is
there a good way to read file contents inside of map and reduce calls.
I am not asking about what classes (i.e. MapFile.Reader, etc.) I want
to know a consistent way to do it. Right now I am using a static
reference in the main class, called by an inner class. This works well
on one machine but I don't think it is going to work across nodes. Here
is how I have it now:
class x {
private static MapFile.Reader reader = null;
public x() {
reader = new MapFile.Reader...
create mapreduce job
setMapperClass(InnerX.class);
submit job
close reader.
}
class InnerX {
map(WritableComparable key, Writable value, OutputCollector output,
Reporter reporter) {
do something with reader...
}
}
}
Is there a better way to do this?
Dennis
-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
