qi wu wrote:
I wonder why so much space are required by linkdb reduce job, can I
config some nutch or hadoop setting to reduce the disk space usage
for linkdb? Any hints for me to overcome the problem? //bow
* what is you limit of links per url (db.max.inlinks property)? You
could try lowering it.
* Please try the following modification: somewhere around
LinkDb.java:283 add the following line:
job.setCombinerClass(LinkDb.class);
Recompile and re-run.
* Also, as others suggested, you may want to turn on compression.
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
