Nutch 0.7.2 supports distributed searching, but its not exactly optimized. I wouldn't use it until you reach a segment (index) upwards of 20 million documents, then partition everything above that into consecutive 20 million (or less) document segments. This way each search server would have no more then 20 million documents indexed each.
The above statement also depends on the physical hardware your using. This page might help you out a bit, it was written a long time ago (2 years) but should apply perfectly for the version your using: http://wiki.media-style.com/display/nutchDocu/setup+multiple+search+sever ----- Original Message ---- From: Shrinivas Patwardhan <[EMAIL PROTECTED]> To: [email protected] Sent: Thursday, February 8, 2007 1:21:09 AM Subject: nutch 0.7.2 and distributed search hello all i just wanted to know if we can use the nutch 0.7.2 version for distributed searching ? or with hadoop ? -- Thanks & Regards Shrinivas Patwardhan
