Hi, the build-in spider at my site(s) (intranet test) hangs at ca. 50% of the index runs and of course does not get much of the sites links and depth (compared to e.g. ht://dig, etc.; urlfilters are set similar) although I run nutch crawl with "-depth 10.000" or "-depth 1.000" (nearly no difference in the amount of sites indexed). Any Ideas to this ?
Maybe is there an opportunity in nutch to get around the hangs by using another crawler/spider than the build in one? Regards Carsten ------------------------------------------------------- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 _______________________________________________ Nutch-general mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-general
