Hi, I've encountered that here nutch is fetching quite a sum or URLs from a long list (about 25.000). But from time to time nutch is "waiting" for 10 seconds or so. Nothing is locked, but system-load is 99,9% then. Is nutch writing fetched data or index to disk at that stage? Is there any way to optimize this step, e.g. by writing more often and performing the write in "background" or caching even more in mem instead of flushing to disk?
Regards, Stefan ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
