Here we're using one machine for fetching (exclusive at the moment) with about 50 fetchers and a local Bind-resolver in caching-nameserver setup. Bandwidth of the fetchers is 5 to 10mbit inbound roughly.
What I see is that during fetching java is taking 99.9% cpu (all userland). At the point where the server "stalls" this changes to 99.9% on system-usage (writing something to disc?). It stalls for about 30 seconds or a bit more. Hmm - I don't know where to look for the cause of these stalls, since you don't see what it really does at that point (in logs or so). PS: Your thoughts on this are very much appreciated. Regards, Stefan Dennis Kubes wrote: > What we were seeing is the dns server cached the addresses in memory > (bind 9x..) and because we were caching so many addresses on a single > dns server it would eat up memory and eventually begin swapping to > disk. When this occurred the server load got up to 1.5 and the iowait > was near 100%. Basically it stalled the box. Requests were still > getting through but it was very slow. Our solution (at least > temporarily was to restart the bind service (not the box just the > daemon) every couple of hours to flush the memory. > As for load on the boxes we are seeing very minimal loads (like .08 > loads and no iowait times). We have about 55 fetchers running (5 on > each box with 11 nodes) and right now we are bandwidth bound on a 2Mbps > pipe. So maybe it is just that we don't have enough load on each > machine to see the kind of waits that you are seeing. Is your system > distributed or on a single machine ? > > Dennis > > Stefan Neufeind wrote: >> Hi Dennis, >> >> thank you for the answer. Hmm, could theoretically be. But to prevent >> this the server already does resolving completely on his local machine. >> Also I wonder about the CPU-load moving to "system" - I suspected heavy >> disk-access or so ... but I don't know how/when the fetcher writes data >> to disk etc. >> >> >> >> Regards, >> Stefan >> >> Dennis Kubes wrote: >> >>> Is this possibly a dns issue. We are running a 5M page crawl and are >>> seeing very heavy DNS load. Just a thought. >>> >>> Dennis >>> >>> Stefan Neufeind wrote: >>> >>>> Hi, >>>> >>>> I've encountered that here nutch is fetching quite a sum or URLs from a >>>> long list (about 25.000). But from time to time nutch is "waiting" for >>>> 10 seconds or so. Nothing is locked, but system-load is 99,9% then. Is >>>> nutch writing fetched data or index to disk at that stage? Is there any >>>> way to optimize this step, e.g. by writing more often and performing >>>> the >>>> write in "background" or caching even more in mem instead of >>>> flushing to disk? ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
