I have also notices similar problems here. I'm running only one fetching thread and the fetchg randomly stops for some reason. I once managed to open the lock by restarting the apache server which the fetcher was crawling, but that's just once :(
I also don't see any problems with dns queries, so your idea didn't work here either. It's strance, because nutch should have a socket timeout, which works in most cases, but not in these freezings. I'm still looking and studying what could cause this. - Juho Mäkinen, http://www.juhonkoti.net On 6/23/05, Sami Siren <[EMAIL PROTECTED]> wrote: > I have experienced similar random freezing in fetcher but after setting > up a local caching dns these problems went away. > > At least in my case the problem was due to connectivity to (some) remote > name servers. You can verify if this is your problem by doing something > like netstat -na|grep ":53 " while you suspect to have a frozen fetch > and look for connections that will not go away. > > -- > Sami Siren > > [EMAIL PROTECTED] wrote: > > Anyone experiencing freezes when fetching with 50 threads ? > > If I use 5 threads everything is fine - if i raise it to 10 > > it freezes and random times when fetching a segment. > > > > Any ideas? > > _____________________________________________________________________ > > For super low premiums, click here http://www.dialdirect.co.za/quote > > > >
