Hi, Forgot to say the the problem doesn't occur if I crawl the same files on the local file system.
Thanks! Luca Luca Rondanini wrote: > Hi all, > > First of all....I've read all the posts regarding this problem in the > mailing list!! :) > > I'm try to index more than 200k documents. I'm reading those documents > through an nfs mount partition. Everything seems fine till we arrive at > 40k-50k documents....then the fetcher fails with the error "Hung Threads"!! > > These are the configurations that i've tried: > > 1) topN=20.000 > fetcher.threads=10 > ulimit -n=1024 > MergeFactor=20 > file.limit=1M > > ----> Hung Threads > > 2) topN=5000 > fetcher.threads=10 > ulimit -n=1024 > MergeFactor=20 > file.limit=1M > > ----> Hung Threads > > > 3) topN=5000 > fetcher.threads=5 > ulimit -n=1024 > MergeFactor=20 > file.limit=1M > > ----> Too many open file > > > 4) topN=5000 > fetcher.threads=5 > ulimit -n=4096 > MergeFactor=10 > file.limit=1M > > ----> Hung Threads > > > > Can anyone please give me a clue as to what is going on?!? > Thanks, > Luca ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
