Dear Stephan,
Thanks for you fast answer.
I think there are some general 'security' hole in nutch. E.g.: if I make queries with hitsPerPage=10000, or if the user press F5 key in IE for long time.
In my situation the problem is 'paginating' like google (pages: 1-10). If the isTotalIsExact() results false -> research with hitsPerPage * 10.
I think I will set maxHitsPerSite value to 0 for a week, and I will try to reanalize how to reprograming the 'paginating'.
Thanks, Ferenc
Stefan Groschupf wrotte:
I notice similar behaviors.
I guess the backend servers does not answering fast enough.
I was thinking about to have multiple search server groups that have identical content and then query groups in a round robbing style.
What people think about this idea?
It is already easy to setup multiple tomcat that use different search servers and simply split traffic by adding 2 or n ip to your dns for the same domain.
Stefan
Am 18.05.2005 um 16:59 schrieb [EMAIL PROTECTED]:
Dear Users!
Firstly sorry my bad English.
I read Stephans great documentation at http://wiki.media-style.com/ display/nutchDocu/.
I maked a frontend (P4 3 GByte RAM, Tomcat 5.5.7 java 1.4.08) with 3 backend with 12 million pages ( 4million / backend AMD64 4 GByte RAM 64 bit linux with jdk 1.5_03).
When I start using it with 3-5 queries / sec, after 1-2 minute the frontend does'nt answer to the requests.
In the Tomcat manager / status I see there is many thread busy (150 and it increasses, now 241), and these are with Stage 'S' (Service).
The backend with usage: top 40-60 % CPU. The frontend with usage: 5% CPU.
Have you any idea what is the problem?
Best Regards, Ferenc
--------------------------------------------------------------- company: http://www.media-style.com forum: http://www.text-mining.org blog: http://www.find23.net
