Author: Alexander Barkov Email: [EMAIL PROTECTED] Message: Probably the reason is a memory leak in indexer. Type "top" and check, how much memory does indexer eat.
> Hi, > > While running 10 indexers simultaneously in cache-mode networking within 2 hours to >48 hours will completely stop on the machine. > > I cannot access the computer remotely or access the network from the computer and of >course the indexer gets timeouts for the site's its trying to crawl. > > I thought perhaps it was all the dns querries so I started up named and had it be >its own dns server but it did not fix it. > > This computer just does the crawling for another computer to serve the data, so >there is very little running on it at the moment to interefere with the crawling. > > Database is 3.8GB (shows no errors) > amd +1800 w/512MB > freebsd 4.5 > mysql 4.0.1 > mnoGoSearch-3.1.19 > > > Database statistics > > Status Expired Total > ----------------------------- > 0 1897472 1899198 Not indexed yet > 1 0 8 Unknown status > 200 0 250278 OK > 204 0 143 Unknown status > 300 0 89 Multiple Choices > 301 0 8120 Moved Permanently > 302 0 18993 Moved Temporarily > 303 0 3 See Other > 304 0 191247 Not Modified > 400 0 578 Bad Request > 401 0 377 Unauthorized > 402 0 1 Payment Required > 403 0 752 Forbidden > 404 0 37838 Not found > 415 0 26 Unsupported Media Type > 500 0 661 Internal Server Error > 502 0 13 Bad Gateway > 503 0 1961 Service Unavailable > 504 3 10002 Gateway Timeout > 505 0 2 Protocol Version Not Supported > 508 0 9 Unknown status > ----------------------------- > Total 1897475 2420299 > > Best Regards, > Jason > > > ps: Is there anyway to get the indexer to not hit a piticular webserver/ip more >than ten times out of a thousand indexer requests? > Just to be much more considerate of the utilization of other people's webservers. > > > Reply: <http://www.mnogosearch.org/board/message.php?id=4459> ___________________________________________ If you want to unsubscribe send "unsubscribe general" to [EMAIL PROTECTED]
