Author: Alexander Barkov Email: [email protected] Message: > Hello, > > I've tried this : > > ./indexer -p 0 > > but it doesn't work :( > The indexer sleeps for at least one seconde after each URL.
With -p0 it does not do any delays between URLs. I guess the bottleneck is in the connection, or in the remote site. To speed up crawling performance, you can run multiple crawling threads in parallel, for example: indexer -N5 Make sure not to put the the remote site down though. > > It seems impossible to index faster than 1s after each url. > > To index 300 000 document on my website for example, the crawl takes 2 full > days ! > > Is there a solution ? Reply: <http://www.mnogosearch.org/board/message.php?id=21762> _______________________________________________ General mailing list [email protected] http://lists.mnogosearch.org/listinfo/general
