Hi guys,
Sometimes we get a bot crawling our search function on our retail web site. The ebay crawler loves to do this (Request.UserAgent: Terapeakbot). They just do a star search and then iterate through page after page. I've noticed that when they get to higher page numbers like page 9000, the searches are taking more than 20 seconds. Is this expected behavior? We're requesting standard facets with the search as well as incorporating boosting by function query. Our index is almost 15 million docs now and we're on Solr 3.6.1, this isn't causing any errors to occur at the solr layer but our web layer times out the search after 20 seconds and logs the exception. Thanks Robi
