If you use StreamingUpdateSolrServer it POSTs all the docs in a single request. 10 million docs may be a bit too much for a single request. I guess you should batch it in multiple requests of smaller chunks,
It is likely that the CPU is really hot when the autowarming is hapening. getting a decent search perf w/o autowarming is not easy . autowarmCount is an attribute of a cache .see here http://wiki.apache.org/solr/SolrCaching On Mon, Apr 13, 2009 at 3:32 AM, vivek sar <vivex...@gmail.com> wrote: > Thanks Shalin. > > I noticed couple more things. As I index around 100 million records a > day, my Indexer is running pretty much at all times throughout the > day. Whenever I run a search query I usually get "connection reset" > when the commit is happening and get "blank page" when the > auto-warming of searchers is happening. Here are my questions, > > 1) Is this coincidence or a known issue? Can't we search while commit > or auto-warming is happening? > 2) How do I stop auto-warming? My search traffic is very low so I'm > trying to turn off auto-warming after commit has happened - is there > anything in the solrconfig.xml to do that? > 3) What would be the best strategy for searching in my scenario where > commits may be happening all the time (I commit every 50K records - so > every 30-60 sec there is a commit happening followed by auto-warming > that takes 40 sec)? > > Search frequency is pretty low for us, but we want to make sure that > whenever it happens it is fast enough and returns result (instead of > exception or a blank screen). > > Thanks for all the help. > > -vivek > > > > On Sat, Apr 11, 2009 at 1:48 PM, Shalin Shekhar Mangar > <shalinman...@gmail.com> wrote: >> On Sun, Apr 12, 2009 at 2:15 AM, vivek sar <vivex...@gmail.com> wrote: >> >>> >>> The problem is I don't see any error message in the catalina.out. I >>> don't even see the request coming in - I simply get blank page on >>> browser. If I keep trying the request goes through and I get respond >>> from Solr, but then it become unresponsive again or sometimes throws >>> "connection reset" error. I'm not sure why would it work sometimes and >>> not the other times for the same query. As soon as I stop the Indexer >>> process things start working fine. Any way I can debug this problem? >>> >> >> I'm not sure. I've never seen this issue myself. >> >> Could you try using the bundled jetty instead of Tomcat or on a different >> box just to make sure this is not an environment specific issue? >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> > -- --Noble Paul
