Hi Eric, I totally agree. That's what I also figured ultimately. One thing I am not clear. The replication is supposed to be incremental ? But looks like it is trying to replicate the whole index. May be I am changing the index so frequently, it is triggering auto merge and a full replication ? I am thinking in right direction?
I see that when I start the solr search instance before I start feeding the solr Index, my searches are fine BUT it is using the old searcher so I am not seeing the updates in the result. So now I am trying to change my architecture. I am going to have a core dedicated to receive daily updates, which is going to be 5 million docs and size is going to be little less than 5 G, which is small and replication will be faster? I will search both the cores i.e. old data and the daily updates and do a field collapsing on my unique id so that I do not return duplicate results .I haven't tried grouping results ; so not sure about the performance. Any suggestion ? Eventually I have to use Solr trunk like you suggested. Thank you for your help, On Wed, Jul 18, 2012 at 10:28 AM, Erick Erickson [via Lucene] < ml-node+s472066n3995754...@n3.nabble.com> wrote: > bq: This index is only used for searching and being replicated every 7 sec > from > the master. > > This is a red-flag. 7 second replication times are likely forcing your > app to spend > all its time opening new searchers. Your cached filter queries are > likely rarely being re-used > because they're being thrown away every 7 seconds. This assumes you're > changing your master index frequently. > > If you need near real time, consider Solr trunk and SolrCloud, but > trying to simulate > NRT with very short replication intervals is usually a bad idea. > > A quick test would be to disable replication for a bit (or lengthen it > to, say, 10 minutes) > > Best > Erick > > On Tue, Jul 17, 2012 at 10:47 PM, Fuad Efendi <[hidden > email]<http://user/SendEmail.jtp?type=node&node=3995754&i=0>> > wrote: > > > > >> FWIW, when asked at what point one would want to split JVMs and shard, > >> on the same machine, Grant Ingersoll mentioned 16GB, and precisely for > >> GC cost reasons. You're way above that. > > > > - his index is 75G, and Grant mentioned RAM heap size; we can use > terabytes > > of index with 16Gb memory. > > > > > > > > > > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://lucene.472066.n3.nabble.com/Using-Solr-3-4-running-on-tomcat7-very-slow-search-tp3995436p3995754.html > To unsubscribe from Using Solr 3.4 running on tomcat7 - very slow search, > click > here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3995436&code=bW91bmFuZGlAZ21haWwuY29tfDM5OTU0MzZ8Mjg1MTA5MTUw> > . > NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://lucene.472066.n3.nabble.com/Using-Solr-3-4-running-on-tomcat7-very-slow-search-tp3995436p3995774.html Sent from the Solr - User mailing list archive at Nabble.com.
