If you index from the outside (i.e. not using DIH) you have more control: * how many threads you use * how you batch documents * how much you wait between indexing batches ...
Otis -- Solr & ElasticSearch Support http://sematext.com/ On Fri, Jan 4, 2013 at 6:25 PM, Marcin Rzewucki <mrzewu...@gmail.com> wrote: > Thanks. I guess you're right - it's normal behaviour. Are there some > guidelines how to use ramBufferSizeMB or only by testing ? Do you know if > DIH is "gentler" than indexing via REST or solrj API ? > Kind regards. > > On 4 January 2013 23:14, Otis Gospodnetic <otis.gospodne...@gmail.com > >wrote: > > > Hi, > > > > I think what you are seeing is a general thing. Regular search is slower > > while there is indexing, too, of course. > > So maybe it's best to mentally decouple indexing part here and simply > make > > your calls as fast as possible without indexing. Then you can add > indexing > > and play with things like ramBufferSizeMB and anything else that has the > > potential of making indexing "gentler" on resources, be that CPU or disk > > or... > > > > Otis > > -- > > Solr & ElasticSearch Support > > http://sematext.com/ > > > > > > > > > > > > On Fri, Jan 4, 2013 at 4:44 PM, Marcin Rzewucki <mrzewu...@gmail.com> > > wrote: > > > > > Hi all, > > > > > > I'm using SolrCloud4x. I've experienced some problem with > StatsComponent. > > > It looks like query time increases during indexing. For example the > > > following query: > > > > > > > > > > > > http://host:8983/solr/core/select?q=*:*&wt=xml&shards.tolerant=true&stats=true&stats.field=total_assets_stl&stats.field=eur_total_revenues_stl&stats.field=eur_total_assets_stl&stats.field=total_revenues_stl&stats.field=usd_total_revenues_stl&stats.field=usd_total_assets_stl&stats.field=eur_total_assets_stl&rows=0 > > > > > > takes less than 1s when there's no indexing in background and more than > > 1s > > > to couple of seconds while indexing. I'm using Trie fields with > > > precisionStep set to -1 (it was precisionStep="8" before, but query > times > > > were much worse, so I changed it). My SolrCloud uses m1.large nodes in > > AWS > > > (7.5 GiB), mmap for index reading and 2GB for JVM, default settings for > > > cache. I wonder what is the reason that StatsComponent is much slower > > > during indexing ? Or is it normal behaviour ? Is it possible to improve > > it > > > ? Any ideas are welcome. > > > > > > Thanks! > > > > > >
