bq: we post json documents through the curl it takes the time (same time i would like to say that we are not hard committing ). that curl takes time i.e. 1.3 sec.
OK, I'm really confused. _what_ is taking 1.3 seconds? When you said commit, I was thinking of Solr's commit operation, which is totally distinct from just adding a doc to the index. But I read the above statement as you're saying it takes 1.3 seconds just to send a doc to Solr. Let's see the exact curl command you're using please? Best, Erick On Thu, Aug 11, 2016 at 5:32 AM, Emir Arnautovic <emir.arnauto...@sematext.com> wrote: > Hi Midas, > > 1. How many indexing threads? > 2. Do you batch documents and what is your batch size? > 3. How frequently do you commit? > > I would recommend: > 1. Move commits to Solr (set auto soft commit to max allowed time) > 2. Use batches (bulks) > 3. tune bulk size and number of threads to achieve max performance. > > Thanks, > Emir > > > > On 11.08.2016 08:21, Midas A wrote: >> >> Emir, >> >> other queries: >> >> a) Solr cloud : NO >> b) <filterCache class="solr.FastLRUCache" >> size="5000" initialSize="5000" autowarmCount="10"/> >> c) <queryResultCache class="solr.LRUCache" >> size="1000" initialSize="1000" autowarmCount="10"/> >> d) <documentCache class="solr.LRUCache" >> size="1000" initialSize="1000" autowarmCount="10"/> >> e) we are using multi threaded system. >> >> On Thu, Aug 11, 2016 at 11:48 AM, Midas A <test.mi...@gmail.com> wrote: >> >>> Emir, >>> >>> we post json documents through the curl it takes the time (same time i >>> would like to say that we are not hard committing ). that curl takes time >>> i.e. 1.3 sec. >>> >>> On Wed, Aug 10, 2016 at 2:29 PM, Emir Arnautovic < >>> emir.arnauto...@sematext.com> wrote: >>> >>>> Hi Midas, >>>> >>>> According to your autocommit configuration and your worry about commit >>>> time I assume that you are doing explicit commits from client code and >>>> that >>>> 1.3s is client observed commit time. If that is the case, than it might >>>> be >>>> opening searcher that is taking time. >>>> >>>> How do you index data - single threaded or multithreaded? How frequently >>>> do you commit from client? Can you let Solr do soft commits instead of >>>> explicitly committing? Do you have warmup queries? Is this SolrCloud? >>>> What >>>> is number of servers (what spec), shards, docs? >>>> >>>> In any case monitoring can give you more info about server/Solr behavior >>>> and help you diagnose issues more easily/precisely. One such monitoring >>>> tool is our SPM <http://sematext.com/spm>. >>>> >>>> Regards, >>>> Emir >>>> >>>> -- >>>> Monitoring * Alerting * Anomaly Detection * Centralized Log Management >>>> Solr & Elasticsearch Support * http://sematext.com/ >>>> >>>> On 10.08.2016 05:20, Midas A wrote: >>>> >>>>> Thanks for replying >>>>> >>>>> index size:9GB >>>>> 2000 docs/sec. >>>>> >>>>> Actually earlier it was taking less but suddenly it has increased . >>>>> >>>>> Currently we do not have any monitoring tool. >>>>> >>>>> On Tue, Aug 9, 2016 at 7:00 PM, Emir Arnautovic < >>>>> emir.arnauto...@sematext.com> wrote: >>>>> >>>>> Hi Midas, >>>>>> >>>>>> Can you give us more details on your index: size, number of new docs >>>>>> between commits. Why do you think 1.3s for commit is to much and why >>>>>> do >>>>>> you >>>>>> need it to take less? Did you do any system/Solr monitoring? >>>>>> >>>>>> Emir >>>>>> >>>>>> >>>>>> On 09.08.2016 14:10, Midas A wrote: >>>>>> >>>>>> please reply it is urgent. >>>>>>> >>>>>>> On Tue, Aug 9, 2016 at 11:17 AM, Midas A <test.mi...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>> Hi , >>>>>>> >>>>>>>> commit is taking more than 1300 ms . what should i check on server. >>>>>>>> >>>>>>>> below is my configuration . >>>>>>>> >>>>>>>> <autoCommit> <maxTime>${solr.autoCommit.maxTime:15000}</maxTime> < >>>>>>>> openSearcher>false</openSearcher> </autoCommit> <autoSoftCommit> >>>>>>>> <maxTime> >>>>>>>> ${solr.autoSoftCommit.maxTime:-1}</maxTime> </autoSoftCommit> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>> >>>>>> Monitoring * Alerting * Anomaly Detection * Centralized Log Management >>>>>> Solr & Elasticsearch Support * http://sematext.com/ >>>>>> >>>>>> >>>>>> > > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch Support * http://sematext.com/ >
