There are a bunch of variables. If there are too many merge threads going on, for instance, then the commit will block until one of the merge threads finishes. It could well be that the one you identify as “slow” is coincidentally after the hard commit, which are could accumulate for 10 minutes or 20K documents.
A 100ms autocommit time is far too rapid. Theres quite a bit of work done for opening a new searcher. I _strongly_ recommend you make this longer. It’s actually quite rare that users even notice 5 second delays. Your “slow” commit is taking 2 seconds, which isn’t all that long. I see you’ve commented out your autowarm settings, but if you ever uncomment them, start with autowarm counts in the 10-20 range. With the settings you have, if you un-commented them you’d be getting 2,560 fq clauses and the most recent 10,240 queries have been re-executed. Best, Erick > On Aug 26, 2020, at 10:04 AM, vishal patel <vishalpatel200...@outlook.com> > wrote: > > Thanks for your quick reply. > > Commit is not called from client side. > We do not use any cache. Here is my solrconfig.xml : > https://drive.google.com/file/d/1LwA1d4OiMhQQv806tR0HbZoEjA8IyfdR/view > > We give set SOLR_OPTS=%SOLR_OPTS% -Dsolr.autoSoftCommit.maxTime=100 because > we want quick view after indexing. > > My main question is why it is taking time for only form > collection(commitScheduler-124-thread-1)? > Why it is committing fast for other > collection(commitScheduler-115-thread-1,commitScheduler-123-thread-1,commitScheduler-121-thread-1,commitScheduler-128-thread-1)? > My Solrconfig.xml is same for all collection. > Does it depend on index size of documents which I > send(FORM1056159510735194,FORM1056159810735197,FORM1056160010735199)? > Does it depend on index size of form collection? > > Regards, > Vishal Patel > ________________________________ > From: Erick Erickson <erickerick...@gmail.com> > Sent: Wednesday, August 26, 2020 5:36 PM > To: solr-user@lucene.apache.org <solr-user@lucene.apache.org> > Subject: Re: Slow commit in Solr 6.1.0 > > It depends on how the commit is called. You have openSearcher=true, which > means the call > won’t return until all your autowarming is done. This _looks_ like it might > be a commit > called from a client, which you should not do. > > It’s also suspicious that these are soft commits 1 second apart. The other > possibility is that > you have your autoSoftCommit interval set to 1000 ms. This is usually far too > fast. > > Here’s more than you want to know about how commits work: > > https://lucidworks.com/post/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ > > Best, > Erick > >> On Aug 26, 2020, at 5:29 AM, vishal patel <vishalpatel200...@outlook.com> >> wrote: >> >> I am using solr 6.1.0. We have 2 shards and each has one replica. >> When I checked shard1 log, I found that commit process was going to slow for >> some collection. >> >> Slow commit: >> 2020-08-25 09:08:10.328 INFO (commitScheduler-124-thread-1) [c:forms >> s:shard1 r:core_node1 x:forms] o.a.s.u.DirectUpdateHandler2 start >> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false} >> 2020-08-25 09:08:11.424 INFO (commitScheduler-124-thread-1) [c:forms >> s:shard1 r:core_node1 x:forms] o.a.s.s.SolrIndexSearcher Opening >> [Searcher@5def3a5a[forms] main] >> 2020-08-25 09:08:11.932 INFO (commitScheduler-124-thread-1) [c:forms >> s:shard1 r:core_node1 x:forms] o.a.s.u.DirectUpdateHandler2 end_commit_flush >> >> 2020-08-25 09:08:11.935 INFO (commitScheduler-124-thread-1) [c:forms >> s:shard1 r:core_node1 x:forms] o.a.s.u.DirectUpdateHandler2 start >> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false} >> 2020-08-25 09:08:13.676 INFO (commitScheduler-124-thread-1) [c:forms >> s:shard1 r:core_node1 x:forms] o.a.s.s.SolrIndexSearcher Opening >> [Searcher@15c82f14[forms] main] >> 2020-08-25 09:08:14.071 INFO (commitScheduler-124-thread-1) [c:forms >> s:shard1 r:core_node1 x:forms] o.a.s.u.DirectUpdateHandler2 end_commit_flush >> >> I have found that all threads other than commitScheduler-124-thread-1 were >> fast.(commitScheduler-115-thread-1,commitScheduler-123-thread-1,commitScheduler-121-thread-1,commitScheduler-128-thread-1) >> what is the reason for slow commit? Why other threads are not slow? >> >> Here is my full log: >> shard1: >> https://drive.google.com/file/d/1jim55pbYxQPORpGJSNjmek5OdraGX79h/view >> shard1 replica: >> https://drive.google.com/file/d/1o34kEO1ZZwPE6eF1ppxsQnwL7wLqcx-o/view >> >> Regards, >> Vishal Patel >> >> <http://aka.ms/weboutlook> >
