Hi, Thank you Emir, Erick and Shawn for your inputs.
I am currently using SolrCloud and planning to try out commitWithin parameter to reduce hard commits as per your advise. Though, just wanted to double check whether commitWithin have any negative impacts in SolrCloud environment like lag to search from other nodes in SolrCloud. Thanks, Leo Prince. On Tue, Oct 17, 2017 at 4:01 AM, Shawn Heisey <apa...@elyograg.org> wrote: > I'm supplementing the other replies you've already gotten. See inline: > > > On 10/13/2017 2:30 AM, Leo Prince wrote: > > I am getting the following errors/warnings from Solr > > 1, ERROR: > > org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: > > > Error opening new searcher. exceeded limit of maxWarmingSearchers=2, > > try again later. 2, PERFORMANCE WARNING: Overlapping > onDeckSearchers=2 > 3, WARN: DistributedUpdateProcessor error sending > See this FAQ entry: > > https://wiki.apache.org/solr/FAQ?highlight=%28ondecksearchers%29#What_ > does_.22exceeded_limit_of_maxWarmingSearchers.3DX.22_mean.3F > > > So my concern is, is there any chance of performance issues when > > number of commits are high at a particular point of time. In our > > application, we are approximating like 100-500 commits can happen > > simultaneously from application and autocommit too for those > > individual requests which are not committing individually after the > > write. > > Autocommit is configured as follows, > > <autoCommit> > <maxTime>15000</maxTime> > <openSearcher>false</openSearcher> > </autoCommit> > The commits generated by this configuration are not opening new > searchers, so they are not connected in any way to the error messages > you're getting, which are about new searchers. Note that this > particular kind of configuration is strongly recommended for ALL Solr > installs using Solr 4.0 and later, so that transaction logs do not grow > out of control. I would personally use a value of at least 60000 for > autoCommit, but there is nothing wrong with a 15 second interval. > > The initial reply you got on this thread mentioned that commits from the > application are discouraged. I don't agree with this statement, but I > will say that the way that people *use* commits from the application is > frequently very wrong, and because of that, switching to fully automatic > soft commits is often the best solution, because they are somewhat > easier to control. > > We have no way of knowing how long it will take to open a new searcher > on your index. It depends on a lot of factors. Whatever that time is, > commits should not be happening on a more frequent basis than that > interval. They should happen *less* frequently than that interval if at > all possible. Depending on exactly how Solr is configured, it might be > possible to reduce the amount of time that a commit with a new searcher > takes to complete. > > Definitely avoid sending a commit after every document. It is generally > also a bad idea to send a commit with every update request. If you want > to do commits manually, then you should index a bunch of data and then > send one commit to make all those changes visible, and not do another > commit until you do another batch of indexing. > > Thanks, > Shawn > >
