Hi Yonik, Thank you for the inputs.
When checked, Upgrading Solr and trying out commitWithin takes a lot of code change from the existing application codebase, hence we are planning to use autoSoftCommit as 1 second and maintaining autoCommit as 15 seconds. So we can maintain our NRT demands. Hence we can avoid frequent and numerous hard commits while we can search new docs written NRT. Is there any known negative impacts in setting up autoSoftCommit as 1 second other than RAM usage..? Thanks in advance. Regards, Leo Prince. On Tue, Oct 17, 2017 at 7:29 PM, Yonik Seeley <ysee...@gmail.com> wrote: > Related: maxWarmingSearchers behavior was fixed (block for another > commit to succeed first rather than fail) in Solr 6.4 and later. > https://issues.apache.org/jira/browse/SOLR-9712 > > Also, if any of your "realtime" search requests only involve > retrieving certain documents by ID, then you can use "realtime get" > without opening a new searcher. > > -Yonik > > > On Tue, Oct 17, 2017 at 9:45 AM, Leo Prince > <leoprince.discussi...@gmail.com> wrote: > > Hi, > > > > Thank you Emir, Erick and Shawn for your inputs. > > > > I am currently using SolrCloud and planning to try out commitWithin > > parameter to reduce hard commits as per your advise. Though, just wanted > to > > double check whether commitWithin have any > > negative impacts in SolrCloud environment like lag to search from other > > nodes in SolrCloud. > > > > Thanks, > > Leo Prince. > > > > On Tue, Oct 17, 2017 at 4:01 AM, Shawn Heisey <apa...@elyograg.org> > wrote: > > > >> I'm supplementing the other replies you've already gotten. See inline: > >> > >> > >> On 10/13/2017 2:30 AM, Leo Prince wrote: > >> > I am getting the following errors/warnings from Solr > > 1, ERROR: > > >> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: > > >> > Error opening new searcher. exceeded limit of maxWarmingSearchers=2, > >> > try again later. 2, PERFORMANCE WARNING: Overlapping > > onDeckSearchers=2 > >> 3, WARN: DistributedUpdateProcessor error sending > >> See this FAQ entry: > >> > >> https://wiki.apache.org/solr/FAQ?highlight=%28ondecksearchers%29#What_ > >> does_.22exceeded_limit_of_maxWarmingSearchers.3DX.22_mean.3F > >> > >> > So my concern is, is there any chance of performance issues when > > >> number of commits are high at a particular point of time. In our > > >> application, we are approximating like 100-500 commits can happen > > >> simultaneously from application and autocommit too for those > > >> individual requests which are not committing individually after the > > >> write. > > Autocommit is configured as follows, > > <autoCommit> > >> <maxTime>15000</maxTime> > <openSearcher>false</openSearcher> > >> </autoCommit> > >> The commits generated by this configuration are not opening new > >> searchers, so they are not connected in any way to the error messages > >> you're getting, which are about new searchers. Note that this > >> particular kind of configuration is strongly recommended for ALL Solr > >> installs using Solr 4.0 and later, so that transaction logs do not grow > >> out of control. I would personally use a value of at least 60000 for > >> autoCommit, but there is nothing wrong with a 15 second interval. > >> > >> The initial reply you got on this thread mentioned that commits from the > >> application are discouraged. I don't agree with this statement, but I > >> will say that the way that people *use* commits from the application is > >> frequently very wrong, and because of that, switching to fully automatic > >> soft commits is often the best solution, because they are somewhat > >> easier to control. > >> > >> We have no way of knowing how long it will take to open a new searcher > >> on your index. It depends on a lot of factors. Whatever that time is, > >> commits should not be happening on a more frequent basis than that > >> interval. They should happen *less* frequently than that interval if at > >> all possible. Depending on exactly how Solr is configured, it might be > >> possible to reduce the amount of time that a commit with a new searcher > >> takes to complete. > >> > >> Definitely avoid sending a commit after every document. It is generally > >> also a bad idea to send a commit with every update request. If you want > >> to do commits manually, then you should index a bunch of data and then > >> send one commit to make all those changes visible, and not do another > >> commit until you do another batch of indexing. > >> > >> Thanks, > >> Shawn > >> > >> >
