Re: Concern on solr commit

On 10/18/2017 3:09 AM, Leo Prince wrote: > Is there any known negative impacts in setting up autoSoftCommit as 1 > second other than RAM usage..? For most users, setting autoSoftCommit to one second is a BAD idea.  In many indexes, commits take longer than one second to complete.  If you do heavy

Re: Concern on solr commit

Hi Leo, If you gracefully shut down Solr documents will be committed. Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 20 Oct 2017, at 08:44, Leo Prince wrote: > >

Re: Concern on solr commit

Thank you Yonik. Since we are using SoftCommits, the docs written will be in RAM until a AutoCommit to reflect onto Disk, I just wanted to know what happens when Solr restarts. Being said, I am using 4.10 and tomcat is handling the Solr, when we restart the tomcat service just before an

Re: Concern on solr commit

On Wed, Oct 18, 2017 at 5:09 AM, Leo Prince wrote: > Is there any known negative impacts in setting up autoSoftCommit as 1 > second other than RAM usage..? Briefly: Don't use autowarming (but keep caches enabled!) Use docValues for fields you will facet and sort

Re: Concern on solr commit

Hi Yonik, Thank you for the inputs. When checked, Upgrading Solr and trying out commitWithin takes a lot of code change from the existing application codebase, hence we are planning to use autoSoftCommit as 1 second and maintaining autoCommit as 15 seconds. So we can maintain our NRT demands.

Re: Concern on solr commit

Related: maxWarmingSearchers behavior was fixed (block for another commit to succeed first rather than fail) in Solr 6.4 and later. https://issues.apache.org/jira/browse/SOLR-9712 Also, if any of your "realtime" search requests only involve retrieving certain documents by ID, then you can use

Re: Concern on solr commit

Hi, Thank you Emir, Erick and Shawn for your inputs. I am currently using SolrCloud and planning to try out commitWithin parameter to reduce hard commits as per your advise. Though, just wanted to double check whether commitWithin have any negative impacts in SolrCloud environment like lag to

Re: Concern on solr commit

I'm supplementing the other replies you've already gotten.  See inline: On 10/13/2017 2:30 AM, Leo Prince wrote: > I am getting the following errors/warnings from Solr > > 1, ERROR: > org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: > > Error opening new searcher. exceeded

Re: Concern on solr commit

Emir is spot on. Here's another thing though. You cannot see a document until after the commit happens as you well know. Pay close attention to this part of the error message: "Error opening new searcher. exceeded limit of maxWarmingSearchers=2, try again later." The commit generating this error

Re: Concern on solr commit

Hi Leo, Errors that you are seeing are related to frequent commits - new commits is issued before searcher for previous commit is opened and warmed. I haven’t looked in indexing code in a while, but if assume that it did not change, commits and writes are mutually exclusive - guarded by the

Re: Concern on solr commit

Hi Emir, Thanks for the response. We have specific near realtime search requirements, that is why we are explicitly invoking Solr commits. However we will try to improve on reducing the commits from application. In the meantime, the errors/warnings I mentioned in my previous mail; are they

Re: Concern on solr commit

Hi Leo, It is considered a bad practice to commit from your application. You should let Solr handle commits. There is a great article about soft and hard commits: https://lucidworks.com/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

Concern on solr commit

Hi, I am new to community and thank you for letting me in. Let me get into my concern real quick. Please find my OS and Solr versions Ubuntu 14.04.4 LTS solr-spec 4.10.2 solr-impl 4.10.2 1634293 - mike - 2014-10-26 05:56:21 lucene-spec 4.10.2 lucene-impl 4.10.2 1634293 - mike - 2014-10-26