Re: Different update handlers for auto commit configuration
Thanks for the clarification, I indeed mixed it with UpdateRequestHandler. On Mon, Dec 1, 2014 at 11:24 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : I thought that the auto commit is per update handler because they are : configured within the update handler tag. updateHandler is not the same thing as a requestHandler that does updates. there can be many Update request handlers configured, but there is only ever one updateHandler/ in a SolrCore. -Hoss http://www.lucidworks.com/
Re: Different update handlers for auto commit configuration
Thanks for the reply Erik, I thought that the auto commit is per update handler because they are configured within the update handler tag. From a little debugging on SOLR code and it looked as if the Commit Tracker will schedule a soft commit on update if it gets commitwithin value, and if not it will use the autosofcommit timeUpperBound. So, according to my understanding, the autosoftcommit should be equivalent to the commit within. Anyway, it looks like the commitWithin is more flexible, so I'll try it. Regarding the requirement, the problem is that we are suppose to be Near real time. Since we are already in production it's harder to change the requirements. You are right about the caches and warming, the cache is not being used that much and we cannot do warming at all. But, the trigger for the change is that on load time the commit frequency increases and we get the performance warnings. We may have a few commits in the same second. So, as a first step if we set it to 2 we will have a benefit. In addition, on our big cores we have an average of commit per second on 24 hours, so still there's some benefit. On Mon, Dec 1, 2014 at 12:42 AM, Erick Erickson erickerick...@gmail.com wrote: Uhhhm, the soft/hard commit settings are global, not configured in each update handler. How are updates being done? Because if you're using SolrJ, you can just use the server.add(doclist, commitwithin) and it'll just be handled automatically. True, putting a 2 second commitwithin on an online update would pick up some batch updates that happened to come in, but that's probably OK. I'd also be sure that the 2 second requirement is real. Soft commits aren't as expensive as hard commits with openSearcher=true, but they aren't free either. At that fast a commit rate you probably won't get much benefit out of the top-level caches, and you'll be warming an awful lot. FWIW, Erick On Sun, Nov 30, 2014 at 12:32 PM, danny teichthal dannyt...@gmail.com wrote: Hi, On our system we currently initiate a soft commit to SOLR after each business transaction that initiate an update. Hard commits are automatic each 2 minutes. We want to limit the explicit commit and move to autoSoftCommit. Because of business restrictions: Online request should be available for searching after 2 seconds. Update from batch jobs can be available after 10 seconds. (maybe more, currently on discussion). There are some transactions that must be available immediately. Question I thought about creating 3 different update handlers, each containing a different autoSoftCommit configuration. Is this an acceptable solution, are there any downsides in using multiple update handlers? Thanks,
Re: Different update handlers for auto commit configuration
: I thought that the auto commit is per update handler because they are : configured within the update handler tag. updateHandler is not the same thing as a requestHandler that does updates. there can be many Update request handlers configured, but there is only ever one updateHandler/ in a SolrCore. -Hoss http://www.lucidworks.com/
Different update handlers for auto commit configuration
Hi, On our system we currently initiate a soft commit to SOLR after each business transaction that initiate an update. Hard commits are automatic each 2 minutes. We want to limit the explicit commit and move to autoSoftCommit. Because of business restrictions: Online request should be available for searching after 2 seconds. Update from batch jobs can be available after 10 seconds. (maybe more, currently on discussion). There are some transactions that must be available immediately. Question I thought about creating 3 different update handlers, each containing a different autoSoftCommit configuration. Is this an acceptable solution, are there any downsides in using multiple update handlers? Thanks,
Re: Different update handlers for auto commit configuration
Uhhhm, the soft/hard commit settings are global, not configured in each update handler. How are updates being done? Because if you're using SolrJ, you can just use the server.add(doclist, commitwithin) and it'll just be handled automatically. True, putting a 2 second commitwithin on an online update would pick up some batch updates that happened to come in, but that's probably OK. I'd also be sure that the 2 second requirement is real. Soft commits aren't as expensive as hard commits with openSearcher=true, but they aren't free either. At that fast a commit rate you probably won't get much benefit out of the top-level caches, and you'll be warming an awful lot. FWIW, Erick On Sun, Nov 30, 2014 at 12:32 PM, danny teichthal dannyt...@gmail.com wrote: Hi, On our system we currently initiate a soft commit to SOLR after each business transaction that initiate an update. Hard commits are automatic each 2 minutes. We want to limit the explicit commit and move to autoSoftCommit. Because of business restrictions: Online request should be available for searching after 2 seconds. Update from batch jobs can be available after 10 seconds. (maybe more, currently on discussion). There are some transactions that must be available immediately. Question I thought about creating 3 different update handlers, each containing a different autoSoftCommit configuration. Is this an acceptable solution, are there any downsides in using multiple update handlers? Thanks,
commit configuration
Hi, I'm using DIH and want to perform commits each N processed document, how can I do this? thanks in advance -- Anass
Re: commit configuration
http://svn.apache.org/repos/asf/lucene/dev/trunk/solr/example/solr/conf/solrconfig.xml Look for autocommit and maxDocs. Hi, I'm using DIH and want to perform commits each N processed document, how can I do this? thanks in advance
