Re: Triggering snapshooter through web admin interface
Lance, do start a new thread if you run into this problem again and please include as much info as possible. Once a snapshot has been taken, the files it contains should not change so I am not sure why tar was telling you a file had changed while it was being copied. Bill On 9/19/07, Chris Hostetter [EMAIL PROTECTED] wrote: lance: since the topic you are describing is not directly related to triggering a snapshot from the web interface can you please start a new thread with a unique subejct describing in more details exactly what it was you were doing and the problem you encountered? this will make it easier for your problem to get visibility (some people don't read every thread, and archive searching is frequently done by thread, so people looking for similar problems may not realize this new thread is burried inside an old one) -Hoss : Date: Wed, 19 Sep 2007 11:33:30 -0700 : From: Lance Norskog [EMAIL PROTECTED] : Reply-To: solr-user@lucene.apache.org : To: solr-user@lucene.apache.org : Subject: RE: Triggering snapshooter through web admin interface : : Is there a ticket for this yet? I have a bug report and request: I just did : a snapshot while indexing 700 records/sec. and got an inconsistency. I was : tarring off the snapshot and tar reported that a file changed while it was : being copied. The error rolled off my screen, so I cannot report the file : name or extension. : : If a solr command to do a snapshot is implemented, please make sure that it : is 100% consistent. : : Thanks, : : Lance Norskog
RE: Triggering snapshooter through web admin interface
lance: since the topic you are describing is not directly related to triggering a snapshot from the web interface can you please start a new thread with a unique subejct describing in more details exactly what it was you were doing and the problem you encountered? this will make it easier for your problem to get visibility (some people don't read every thread, and archive searching is frequently done by thread, so people looking for similar problems may not realize this new thread is burried inside an old one) -Hoss : Date: Wed, 19 Sep 2007 11:33:30 -0700 : From: Lance Norskog [EMAIL PROTECTED] : Reply-To: solr-user@lucene.apache.org : To: solr-user@lucene.apache.org : Subject: RE: Triggering snapshooter through web admin interface : : Is there a ticket for this yet? I have a bug report and request: I just did : a snapshot while indexing 700 records/sec. and got an inconsistency. I was : tarring off the snapshot and tar reported that a file changed while it was : being copied. The error rolled off my screen, so I cannot report the file : name or extension. : : If a solr command to do a snapshot is implemented, please make sure that it : is 100% consistent. : : Thanks, : : Lance Norskog
RE: Triggering snapshooter through web admin interface
: [Wu, Daniel] That sounds great. Do I need to create a JIRA ticket? Sure, JIRA is a great way to track feature requests (since they can be watched and voted on, and if you want to start on an implementation you can attach patches... http://wiki.apache.org/solr/HowToContribute -Hoss
RE: Triggering snapshooter through web admin interface
There is no way to trigger snapshots taking through Solr's admin interface now. Taking a snapshot is a very light-weight operation. It uses hard links so each snapshot doesn't take up much additional disk space. If you [Wu, Daniel] It is not a concern on the snapshot performance. Rather, it is the data consistency requirement. I was also suggesting a new feature to allow sending messages to Solr through http interface and a mechanism to handling the message on the Solr server; in this case, a message to trigger snapshooter script. It seems to me, a very useful feature to help simplify operational issues. don't want to replicate your index while the big batch job is still running, you can disable snappuller on the slave while the batch job is running and enable it after the batch job has completed. [Wu, Daniel] Yes, it can be done that way but will not be as elegant. The Solr master can be well-known among the applications; however, the slaves could be anywhere. To turn off snapulling requires the knowledge of all the Solr slaves as well as timing of the indexing. It gets ugly when there are multiple environments (e.g. dev, qa, stage, production) and multiple indexes.
RE: Triggering snapshooter through web admin interface
: I was also suggesting a new feature to allow sending messages to Solr : through http interface and a mechanism to handling the message on the : Solr server; in this case, a message to trigger snapshooter script. It : seems to me, a very useful feature to help simplify operational issues. it's been a while since i looked at the SolrEventListener stuff, but i think that would be pretty easy to develop as a plugin. The existing postCommit/postOptimizefirstSearcher/newSearcher event listener tracking are part of hte SolrCore because it needs to know about them when managing the index ... but if you just wanted a way to trigger arbitrary events by name, the utility functions used in SolrCore could be reused by a custom plugin ... then you could reuse things like the RunExecutableListener from your own RequestHandler with the same solrconfig.xml syntax. that would be a pretty cool addition to Solr ... an EventRequestHandler that takes in a single event param and triggers all of the Listeners configured for that even in the solrconfig.xml -Hoss
RE: Triggering snapshooter through web admin interface
-Original Message- From: Chris Hostetter [mailto:[EMAIL PROTECTED] Sent: Monday, September 17, 2007 1:28 PM To: solr-user@lucene.apache.org Subject: RE: Triggering snapshooter through web admin interface : I was also suggesting a new feature to allow sending messages to Solr : through http interface and a mechanism to handling the message on the : Solr server; in this case, a message to trigger snapshooter script. It : seems to me, a very useful feature to help simplify operational issues. it's been a while since i looked at the SolrEventListener stuff, but i think that would be pretty easy to develop as a plugin. The existing postCommit/postOptimizefirstSearcher/newSearcher event listener tracking are part of hte SolrCore because it needs to know about them when managing the index ... but if you just wanted a way to trigger arbitrary events by name, the utility functions used in SolrCore could be reused by a custom plugin ... then you could reuse things like the RunExecutableListener from your own RequestHandler with the same solrconfig.xml syntax. that would be a pretty cool addition to Solr ... an EventRequestHandler that takes in a single event param and triggers all of the Listeners configured for that even in the solrconfig.xml -Hoss [Wu, Daniel] That sounds great. Do I need to create a JIRA ticket?
Triggering snapshooter through web admin interface
Hi, I am not sure if this can be done. Let's say if periodically there is a big batch to be indexed and we don't want to replicate the data befor the batch is completely indexed. We would like to avoid post commit hook as we will be periodically committing to reduce the memory usage and we also would like to avoid post optimize hook as optimizing causes the replication of new big optimized file every time. Is there a way to trigger snap shooting through Solr's admin interface? For that matter, is there a generic way to trigger the Solr server to execute some pre-defined scripts? I know it can be achieved with some ssh scripts. It seems more elegant to be able to do so through the admin interface so the client doesn't need to know the physical location of the Solr installation. Daniel
