Re: Dinamically change master
Back with my master resiliency need, talking with Upayavira we discovered we were proposing the same solution :-) This can be useful if you don't have a VIP with master/backup polling policy. It goes like this: there are 2 host for indexing, one is the main and one is the backup one, the backup one is slave of the main one and the main one is also master of N hosts which will be used for searching. If the main master goes down then the backup one will be used for indexing and/or serving search slaves. This last feature can be done defining an external properties file for each search slave which will contain the URL to master (pointed inside the replication request handler tag of solrconfig.xml), so if these search slaves run on multi core one has only to change properties file URL and issue a http://SLAVEURL/solr/admin/cores?action=RELOADcore=core0 to get polling the backup master. Cheers, Tommaso 2010/12/1 Tommaso Teofili tommaso.teof...@gmail.com Thanks Upayavira, that sounds very good. p.s.: I read that page some weeks ago and didn't get back to check on it. 2010/12/1 Upayavira u...@odoko.co.uk Note, all extracted from http://wiki.apache.org/solr/SolrReplication You'd put: requestHandler name=/replication class=solr.ReplicationHandler lst name=master !--Replicate on 'startup' and 'commit'. 'optimize' is also a valid value for replicateAfter. -- str name=replicateAfterstartup/str str name=replicateAftercommit/str /lst /requestHandler into every box you want to be able to act as a master, then use: http://slave_host:port /solr/replication?command=fetchindexmasterUrl=your master URL As the above page says better than I can, It is possible to pass on extra attribute 'masterUrl' or other attributes like 'compression' (or any other parameter which is specified in the lst name=slave tag) to do a one time replication from a master. This obviates the need for hardcoding the master in the slave. HTH, Upayavira On Wed, 01 Dec 2010 06:24 +0100, Tommaso Teofili tommaso.teof...@gmail.com wrote: Hi Upayavira, this is a good start for solving my problem, can you please tell how does such a replication URL look like? Thanks, Tommaso 2010/12/1 Upayavira u...@odoko.co.uk Hi Tommaso, I believe you can tell each server to act as a master (which means it can have its indexes pulled from it). You can then include the master hostname in the URL that triggers a replication process. Thus, if you triggered replication from outside solr, you'd have control over which master you pull from. Does this answer your question? Upayavira On Tue, 30 Nov 2010 09:18 -0800, Ken Krugler kkrugler_li...@transpac.com wrote: Hi Tommaso, On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: Hi all, in a replication environment if the host where the master is running goes down for some reason, is there a way to communicate to the slaves to point to a different (backup) master without manually changing configuration (and restarting the slaves or their cores)? Basically I'd like to be able to change the replication master dinamically inside the slaves. Do you have any idea of how this could be achieved? One common approach is to use VIP (virtual IP) support provided by load balancers. Your slaves are configured to use a VIP to talk to the master, so that it's easy to dynamically change which master they use, via updates to the load balancer config. -- Ken -- Ken Krugler +1 530-210-6378 http://bixolabs.com e l a s t i c w e b m i n i n g
Re: Dinamically change master
Note, all extracted from http://wiki.apache.org/solr/SolrReplication You'd put: requestHandler name=/replication class=solr.ReplicationHandler lst name=master !--Replicate on 'startup' and 'commit'. 'optimize' is also a valid value for replicateAfter. -- str name=replicateAfterstartup/str str name=replicateAftercommit/str /lst /requestHandler into every box you want to be able to act as a master, then use: http://slave_host:port/solr/replication?command=fetchindexmasterUrl=your master URL As the above page says better than I can, It is possible to pass on extra attribute 'masterUrl' or other attributes like 'compression' (or any other parameter which is specified in the lst name=slave tag) to do a one time replication from a master. This obviates the need for hardcoding the master in the slave. HTH, Upayavira On Wed, 01 Dec 2010 06:24 +0100, Tommaso Teofili tommaso.teof...@gmail.com wrote: Hi Upayavira, this is a good start for solving my problem, can you please tell how does such a replication URL look like? Thanks, Tommaso 2010/12/1 Upayavira u...@odoko.co.uk Hi Tommaso, I believe you can tell each server to act as a master (which means it can have its indexes pulled from it). You can then include the master hostname in the URL that triggers a replication process. Thus, if you triggered replication from outside solr, you'd have control over which master you pull from. Does this answer your question? Upayavira On Tue, 30 Nov 2010 09:18 -0800, Ken Krugler kkrugler_li...@transpac.com wrote: Hi Tommaso, On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: Hi all, in a replication environment if the host where the master is running goes down for some reason, is there a way to communicate to the slaves to point to a different (backup) master without manually changing configuration (and restarting the slaves or their cores)? Basically I'd like to be able to change the replication master dinamically inside the slaves. Do you have any idea of how this could be achieved? One common approach is to use VIP (virtual IP) support provided by load balancers. Your slaves are configured to use a VIP to talk to the master, so that it's easy to dynamically change which master they use, via updates to the load balancer config. -- Ken -- Ken Krugler +1 530-210-6378 http://bixolabs.com e l a s t i c w e b m i n i n g
Re: Dinamically change master
Thanks Upayavira, that sounds very good. p.s.: I read that page some weeks ago and didn't get back to check on it. 2010/12/1 Upayavira u...@odoko.co.uk Note, all extracted from http://wiki.apache.org/solr/SolrReplication You'd put: requestHandler name=/replication class=solr.ReplicationHandler lst name=master !--Replicate on 'startup' and 'commit'. 'optimize' is also a valid value for replicateAfter. -- str name=replicateAfterstartup/str str name=replicateAftercommit/str /lst /requestHandler into every box you want to be able to act as a master, then use: http://slave_host:port/solr/replication?command=fetchindexmasterUrl=your master URL As the above page says better than I can, It is possible to pass on extra attribute 'masterUrl' or other attributes like 'compression' (or any other parameter which is specified in the lst name=slave tag) to do a one time replication from a master. This obviates the need for hardcoding the master in the slave. HTH, Upayavira On Wed, 01 Dec 2010 06:24 +0100, Tommaso Teofili tommaso.teof...@gmail.com wrote: Hi Upayavira, this is a good start for solving my problem, can you please tell how does such a replication URL look like? Thanks, Tommaso 2010/12/1 Upayavira u...@odoko.co.uk Hi Tommaso, I believe you can tell each server to act as a master (which means it can have its indexes pulled from it). You can then include the master hostname in the URL that triggers a replication process. Thus, if you triggered replication from outside solr, you'd have control over which master you pull from. Does this answer your question? Upayavira On Tue, 30 Nov 2010 09:18 -0800, Ken Krugler kkrugler_li...@transpac.com wrote: Hi Tommaso, On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: Hi all, in a replication environment if the host where the master is running goes down for some reason, is there a way to communicate to the slaves to point to a different (backup) master without manually changing configuration (and restarting the slaves or their cores)? Basically I'd like to be able to change the replication master dinamically inside the slaves. Do you have any idea of how this could be achieved? One common approach is to use VIP (virtual IP) support provided by load balancers. Your slaves are configured to use a VIP to talk to the master, so that it's easy to dynamically change which master they use, via updates to the load balancer config. -- Ken -- Ken Krugler +1 530-210-6378 http://bixolabs.com e l a s t i c w e b m i n i n g
Re: Dinamically change master
Hi Tommaso, On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: Hi all, in a replication environment if the host where the master is running goes down for some reason, is there a way to communicate to the slaves to point to a different (backup) master without manually changing configuration (and restarting the slaves or their cores)? Basically I'd like to be able to change the replication master dinamically inside the slaves. Do you have any idea of how this could be achieved? One common approach is to use VIP (virtual IP) support provided by load balancers. Your slaves are configured to use a VIP to talk to the master, so that it's easy to dynamically change which master they use, via updates to the load balancer config. -- Ken -- Ken Krugler +1 530-210-6378 http://bixolabs.com e l a s t i c w e b m i n i n g
Re: Dinamically change master
Hi, Thanks Jacob and Ken for your replies. I am not able to change project architecture to add Lucandra even if it looks like a nice solution. Going the VIP way can definitely an option even if I'd be more keen to solve that inside Solr. I am thinking to try and play with Collection Distribution scripts or doing some custom stuff. But, please, if you have any other advice let me know. Thanks again. Tommaso 2010/11/30 Ken Krugler kkrugler_li...@transpac.com Hi Tommaso, On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: Hi all, in a replication environment if the host where the master is running goes down for some reason, is there a way to communicate to the slaves to point to a different (backup) master without manually changing configuration (and restarting the slaves or their cores)? Basically I'd like to be able to change the replication master dinamically inside the slaves. Do you have any idea of how this could be achieved? One common approach is to use VIP (virtual IP) support provided by load balancers. Your slaves are configured to use a VIP to talk to the master, so that it's easy to dynamically change which master they use, via updates to the load balancer config. -- Ken -- Ken Krugler +1 530-210-6378 http://bixolabs.com e l a s t i c w e b m i n i n g
Re: Dinamically change master
Hi Tommaso, I believe you can tell each server to act as a master (which means it can have its indexes pulled from it). You can then include the master hostname in the URL that triggers a replication process. Thus, if you triggered replication from outside solr, you'd have control over which master you pull from. Does this answer your question? Upayavira On Tue, 30 Nov 2010 09:18 -0800, Ken Krugler kkrugler_li...@transpac.com wrote: Hi Tommaso, On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: Hi all, in a replication environment if the host where the master is running goes down for some reason, is there a way to communicate to the slaves to point to a different (backup) master without manually changing configuration (and restarting the slaves or their cores)? Basically I'd like to be able to change the replication master dinamically inside the slaves. Do you have any idea of how this could be achieved? One common approach is to use VIP (virtual IP) support provided by load balancers. Your slaves are configured to use a VIP to talk to the master, so that it's easy to dynamically change which master they use, via updates to the load balancer config. -- Ken -- Ken Krugler +1 530-210-6378 http://bixolabs.com e l a s t i c w e b m i n i n g
Re: Dinamically change master
Hi Upayavira, this is a good start for solving my problem, can you please tell how does such a replication URL look like? Thanks, Tommaso 2010/12/1 Upayavira u...@odoko.co.uk Hi Tommaso, I believe you can tell each server to act as a master (which means it can have its indexes pulled from it). You can then include the master hostname in the URL that triggers a replication process. Thus, if you triggered replication from outside solr, you'd have control over which master you pull from. Does this answer your question? Upayavira On Tue, 30 Nov 2010 09:18 -0800, Ken Krugler kkrugler_li...@transpac.com wrote: Hi Tommaso, On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: Hi all, in a replication environment if the host where the master is running goes down for some reason, is there a way to communicate to the slaves to point to a different (backup) master without manually changing configuration (and restarting the slaves or their cores)? Basically I'd like to be able to change the replication master dinamically inside the slaves. Do you have any idea of how this could be achieved? One common approach is to use VIP (virtual IP) support provided by load balancers. Your slaves are configured to use a VIP to talk to the master, so that it's easy to dynamically change which master they use, via updates to the load balancer config. -- Ken -- Ken Krugler +1 530-210-6378 http://bixolabs.com e l a s t i c w e b m i n i n g
