I've got some ZooKeeper work that I did on the plane the other day
that I haven't had the time to write up just yet.
I've got the start for:
1. Automatic setting of Shards param for distributed search and new
shards can automatically register themselves and be added to the mix
(this is the furthest along, but is dependent on the firstSearcher/
newSearcher problem with distributed search that I mentioned in a
diff. thread)
2. Start on way to bring replicants and masters on and offline and do
rebalancing
3. RequestHandler based administration to add/delete/list/status
ZooKeeper and it's information
I think the nice thing about all of it is it pretty much works
seamlessly with existing Solr components and request handlers. I need
to bring the patch up to date and fill in a couple of pieces, but will
post soon as a start.
-Grant
On Jul 14, 2009, at 12:12 PM, Yonik Seeley wrote:
On Tue, Jul 14, 2009 at 11:49 AM, Eric
Pugh<ep...@opensourceconnections.com> wrote:
Isn't the pool of slave Solr's behind a load balancer similar to SOLR
automatic failover? Guess I am not sure what you mean? Seems like
what
ZooKeeper does
(http://hadoop.apache.org/zookeeper/docs/current/
zookeeperOver.html) is
analogous to the Solr master/pool of slave servers.
Right now, the task of keeping track of what servers are in a cluster
and what data they have is pushed off to the user.
Zookeeper would keep track of that (and would also serve as a single
point of configuration).
Wouldn't it be nice to be able to start up another server like so:
java -jar start.jar -Dcluster_id=foo
And just have everything work (no updating loadbalancer configuration,
other clients, etc).
Then there is master failover - a somewhat harder problem... or very
much harder, depending on how one approaches it.
-Yonik
http://www.lucidimagination.com
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
