[
https://issues.apache.org/jira/browse/SOLR-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784317#action_12784317
]
Mark Miller edited comment on SOLR-1277 at 12/1/09 6:18 PM:
------------------------------------------------------------
I'd like to reboot this issue a bit. I've started a fresh experimental pass.
I think we want to move ZooKeeper's home out of SolrCore and into CoreContainer
- one of the big benefits of using ZooKeeper is the ability to config from a
central location. This implies that a ZooKeeper client should be up and running
before any SolrCore is.
So this new patch explores that path - its just an initial stab in that
direction, so a lot is left undone, but I figured I'd put it up to get feedback
on the new approach.
This initial pass allows basic loading of config/schema from ZooKeeper and adds
some ground work for ZooKeeper unit tests.
Some simple notes below:
{noformat}
ZooKeeper Integration
Features:
load solrconfig/schema from zookeeper (currently no other config files - want
to be able to load any config from ZK)
TODO: handle distributed search node choices, fault tolerance
TODO: integrate with replication
TODO: fault tolerant indexing
TODO: more ;)
Startup?
System Properties? -DzkHost=localhost:2127 {-DzkPath=/solr} - if no zkHost
found, run non ZooKeeper mode
Optional Properties? -Dsolrconfig=solrconfig.xml -Dschema=schema.xml -
multicore config: comes from solr.xml
Groups? -Dgroup= ? -Dshard= ? How about multicore? Set in solr.xml?
Distinguish master from slave?
Another sys property?
Cache zk info locally and 'watch' for changes
Layout - configurable prefix - defaults to solr
/solr/conf
/solrconfig.xml
/schema.xml
multicore?
/solr/core0/conf
/solr/core1/conf
ZooKeeper Limitations:
1 MB of data per node
wants zk log on its own device (not just partition)
(perhaps not so necessary with fewer nodes?)
must not swap to maintain performance
{noformat}
was (Author: [email protected]):
I'd like to reboot this issue a bit. I've started a fresh experimental pass.
I think we want to move ZooKeeper's home out of SolrCore and into CoreContainer
- one of the big benefits of using ZooKeeper is the ability to config from a
central location. This implies that a ZooKeeper client should be up and running
before any SolrCore is.
So this new patch explores that path - its just an initial stab in that
direction, so a lot is left on done, but I figured I'd put it up to get
feedback on the new approach.
This initial pass allows basic loading of config/schema from ZooKeeper and adds
some ground work for ZooKeeper unit tests.
Some simple notes below:
{noformat}
ZooKeeper Integration
Features:
load solrconfig/schema from zookeeper (currently no other config files - want
to be able to load any config from ZK)
TODO: handle distributed search node choices, fault tolerance
TODO: integrate with replication
TODO: fault tolerant indexing
TODO: more ;)
Startup?
System Properties? -DzkHost=localhost:2127 {-DzkPath=/solr} - if no zkHost
found, run non ZooKeeper mode
Optional Properties? -Dsolrconfig=solrconfig.xml -Dschema=schema.xml -
multicore config: comes from solr.xml
Groups? -Dgroup= ? -Dshard= ? How about multicore? Set in solr.xml?
Distinguish master from slave?
Another sys property?
Cache zk info locally and 'watch' for changes
Layout - configurable prefix - defaults to solr
/solr/conf
/solrconfig.xml
/schema.xml
multicore?
/solr/core0/conf
/solr/core1/conf
ZooKeeper Limitations:
1 MB of data per node
wants zk log on its own device (not just partition)
(perhaps not so necessary with fewer nodes?)
must not swap to maintain performance
{noformat}
> Implement a Solr specific naming service (using Zookeeper)
> ----------------------------------------------------------
>
> Key: SOLR-1277
> URL: https://issues.apache.org/jira/browse/SOLR-1277
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.5
>
> Attachments: log4j-1.2.15.jar, SOLR-1277.patch, SOLR-1277.patch,
> zookeeper-3.2.1.jar
>
> Original Estimate: 672h
> Remaining Estimate: 672h
>
> The goal is to give Solr server clusters self-healing attributes
> where if a server fails, indexing and searching don't stop and
> all of the partitions remain searchable. For configuration, the
> ability to centrally deploy a new configuration without servers
> going offline.
> We can start with basic failover and start from there?
> Features:
> * Automatic failover (i.e. when a server fails, clients stop
> trying to index to or search it)
> * Centralized configuration management (i.e. new solrconfig.xml
> or schema.xml propagates to a live Solr cluster)
> * Optionally allow shards of a partition to be moved to another
> server (i.e. if a server gets hot, move the hot segments out to
> cooler servers). Ideally we'd have a way to detect hot segments
> and move them seamlessly. With NRT this becomes somewhat more
> difficult but not impossible?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
