Re: SOLR cloud (5.2.1) recovery

2015-08-18 Thread Erick Erickson
First, do not think in terms of cores, think replicas ;). And do not,
use the core admin bits of the admin UI to do any SolrCloud-related
operations. It's possible, but far too easy to get wrong.

Use the collections API instead.

Second, 600 collections, assuming all on a single cluster is
getting pretty big. Solr will start to have issues as the number
of collections increases much more.

But to your point:
The simplest thing to do is use the ADDREPLICA and
DELETEREPLICA commands from the collections API and be
done with it. For ADDREPLICA, you can specify which node it
is supposed to appear on. So let's say one of your nodes is just
gone that contained a replica for collection1, shard1. Hardware
fault or similar, and it's not coming back.

You bring up a new Solr node to replace it and:
1 issue an ADDREPLICA for collection1, shard1 that specifies
the new node. Just wait for a while and it'll replicate the necessary
data from the leader and start serving queries. You do not have to
stop indexing during this process, although you might want to. Nor
do you have to stop querying.

2 issue a DELETEREPLICA for the old replica. That'll clear it out
of the Zookeeper cluster state.

The order of these streps is irrelevant.

bq: or if accidentally i delete the data dir

You shouldn't have to do anything except restart the node. It should
replicate the index from the leader. That said, you shouldn't have to
manipulate the directories under SOLR_HOME manually, if you
find yourself doing this regularly it's a red flag..

There's no single command to say add a replica on nodeX for every
replica on nodeY though. So you'd have to do this for each replica on
the disappeared node. All the necessary information is present in the
clusterstate, and such a command is possible, but nobody's done it yet.
It'd be possible to create an external utility to do this too.

Best,
Erick

On Tue, Aug 18, 2015 at 10:07 AM, Olivier Damiot
olivier.dam...@gmail.com wrote:
 hello,

 i'am a bit confused about how solr cloud recovery is supposed to work
 exactly in the case of loosing a single node completely.

 My 600 collections are created with
 numShards=3replicationFactor=3maxShardsPerNode=3

 However, how do i configure a new node to take the place of the dead
 node, or if accidentally i delete the data dir ?

 I bring up a new node which is completely empty (empty data dir),
 install solr, and connect it to zookeeper.Is it supposed to work
 automatically from there? All my shards/replicas on this node as down
 (i suppose because there is no cores in data dir).

 Do I need to recreate the cores first?

 Can i copy/paste data directory from another node to this one ? I
 think no because i should rename all variables in core.properties
 which are dedicated for each node (like name or coreNodeName)

 thanks,

 Olivier Damiot


SOLR cloud (5.2.1) recovery

2015-08-18 Thread Olivier Damiot
hello,

i'am a bit confused about how solr cloud recovery is supposed to work
exactly in the case of loosing a single node completely.

My 600 collections are created with
numShards=3replicationFactor=3maxShardsPerNode=3

However, how do i configure a new node to take the place of the dead
node, or if accidentally i delete the data dir ?

I bring up a new node which is completely empty (empty data dir),
install solr, and connect it to zookeeper.Is it supposed to work
automatically from there? All my shards/replicas on this node as down
(i suppose because there is no cores in data dir).

Do I need to recreate the cores first?

Can i copy/paste data directory from another node to this one ? I
think no because i should rename all variables in core.properties
which are dedicated for each node (like name or coreNodeName)

thanks,

Olivier Damiot