Thanks Hrishikesh! Looking forward to hearing from you. On Fri, Sep 16, 2016 at 9:30 PM, Hrishikesh Gadre <gadre.s...@gmail.com> wrote:
> Hi Stephen, > > Thanks for the update. I filed SOLR-9527 > <https://issues.apache.org/jira/browse/SOLR-9527> for tracking purpose. I > will take a look and get back to you. > > Thanks > Hrishikesh > > On Fri, Sep 16, 2016 at 2:56 PM, Stephen Lewis <sle...@panopto.com> wrote: > > > Hello, > > > > I've tried this on both solr 6.1 and 6.2, with the same result. You are > > right that the collections API offering collection level backup/restore > > from remote server is a new feature. > > > > After some more experimentation, I am fairly sure that this is a bug > which > > is specific to the leaders in backup restore. After I ran a command to > > restore a backup of the collection "foo" (which has maxShardsPerNode set > to > > 1 as well) with a replication factor of 2, I see consistently that the > > followers (replica > 1) are correctly distributed, but all of the leaders > > are brought up hosted on one shard. > > > > *Repro* > > > > *create * > > http://solr.test:8983/solr/admin/collections?action= > > CREATE&name=foo&numShards=3&maxShardsPerNode=1&collection. > > configName=test-one > > (after creation, all shards are on different nodes as expected) > > > > *backup* > > http://solr.test:8983/solr/admin/collections?action= > > BACKUP&name=foo-2&collection=foo&async=foo-2 > > > > *delete* > > http://solr.test:8983/solr/admin/collections?action=DELETE&name=foo > > > > *restore* > > Result: All leaders are hosted on node, followers are spread about. > > > > { > > "responseHeader" : { "status" : 0,"QTime" : 7}, > > "cluster" : { > > "collections" : { > > "foo" : { > > "replicationFactor" : "2", > > "shards" : { > > "shard2" : { > > "range" : "d5550000-2aa9ffff", > > "state" : "active", > > "replicas" : { > > "core_node1" : { > > "core" : "foo_shard2_replica0", > > "base_url" : "http://IP1:8983/solr";, > > "node_name" : "IP1:8983_solr", > > "state" : "active", > > "leader" : "true" > > }, > > "core_node4" : { > > "core" : "foo_shard2_replica1", > > "base_url" : "http://IP2:8983/solr";, > > "node_name" : "IP2:8983_solr", > > "state" : "recovering" > > } > > } > > }, > > "shard3" : { > > "range" : "2aaa0000-7fffffff", > > "state" : "active", > > "replicas" : { > > "core_node2" : { > > "core" : "foo_shard3_replica0", > > "base_url" : "http://IP1:8983/solr";, > > "node_name" : "IP1:8983_solr", > > "state" : "active", > > "leader" : "true" > > }, > > "core_node5" : { > > "core" : "foo_shard3_replica1", > > "base_url" : "http://IP3:8983/solr";, > > "node_name" : "IP3:8983_solr", > > "state" : "recovering" > > } > > } > > }, > > "shard1" : { > > "range" : "80000000-d554ffff", > > "state" : "active", > > "replicas" : { > > "core_node3" : { > > "core" : "foo_shard1_replica0", > > "base_url" : "http://IP1:8983/solr";, > > "node_name" : "IP1:8983_solr", > > "state" : "active", > > "leader" : "true" > > }, > > "core_node6" : { > > "core" : "foo_shard1_replica1", > > "base_url" : "http://IP4:8983/solr";, > > "node_name" : "IP4:8983_solr", > > "state" : "recovering" > > } > > } > > } > > }, > > "router" : { > > "name" : "compositeId" > > }, > > "maxShardsPerNode" : "1", > > "autoAddReplicas" : "false", > > "znodeVersion" : 204, > > "configName" : "test-one" > > } > > }, > > "properties" : { > > "location" : "/mnt/solr_backups" > > }, > > "live_nodes" : [ > > "IP5:8983_solr", > > "IP3:8983_solr", > > "IP6:8983_solr", > > "IP4:8983_solr", > > "IP7:8983_solr", > > "IP1:8983_solr", > > "IP8:8983_solr", > > "IP9:8983_solr", > > "IP2:8983_solr"] > > } > > } > > > > > > On Fri, Sep 16, 2016 at 2:07 PM, Reth RM <reth.ik...@gmail.com> wrote: > > > > > Which version of solr? Afaik, until 6.1, solr backup and restore > command > > > apis required to do separate backup for each shard, and then restore in > > > similar lines( both go for each). 6.1 version seems to have new feature > > of > > > backing up entire collection records and then restoring it back to new > > > collection setup(did not try yet). > > > > > > > > > On Thu, Sep 15, 2016 at 1:45 PM, Stephen Lewis <sle...@panopto.com> > > wrote: > > > > > > > Hello, > > > > > > > > I have a solr cloud cluster in a test environment running 6.1 where I > > am > > > > looking at using the collections API BACKUP and RESTORE commands to > > > manage > > > > data integrity. > > > > > > > > When restoring from a backup, I'm finding the same behavior occurs > > every > > > > time; after the restore command, all shards are being hosted on one > > node. > > > > What's especially surprising about this is that there are 6 live > nodes > > > > beforehand, the collection has maxShardsPerNode set to 1, and this > > occurs > > > > even if I pass through the parameter maxShardsPerNode=1 to the API > > call. > > > Is > > > > there perhaps somewhere else I need to configure something, or > another > > > step > > > > I am missing? If perhaps I'm misunderstanding the intention of these > > > > parameters, could you clarify for me and let me know how to support > > > > restoring different shards on different nodes? > > > > > > > > Full repro below. > > > > > > > > Thanks! > > > > > > > > > > > > *Repro* > > > > > > > > *Cluster state before* > > > > > > > > http://54.85.30.39:8983/solr/admin/collections?action= > > > > CLUSTERSTATUS&wt=json > > > > > > > > { > > > > "responseHeader" : {"status" : 0,"QTime" : 4}, > > > > "cluster" : { > > > > "collections" : {}, > > > > "live_nodes" : [ > > > > "172.18.7.153:8983_solr", > > > > "172.18.2.20:8983_solr", > > > > "172.18.10.88:8983_solr", > > > > "172.18.6.224:8983_solr", > > > > "172.18.8.255:8983_solr", > > > > "172.18.2.21:8983_solr"] > > > > } > > > > } > > > > > > > > > > > > *Restore Command (formatted for ease of reading)* > > > > > > > > http://54.85.30.39:8983/solr/admin/collections?action=RESTORE > > > > > > > > &collection=panopto > > > > &async=backup-4 > > > > > > > > &location=/mnt/beta_solr_backups > > > > &name=2016-09-02 > > > > > > > > &maxShardsPerNode=1 > > > > > > > > <response> > > > > <lst name="responseHeader"> > > > > <int name="status">0</int> > > > > <int name="QTime">16</int> > > > > </lst> > > > > <str name="requestid">backup-4</str> > > > > </response> > > > > > > > > > > > > *Cluster state after* > > > > > > > > http://54.85.30.39:8983/solr/admin/collections?action= > > > > CLUSTERSTATUS&wt=json > > > > > > > > { > > > > "responseHeader" : {"status" : 0,"QTime" : 8}, > > > > "cluster" : { > > > > "collections" : { > > > > "panopto" : { > > > > "replicationFactor" : "1", > > > > "shards" : { > > > > "shard2" : { > > > > "range" : "0-7fffffff", > > > > "state" : "construction", > > > > "replicas" : { > > > > "core_node1" : { > > > > "core" : "panopto_shard2_replica0", > > > > "base_url" : "http://172.18.2.21:8983/solr";, > > > > "node_name" : "172.18.2.21:8983_solr", > > > > "state" : "active", > > > > "leader" : "true" > > > > } > > > > } > > > > }, > > > > "shard1" : { > > > > "range" : "80000000-ffffffff", > > > > "state" : "construction", > > > > "replicas" : { > > > > "core_node2" : { > > > > "core" : "panopto_shard1_replica0", > > > > "base_url" : "http://172.18.2.21:8983/solr";, > > > > "node_name" : "172.18.2.21:8983_solr", > > > > "state" : "active", > > > > "leader" : "true" > > > > } > > > > } > > > > } > > > > }, > > > > "router" : { > > > > "name" : "compositeId" > > > > }, > > > > "maxShardsPerNode" : "1", > > > > "autoAddReplicas" : "false", > > > > "znodeVersion" : 44, > > > > "configName" : "panopto" > > > > } > > > > }, > > > > "live_nodes" : ["172.18.7.153:8983_solr", "172.18.2.20:8983 > _solr", > > > > "172.18.10.88:8983_solr", "172.18.6.224:8983_solr", " > 172.18.8.255:8983 > > > > _solr", > > > > "172.18.2.21:8983_solr"] > > > > } > > > > } > > > > > > > > > > > > > > > > > > > > -- > > > > Stephen > > > > > > > > (206)753-9320 > > > > stephen-lewis.net > > > > > > > > > > > > > > > -- > > Stephen > > > > (206)753-9320 > > stephen-lewis.net > > > -- Stephen (206)753-9320 stephen-lewis.net
