bq: As a side note, we do this for our customers as that's baked into our cloud provisioning software,
Exactly, nothing OOB is there, but all the data is available, you "just" have to write a tool that knows where to look ;) That said, this would certainly be something that would have to be optional and, IMO, not on by default. It'd need some design effort, as I'd guess it'd be on a per-collection basis thus turned on in solrconfig.xml. Hmmm, or perhaps this would be some kind of list of nodes in Zookeeper. Or.. So when a new Solr node came up for the first time, it'd need to find out which collections were configured with this option either through enumerating all the collections and checking their states or looking in their solrconfig files or enumerating a list of children in Zookeeper or.... Plus you'd need some kind of safeguards in place to handle bringing up, say, 10 new Solr instances one at a time so all the new replicas didn't end up on the first node you added. And so on.... The more I think about all that the less I like it; it seems that custom utilities on a per-collection basis make more sense. And yeah, autoAddReplicas is exactly for that on HDFS. Using autoAddReplicas for non shared filesystems doesn't really make sense IMO. The supposition is that the Solr node has gone away. How would some _other_ Solr instance on some _other_ node know where to look for the index? Best, Erick On Wed, Dec 9, 2015 at 11:37 AM, Sameer Maggon <sam...@measuredsearch.com> wrote: > Erick, > > Typically, while creating collections, a replicationFactor is specified. > Thus, the meta data about the collection does have information about what > the "desired" replicationFactor is for the collection. If that's the case, > when a Solr node joins the cluster, there could be a pro-active add-replica > operation that can be initiated if the Solr detects that the current > replicas are less than the desired replicationFactor and pull the > collection data from the leader. > > Isn't that what the attribute "autoAddReplicas" does for HDFS - can this be > done for non-shared filesystem? As a side note, we do this for our > customers as that's baked into our cloud provisioning software, but it > would be nice if Solr supports that OOTB. Are there any underlying flaws of > doing that? > > Thanks, > -- > > *Sameer Maggon* > www.measuredsearch.com > <https://mailtrack.io/trace/link/663333fad5b85359bf1b21be04166edea6c7d13e?url=http%3A%2F%2Fmeasuredsearch.com%2F&signature=6dbc74f0abef4882> > | > Deploy, Scale & Manage Solr in the cloud of your choice. > > > On Wed, Dec 9, 2015 at 11:19 AM, Erick Erickson <erickerick...@gmail.com> > wrote: > >> Not that I know of. The two systems are somewhat disconnected. >> AWS doesn't know that Solr lives on those nodes, it's just spinning >> one up, right? Albeit with Solr running. >> >> There's nothing in Solr that auto-detects the existence of a new >> Solr node and automagically assigns collections and/or replicas. >> >> How would either system intuit that this new node is replacing >> something else and "do the right thing"? >> >> I'll tell you how, by interrogating Zookeeper and seeing that for some >> specific collection, shardX had fewer replicas than other shards and >> issuing the Collections API ADDREPLICA command. >> >> But now there are _three_ systems that need to be coordinated and >> doing the right thing in your situation would be the wrong thing in >> another. The last thing many sys ops want is having replicas started >> without their knowledge. >> >> And on top of that, I have doubts about the model. Having AWS >> elastically spin up a new replica is a heavyweight operation from >> Solr's perspective. I mean this potentially copies a many G set of >> index files from one place to another which could take a long time, >> is that really what's desired here? >> >> I have seen some folks spin up/down Solr instances based on a >> schedule if they know roughly when the peak load will be, but again >> there's nothing built in to handle this. >> >> Best, >> Erick >> >> On Wed, Dec 9, 2015 at 10:15 AM, Ugo Matrangolo >> <ugo.matrang...@gmail.com> wrote: >> > Hi, >> > >> > I was trying to setup a SolrCloud cluster in AWS backed by an ASG (auto >> > scaling group) serving a replicated collection. I have just came across a >> > case when one of the Solr node became unresponsive with AWS killing it >> and >> > spinning a new one. >> > >> > Unfortunately, this new Solr node did not join as a replica of the >> existing >> > collection requiring human intervention to configure it as a new replica. >> > >> > I was wondering if there is around something that will make this process >> > fully automated by detecting that a new node just joined the cluster and >> > instructing it (e.g. via Collections API) to join as a replica of a given >> > collection. >> > >> > Best >> > Ugo >>
