Re: Fully automated replica creation in AWS

2015-12-09 Thread Erick Erickson
Not that I know of. The two systems are somewhat disconnected. AWS doesn't know that Solr lives on those nodes, it's just spinning one up, right? Albeit with Solr running. There's nothing in Solr that auto-detects the existence of a new Solr node and automagically assigns collections and/or

Fully automated replica creation in AWS

2015-12-09 Thread Ugo Matrangolo
Hi, I was trying to setup a SolrCloud cluster in AWS backed by an ASG (auto scaling group) serving a replicated collection. I have just came across a case when one of the Solr node became unresponsive with AWS killing it and spinning a new one. Unfortunately, this new Solr node did not join as a

Re: Fully automated replica creation in AWS

2015-12-09 Thread Erick Erickson
bq: As a side note, we do this for our customers as that's baked into our cloud provisioning software, Exactly, nothing OOB is there, but all the data is available, you "just" have to write a tool that knows where to look ;) That said, this would certainly be something that would have to be

Re: Fully automated replica creation in AWS

2015-12-09 Thread Jeff Wartes
It’s a pretty common misperception that since solr scales, you can just spin up new nodes and be done. Amazon ElasticSearch and older solrcloud getting-started docs encourage this misperception, as does the HDFS-only autoAddReplicas flag. I agree that auto-scaling should be approached carefully,

Re: Fully automated replica creation in AWS

2015-12-09 Thread Sameer Maggon
Erick, Typically, while creating collections, a replicationFactor is specified. Thus, the meta data about the collection does have information about what the "desired" replicationFactor is for the collection. If that's the case, when a Solr node joins the cluster, there could be a pro-active

Re: Fully automated replica creation in AWS

2015-12-09 Thread Jean-Sebastien Vachon
2015 2:19 PM To: solr-user Subject: Re: Fully automated replica creation in AWS Not that I know of. The two systems are somewhat disconnected. AWS doesn't know that Solr lives on those nodes, it's just spinning one up, right? Albeit with Solr running. There's nothing in Solr that auto-d