What tools do you use for the "auto setup"? How do you get your config automatically uploaded to zk?
On Tue, Sep 22, 2015 at 2:35 PM, Gili Nachum <gilinac...@gmail.com> wrote: > Our auto setup sequence is: > 1.deploy 3 zk nodes > 2. Deploy solr nodes and start them connecting to zk. > 3. Upload collection config to zk. > 4. Call create collection rest api. > 5. Done. SolrCloud ready to work. > > Don't yet have automation for replacing or adding a node. > On Sep 22, 2015 18:27, "Steve Davids" <sdav...@gmail.com> wrote: > > > Hi, > > > > I am trying to come up with a repeatable process for deploying a Solr > Cloud > > cluster from scratch along with the appropriate security groups, auto > > scaling groups, and custom Solr plugin code. I saw that LucidWorks > created > > a Solr Scale Toolkit but that seems to be more of a one-shot deal than > > really setting up your environment for the long-haul. Here is were we are > > at right now: > > > > 1. ZooKeeper ensemble is easily brought up via a Cloud Formation > Script > > 2. We have an RPM built to lay down the Solr distribution + Custom > > plugins + Configuration > > 3. Solr machines come up and connect to ZK > > > > Now, we are using Puppet which could easily create the core.properties > file > > for the corresponding core and have ZK get bootstrapped but that seems to > > be a no-no these days... So, can anyone think of a way to get ZK > > bootstrapped automatically with pre-configured Collection configurations? > > Also, is there a recommendation on how to deal with machines that are > > coming/going? As I see it machines will be getting spun up and terminated > > from time to time and we need to have a process of dealing with that, the > > first idea was to just use a common node name so if a machine was > > terminated a new one can come up and replace that particular node but on > > second thought it would seem to require an auto scaling group *per* node > > (so it knows what node name it is). For a large cluster this seems crazy > > from a maintenance perspective, especially if you want to be elastic with > > regard to the number of live replicas for peak times. So, then the next > > idea was to have some outside observer listen to when new ec2 instances > are > > created or terminated (via CloudWatch SQS) and make the appropriate API > > calls to either add the replica or delete it, this seems doable but > perhaps > > not the simplest solution that could work. > > > > I was hoping others have already gone through this and have valuable > advice > > to give, we are trying to setup Solr Cloud the "right way" so we don't > get > > nickel-and-dimed to death from an O&M perspective. > > > > Thanks, > > > > -Steve > > >
