Hello Pradeep, thank you for sharing your experience, will certainly consider it.
On Thu, Jul 20, 2017 at 9:29 AM, Pradeep Gollakota <pradeep...@gmail.com> wrote: > Luigi, > > I strongly urge you to consider a 5 node ZK deployment. I've always done > that in the past for resiliency during maintenance. In a 3 node cluster, > you can only tolerate one "failure", so if you bring one node down for > maintenance and another node crashes during said maintenance, your ZK > cluster is down. All the deployments I've had were 5 nodes of ZK and 5 > nodes of Kafka. > > - Pradeep > > On Thu, Jul 20, 2017 at 9:12 AM, Luigi Tagliamonte < > luigi.tagliamont...@gmail.com> wrote: > > > Yes Andrey, > > you can use an ENI without EIP on AWS if you only want a private address. > > > > After some consideration, I think that growing the zookeeper cluster more > > than 3 nodes is really unlikely so I think that I will attach 3 ENI to 3 > > servers in autoscaling and I will configure Kafka in using this 3 IPs. > > In this way I can get rid of the additional ELB/Haproxy layer, if I will > > ever need to grow the zk ensemble I will re-engineering the solution. > > > > I'm wondering if reusing an old IP on a brand new zk node will create > > issues in the ensemble. > > Is anybody here aware of possible drawbacks? > > > > On Wed, Jul 19, 2017 at 11:58 PM, Andrey Dyachkov < > > andrey.dyach...@gmail.com > > > wrote: > > > > > The problem with EIP it is a public ip. > > > Another option is to have the secondary interface attached to the > > instance > > > on start(or a bit later) with the private static ip, but we are > > > investigating the possibility. > > > On Wed 19. Jul 2017 at 23:38, Luigi Tagliamonte < > > > luigi.tagliamont...@gmail.com> wrote: > > > > > > > Hello Andrey, > > > > I see that the ELB is not going to help directly with the bug, but > > > > introduces a nice layer that makes zookeeper DNS management easier. > > > > Introducing and ELB I don't have to deal with keep DNS in sync for > all > > > the > > > > servers in the zk ensemble. > > > > For the moment I can use an HAproxy with EIP and when the bug is > > solved I > > > > can move to ELB. > > > > What do you think about it? > > > > Regards > > > > L. > > > > > > > > On Wed, Jul 19, 2017 at 2:16 PM, Andrey Dyachkov < > > > > andrey.dyach...@gmail.com> > > > > wrote: > > > > > > > > > Hi, > > > > > I have just posted almost the same question in dev list. > > > > > Zookeeper client resolves address only once, on start, introducing > > ELB > > > > > won't really help here (ELBs can be replaced, which involved ip > > > change), > > > > > but I am eager to know if there is a solution for that. > > > > > > > > > > On Wed, 19 Jul 2017 at 23:08 Luigi Tagliamonte < > > > > > luigi.tagliamont...@gmail.com> wrote: > > > > > > > > > > > Hello, Users! > > > > > > I'm designing a Kafka deployment on AWS and it's my first time > > > working > > > > > with > > > > > > Kafka and Zookeeper so I've collected a lot of info so far but > also > > > > some > > > > > > questions that I would love to submit to a much expert audience > > like > > > > you. > > > > > > > > > > > > I have been experimenting with exhibitor and zookeeper in auto > > > scaling > > > > > > group and the exhibitor orchestration seems to work so far. > > > > > > > > > > > > I was trying to find a way to configure zookeeper servers in > Kafka > > > conf > > > > > and > > > > > > do not have to reconfigure them in case a zookeeper node needs to > > be > > > > > > replaced/dies, so i taught of course of using DNS but then I read > > > that > > > > > > zkclient library used by Kafka has this bug: > > > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-2184. > > > > > > > > > > > > So I'm now thinking about using an ELB in front of the zookeeper > > > > cluster. > > > > > > Teorically on how zookeeper client should work there should be no > > > > problem > > > > > > but I'm wondering if any of you used that and how is the outcome? > > > > > > > > > > > -- > > > > > > > > > > With great enthusiasm, > > > > > Andrey > > > > > > > > > > > > -- > > > > > > With great enthusiasm, > > > Andrey > > > > > >
