For low volume zookeeper doesn't seem to use many resources.   I would put it 
on nodejs server as that will have less IO and heavy IO could impact zookeeper. 
 Or, you could put some ZK nodes on nodejs and some on DB servers to hedge your 
bets.   As always, you'll find out a lot once you actually start running it in 
production.

-Dave

-----Original Message-----
From: Sachin Mittal [mailto:sjmit...@gmail.com]
Sent: Thursday, December 1, 2016 6:03 AM
To: users@kafka.apache.org
Subject: Re: I need some help with the production server architecture

Folks any help on this.

Just to put it in simple terms, since we have limited resources available to us 
what is better option 1. run zookeeper on servers running the nodejs web server 
or db server.
2. what about kafka brokers.

Thanks
Sachin


On Tue, Nov 29, 2016 at 1:06 PM, Sachin Mittal <sjmit...@gmail.com> wrote:

> Hi,
> Sometime back i was informed on the group that in production we should
> never run kafka on same physical machine. So based on that I have a
> question on how to divide the server nodes we have to run zookeper and
> kafka brokers.
>
> I have a following setup
> Data center 1
> Lan 1 (3 VMs)
> 192.168.xx.yy1
> 192.168.xx.yy2
> 192.168.xx.yy3
> Right now here we are running a cluster of 3 nodejs web servers.
> These collect data from web and write to kafka queue. Each VM has 70
> GB of space.
>
> Lan 2 (3 VMs)
> 192.168.zz.aa1
> 192.168.zz.aa2
> 192.168.zz.aa3
> These are served the cluster of our database server. Each VM has 400
> GB of space.
>
> Date center 2
> Lan 1 (3 VMs)
> 192.168.yy.bb1
> 192.168.yy.bb2
> 192.168.yy.bb3
> Three new machines where we plan to run a cluster of new database to
> be served as sink of kafka stream applications. Each VM has 400 GB of space.
> These have connectivity only between Lan 2 of Data center 1 with a
> 100MBs of data transfer rate.
>
> Each VM has a 4 core processor and 16 GB of RAM. They all run linux.
>
> Now I would like my topics to be replicated with a factor of 3. Since
> we don't foresee much volume of data, I don't want it to be partitioned.
>
> Also we would like one server to be used as streaming application
> server, where we can run one or more kafka stream applications to
> process the topics and write to the new database.
>
>  So please let me know what is a suitable division to run brokers and
> zookeeper.
>
>
> Thanks
> Sachin
>
>
>
>
This e-mail and any files transmitted with it are confidential, may contain 
sensitive information, and are intended solely for the use of the individual or 
entity to whom they are addressed. If you have received this e-mail in error, 
please notify the sender by reply e-mail immediately and destroy all copies of 
the e-mail and any attachments.

Reply via email to