To be honest, my advice would be to consider docker
- install docker
- create a swarm with both nodes
- deploy a zookeeper service with 3 instances
- deploy a kafka service with 3 instances
On 19/07/17 15:19, M. Manna wrote:
Hello,
Thanks for the response. So as a workaround, can I do the following:
1) Node 1 - zookeeper 1 + 2 +3 (brokers 1,2,3)
2) Node 2 - zookeeper 4 + 5 + 6 (brokers 4,5,6)
Do you recommend this way?
Regards,
On 19 July 2017 at 14:12, mosto...@gmail.com <mosto...@gmail.com> wrote:
AFAIK you cant deploy a 2 node cluster, or you'll have split brains issues
"out of the box"
On 19/07/17 15:11, M. Manna wrote:
Hello,
After running an emulated 3 node cluster on my local pc, I am now trying
to
deploy a 2-node cluster on two of my remote machines. The following is the
server configuration:
broker.id=1
advertised.listeners=PLAINTEXT://0.0.0.0:9092
listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL
num.network.threads=9
num.io.threads=24
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=999999999
log.dirs=/kafka1
num.partitions=1
num.recovery.threads.per.data.dir=1
log.retention.hours=2
log.retention.minutes=15
log.retention.bytes=20971520
log.segment.bytes=10485760
log.roll.hours=1
log.retention.check.interval.ms=300000
offsets.retention.minutes=2880
offsets.retention.check.interval.ms=300000
log.cleanup.policy=delete
zookeeper.connect=available-server-1:2181,available-server-2:2181
zookeeper.connection.timeout.ms=35000
zookeeper.session.timeout.ms=20000
compression.type=gzip
delete.topic.enable=true
kafka.metrics.polling.interval.secs=5
kafka.metrics.reporters=kafka.metrics.KafkaCSVMetricsReporter
kafka.csv.metrics.dir=/tmp/kafka_metrics
kafka.csv.metrics.reporter.enabled=false
controlled.shutdown.max.retries=10
controlled.shutdown.retry.backoff.ms=3000
request.timeout.ms=45000
We are starting the clusters in the following order
1) available-server-1 (execute zookeeper-server-start followed by
kafka-server-start)
2) available-server-2 (execute zookeeper-server-start followed by
kafka-server-start)
The brokers cannot start up and need to be started separately. Is there
any
particular order the cluster nodes should be started?
Best Regards,
