We have a 0.9 kafka cluster consists of 7 kafka-brokers colocated with 7
Producers/consumers are going full tilt at this cluster and we measure
delays in response time while we run disruptive tests
Delays in response time are 'acceptable' (a few seconds max) for the
following disruptions on 1 or 2 of these hosts
- Soft kill kafka-broker/zookepeer
- Hard kill kafka-broker/zookepeer
- soft/hard reboot host
However when we disrupt the network for 1 or 2 hosts using a sequence such
as below, we do see delays of ~ 30 seconds (1 host = topic leader) to ~ 60
seconds (2 hosts including topic leader)
- Shut down a network interface, '/etc/init.d/network stop'.
- Sleep for a few minutes
- Start network interface, '/etc/init.d/network start'
Those delays suggest that perhaps there is a 30 second timeout somewhere
that explains the 30 second delay, and it is conjectured that shortening
this timeout may possibly reduce the resultant delay.
We have tried changing various timeouts such as those listed below, but
have had no success so far
Searches have not yielded any solutions. Any help/guidance is greatly
Thanks and regards,
*Principal Software Engineer |Service Cloud | Salesforce.com | Landmark
10th @ HQ | 510-859-6975*