Re: Improving crash-restart catchup speed

2020-12-18 Thread Gokul Ramanan Subramanian
Thanks. Makes sense. On Wed, Dec 16, 2020 at 2:02 AM Haruki Okada wrote: > Hi. > One possible solution I can imagine is to use replication throttle. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-73+Replication+Quotas > > You can set lower replication quota for other brokers -> failed

Re: Improving crash-restart catchup speed

2020-12-15 Thread Haruki Okada
Hi. One possible solution I can imagine is to use replication throttle. https://cwiki.apache.org/confluence/display/KAFKA/KIP-73+Replication+Quotas You can set lower replication quota for other brokers -> failed broker traffic during catching-up to prevent exhausting storage throughput.

Improving crash-restart catchup speed

2020-12-15 Thread Gokul Ramanan Subramanian
Hi. When a broker crashes and restarts after a long time (say 30 minutes), the broker has to do some work to sync all its replicas with the leaders. If the broker is a preferred replica for some partition, the broker may become the leader as part of a preferred replica leader election, while it