Sushant Mahajan created KAFKA-21356:
---------------------------------------
Summary: Performance issue in share fetch with share group DLQ
enabled.
Key: KAFKA-21356
URL: https://issues.apache.org/jira/browse/KAFKA-21356
Project: Kafka
Issue Type: Bug
Reporter: Sushant Mahajan
Assignee: Sushant Mahajan
Enabling DLQ on a share group record could strangulate the overall share fetch
throughput by choking the persister. The causes are:
1. `ShareCoordinatorMetadataCacheHelperImpl.isDlqEnabledOnTopic` returns the
value of a dynamic config so it is called repeatedly. The impl of this method
creates a new LogConfig object, on each invocation and parses entire
TopicConfig. On a system with large number of DLQ requests, these calls add up
consuming CPU and choking the persister.
Fix: Use raw config lookup (shrinks cpu samples at high load from 27% to 0.1%)
2. Share group DLQ requires reading data from the ReplicaManagerLogReader,
which is a disk operation. Currently, this happens in the same thread as the
Persister which means that other share coordinator requests remain queued on a
potentially slow disk read operation, again choking the persister.
Fix: Allocate separate executor to the record readers.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
