[
https://issues.apache.org/jira/browse/KAFKA-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944525#comment-16944525
]
Piotr Smolinski commented on KAFKA-8979:
----------------------------------------
The problem seems to be in the toSent() method of the FetchResponse message:
[https://github.com/apache/kafka/blob/2.3.0/clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java#L421-L435]
The method creates MultiRecordsSend with multiple buffers. Under certain
conditions the buffers are sent immediately. At least two first buffers in the
message could be combined.
> Fetch response is frequently split into multiple TCP/IP packets
> ---------------------------------------------------------------
>
> Key: KAFKA-8979
> URL: https://issues.apache.org/jira/browse/KAFKA-8979
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 2.2.1
> Reporter: Piotr Smolinski
> Priority: Minor
> Attachments: Screenshot 2019-10-04 at 13.58.48.png,
> dump.kafka.consumer.1.pcap
>
>
> Checking the network dumps from Kafka I have observed that some Fetch
> response frames are split into multiple TCP segments. It adds some network
> overhead and forces additional TCP coordination. While it might make sense
> for zero-copy IO, the observed cases do not confirm this.
> In the attached screenshot the 186 octets of Fetch response frame containing
> messages from 4 partitions (no actual messages were included in this case)
> are split into 7 TCP segments of length 8, 14, 12, 38, 38, 38 and 38 octets
> respectively.
> In the observed cases, the frames are typically split into 8 octets (frame
> length /4/ and correlation id /4/) followed by 14 octets (throttle time /4/,
> error code /2/, fetch session id /4/ and count of following data sections
> /4/) subsequently followed by topic preamble.
> Most times the response frames are sent as single packet (as long as they fit
> into IP packet), but the Fetch response frames are showing up split
> frequently enough to indicate some kind of network inefficiency.
> The dumps were analysed with Wireshark built from the source code to leverage
> Kafka 2 dissection.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
