There are two cases that zero-copy fetch thanks to sendfile don't work.
- SSL encryption is enabled
* Need to encrypt on Kafka process before sending to client
-
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/network/SslTransportLayer.java#L946
* Unlike plaintext transport layer which directly write to socket:
-
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/network/PlaintextTransportLayer.java#L214
- Message down conversion happens
* refs:
https://kafka.apache.org/documentation/#upgrade_10_performance_impact
Does your environment match above cases?
2020年9月1日(火) 9:05 Ming Liu <[email protected]>:
> Hi Kafka dev community,
> As we know, one major reason that Kafka is fast is because it is using
> sendfile() for zero copy, as what it described at
> https://kafka.apache.org/documentation/#producerconfigs,
>
>
>
> *This combination of pagecache and sendfile means that on a Kafka cluster
> where the consumers are mostly caught up you will see no read activity on
> the disks whatsoever as they will be serving data entirely from cache.*
>
> However, when I ran tracing on all my kafka brokers, I didn't get a
> single sendfile system call, why is this? Does it eventually translate to
> plain read/write syscalls?
>
> sudo ./syscount -p 126806 -d 30
> Tracing syscalls, printing top 10... Ctrl+C to quit.
> [17:44:10]
> SYSCALL COUNT
> epoll_wait 108482
> write 107165
> epoll_ctl 95058
> futex 86716
> read 86388
> pread 26910
> fstat 9213
> getrusage 120
> close 27
> open 21
>
--
========================
Okada Haruki
[email protected]
========================
