[
https://issues.apache.org/jira/browse/KAFKA-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487841#comment-13487841
]
Jun Rao commented on KAFKA-593:
-------------------------------
Here is the issue. We rolled a new segment in the follower. The follower in one
fetch gets 10k bytes of data and appends to its log. This won't add any index
entry since it's the very first append to this segment. After the append, the
log rolled since the max segment size is reached. This leaves an empty index.
Technically, the logic is still correct. It does mean that index entries may
not be generated as frequently as one expects, depending on the fetch size used
in the follower fetcher thread and how far behind a follower is. This may
impact consumer performance a bit.
> Empty log index file created when it shouldn't be empty
> -------------------------------------------------------
>
> Key: KAFKA-593
> URL: https://issues.apache.org/jira/browse/KAFKA-593
> Project: Kafka
> Issue Type: Bug
> Reporter: Yang Ye
> Attachments: kafka_583_zk_kafka_data.tar.gz
>
>
> We have met empty index file during system test when it shouldn't be empty.
> In this case, there're around 100 messages in each segment, each of size
> around 100 bytes, given the "logIndexIntervalBytes" 4096, there should be at
> least 2 log index entries, but we see empty index file. The kafka and
> zookeeper logs are attached
> [yye@yye-ld kafka_server_3_logs]$ cd test_1-2/
> [yye@yye-ld test_1-2]$ ls -l
> total 84
> -rw-r--r-- 1 yye eng 8 Oct 29 15:22 00000000000000000000.index
> -rw-r--r-- 1 yye eng 10248 Oct 29 15:22 00000000000000000000.log
> -rw-r--r-- 1 yye eng 8 Oct 29 15:22 00000000000000000100.index
> -rw-r--r-- 1 yye eng 10296 Oct 29 15:22 00000000000000000100.log
> -rw-r--r-- 1 yye eng 0 Oct 29 15:23 00000000000000000200.index
> -rw-r--r-- 1 yye eng 10293 Oct 29 15:23 00000000000000000200.log
> -rw-r--r-- 1 yye eng 0 Oct 29 15:23 00000000000000000300.index
> -rw-r--r-- 1 yye eng 10274 Oct 29 15:23 00000000000000000300.log
> -rw-r--r-- 1 yye eng 0 Oct 29 15:23 00000000000000000399.index
> -rw-r--r-- 1 yye eng 10276 Oct 29 15:23 00000000000000000399.log
> -rw-r--r-- 1 yye eng 0 Oct 29 15:23 00000000000000000498.index
> -rw-r--r-- 1 yye eng 10256 Oct 29 15:23 00000000000000000498.log
> -rw-r--r-- 1 yye eng 10485760 Oct 29 15:23 00000000000000000596.index
> -rw-r--r-- 1 yye eng 3564 Oct 29 15:23 00000000000000000596.log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
