[
https://issues.apache.org/jira/browse/KAFKA-6172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dong Lin resolved KAFKA-6172.
-----------------------------
Resolution: Fixed
> Cache lastEntry in TimeIndex to avoid unnecessary disk access
> -------------------------------------------------------------
>
> Key: KAFKA-6172
> URL: https://issues.apache.org/jira/browse/KAFKA-6172
> Project: Kafka
> Issue Type: Improvement
> Reporter: Dong Lin
> Assignee: Dong Lin
> Fix For: 1.0.1
>
>
> LogSegment.close() calls timeIndex.maybeAppend(...), which in turns make a
> number of calls to timeIndex.lastEntry(). Currently timeIndex.lastEntry()
> involves disk seek operation because it tries to read the content of the last
> few bytes of the index files on the disk. This slows down the broker shutdown
> process.
> Here is the time of LogManager.shutdown() in various settings. In all these
> tests, broker has roughly 6k partitions and 20k segments.
> - If broker does not have this patch and `log.dirs` is configured with 1 JBOD
> log directory, LogManager.shutdown() takes 15 minutes (roughly 900 seconds).
> - If broker does not have this patch and `log.dirs` is configured with 10
> JBOD log directories, LogManager.shutdown() takes 84 seconds.
> - If broker have this patch and `log.dirs` is configured with 10 JBOD log
> directories, LogManager.shutdown() takes 24 seconds.
> Thus we expect to save 71% time in LogManager.shutdown() by having this
> optimization. This patch intends to reduce the broker shutdown time by
> caching the lastEntry in memory so that broker does not have to always read
> disk to get the lastEntry.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
