[
https://issues.apache.org/jira/browse/KAFKA-6172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dong Lin updated KAFKA-6172:
----------------------------
Description:
LogSegment.close() calls timeIndex.maybeAppend(...), which in turns make a
number of calls to timeIndex.lastEntry(). Currently timeIndex.lastEntry()
involves disk seek operation because it tries to read the content of the last
few bytes of the index files on the disk. This slows down the broker shutdown
process.
Here is the time of LogManager.shutdown() in various settings. In all these
tests, broker has roughly 6k partitions and 20k segments.
- If broker does not have this patch and `log.dirs` is configured with 1 JBOD
log directory, LogManager.shutdown() takes 15 minutes (roughly 900 seconds).
- If broker does not have this patch and `log.dirs` is configured with 10 JBOD
log directories, LogManager.shutdown() takes 84 seconds.
- If broker does not have this patch and `log.dirs` is configured with 10 JBOD
log directories, LogManager.shutdown() takes 24 seconds.
Thus we expect to save 71% time in LogManager.shutdown() by having this
optimization. This patch intends to reduce the broker shutdown time by caching
the lastEntry in memory so that broker does not have to always read disk to get
the lastEntry.
was:
LogSegment.close() calls timeIndex.maybeAppend(...), which in turns make a
number of calls to timeIndex.lastEntry(). Currently timeIndex.lastEntry()
involves disk seek operation because it tries to read the content of the last
few bytes of the index files on the disk. This slows down the broker shutdown
process.
For a given broker of 6k partitions and 19k segments, we find that
LogManager.shutdown() takes 15 minutes. The broker is configured to use 10
threads to close log in parallel. According to the thread dump taken while the
broker is in the process of LogManager.shutdown(), roughly 5 out of the 10
threads are in RUNNABLE state at TimeIndex.lastEntry(). This suggests that
TimeIndex.lastEntry() is very likely costing a lot of shutdown time.
This patch intends to reduce the broker shutdown time by caching the lastEntry
in memory so that broker does not have to always read disk to get the lastEntry.
> Cache lastEntry in TimeIndex to avoid unnecessary disk access
> -------------------------------------------------------------
>
> Key: KAFKA-6172
> URL: https://issues.apache.org/jira/browse/KAFKA-6172
> Project: Kafka
> Issue Type: Improvement
> Reporter: Dong Lin
> Assignee: Dong Lin
>
> LogSegment.close() calls timeIndex.maybeAppend(...), which in turns make a
> number of calls to timeIndex.lastEntry(). Currently timeIndex.lastEntry()
> involves disk seek operation because it tries to read the content of the last
> few bytes of the index files on the disk. This slows down the broker shutdown
> process.
> Here is the time of LogManager.shutdown() in various settings. In all these
> tests, broker has roughly 6k partitions and 20k segments.
> - If broker does not have this patch and `log.dirs` is configured with 1 JBOD
> log directory, LogManager.shutdown() takes 15 minutes (roughly 900 seconds).
> - If broker does not have this patch and `log.dirs` is configured with 10
> JBOD log directories, LogManager.shutdown() takes 84 seconds.
> - If broker does not have this patch and `log.dirs` is configured with 10
> JBOD log directories, LogManager.shutdown() takes 24 seconds.
> Thus we expect to save 71% time in LogManager.shutdown() by having this
> optimization. This patch intends to reduce the broker shutdown time by
> caching the lastEntry in memory so that broker does not have to always read
> disk to get the lastEntry.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
