Jeff Kim created KAFKA-14407:
--------------------------------
Summary: Able to access offset index memory map even after memory
map flag is turned off
Key: KAFKA-14407
URL: https://issues.apache.org/jira/browse/KAFKA-14407
Project: Kafka
Issue Type: Bug
Reporter: Jeff Kim
Original investigation:
[https://github.com/apache/kafka/pull/12783#issuecomment-1315804689]
we have mechanisms in place to check before accessing a memory map. an example:
private def fetchHighWatermarkMetadata: LogOffsetMetadata = {
localLog.checkIfMemoryMappedBufferClosed()
...
}
which makes it seem okay to read from the offset index. however, we should not
allow any disk io if an offset index is unmapped. moreover, if a local log is
closed, we should set `isMemoryMappedBufferClosed` to true in
`LocalLog.close()`:
private[log] def close(): Unit = {
maybeHandleIOException(s"Error while renaming dir for $topicPartition in
dir ${dir.getParent}") {
checkIfMemoryMappedBufferClosed()
segments.close()
// isMemoryMappedBufferClosed = true?
}
}
note that besides altering replica log dir, UnifiedLog.close() is only called
during shutdown.
i'm not sure if this is a bug but accessing the high watermark (by reading from
the actual segment via offset index) will result in a NPE because
isMemoryMappedBufferClosed is false.
java.lang.NullPointerException
at kafka.log.OffsetIndex.$anonfun$lookup$1(OffsetIndex.scala:90)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
