jolshan commented on a change in pull request #9590:
URL: https://github.com/apache/kafka/pull/9590#discussion_r610089215
##########
File path: core/src/main/scala/kafka/log/Log.scala
##########
@@ -760,6 +768,12 @@ class Log(@volatile private var _dir: File,
// must fall within the range of existing segment(s). If we cannot find
such a segment, it means the deletion
// of that segment was successful. In such an event, we should simply
rename the .swap to .log without having to
// do a replace with an existing segment.
+ //
+ // For case 1 (log cleaning), we may have old segments before or after
the swap segment that were cleaned.
+ // Unfortunately, since the baseOffset and the readNextOffset were
changed, these segments will not be removed on
+ // recovery if they were not yet given a DeletedFileSuffix. A subsequent
cleaning that succeeds will correctly remove these segments.
+ // ie. segments [0, 1000), [1000, 2000), [2000, 3000) cleaned into
[1500, 1750).swap without marking old segments with DeletedFileSuffix
Review comment:
Yes. That is right. `logSegments` does the following:
```
Get all segments beginning with the segment that includes "from" and ending
with the segment
that includes up to "to-1" or the end of the log (if to > logEndOffset).
```
So that grabs [1000,2000) and [2000,3000) as you said. Both of the read
next offsets of these segments are larger than 1500. So these two segments are
replaced with [1500,2500)
I can include this example and the simple summary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
