I am wondering if the following sequence of events is possible: 1) A given partition has both current and future replica on this broker. 2) The background scheduler thread executes `cleanupLogs()` and plans to call `log.deleteOldSegments` for the future log of this partition. 3) The ReplicaAlterDirThread decides that the future replica has caught up with the current replica and it executes `Partition.maybeReplaceCurrentWithFutureReplica()` to replace the current log with future log for this partition. 4) CleanerThread executes `cleanOrSleep()` and picks this partition for log compaction. 5) The background scheduler thread starts to execute `log.deleteOldSegments` for this partition while CleanerThread is compacting the same log object.
If this is possible, it is probably safer to also use `pauseCleaningForNonCompactedPartitions()` to make sure that the partition of the future log will not be cleaned concurrently. In terms of implementation, if `cleaner !=null`, then we can call `pauseCleaningForNonCompactedPartitions()` only once, and then for each deletable partition, call `deleteOldSegments()` for both its current log and future log (if exists). What do you think? [ Full content available at: https://github.com/apache/kafka/pull/5591 ] This message was relayed via gitbox.apache.org for [email protected]
