Hi I have run into a strange behaviour. I have a topic that has retention.ms and segment.ms set to one week. Hasn't been any traffic there for a long time, more than a week. So the logfile was empty. Name och the old logfile was 00000000000000069531.log. We have a job that checks the offset and fetches latest records if there are any, latest record fetched had offset 69530. So that indicates that no records had actually been added to the partition.
Shortly after the restart, due to a operator upgrade, of the cluster the file got swiped and a new one created. [LocalLog partition=o01.qdpo01.qto01hah-1, dir=/var/lib/kafka/data/kafka-log1/o01.qdpo01.qto01hah-1] Rolled new log segment at offset 69532 in 1 ms. And also: [UnifiedLog partition=o01.qdpo01.qto01hah-1, dir=/var/lib/kafka/data/kafka-log1] Deleting segment LogSegment(baseOffset=69531, size=3819, lastModifiedTime=1783335231543, largestRecordTimestamp=0) due to log retention time 604800000ms breach based on the largest record timestamp in the segment What I don't understand is why this triggered an increase in the offset. I mean the possible record on offset 69531 should have been newer than the last job checking for updated offsets. And if that would have been there the old logfile would contain an event that wasn't old due to retention.ms and hence the file shouldn't have been deleted. On the broker node only the new 00000000000000069532.log is present. So the question I have is what can have triggered this increase in offset? BR Per
