[
https://issues.apache.org/jira/browse/KAFKA-16782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17862430#comment-17862430
]
Roland Sommer commented on KAFKA-16782:
---------------------------------------
As both affected partitions kept filling up, I was forced to issue a manual
`kafka-delete-records.sh` command. I chose a timestamp older than what should
have been deleted automatically. After issuing the command all the segments
that should have been deleted anyway were removed, too.
> Some partition's segments are suddenly not deleted anymore
> ----------------------------------------------------------
>
> Key: KAFKA-16782
> URL: https://issues.apache.org/jira/browse/KAFKA-16782
> Project: Kafka
> Issue Type: Bug
> Reporter: Roland Sommer
> Priority: Major
>
> I recently discovered an odd behaviour in one of our kafka clusters
> (KRaft-based, v3.7.0):
> We have a topic for distributed log collection with 48 partitions. Retention
> is set to 84 hours, we have the default {{cleanup.policy=delete}} in place.
> For all but two partitions this works as expected. In two partition
> directories there are files going back to january and consuming the specific
> partitions yields data from january (showing it's not only the files lying
> around, they are actually processed).
> Topic settings as per {{kafka-topics.sh --describe}}:
> {{Topic: syslog TopicId: AeJLnYPnQFOtMc0ZjpH7sw PartitionCount: 48
> ReplicationFactor: 2 Configs:
> compression.type=snappy,cleanup.policy=delete,segment.bytes=1073741824,retention.ms=302400000,max.message.bytes=2097152}}
> Searching the cluster logs, there is no indicator of what could be the reason
> here (at least I did not spot anything suspicious up until now). Up to the
> time were deletion stopped, there are log entries showing the deleteion of
> old log segments, but that simply stopped. As far as I can see, there has not
> been any change on the cluster at that point.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
