[
https://issues.apache.org/jira/browse/KAFKA-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiangtao Liu updated KAFKA-8270:
--------------------------------
Description:
Kafka log cannot be deleted after the configured retention hours (12 hours for
log retention).
what's our Kafka cluster?
There are 6 brokers deployed with Kafka version 1.1.1.
Is it reproducible?
I am not sure since our Kafka cluster is working well over 1.5 years without
retention issue until 4/13/2019 ~ 4/20/2019.
is it related to the active segment?
as I know Kafka will not delete an active segment, my case those old logs are
not activated, they should be inactivated.
What's the current status?
Those old logs have been deleted after I manually rolling Kafka server with
retention hours adjustment, now our Kafka storage is back to normal, please
check the screenshot attached with this ticket.
A sample old log shared here to a better understanding of the retention hours
not working issue.
```
// it has been there from 4/12
-rw-r--r-- 1 root root 136866627{color:#d04437} Apr 12{color} 04:33
00000000002581377820.log
// It was still being opened by Kafka when I check it with the tool lsof on
{color:#d04437}4/19/2019 before server rolling restart with retention hours
adjustment{color}.
java 20281 0 1678u REG 202,32 136866627 1074562295
/kafka/data/.../00000000002581377820.log
```
was:
Kafka log cannot be deleted after the configured retention hours (12 hours for
log retention).
what's our Kafka cluster?
There are 6 brokers deployed with Kafka version 1.1.1.
Is it reproducible?
I am not sure since our Kafka cluster is working well over 1.5 years without
retention issue until 4/13/2019 ~ 4/20/2019.
is it related to the active segment?
as I know Kafka will not delete an active segment, my case those old logs are
not activated, they should be inactivated.
What's current status?
Those old logs have been deleted after I manually rolling Kafka server with
retention hours adjustment.
A sample old log
```
// it has been there from 4/12
-rw-r--r-- 1 root root 136866627{color:#d04437} Apr 12{color} 04:33
00000000002581377820.log
// It was still being opened by Kafka when I check it with the tool lsof on
{color:#d04437}4/19/2019 before server rolling restart with retention hours
adjustment{color}.
java 20281 0 1678u REG 202,32 136866627 1074562295
/kafka/data/.../00000000002581377820.log
```
> Kafka retention hour is not working
> -----------------------------------
>
> Key: KAFKA-8270
> URL: https://issues.apache.org/jira/browse/KAFKA-8270
> Project: Kafka
> Issue Type: Bug
> Components: log
> Reporter: Jiangtao Liu
> Assignee: Richard Yu
> Priority: Major
> Labels: storage
> Attachments: Screen Shot 2019-04-20 at 10.57.59 PM.png
>
>
> Kafka log cannot be deleted after the configured retention hours (12 hours
> for log retention).
>
> what's our Kafka cluster?
> There are 6 brokers deployed with Kafka version 1.1.1.
>
> Is it reproducible?
> I am not sure since our Kafka cluster is working well over 1.5 years without
> retention issue until 4/13/2019 ~ 4/20/2019.
>
> is it related to the active segment?
> as I know Kafka will not delete an active segment, my case those old logs
> are not activated, they should be inactivated.
>
> What's the current status?
> Those old logs have been deleted after I manually rolling Kafka server with
> retention hours adjustment, now our Kafka storage is back to normal, please
> check the screenshot attached with this ticket.
> A sample old log shared here to a better understanding of the retention hours
> not working issue.
> ```
> // it has been there from 4/12
> -rw-r--r-- 1 root root 136866627{color:#d04437} Apr 12{color} 04:33
> 00000000002581377820.log
> // It was still being opened by Kafka when I check it with the tool lsof on
> {color:#d04437}4/19/2019 before server rolling restart with retention hours
> adjustment{color}.
> java 20281 0 1678u REG 202,32 136866627 1074562295
> /kafka/data/.../00000000002581377820.log
> ```
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
