[
https://issues.apache.org/jira/browse/HBASE-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wayne updated HBASE-3489:
-------------------------
Attachment: oldlog.txt
Attached are the name node entries with .oldlog. After running on .90 for 5
days I had more than 18+ TB of old logs. The data size was only ~2TB. Having
not even known about this until 2 days ago I had mis-calculated our required
production cluster size (good news). I think these logs should be deleted
normally after 1 day and at a pace able to keep up with heavy writes.
There is I believe a bug from replication in .90 as no logs seemed to get
deleted, but even if it worked as designed I would have to wait until 7 days
and at the rate I was load testing the 40TB limit of our test cluster might
have been reached with actual data of only 10% of that. I question the 7 day
limit as a good default. It causes novices like myself to think the data size
is a lot bigger than it is. I was even convinced lzo compression was not
working due to the spike in disk usage.
> .oldlogs not being cleaned out
> ------------------------------
>
> Key: HBASE-3489
> URL: https://issues.apache.org/jira/browse/HBASE-3489
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.0
> Environment: 10 Nodes Write Heavy Cluster
> Reporter: Wayne
> Attachments: oldlog.txt
>
>
> The .oldlogs folder is never being cleaned up. The
> hbase.master.logcleaner.ttl has been set to clean up the old logs but the
> clean up is never kicking in. The limit of 10 files is not the problem. After
> running for 5 days not a single log file has ever been deleted and the
> logcleaner is set to 2 days (from the default of 7 days). It is assumed that
> the replication changes that want to be sure to keep these logs around if
> needed have caused the cleanup to be blocked. There is no replication defined
> (knowingly).
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
