[
https://issues.apache.org/jira/browse/HBASE-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16305598#comment-16305598
]
Peter Somogyi commented on HBASE-18309:
---------------------------------------
bq. Please open new issue.
[~stack], new issue about changing log level?
bq. This is designed, because ttl is not expired, and file is kept which
leading to "Couldn't delete file completely under...", it is not an issue.
Yes, that is by design, but I think WARN log level is too much for this. Maybe
INFO?
bq. If we view it from result, it's ok, but if from logic, it is wrong, because
it's not ReplicationLogCleaner's duty, it's an issue.
Agree, this is not correct behaviour.
> Support multi threads in CleanerChore
> -------------------------------------
>
> Key: HBASE-18309
> URL: https://issues.apache.org/jira/browse/HBASE-18309
> Project: HBase
> Issue Type: Improvement
> Reporter: binlijin
> Assignee: Reid Chan
> Fix For: 3.0.0, 2.0.0-beta-1
>
> Attachments: HBASE-18309.addendum.patch,
> HBASE-18309.master.001.patch, HBASE-18309.master.002.patch,
> HBASE-18309.master.004.patch, HBASE-18309.master.005.patch,
> HBASE-18309.master.006.patch, HBASE-18309.master.007.patch,
> HBASE-18309.master.008.patch, HBASE-18309.master.009.patch,
> HBASE-18309.master.010.patch, HBASE-18309.master.011.patch,
> HBASE-18309.master.012.patch, space_consumption_in_archive.png
>
>
> There is only one thread in LogCleaner to clean oldWALs and in our big
> cluster we find this is not enough. The number of files under oldWALs reach
> the max-directory-items limit of HDFS and cause region server crash, so we
> use multi threads for LogCleaner and the crash not happened any more.
> What's more, currently there's only one thread iterating the archive
> directory, and we could use multiple threads cleaning sub directories in
> parallel to speed it up.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
