[
https://issues.apache.org/jira/browse/HBASE-17280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15839978#comment-15839978
]
Ajay Jadhav commented on HBASE-17280:
-------------------------------------
I see some merge conflicts for the master branch.
So, I'll add the master patch soon. I have attached the branch-1.2 patch.
Actually, the patch is based on branch 1.2.2. There I saw the following
cleaners in HMaster:
private ReplicationZKLockCleanerChore replicationZKLockCleanerChore;
private LogCleaner logCleaner;
private HFileCleaner hfileCleaner;
Our main intention with this patch was to avoid running the expensive cleaner
chore during peak times. During our experimentation, we saw a lot of HFiles and
WAL log related files getting created inside archive dir (didn't see ZKlock
related files). Since we were replacing hdfs with S3, these delete calls will
take forever to complete.
As for setCleanerRunning(): We want to enable/ disable the cleaner entirely.
So, when the user run cleaner_switch, we enable/ disable both the cleaners.
They both go down/ up together.
Both HFileCleaner and LogCleaner are implementing cleanerchore which is doing
this expensive list and delete files from the dir which we want to avoid due to
underlying filesystem.
ReplicationZKLockCleanerChore seems like is trying to clean replication locks
on zk and directly implements the scheduledchore.
> Add mechanism to control hbase cleaner behavior
> -----------------------------------------------
>
> Key: HBASE-17280
> URL: https://issues.apache.org/jira/browse/HBASE-17280
> Project: HBase
> Issue Type: Improvement
> Components: Client, hbase, shell
> Affects Versions: 2.0.0, 1.2.0
> Reporter: Ajay Jadhav
> Priority: Minor
> Fix For: 2.0.0, 1.2.0
>
> Attachments: HBASE-17280.branch-1.2.patch,
> HBASE-17280.branch-2.0.patch, HBASE-17280.v1-branch-1.2.patch,
> HBASE-17280.v2-branch-1.2.patch, HBASE-17280.v2-branch-2.patch
>
>
> Cleaner is used to get rid of archived HFiles and old WALs in HBase.
> In the case of heavy workload, cleaner can affect query performance by
> creating a lot of connections to perform costly reads/ writes against
> underlying filesystem.
> This patch allows the user to control HBase cleaner behavior by providing
> shell commands to enable/ disable and manually run it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
