[
https://issues.apache.org/jira/browse/HBASE-21011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569257#comment-16569257
]
Tak Lon (Stephen) Wu commented on HBASE-21011:
----------------------------------------------
Thanks for your comment [~reidchan], let me try to clarify below and explain
the use case of this change. apologized for the misleading description (will
update it and please see the [PR #89|https://github.com/apache/hbase/pull/89]
to match the explanation )
{quote} Should be first oldwals then hfiles, it depends on the order it is put
in {{ChoreService}}. So far, oldwals comes first. {quote}
so, one of our use cases is that cleaner chore is disabled, LogCleaner and
HFileCleaner are not running in the background such that the user will need to
execute admin command {{cleaner_chore_run}} to clean up oldwals and hfiles when
they get larger. HBASE-18083 helps cleaning the HFiles faster if they use more
threads as well as HBASE-18309 for oldwals , but this JIRA is especially for
those who disables them by default.
{quote} In what request? they are periodically run without any request {quote}
I was too nerd about wording, but the request mean {{RunCleanerChoreRequest}},
I will update the description to when running admin command
{{cleaner_chore_run}}.
{quote} They are independent two chores, one succeed or not does nothing to the
other. {quote}
So, when the case of cleaner chore(s) is default disabled and when admin issues
{{cleaner_chore_run}}, it will trigger
[{{runCleaner}}|https://github.com/taklwu/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java#L1414-L1420]
for submitting a task for hfiles and oldwals cleaning, but the
{{runCleaner()}} is a block call only return when the task/thread join back.
You're right that the chores are independently running if they're enabled, but
this change is only target for when they're default disabled (major use case)
or if anyone would like use the admin CLI to submit a task in queue (but this
latter use case is useless as you observed)
{quote} Even if you provide a cli, it makes no sense to the cleaning speed.
{quote}
Apologized again on the wrong description, and yes, it won't speedup anything
but only provide a use case for admin/user to run cleaner when cleaner chore is
disabled by default (I cannot tell why one want to disable it, but
{{cleaner_chore_switch}} exists and user has the potential to hit this use case)
I will update the description to make it clear this change is only for when
cleaner chore is disabled, and this is an admin CLI change. please let me know
if this make sense.
> Provide CLI option to run oldwals and hfiles cleaner separately
> ---------------------------------------------------------------
>
> Key: HBASE-21011
> URL: https://issues.apache.org/jira/browse/HBASE-21011
> Project: HBase
> Issue Type: Improvement
> Components: Admin, Client
> Affects Versions: 3.0.0, 1.4.6, 2.1.1
> Reporter: Tak Lon (Stephen) Wu
> Assignee: Tak Lon (Stephen) Wu
> Priority: Minor
> Fix For: 3.0.0
>
>
> Existing logic of cleaner chore is first execute HFiles cleaner and then
> oldwals cleaner in a request, and only return succeed if both completes.
> There is a use case of running only oldwals cleaner because oldwals uses all
> the disk space, and running HFiles cleaner is too slow because either the
> amount of old HFiles or directories are too much. So, this change provide the
> flexibility for those disabled cleaner by default or would like to execute
> admin command to run oldwals and HFiles cleaning procedure individually.
> NOTE that we keep the default as running both of them for backward
> compatibility, e.g. the proposed admin CLI options are
> {noformat}
> hbase> cleaner_chore_run
> hbase> cleaner_chore_run 'hfiles'
> hbase> cleaner_chore_run 'oldwals'
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
