[
https://issues.apache.org/jira/browse/HBASE-21011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16577029#comment-16577029
]
Tak Lon (Stephen) Wu edited comment on HBASE-21011 at 8/11/18 5:08 AM:
-----------------------------------------------------------------------
Thanks [~reidchan] for your suggestion about using {{hdfs -rm}} but I don't
think that would be a good suggestion to operator due to any error-prone
entering (pointing to directory other than oldWALs or HFiles archive directory)
while using {{-rm}}. I hope not but it could lead to data loss disaster (that's
why I don't suggest operator to use hdfs command as workaround if possible).
But I do agree with you about the point of `endless requirement`, once I get
back from our use cases, I will close this item.
Also, I think part of the HBase on Cloud should be related HBASE-20952 about
WAL interface decouple from HDFS (although I didn't put any comment), but IMHO
backward compatibility should not only related this {{run_cleaner_chore}} admin
CLI but all related components that handles WAL. Anyway, I will keep this in
mind for future improvements/changes related to WAL.
was (Author: taklwu):
Thanks [~reidchan] for your suggestion about using {{hdfs -rm}} but I don't
think that would be a good suggestion to operator due to any error-prone
entering while using {{-rm}} could lead to data loss disaster. But I do agree
with you about the point of `endless requirement`, once I get back from our use
cases, I will close this item.
Also, I think part of the HBase on Cloud should be related HBASE-20952 about
WAL interface decouple from HDFS (although I didn't put any comment), but IMHO
backward compatibility should not only related this {{run_cleaner_chore}} admin
CLI but all related components that handles WAL. Anyway, I will keep this in
mind for future improvements/changes related to WAL.
> Provide CLI option to run oldwals and hfiles cleaner separately when cleaner
> chore is disabled
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-21011
> URL: https://issues.apache.org/jira/browse/HBASE-21011
> Project: HBase
> Issue Type: Improvement
> Components: Admin, Client
> Affects Versions: 3.0.0, 1.4.6, 2.1.1
> Reporter: Tak Lon (Stephen) Wu
> Assignee: Tak Lon (Stephen) Wu
> Priority: Minor
> Attachments: HBASE-21011.master.001.patch,
> HBASE-21011.master.002.patch, HBASE-21011.master.003.patch,
> HBASE-21011.master.004.patch
>
>
> There is a corner case when cleaner chore for HFiles and oldwals is disabled,
> admin/user needs to manually execute admin command {{cleaner_chore_run}} to
> clean the old HFiles and oldwals. Existing logic of {{cleaner_chore_run}} is
> to [firstly trigger the HFiles cleaner and then oldwals
> cleaner|https://github.com/taklwu/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java#L1414-L1420],
> and only return succeed if both completes.
> but when running this {{cleaner_chore_run}} command, there is a potential use
> case that admin would like trigger the cleaner for only oldwals or hfiles but
> still keep the automatic cleaner chore disabled. So, this change aims to
> provide support for this corner case, and provide flexibility for those user
> with cleaner chore disabled by default to execute admin CLI to run oldwals
> and HFiles cleaning procedure individually.
> NOTE that {{cleaner_chore_run}} was introduced in HBASE-17280, this patch
> added options 'hfiles' and 'oldwals' to it. Also fix default behavior of
> {{cleaner_chore_run}} will be only ran when cleaner chore is set to disabled,
> e.g. the proposed admin CLI options are
> {noformat}
> hbase> cleaner_chore_run # this was introduced in HBASE-17280,
> but changed the behavior to only ran when cleaner chore is set to disabled
> hbase> cleaner_chore_run 'hfiles' # added, ran when cleaner chore is set
> to disabled
> hbase> cleaner_chore_run 'oldwals' # added, ran when cleaner chore is set
> to disabled
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
