[
https://issues.apache.org/jira/browse/HBASE-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518952#comment-16518952
]
Yu Li commented on HBASE-20691:
-------------------------------
bq. CommonFSUtils should be updated to check against
DEFER_TO_HDFS_STORAGE_POLICY
It should check against the default storage policy and let through if so.
Currently the default policy is set to "NONE", and we give it a comprehensive
constant name {{DEFER_TO_HDFS_STORAGE_POLICY}}
bq. the test should be in TestCommonFSUtils
Agree that they should be. Checking commit history, the set-storage related
test cases were added in {{TestFSUtils}} by HBASE-13498 and somehow left there
during the code refactor in HBASE-18784... How about we open a follow-on JIRA
to move all set-storage test cases into {{TestCommonFSUtils}} after this one?
bq. Shouldn't this second invocation have thrown an IOException?
Personally I think it's OK to let it fail silently with some warning log if the
given policy is invalid or the set policy attempt failed, as the current
implementation does. Throwing an IOE and cause region fail to open is too much
IMHO.
> Storage policy should allow deferring to HDFS
> ---------------------------------------------
>
> Key: HBASE-20691
> URL: https://issues.apache.org/jira/browse/HBASE-20691
> Project: HBase
> Issue Type: Bug
> Components: Filesystem Integration, wal
> Affects Versions: 1.5.0, 2.0.0
> Reporter: Sean Busbey
> Assignee: Yu Li
> Priority: Minor
> Fix For: 2.1.0, 1.5.0
>
> Attachments: HBASE-20691.patch, HBASE-20691.v2.patch,
> HBASE-20691.v3.patch
>
>
> In HBase 1.1 - 1.4 we can defer storage policy decisions to HDFS by using
> "NONE" as the storage policy in hbase configs.
> As described on this [dev@hbase thread "WAL storage policies and interactions
> with Hadoop admin
> tools."|https://lists.apache.org/thread.html/d220726fab4bb4c9e117ecc8f44246402dd97bfc986a57eb22311117@%3Cdev.hbase.apache.org%3E]
> we no longer have that option in 2.0.0 and 1.5.0 (as the branch is now).
> Additionally, we can't set the policy to HOT in the event that HDFS has
> changed the policy for a parent directory of our WALs.
> We should put back that ability. Presuming this is done by re-adopting the
> "NONE" placeholder variable, we need to ensure that value doesn't get passed
> to HDFS APIs. Since it isn't a valid storage policy attempting to use it will
> cause a bunch of logging churn (which will be a regression of the problem
> HBASE-18118 sought to fix).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
