[
https://issues.apache.org/jira/browse/HBASE-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16517662#comment-16517662
]
Sean Busbey commented on HBASE-20691:
-------------------------------------
CommonFSUtils should be updated to check against
{{DEFER_TO_HDFS_STORAGE_POLICY}} rather than against a passed in default of
{{DEFAULT_WAL_STORAGE_POLICY}}. They're the same thing now, but they won't
necessarily be.
Since the {{setStoragePolicy}} code is in CommonFSUtils, the test should be in
TestCommonFSUtils.
{code}
354 LOG.debug("Before set storage policy to NONE");
355 FSUtils.setStoragePolicy(testFs, conf, new Path("non-exist"),
HConstants.WAL_STORAGE_POLICY,
356 HConstants.DEFAULT_WAL_STORAGE_POLICY);
357 LOG.debug("After set storage policy to NONE");
358 conf.set(HConstants.WAL_STORAGE_POLICY, "HOT");
359 // warning log is expected when passing some valid policy
360 FSUtils.setStoragePolicy(testFs, conf, new Path("non-exist"),
HConstants.WAL_STORAGE_POLICY,
361 HConstants.DEFAULT_WAL_STORAGE_POLICY);
{code}
Shouldn't this second invocation have thrown an IOException?
> Storage policy should allow deferring to HDFS
> ---------------------------------------------
>
> Key: HBASE-20691
> URL: https://issues.apache.org/jira/browse/HBASE-20691
> Project: HBase
> Issue Type: Bug
> Components: Filesystem Integration, wal
> Affects Versions: 1.5.0, 2.0.0
> Reporter: Sean Busbey
> Assignee: Yu Li
> Priority: Minor
> Fix For: 2.1.0, 1.5.0
>
> Attachments: HBASE-20691.patch, HBASE-20691.v2.patch,
> HBASE-20691.v3.patch
>
>
> In HBase 1.1 - 1.4 we can defer storage policy decisions to HDFS by using
> "NONE" as the storage policy in hbase configs.
> As described on this [dev@hbase thread "WAL storage policies and interactions
> with Hadoop admin
> tools."|https://lists.apache.org/thread.html/d220726fab4bb4c9e117ecc8f44246402dd97bfc986a57eb22311117@%3Cdev.hbase.apache.org%3E]
> we no longer have that option in 2.0.0 and 1.5.0 (as the branch is now).
> Additionally, we can't set the policy to HOT in the event that HDFS has
> changed the policy for a parent directory of our WALs.
> We should put back that ability. Presuming this is done by re-adopting the
> "NONE" placeholder variable, we need to ensure that value doesn't get passed
> to HDFS APIs. Since it isn't a valid storage policy attempting to use it will
> cause a bunch of logging churn (which will be a regression of the problem
> HBASE-18118 sought to fix).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
