[
https://issues.apache.org/jira/browse/HBASE-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16519029#comment-16519029
]
Yu Li edited comment on HBASE-20691 at 6/21/18 7:45 AM:
--------------------------------------------------------
bq. Right now the code in CommonFSUtils just checks against the default passed
into the call to setStoragePolicy, not against any constant. That's incorrect
Ok, got your point, it follows this mode since introduced by HBASE-12848. Let
me remove the {{setStoragePolicy}} method with the *defaultPolicy* parameter
and directly check against constant in the {{setStoragePolicy}} method with 3
parameters.
bq. I don't mean in the region server, I mean just here in this test.
Ah I see, the test case here simply tries to prove the HDFS api won't be called
if we tries to set the storage policy to default, and vice versa. Please check
the new patch and I think it will be much more clear. Please note that the
IOException thrown will be caught and logged as a warning like below (I guess
you ignored the UT result I pasted above sir, so allow me to repeat):
{noformat}
2018-06-08 22:59:39,063 WARN [Time-limited test] util.CommonFSUtils(572):
Unable to set storagePolicy=HOT for path=non-exist. DEBUG log level might have
more details.
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:563)
at
org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:524)
at
org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:484)
at
org.apache.hadoop.hbase.util.TestFSUtils.verifyNoHDFSApiInvocationForDefaultPolicy(TestFSUtils.java:356)
at
org.apache.hadoop.hbase.util.TestFSUtils.testSetStoragePolicyDefault(TestFSUtils.java:341)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...
Caused by: java.io.IOException: The setStoragePolicy method is invoked
unexpectedly
at
org.apache.hadoop.hbase.util.TestFSUtils$AlwaysFailSetStoragePolicyFileSystem.setStoragePolicy(TestFSUtils.java:364)
... 30 more
{noformat}
was (Author: carp84):
bq. Right now the code in CommonFSUtils just checks against the default passed
into the call to setStoragePolicy, not against any constant. That's incorrect
Ok, got your point, it follows this mode since introduced by HBASE-12848. Let
me remove the {{setStoragePolicy}} method with the *defaultPolicy* parameter
and directly check against constant in the {{setStoragePolicy}} method with 3
parameters.
bq. I don't mean in the region server, I mean just here in this test.
Ah I see, the test case here simply tries to prove the HDFS api won't be called
if we tries to set the storage policy to default, and vice versa. Please check
the new patch and I think it will be much more clear
> Storage policy should allow deferring to HDFS
> ---------------------------------------------
>
> Key: HBASE-20691
> URL: https://issues.apache.org/jira/browse/HBASE-20691
> Project: HBase
> Issue Type: Bug
> Components: Filesystem Integration, wal
> Affects Versions: 1.5.0, 2.0.0
> Reporter: Sean Busbey
> Assignee: Yu Li
> Priority: Minor
> Fix For: 2.1.0, 1.5.0
>
> Attachments: HBASE-20691.patch, HBASE-20691.v2.patch,
> HBASE-20691.v3.patch, HBASE-20691.v4.patch
>
>
> In HBase 1.1 - 1.4 we can defer storage policy decisions to HDFS by using
> "NONE" as the storage policy in hbase configs.
> As described on this [dev@hbase thread "WAL storage policies and interactions
> with Hadoop admin
> tools."|https://lists.apache.org/thread.html/d220726fab4bb4c9e117ecc8f44246402dd97bfc986a57eb22311117@%3Cdev.hbase.apache.org%3E]
> we no longer have that option in 2.0.0 and 1.5.0 (as the branch is now).
> Additionally, we can't set the policy to HOT in the event that HDFS has
> changed the policy for a parent directory of our WALs.
> We should put back that ability. Presuming this is done by re-adopting the
> "NONE" placeholder variable, we need to ensure that value doesn't get passed
> to HDFS APIs. Since it isn't a valid storage policy attempting to use it will
> cause a bunch of logging churn (which will be a regression of the problem
> HBASE-18118 sought to fix).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
