[ 
https://issues.apache.org/jira/browse/HBASE-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506121#comment-16506121
 ] 

Yu Li edited comment on HBASE-20691 at 6/8/18 3:07 PM:
-------------------------------------------------------

Below is what I observed in my local run with patch v3, FYI.
{noformat}
2018-06-08 22:59:39,059 DEBUG [Time-limited test] util.TestFSUtils(350): Before 
set storage policy to NONE
2018-06-08 22:59:39,062 DEBUG [Time-limited test] util.TestFSUtils(353): After 
set storage policy to NONE
2018-06-08 22:59:39,063 WARN  [Time-limited test] util.CommonFSUtils(572): 
Unable to set storagePolicy=HOT for path=non-exist. DEBUG log level might have 
more details.
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:563)
        at 
org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:524)
        at 
org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:484)
        at 
org.apache.hadoop.hbase.util.TestFSUtils.verifyNoHDFSApiInvocationForDefaultPolicy(TestFSUtils.java:356)
        at 
org.apache.hadoop.hbase.util.TestFSUtils.testSetStoragePolicyDefault(TestFSUtils.java:341)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        ...
Caused by: java.io.IOException: The setStoragePolicy method is invoked 
unexpectedly
        at 
org.apache.hadoop.hbase.util.TestFSUtils$AlwaysFailSetStoragePolicyFileSystem.setStoragePolicy(TestFSUtils.java:364)
        ... 30 more
{noformat}


was (Author: carp84):
Below is what I observed in my local run, FYI.
{noformat}
2018-06-08 22:59:39,059 DEBUG [Time-limited test] util.TestFSUtils(350): Before 
set storage policy to NONE
2018-06-08 22:59:39,062 DEBUG [Time-limited test] util.TestFSUtils(353): After 
set storage policy to NONE
2018-06-08 22:59:39,063 WARN  [Time-limited test] util.CommonFSUtils(572): 
Unable to set storagePolicy=HOT for path=non-exist. DEBUG log level might have 
more details.
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:563)
        at 
org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:524)
        at 
org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:484)
        at 
org.apache.hadoop.hbase.util.TestFSUtils.verifyNoHDFSApiInvocationForDefaultPolicy(TestFSUtils.java:356)
        at 
org.apache.hadoop.hbase.util.TestFSUtils.testSetStoragePolicyDefault(TestFSUtils.java:341)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        ...
Caused by: java.io.IOException: The setStoragePolicy method is invoked 
unexpectedly
        at 
org.apache.hadoop.hbase.util.TestFSUtils$AlwaysFailSetStoragePolicyFileSystem.setStoragePolicy(TestFSUtils.java:364)
        ... 30 more
{noformat}

> Storage policy should allow deferring to HDFS
> ---------------------------------------------
>
>                 Key: HBASE-20691
>                 URL: https://issues.apache.org/jira/browse/HBASE-20691
>             Project: HBase
>          Issue Type: Bug
>          Components: Filesystem Integration, wal
>    Affects Versions: 1.5.0, 2.0.0
>            Reporter: Sean Busbey
>            Assignee: Yu Li
>            Priority: Minor
>             Fix For: 2.1.0, 1.5.0
>
>         Attachments: HBASE-20691.patch, HBASE-20691.v2.patch, 
> HBASE-20691.v3.patch
>
>
> In HBase 1.1 - 1.4 we can defer storage policy decisions to HDFS by using 
> "NONE" as the storage policy in hbase configs.
> As described on this [dev@hbase thread "WAL storage policies and interactions 
> with Hadoop admin 
> tools."|https://lists.apache.org/thread.html/d220726fab4bb4c9e117ecc8f44246402dd97bfc986a57eb22311117@%3Cdev.hbase.apache.org%3E]
>  we no longer have that option in 2.0.0 and 1.5.0 (as the branch is now). 
> Additionally, we can't set the policy to HOT in the event that HDFS has 
> changed the policy for a parent directory of our WALs.
> We should put back that ability. Presuming this is done by re-adopting the 
> "NONE" placeholder variable, we need to ensure that value doesn't get passed 
> to HDFS APIs. Since it isn't a valid storage policy attempting to use it will 
> cause a bunch of logging churn (which will be a regression of the problem 
> HBASE-18118 sought to fix).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to