?????? HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy

zheng wang Wed, 22 Jul 2020 00:08:15 -0700

Are you sure you are using hadoop3.0.0?




------------------&nbsp;????????&nbsp;------------------
??????:                                                                         
                                               "user"                           
                                                         
<[email protected]&gt;;
????????:&nbsp;2020??7??22??(??????) ????3:17
??????:&nbsp;"Hbase-User"<[email protected]&gt;;

????:&nbsp;Re: HBase 2.1.0 - NoSuchMethodException 
org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy



Thanks for the pointer.

One more query in I am getting this error from CommonFSUtils.
invokeSetStoragePolicy()
<https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533&gt;.
I am using hadoop 3.0.0 and in FilterFileSystem (which LocalFileSystem
extends from) I do see the method&nbsp; setStoragePolicy
<https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637&gt;
.

Can someone explain how is this NoSuchMethodException is being thrown or I
am looking at the wrong code path for LocalFileSystem?

On Tue, Jul 21, 2020 at 7:04 PM Sean Busbey <[email protected]&gt; wrote:

&gt; that is the detail message of why it determined that the FileSystem
&gt; doesn't support setStoragePolicy. As opposed to e.g. a security
&gt; manager denying access to introspect the methods available.
&gt;
&gt; On Tue, Jul 21, 2020 at 8:24 AM Debraj Manna <[email protected]&gt;
&gt; wrote:
&gt; &gt;
&gt; &gt; I understood the "util.CommonFSUtils: FileSystem doesn't support
&gt; &gt; setStoragePolicy;" part.
&gt; &gt;
&gt; &gt; Can you let me know why it is saying "java.lang.NoSuchMethodException:
&gt; &gt; org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy2020-07-20
&gt; &gt; 06:02:24,859 WARN&nbsp; [StoreOpener-1588230740-1]" ?
&gt; &gt;
&gt; &gt;
&gt; &gt; On Tue, Jul 21, 2020 at 6:43 PM zheng wang <[email protected]&gt; wrote:
&gt; &gt;
&gt; &gt; &gt; This log info just as a warning that cant make it disappear for 
now,
&gt; but
&gt; &gt; &gt; will not impact anything, so you can just ignore it in local 
mode.
&gt; &gt; &gt;
&gt; &gt; &gt;
&gt; &gt; &gt; ------------------&amp;nbsp;????????&amp;nbsp;------------------
&gt; &gt; &gt; ??????:
&gt; &gt; 
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 "user"
&gt; &gt; 
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 <
&gt; &gt; &gt; [email protected]&amp;gt;;
&gt; &gt; &gt; ????????:&amp;nbsp;2020??7??21??(??????) ????9:19
&gt; &gt; &gt; ??????:&amp;nbsp;"Hbase-User"<[email protected]&amp;gt;;
&gt; &gt; &gt;
&gt; &gt; &gt; ????:&amp;nbsp;Re: HBase 2.1.0 - NoSuchMethodException
&gt; &gt; &gt; org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
&gt; &gt; &gt;
&gt; &gt; &gt;
&gt; &gt; &gt;
&gt; &gt; &gt; Thanks for replying.
&gt; &gt; &gt;
&gt; &gt; &gt; Yes it is a single node hbase cluster. I am not specifying any 
storage
&gt; &gt; &gt; policy. Looking at the HStore
&gt; &gt; &gt; <
&gt; &gt; &gt;
&gt; 
https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt
&gt; &gt; &gt; ;
&gt; &gt; &gt; code it appears even if no storage policy is specified it will 
take
&gt; HOT.
&gt; &gt; &gt;
&gt; &gt; &gt; Can you explain this a bit more how can I get around this error 
or in a
&gt; &gt; &gt; single node hbase cluster I should be ignoring this?
&gt; &gt; &gt;
&gt; &gt; &gt;
&gt; &gt; &gt; On Tue, Jul 21, 2020 at 3:03 PM zheng wang 
<[email protected]&amp;gt; wrote:
&gt; &gt; &gt;
&gt; &gt; &gt; &amp;gt; LocalFileSystem? The&amp;amp;nbsp;setStoragePolicy 
could only be used
&gt; in
&gt; &gt; &gt; &amp;gt; distributed hdfs.
&gt; &gt; &gt; &amp;gt; &amp;amp;nbsp;
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; &amp;gt; 
------------------&amp;amp;nbsp;????????&amp;amp;nbsp;------------------
&gt; &gt; &gt; &amp;gt; ??????:
&gt; &gt; &gt;
&gt; 
&amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
&gt; &gt; &gt; "user"
&gt; &gt; &gt;
&gt; 
&amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
&gt; &gt; &gt; <
&gt; &gt; &gt; &amp;gt; [email protected]&amp;amp;gt;;
&gt; &gt; &gt; &amp;gt; ????????:&amp;amp;nbsp;2020??7??21??(??????) ????5:58
&gt; &gt; &gt; &amp;gt; 
??????:&amp;amp;nbsp;"Hbase-User"<[email protected]&amp;amp;gt;;
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; &amp;gt; ????:&amp;amp;nbsp;HBase 2.1.0 - NoSuchMethodException
&gt; &gt; &gt; &amp;gt; org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; &amp;gt; Hi
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; &amp;gt; I am using HBase 2.1.0 with Hadoop 3.0.0. In hbase 
master logs I
&gt; am
&gt; &gt; &gt; &amp;gt; seeing a warning like below
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; &amp;gt; 2020-07-20 06:02:24,859 WARN&amp;amp;nbsp; 
[StoreOpener-1588230740-1]
&gt; &gt; &gt; &amp;gt; util.CommonFSUtils: FileSystem doesn't support 
setStoragePolicy;
&gt; &gt; &gt; &amp;gt; HDFS-6584, HDFS-9345 not available. This is normal and 
expected on
&gt; &gt; &gt; &amp;gt; earlier Hadoop versions.
&gt; &gt; &gt; &amp;gt; java.lang.NoSuchMethodException:
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt;
&gt; 
org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path,
&gt; &gt; &gt; &amp;gt; java.lang.String)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt; java.lang.Class.getDeclaredMethod(Class.java:2130)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt;
&gt; 
org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:577)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt;
&gt; 
org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:558)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt;
&gt; 
org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:526)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt;
&gt; 
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:194)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt;
&gt; 
org.apache.hadoop.hbase.regionserver.HStore.<init&amp;amp;gt;(HStore.java:264)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt;
&gt; 
org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt;
&gt; org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt;
&gt; org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt; java.util.concurrent.FutureTask.run(FutureTask.java:266)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt; java.util.concurrent.FutureTask.run(FutureTask.java:266)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt;
&gt; 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt;
&gt; 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; 
&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
&gt; at
&gt; &gt; &gt; &amp;gt; java.lang.Thread.run(Thread.java:748)
&gt; &gt; &gt; &amp;gt;
&gt; &gt; &gt; &amp;gt; As per my understanding, this error should not be 
coming with
&gt; Hadoop
&gt; &gt; &gt; &amp;gt; 3.0.0. Can someone let me know if my understanding is 
correct or
&gt; what
&gt; &gt; &gt; &amp;gt; could be going wrong here?
&gt;
&gt;
&gt;
&gt; --
&gt; Sean
&gt;

?????? HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy

Reply via email to