Thanks all my doubts are resolved now. On Wed, Jul 22, 2020 at 4:46 PM zheng wang <[email protected]> wrote:
> And getMethod does. > > > > > ------------------ 原始邮件 ------------------ > 发件人: > "user" > < > [email protected]>; > 发送时间: 2020年7月22日(星期三) 晚上6:44 > 收件人: "Hbase-User"<[email protected]>; > > 主题: Re: HBase 2.1.0 - NoSuchMethodException > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy > > > > Yes I am using hadoop 3.0.0. > > On Wed, Jul 22, 2020 at 12:31 PM zheng wang <[email protected]> wrote: > > > Are you sure you are using hadoop3.0.0? > > > > > > > > > > ------------------&nbsp;原始邮件&nbsp;------------------ > > 发件人: > > > "user" > > > < > > [email protected]&gt;; > > 发送时间:&nbsp;2020年7月22日(星期三) 凌晨3:17 > > 收件人:&nbsp;"Hbase-User"<[email protected]&gt;; > > > > 主题:&nbsp;Re: HBase 2.1.0 - NoSuchMethodException > > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy > > > > > > > > Thanks for the pointer. > > > > One more query in I am getting this error from CommonFSUtils. > > invokeSetStoragePolicy() > > < > > > https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533&gt > > > <https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533&gt>>; > ;. > > I am using hadoop 3.0.0 and in FilterFileSystem (which LocalFileSystem > > extends from) I do see the method&nbsp; setStoragePolicy > > < > > > https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637&gt > > > <https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637&gt>>; > ; > > . > > > > Can someone explain how is this NoSuchMethodException is being thrown > or I > > am looking at the wrong code path for LocalFileSystem? > > > > On Tue, Jul 21, 2020 at 7:04 PM Sean Busbey <[email protected]&gt; > wrote: > > > > &gt; that is the detail message of why it determined that the > FileSystem > > &gt; doesn't support setStoragePolicy. As opposed to e.g. a > security > > &gt; manager denying access to introspect the methods available. > > &gt; > > &gt; On Tue, Jul 21, 2020 at 8:24 AM Debraj Manna < > > [email protected]&gt; > > &gt; wrote: > > &gt; &gt; > > &gt; &gt; I understood the "util.CommonFSUtils: FileSystem > doesn't support > > &gt; &gt; setStoragePolicy;" part. > > &gt; &gt; > > &gt; &gt; Can you let me know why it is saying > > "java.lang.NoSuchMethodException: > > &gt; &gt; > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy2020-07-20 > > &gt; &gt; 06:02:24,859 WARN&nbsp; > [StoreOpener-1588230740-1]" ? > > &gt; &gt; > > &gt; &gt; > > &gt; &gt; On Tue, Jul 21, 2020 at 6:43 PM zheng wang < > [email protected]&gt; > > wrote: > > &gt; &gt; > > &gt; &gt; &gt; This log info just as a warning that cant > make it disappear > > for now, > > &gt; but > > &gt; &gt; &gt; will not impact anything, so you can just > ignore it in > > local mode. > > &gt; &gt; &gt; > > &gt; &gt; &gt; > > &gt; &gt; &gt; > ------------------&amp;nbsp;原始邮件&amp;nbsp;------------------ > > &gt; &gt; &gt; 发件人: > > &gt; &gt; > > > &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; > > "user" > > &gt; &gt; > > > &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; > > < > > &gt; &gt; &gt; [email protected]&amp;gt;; > > &gt; &gt; &gt; 发送时间:&amp;nbsp;2020年7月21日(星期二) 晚上9:19 > > &gt; &gt; &gt; 收件人:&amp;nbsp;"Hbase-User"< > [email protected]&amp;gt;; > > &gt; &gt; &gt; > > &gt; &gt; &gt; 主题:&amp;nbsp;Re: HBase 2.1.0 - > NoSuchMethodException > > &gt; &gt; &gt; > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy > > &gt; &gt; &gt; > > &gt; &gt; &gt; > > &gt; &gt; &gt; > > &gt; &gt; &gt; Thanks for replying. > > &gt; &gt; &gt; > > &gt; &gt; &gt; Yes it is a single node hbase cluster. I > am not specifying > > any storage > > &gt; &gt; &gt; policy. Looking at the HStore > > &gt; &gt; &gt; < > > &gt; &gt; &gt; > > &gt; > > > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt > > > <https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt>>; > &gt > > < > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt&gt> > ;; > > &gt; &gt; ; > > &gt; &gt; &gt; code it appears even if no storage policy > is specified it > > will take > > &gt; HOT. > > &gt; &gt; &gt; > > &gt; &gt; &gt; Can you explain this a bit more how can I > get around this > > error or in a > > &gt; &gt; &gt; single node hbase cluster I should be > ignoring this? > > &gt; &gt; &gt; > > &gt; &gt; &gt; > > &gt; &gt; &gt; On Tue, Jul 21, 2020 at 3:03 PM zheng wang > <[email protected]&amp;gt; > > wrote: > > &gt; &gt; &gt; > > &gt; &gt; &gt; &amp;gt; LocalFileSystem? > The&amp;amp;nbsp;setStoragePolicy > > could only be used > > &gt; in > > &gt; &gt; &gt; &amp;gt; distributed hdfs. > > &gt; &gt; &gt; &amp;gt; &amp;amp;nbsp; > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; &amp;gt; > > > ------------------&amp;amp;nbsp;原始邮件&amp;amp;nbsp;------------------ > > &gt; &gt; &gt; &amp;gt; 发件人: > > &gt; &gt; &gt; > > &gt; > > > &amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; > > &gt; &gt; &gt; "user" > > &gt; &gt; &gt; > > &gt; > > > &amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; > > &gt; &gt; &gt; < > > &gt; &gt; &gt; &amp;gt; [email protected] > &amp;amp;gt;; > > &gt; &gt; &gt; &amp;gt; > 发送时间:&amp;amp;nbsp;2020年7月21日(星期二) 下午5:58 > > &gt; &gt; &gt; &amp;gt; > 收件人:&amp;amp;nbsp;"Hbase-User"< > > [email protected]&amp;amp;gt;; > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; &amp;gt; 主题:&amp;amp;nbsp;HBase > 2.1.0 - > > NoSuchMethodException > > &gt; &gt; &gt; &amp;gt; > > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; &amp;gt; Hi > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; &amp;gt; I am using HBase 2.1.0 with > Hadoop 3.0.0. In hbase > > master logs I > > &gt; am > > &gt; &gt; &gt; &amp;gt; seeing a warning like below > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; &amp;gt; 2020-07-20 06:02:24,859 > WARN&amp;amp;nbsp; > > [StoreOpener-1588230740-1] > > &gt; &gt; &gt; &amp;gt; util.CommonFSUtils: > FileSystem doesn't support > > setStoragePolicy; > > &gt; &gt; &gt; &amp;gt; HDFS-6584, HDFS-9345 not > available. This is normal > > and expected on > > &gt; &gt; &gt; &amp;gt; earlier Hadoop versions. > > &gt; &gt; &gt; &amp;gt; > java.lang.NoSuchMethodException: > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > &gt; > > > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path, > > &gt; &gt; &gt; &amp;gt; java.lang.String) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > java.lang.Class.getDeclaredMethod(Class.java:2130) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > &gt; > > > org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:577) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > &gt; > > > org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:558) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > &gt; > > > org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:526) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > &gt; > > > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:194) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > &gt; > > > org.apache.hadoop.hbase.regionserver.HStore.<init&amp;amp;gt;(HStore.java:264) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > &gt; > > > org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > > &gt; > org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > > &gt; > org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > > java.util.concurrent.FutureTask.run(FutureTask.java:266) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > > java.util.concurrent.FutureTask.run(FutureTask.java:266) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > &gt; > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > &gt; > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; > > > &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp; > > &gt; at > > &gt; &gt; &gt; &amp;gt; > java.lang.Thread.run(Thread.java:748) > > &gt; &gt; &gt; &amp;gt; > > &gt; &gt; &gt; &amp;gt; As per my understanding, this > error should not be > > coming with > > &gt; Hadoop > > &gt; &gt; &gt; &amp;gt; 3.0.0. Can someone let me > know if my understanding > > is correct or > > &gt; what > > &gt; &gt; &gt; &amp;gt; could be going wrong here? > > &gt; > > &gt; > > &gt; > > &gt; -- > > &gt; Sean > > &gt;
