[ https://issues.apache.org/jira/browse/HBASE-22951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919796#comment-16919796 ]
stack commented on HBASE-22951: ------------------------------- Fails. hbck1 always wanted to talk to hdfs. hbck2 does now since we backfilled checks of hdfs. Let me see if a 'shaded' or cut-down hdfs client to add to CP by default. Thanks for taking a look. > [HBCK2] hbase hbck throws IOE "No FileSystem for scheme: hdfs" > -------------------------------------------------------------- > > Key: HBASE-22951 > URL: https://issues.apache.org/jira/browse/HBASE-22951 > Project: HBase > Issue Type: Bug > Reporter: stack > Priority: Major > > Input appreciated on this one. > If I do the below, passing a config that is pointing at a HDFS, I get the > below (If I run w/o, hbck just picks up the wrong fs -- the local fs). > {code} > $ /vagrant/hbase/bin/hbase --config hbase-conf hbck > 2019-08-30 05:04:54,467 WARN [main] util.NativeCodeLoader: Unable to load > native-hadoop library for your platform... using builtin-java classes where > applicable > Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs > at > org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2799) > at > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2810) > at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100) > at > org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2849) > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2831) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:389) > at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356) > at > org.apache.hadoop.hbase.util.CommonFSUtils.getRootDir(CommonFSUtils.java:361) > at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3605) > {code} > Its because the CLASSPATH is carefully curated so as to use shaded client > only; there are no hdfs classes on the CLASSPATH intentionally. > So, how to fix? Happens whether hbck1 or hbck2 (you have to do a hdfs > operation for hbck2 to trigger same issue). > Could be careful in hbck2 and note that if fs operation, you need to add hdfs > jars to CLASSPATH so hbck2 can go against hdfs. > If add the ' --internal-classpath' flag, then all classes are put on the > CLASSPATH for hbck(2) (including the hdfs client jar which got the hdfs > implementation after 2.7.2 was released) and stuff 'works'. > Could edit the bin/hbase script and make it so hdfs classes are added to the > hbck CLASSPATH? Could see if could do hdfs client-only? > Anyways, putting this up for now. Others may have opinions. Thanks. -- This message was sent by Atlassian Jira (v8.3.2#803003)