Wei-Chiu Chuang commented on HBASE-20145:

Thanks [~rohithsharma] now I understand why:

Turns out that if you point env HADOOP_HOME to a Hadoop3 path with hbase2.0 
convenience binary, the classpath will contain both Hadoop 2.7.4 and Hadoop 3 
jar files. Effectively causing class conflicts.

Since Hadoop 3 jars are available, HBase is able to query StreamCapabilities of 
the output stream. However, because Hadoop 2.7.4 jars is loaded before Hadoop 3 
jars in classpath, it loaded the HdfsDataOutputStream class implemented in 
Hadoop 2.7.4, which does not inherit from StreamCapabilities.
if (StreamCapabilities.CLASS.isAssignableFrom(stream.getClass())) {
        try {
          LOG.warn("stream type = " + 
          result = ((Boolean)StreamCapabilities.METHOD.invoke(stream, 
        } catch 
            exception) {
          LOG.warn("Your Hadoop installation's StreamCapabilities 
implementation doesn't match " +
              "our understanding of how it's supposed to work. Please file a 
JIRA and include " +
              "the following stack trace. In the mean time we're interpreting 
this behavior " +
              "difference as a lack of capability support, which will probably 
cause a failure.",

This is not a problem in the code itself. But the code can be improved to log 
warning message when output stream class does not inherit from 

> HMaster start fails with IllegalStateException when HADOOP_HOME is set
> ----------------------------------------------------------------------
>                 Key: HBASE-20145
>                 URL: https://issues.apache.org/jira/browse/HBASE-20145
>             Project: HBase
>          Issue Type: Bug
>         Environment: HBase-2.0-beta1.
> Hadoop trunk version.
> java version "1.8.0_144"
>            Reporter: Rohith Sharma K S
>            Assignee: Wei-Chiu Chuang
>            Priority: Critical
> It is observed that HMaster start is failed when HADOOP_HOME is set as env 
> while starting HMaster. HADOOP_HOME is pointing to Hadoop trunk version.
> {noformat}
> 2018-03-07 16:59:52,654 ERROR [master//] master.HMaster: 
> Failed to become active master
> java.lang.IllegalStateException: The procedure WAL relies on the ability to 
> hsync for proper operation during component failures, but the underlying 
> filesystem does not support doing so. Please check the config value of 
> 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness 
> and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount 
> that can provide it.
>     at 
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1036)
>     at 
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:374)
>     at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:532)
>     at 
> org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1232)
>     at 
> org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1145)
>     at 
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:837)
>     at 
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2026)
>     at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:547)
>     at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The same configs is working in HBase-1.2.6 build properly. 

This message was sent by Atlassian JIRA

Reply via email to