[
https://issues.apache.org/jira/browse/HBASE-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437475#comment-16437475
]
Wei-Chiu Chuang edited comment on HBASE-20145 at 4/13/18 4:53 PM:
------------------------------------------------------------------
Thanks [~rohithsharma] now I understand why:
Turns out that if you point env HADOOP_HOME to a Hadoop3 path with hbase2.0
convenience binary, the classpath will contain both Hadoop 2.7.4 and Hadoop 3
jar files. Effectively causing class conflicts.
Since Hadoop 3 jars are available, HBase is able to query StreamCapabilities of
the output stream. However, because Hadoop 2.7.4 jars is loaded before Hadoop 3
jars in classpath, it loaded the HdfsDataOutputStream class implemented in
Hadoop 2.7.4, which does not inherit from StreamCapabilities.
{code:java}
if (StreamCapabilities.CLASS.isAssignableFrom(stream.getClass())) {
try {
result = ((Boolean)StreamCapabilities.METHOD.invoke(stream,
capability)).booleanValue();
} catch
(IllegalAccessException|IllegalArgumentException|InvocationTargetException
exception) {
LOG.warn("Your Hadoop installation's StreamCapabilities
implementation doesn't match " +
"our understanding of how it's supposed to work. Please file a
JIRA and include " +
"the following stack trace. In the mean time we're interpreting
this behavior " +
"difference as a lack of capability support, which will probably
cause a failure.",
exception);
}
}
{code}
This is not a problem in the code itself. But the code can be improved to log a
warning message when output stream class does not inherit from
StreamCapabilities.
was (Author: jojochuang):
Thanks [~rohithsharma] now I understand why:
Turns out that if you point env HADOOP_HOME to a Hadoop3 path with hbase2.0
convenience binary, the classpath will contain both Hadoop 2.7.4 and Hadoop 3
jar files. Effectively causing class conflicts.
Since Hadoop 3 jars are available, HBase is able to query StreamCapabilities of
the output stream. However, because Hadoop 2.7.4 jars is loaded before Hadoop 3
jars in classpath, it loaded the HdfsDataOutputStream class implemented in
Hadoop 2.7.4, which does not inherit from StreamCapabilities.
{code:java}
if (StreamCapabilities.CLASS.isAssignableFrom(stream.getClass())) {
try {
LOG.warn("stream type = " +
stream.getWrappedStream().getClass().getCanonicalName());
result = ((Boolean)StreamCapabilities.METHOD.invoke(stream,
capability)).booleanValue();
} catch
(IllegalAccessException|IllegalArgumentException|InvocationTargetException
exception) {
LOG.warn("Your Hadoop installation's StreamCapabilities
implementation doesn't match " +
"our understanding of how it's supposed to work. Please file a
JIRA and include " +
"the following stack trace. In the mean time we're interpreting
this behavior " +
"difference as a lack of capability support, which will probably
cause a failure.",
exception);
}
}
{code}
This is not a problem in the code itself. But the code can be improved to log
warning message when output stream class does not inherit from
StreamCapabilities.
> HMaster start fails with IllegalStateException when HADOOP_HOME is set
> ----------------------------------------------------------------------
>
> Key: HBASE-20145
> URL: https://issues.apache.org/jira/browse/HBASE-20145
> Project: HBase
> Issue Type: Bug
> Environment: HBase-2.0-beta1.
> Hadoop trunk version.
> java version "1.8.0_144"
> Reporter: Rohith Sharma K S
> Assignee: Wei-Chiu Chuang
> Priority: Critical
>
> It is observed that HMaster start is failed when HADOOP_HOME is set as env
> while starting HMaster. HADOOP_HOME is pointing to Hadoop trunk version.
> {noformat}
> 2018-03-07 16:59:52,654 ERROR [master//10.200.4.200:16000] master.HMaster:
> Failed to become active master
> java.lang.IllegalStateException: The procedure WAL relies on the ability to
> hsync for proper operation during component failures, but the underlying
> filesystem does not support doing so. Please check the config value of
> 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness
> and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount
> that can provide it.
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1036)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:374)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:532)
> at
> org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1232)
> at
> org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1145)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:837)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2026)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:547)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The same configs is working in HBase-1.2.6 build properly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
