Nixon Rodrigues created ATLAS-5325:
--------------------------------------
Summary: Atlas startup failure in embedded-hbase-solr profile due
to conflicting HBase shaded MapReduce dependency
Key: ATLAS-5325
URL: https://issues.apache.org/jira/browse/ATLAS-5325
Project: Atlas
Issue Type: Bug
Affects Versions: trunk
Reporter: Nixon Rodrigues
Problem Statement
Apache Atlas packaged using the embedded-hbase-solr profile fails to start
during initialization with the following exception:
java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKClientConfig(
Lorg/apache/hadoop/conf/Configuration;
)Lorg/apache/zookeeper/client/ZKClientConfig;
As a result, Atlas is unable to establish connectivity with HBase and startup
fails.
Environment
* Apache Atlas built using embedded-hbase-solr profile
* Embedded HBase
* Embedded Solr
* HBase 2.6.4 based runtime
Root Cause Analysis (RCA)
Investigation revealed a classpath conflict involving HBase shaded dependencies
packaged in the Atlas runtime distribution.
The runtime classpath contains:
hbase-shaded-mapreduce-2.6.4-hadoop3.jar
This artifact includes HBase classes whose method signatures differ from those
expected by the HBase client libraries used by Atlas.
Specifically, during initialization of HBase's ZKConnectionRegistry, Atlas
invokes:
org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKClientConfig(Configuration)
However, the version of ZKConfig resolved at runtime exposes an incompatible
method signature, resulting in:
java.lang.NoSuchMethodError
The exception occurs during creation of ReadOnlyZKClient and prevents Atlas
from completing startup.
Evidence
Bytecode inspection confirmed that the ZKConfig class packaged through the
shaded MapReduce dependency exposes a different API signature than the one
expected by Atlas runtime components.
Removing the conflicting dependency eliminates the NoSuchMethodError and allows
Atlas startup to proceed successfully.
Workaround
Remove the following jar from the Atlas runtime classpath:
hbase-shaded-mapreduce-2.6.4-hadoop3.jar
Example:
mv server/webapp/atlas/WEB-INF/lib/hbase-shaded-mapreduce-2.6.4-hadoop3.jar
/tmp/
After removing this dependency and restarting Atlas:
* Embedded HBase starts successfully
* Embedded Solr starts successfully
* Atlas initializes successfully
* Atlas UI becomes accessible
Possible Fixes
Option 1 (Recommended)
Exclude hbase-shaded-mapreduce-2.6.4-hadoop3.jar from the Atlas distribution
generated by the embedded-hbase-solr profile.
Pros:
* Minimal change
* Resolves startup failure
* Avoids runtime classpath conflicts
Option 2
Review the dependency tree for the embedded-hbase-solr profile and eliminate
duplicate or conflicting HBase shaded artifacts during packaging.
Pros:
* Prevents similar runtime linkage errors
* Produces a cleaner runtime classpath
Option 3
Introduce dependency management rules or exclusions to ensure a single
compatible implementation of HBase Zookeeper classes is present in the final
Atlas distribution.
Pros:
* Long-term maintainable solution
* Reduces risk of future dependency regressions
Resolution
Atlas startup failure was caused by a runtime classpath conflict introduced by
hbase-shaded-mapreduce-2.6.4-hadoop3.jar packaged in the embedded-hbase-solr
distribution. Removing the jar resolves the issue and allows Atlas to start
successfully.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
