Vsevolod Ostapenko created KYLIN-3139:
-----------------------------------------
Summary: Failure in map-reduce job due to undefined hdp.version
variable when using HDP stack and remote HBase cluster
Key: KYLIN-3139
URL: https://issues.apache.org/jira/browse/KYLIN-3139
Project: Kylin
Issue Type: Bug
Components: General
Affects Versions: v2.2.0
Environment: HDP 2.5.6, two cluster setup, Kylin 2.2.0 in a cluster
with Hive only, remote HBase cluster for data storage
Reporter: Vsevolod Ostapenko
Assignee: Vsevolod Ostapenko
Priority: Minor
When running on top of HDP stack and using a setup where Hive and HBase run in
different clusters cube build/refresh fails on the step "Extract Fact Table
Distinct Columns" with the error
{quote}java.lang.IllegalArgumentException: Unable to parse
'/hdp/apps/$\{hdp.version\}/mapreduce/mapreduce.tar.gz#mr-framework' as a URI,
check the setting for mapreduce.application.framework.path{quote}
Based on existing JIRA discussions in Ambari project, it's responsibility of a
service to set hdp.version Java property. When HBase is not installed as a
service in a cluster hbase launcher does not set this property (presumably
because HBase in that case is just a client and not a service).
The only suitable workaround found so far is to set property as part of the
conf/setenv.sh script.
In order to avoid hard coding of the HDP version info, suggested change to
setenv.sh will attempt to detect HDP version at run-time. It should work for
all released HDP version from 2.2.x to 2.6.x
In addition to that, it will also try to locate and set Java native library
path, when running on top of HDP.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
