wangrupeng created KYLIN-4194:
---------------------------------
Summary: Throw KylinConfigCannotInitException at STEP "Extract
Fact Table Distinct Columns" with spark
Key: KYLIN-4194
URL: https://issues.apache.org/jira/browse/KYLIN-4194
Project: Kylin
Issue Type: Bug
Affects Versions: v2.6.4, v3.0.0-beta
Reporter: wangrupeng
Assignee: wangrupeng
Attachments: kylin_spark.log
When I use spark to extract fact table distinct columns values, kylin throw an
exception,
org.apache.kylin.common.KylinConfigCannotInitException: Didn't find KYLIN_CONF
or KYLIN_HOME, please set one of them,
see more details in [^kylin_spark.log] ^^ . I tried cdh5.7 and hdp2.4, both
platforms go this problem.
It seems like an environment variables issue, but when I set KYLIN_CONF or
KYLIN_HOME on each yarn cluster node, it still not work.
As we can see from the log, this exception occurs when TrieDictionaryBuilder
try to build Dictionary<String> objects, but cannot find KylinConfig using
KylinConfig.getInstanceFromEnv() method.
My solution is to add code in spark job by reset
KylinConfig.THREAD_ENV_INSTANCE
from hdfs metadata as the spark job's metadata is stored in hdfs.And finally it
works fine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
