[
https://issues.apache.org/jira/browse/KYLIN-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961941#comment-16961941
]
ASF GitHub Bot commented on KYLIN-4194:
---------------------------------------
nichunen commented on pull request #891: KYLIN-4194 Fix not find KYLIN_CONF or
KYLIN_HOME when build dictionary
URL: https://github.com/apache/kylin/pull/891
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Throw KylinConfigCannotInitException at STEP "Extract Fact Table Distinct
> Columns" with spark
> ---------------------------------------------------------------------------------------------
>
> Key: KYLIN-4194
> URL: https://issues.apache.org/jira/browse/KYLIN-4194
> Project: Kylin
> Issue Type: Bug
> Affects Versions: v3.0.0-beta
> Reporter: wangrupeng
> Assignee: wangrupeng
> Priority: Major
> Attachments: image-2019-10-16-16-13-00-190.png, kylin_spark.log
>
>
> When I use spark to extract fact table distinct columns values, kylin throw
> an exception,
> org.apache.kylin.common.KylinConfigCannotInitException: Didn't find
> KYLIN_CONF or KYLIN_HOME, please set one of them,
> see more details in [^kylin_spark.log] ^^ . I tried cdh5.7 and hdp2.4, both
> platforms got this problem.
> It seems like an environment variables issue, but when I set KYLIN_CONF or
> KYLIN_HOME on each yarn cluster node, it still not work.
> As we can see from the log, this exception occurs when TrieDictionaryBuilder
> try to build Dictionary<String> objects, but cannot find KylinConfig using
> KylinConfig.getInstanceFromEnv() method.
> My solution is to add code in spark job by reset
> KylinConfig.THREAD_ENV_INSTANCE
> from hdfs metadata as the spark job's metadata is stored in hdfs.And finally
> it works fine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
