I can say for sure that KYLIN_CONF property is set to "meta" during a
build job (using MR as engine), because when I first start kylin server
and before any build jobs, I can run a query in Insight tab, while after
a (failed) build job, there will be an error locating "kylin.properties"
for queries and about everything else.
On 14/02/2019 00:38, ShaoFeng Shi wrote:
Hi Nikodimos,
Interesting finding; The method of
"AbstractHadoopJob#loadKylinPropsAndMetadata" only be invoked from a
Hadoop container (mapper/reducer), so the "KYILN_CONF" only be set out
of Kylin's main process; Can you double check your local environment,
see whether "KYLIN_CONF" has already been set, like some shell script,
or bash_profile, etc?
Best regards,
Shaofeng Shi 史少锋
Apache Kylin PMC
Work email: [email protected]
<mailto:[email protected]>
Kyligence Inc: https://kyligence.io/
Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: [email protected]
<mailto:[email protected]>
Join Kylin dev mail group: [email protected]
<mailto:[email protected]>
Nikodimos Nikolaidis <[email protected]
<mailto:[email protected]>> 于2019年2月13日周三 上午7:36写道：
After compiling master branch with your latest commit, which
prints `confFile.getAbsolutePath()`, the ouput was "Failed to read
kylin_hive_conf.xml at
'/opt/apache-kylin-2.6.0-bin/meta/kylin_hive_conf.xml'".
It seems that KYLIN_CONF property is set to $KYLIN_HOME/meta
(possibly at
https://github.com/apache/kylin/blob/master/engine-mr/src/main/java/org/apache/kylin/engine/mr/common/AbstractHadoopJob.java#L478
).
By creating a link `meta` to `conf` in $KYLIN_HOME, build job
worked without errors. But, why is KYLIN_CONF property set like
that? Is there an error in my configuration or is it a bug?
thanks by the way
On 2/11/19 8:55 PM, ShaoFeng Shi wrote:
Usually, this is an environment issue I think; The exception
didn't clearly print the real path, could you modify this line
code to let it print the "confFile"?
https://github.com/apache/kylin/blob/master/core-common/src/main/java/org/apache/kylin/common/util/SourceConfigurationUtil.java#L87
Besides, please also check the kylin.log before and after the
error, there should be some debugging logs.
Best regards,
Shaofeng Shi 史少锋
Apache Kylin PMC
Work email: [email protected]
<mailto:[email protected]>
Kyligence Inc: https://kyligence.io/
Apache Kylin FAQ:
https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: [email protected]
<mailto:[email protected]>
Join Kylin dev mail group: [email protected]
<mailto:[email protected]>
Nikodimos Nikolaidis <[email protected]
<mailto:[email protected]>> 于2019年2月11日周一 下午10:44写道：
Hello,
I am using latest (2.6) version of Kylin and I am following
instructions from
https://kylin.apache.org/docs/tutorial/cube_streaming.html to
build a demo streaming cube.
At a BUILD CUBE job, I can see in monitor tab in "Hive
Cleanup" step an error occurs by the following java exception:
java.lang.RuntimeException: Failed to read kylin_hive_conf.xml
at
org.apache.kylin.common.util.SourceConfigurationUtil.loadXmlConfiguration(SourceConfigurationUtil.java:87)
at
org.apache.kylin.common.util.SourceConfigurationUtil.loadHiveConfiguration(SourceConfigurationUtil.java:61)
at
org.apache.kylin.common.util.HiveCmdBuilder.<init>(HiveCmdBuilder.java:48)
at
org.apache.kylin.source.hive.GarbageCollectionStep.cleanUpIntermediateFlatTable(GarbageCollectionStep.java:61)
at
org.apache.kylin.source.hive.GarbageCollectionStep.doWork(GarbageCollectionStep.java:48)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:70)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Then if I try to run a SQL query in Insight tab the following
exception is thrown:
java.lang.RuntimeException: fail to locate kylin.properties
while executing SQL: "select minute_start, count(*), sum(amount),
sum(qty) from streaming_sales_table group by minute_start order by
minute_start LIMIT 50000"
and generally, it seems that it can't find any Kylin
configuration files afterwards.
I have set properly JAVA_HOME, HADOOP_HOME, HIVE_HOME,
HBASE_HOME, KYLIN_HOME environment variables.
What could be the reason for that?
