1. we were unable to build cube, the issue was later rooted in missing a
spark library, which is weird, because we didnt set up kylin with spark
by that time, we just wanna try it on MR first.
2. When we trying to set up kylin with Spark following the tutorial on
kylin's official web, we cannot find the 'Spark' option in cube engine,
this issue is still under investigation and we dont have much clue on this.
On 2/7/18 3:28 PM, ShaoFeng Shi wrote:
CDH is also supported. But the vendors may have customization on
Spark, so sometimes you may need to use their Spark release or copying
additional jars to the Spark that Kylin ships.
The information you provided is limited; In the first post you said
there is no Spark option in the cube wizard, in the second you fix the
issue by adding spark-network jar, I don't know how they are related,
as the first seems to be the GUI issue.
2018-02-07 9:51 GMT+08:00 谢巍盛 <[email protected]
<mailto:[email protected]>>:
Okay, if that's the case, I assume setting up kylin on hadoop with
apache version instead of CDH/HDP wont have this issue.
Thanks.
On 2/7/18 9:44 AM, Ted Yu wrote:
Since you used a vendor's Kylin package, please consider asking
on their mailing list.
-------- Original message --------
From: 谢巍盛 <[email protected]>
<mailto:[email protected]>
Date: 2/6/18 5:36 PM (GMT-08:00)
To: [email protected] <mailto:[email protected]>
Subject: Re: Unable to build cube with Spark
Hi Shaofeng,
The kylin used on our end is /apache/-/kylin/-/2.2.0/-/bin/-/cdh57/
data source is on Hive.
We managed to fix this issue by adding
spark-network-common_2.10-1.6.0-cdh5.9.2.jar under $SPARK_HOME/lib/
Still, it seems weird to us, we didnt setup Spark for kylin,
somehow kylin will look for this library while building cube. Any
clue?
Best,
Vincent
On 2/7/18 9:08 AM, ShaoFeng Shi wrote:
Hi Weisheng,
Two questions:
What's your Kylin version? and what's the data source of this
cube, Hive or Kafka?
2018-02-06 21:10 GMT+08:00 谢巍盛 <[email protected]
<mailto:[email protected]>>:
Hi,
we follow the step in
http://kylin.apache.org/docs21/tutorial/cube_spark.html
<http://kylin.apache.org/docs21/tutorial/cube_spark.html>
try to build cube with Spark, but couldnt get it worked. The
cube engine on the webUI didnt offer another option other
than MR, it didnt show Spark(beta) as the tutorial shows.
Anyone knows what the problem might be? Or is there any way
to debug this issue? Thanks.
Best,
Vincent
--
Best regards,
Shaofeng Shi 史少锋
--
Best regards,
Shaofeng Shi 史少锋
