OK, thanks for the update! 2018-02-07 15:50 GMT+08:00 谢巍盛 <[email protected]>:
> 1. we were unable to build cube, the issue was later rooted in missing a > spark library, which is weird, because we didnt set up kylin with spark by > that time, we just wanna try it on MR first. > 2. When we trying to set up kylin with Spark following the tutorial on > kylin's official web, we cannot find the 'Spark' option in cube engine, > this issue is still under investigation and we dont have much clue on this. > > > On 2/7/18 3:28 PM, ShaoFeng Shi wrote: > > CDH is also supported. But the vendors may have customization on Spark, so > sometimes you may need to use their Spark release or copying additional > jars to the Spark that Kylin ships. > > The information you provided is limited; In the first post you said there > is no Spark option in the cube wizard, in the second you fix the issue by > adding spark-network jar, I don't know how they are related, as the first > seems to be the GUI issue. > > 2018-02-07 9:51 GMT+08:00 谢巍盛 <[email protected]>: > >> Okay, if that's the case, I assume setting up kylin on hadoop with apache >> version instead of CDH/HDP wont have this issue. >> >> Thanks. >> >> On 2/7/18 9:44 AM, Ted Yu wrote: >> >> Since you used a vendor's Kylin package, please consider asking on their >> mailing list. >> >> -------- Original message -------- >> From: 谢巍盛 <[email protected]> <[email protected]> >> Date: 2/6/18 5:36 PM (GMT-08:00) >> To: [email protected] >> Subject: Re: Unable to build cube with Spark >> >> Hi Shaofeng, >> >> The kylin used on our end is *apache*-*kylin*-*2.2.0*-*bin*-*cdh57* >> data source is on Hive. >> >> We managed to fix this issue by adding >> spark-network-common_2.10-1.6.0-cdh5.9.2.jar >> under $SPARK_HOME/lib/ >> >> Still, it seems weird to us, we didnt setup Spark for kylin, somehow >> kylin will look for this library while building cube. Any clue? >> >> Best, >> Vincent >> >> On 2/7/18 9:08 AM, ShaoFeng Shi wrote: >> >> Hi Weisheng, >> >> Two questions: >> What's your Kylin version? and what's the data source of this cube, Hive >> or Kafka? >> >> 2018-02-06 21:10 GMT+08:00 谢巍盛 <[email protected]>: >> >>> Hi, >>> >>> we follow the step in http://kylin.apache.org/docs21 >>> /tutorial/cube_spark.html try to build cube with Spark, but couldnt get >>> it worked. The cube engine on the webUI didnt offer another option other >>> than MR, it didnt show Spark(beta) as the tutorial shows. Anyone knows what >>> the problem might be? Or is there any way to debug this issue? Thanks. >>> >>> Best, >>> >>> Vincent >>> >>> >> >> >> -- >> Best regards, >> >> Shaofeng Shi 史少锋 >> >> >> >> > > > -- > Best regards, > > Shaofeng Shi 史少锋 > > > -- Best regards, Shaofeng Shi 史少锋
