Sample Cube Building Error during #4 Step Name: Build Dimension Dictionary

2016-09-20 Thread Sunwei 
Hi all:

  I'm using apache-kylin-1.5.4-HBase1.x-bin with hadoop2.6.4 Hive1.2.1
Hbase1.1.4.

  When i build the sample cube "kylin_sales_cube", i got a error at #4 Step
Name: Build Dimension Dictionary

log output is:

java.lang.NoSuchMethodError:
org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(Lorg/apache/hadoop/
hive/serde2/Deserializer;Lorg/apache/hadoop/conf/Configuration;Ljava/util/Pr
operties;Ljava/util/Properties;)V

 at
org.apache.hive.hcatalog.mapreduce.InternalUtil.initializeDeserializer(Inter
nalUtil.java:156)

 at
org.apache.hive.hcatalog.mapreduce.HCatRecordReader.createDeserializer(HCatR
ecordReader.java:127)

 at
org.apache.hive.hcatalog.mapreduce.HCatRecordReader.initialize(HCatRecordRea
der.java:92)

 at
org.apache.hive.hcatalog.data.transfer.impl.HCatInputFormatReader.read(HCatI
nputFormatReader.java:87)

 at
org.apache.kylin.source.hive.HiveTableReader.loadHCatRecordItr(HiveTableRead
er.java:174)

 at
org.apache.kylin.source.hive.HiveTableReader.next(HiveTableReader.java:99)

 at
org.apache.kylin.dict.TableColumnValueEnumerator.moveNext(TableColumnValueEn
umerator.java:43)

 at
org.apache.kylin.dict.DictionaryGenerator$NumberDictBuilder.build(Dictionary
Generator.java:174)

 at
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerato
r.java:81)

 at
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerato
r.java:73)

 at
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.ja
va:321)

 at
org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:185)

 at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGe
neratorCLI.java:50)

 at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGe
neratorCLI.java:41)

 at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob
.java:56)

 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

 at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellEx
ecutable.java:63)

 at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable
.java:112)

 at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChaine
dExecutable.java:57)

 at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable
.java:112)

 at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultS
cheduler.java:136)

 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
42)

 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
17)

 at java.lang.Thread.run(Thread.java:745)

How can i fix it?

thanks ~

Re: Error while building cube from stream

2016-09-20 Thread Alberto Ramón 
I don't know but , can you check this change?: KYLIN-1744
 in V1.3


2016-09-20 14:50 GMT+02:00 Tony Lee :

> Hi,
>
> I was building cube from stream as the document(http://kylin.apache.
> org/docs15/tutorial/cube_streaming.html
>
> ) says.
>
> I was using 1.5.3, and i encounter this error. Same error on 1.5.4.
> Everything fine on 1.5.2.1.
>
> Any idea how to solve this?
>
>
> 2016-09-20 20:31:51,520 INFO  [main KafkaStreamingInput:129]: finish to
> get streaming batch, total message count:30
> 2016-09-20 20:31:51,532 DEBUG [main CubeManager:855]: Reloaded new cube:
> STREAMING_CUBE with reference beingCUBE[name=STREAMING_CUBE] having 1
> segments:KYLIN_2822I1W3CX
> 2016-09-20 20:31:51,536 INFO  [main CubeManager:314]: Updating cube
> instance 'STREAMING_CUBE'
> 2016-09-20 20:31:51,538 WARN  [main StreamingCLI:127]: invalid
> args:streaming start STREAMING_CUBE 147437454_147437460 -start
> 147437454 -end 147437460 -cube STREAMING_CUBE
> 2016-09-20 20:31:51,539 ERROR [main StreamingCLI:103]: error start
> streaming
> java.lang.IllegalStateException: Segments overlap:
> STREAMING_CUBE[FULL_BUILD] and STREAMING_CUBE[FULL_BUILD]
> at org.apache.kylin.cube.CubeValidator.validate(CubeValidator.java:85)
> at org.apache.kylin.cube.CubeManager.updateCubeWithRetry(
> CubeManager.java:358)
> at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:301)
> at org.apache.kylin.cube.CubeManager.appendSegment(CubeManager.java:441)
> at org.apache.kylin.engine.streaming.cube.StreamingCubeBuilder.
> createBuildable(StreamingCubeBuilder.java:118)
> at org.apache.kylin.engine.streaming.OneOffStreamingBuilder$1.run(
> OneOffStreamingBuilder.java:76)
> at org.apache.kylin.engine.streaming.cli.StreamingCLI.
> startOneOffCubeStreaming(StreamingCLI.java:123)
> at org.apache.kylin.engine.streaming.cli.StreamingCLI.
> main(StreamingCLI.java:97)
> 2016-09-20 20:31:51,543 INFO  [Thread-0 ConnectionManager$
> HConnectionImplementation:1678]: Closing zookeeper
> sessionid=0x35708fbc2740013
> 2016-09-20 20:31:51,549 INFO  [Thread-0 ZooKeeper:684]: Session:
> 0x35708fbc2740013 closed
> 2016-09-20 20:31:51,549 INFO  [main-EventThread ClientCnxn:512]:
> EventThread shut down
>
>

Error while building cube from stream

2016-09-20 Thread Tony Lee 
Hi,

I was building cube from stream as the document(
http://kylin.apache.org/docs15/tutorial/cube_streaming.html

) says.

I was using 1.5.3, and i encounter this error. Same error on 1.5.4.
Everything fine on 1.5.2.1.

Any idea how to solve this?


2016-09-20 20:31:51,520 INFO  [main KafkaStreamingInput:129]: finish to get
streaming batch, total message count:30
2016-09-20 20:31:51,532 DEBUG [main CubeManager:855]: Reloaded new cube:
STREAMING_CUBE with reference beingCUBE[name=STREAMING_CUBE] having 1
segments:KYLIN_2822I1W3CX
2016-09-20 20:31:51,536 INFO  [main CubeManager:314]: Updating cube
instance 'STREAMING_CUBE'
2016-09-20 20:31:51,538 WARN  [main StreamingCLI:127]: invalid
args:streaming start STREAMING_CUBE 147437454_147437460 -start
147437454 -end 147437460 -cube STREAMING_CUBE
2016-09-20 20:31:51,539 ERROR [main StreamingCLI:103]: error start streaming
java.lang.IllegalStateException: Segments overlap:
STREAMING_CUBE[FULL_BUILD] and STREAMING_CUBE[FULL_BUILD]
at org.apache.kylin.cube.CubeValidator.validate(CubeValidator.java:85)
at
org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:358)
at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:301)
at org.apache.kylin.cube.CubeManager.appendSegment(CubeManager.java:441)
at
org.apache.kylin.engine.streaming.cube.StreamingCubeBuilder.createBuildable(StreamingCubeBuilder.java:118)
at
org.apache.kylin.engine.streaming.OneOffStreamingBuilder$1.run(OneOffStreamingBuilder.java:76)
at
org.apache.kylin.engine.streaming.cli.StreamingCLI.startOneOffCubeStreaming(StreamingCLI.java:123)
at
org.apache.kylin.engine.streaming.cli.StreamingCLI.main(StreamingCLI.java:97)
2016-09-20 20:31:51,543 INFO  [Thread-0
ConnectionManager$HConnectionImplementation:1678]: Closing zookeeper
sessionid=0x35708fbc2740013
2016-09-20 20:31:51,549 INFO  [Thread-0 ZooKeeper:684]: Session:
0x35708fbc2740013 closed
2016-09-20 20:31:51,549 INFO  [main-EventThread ClientCnxn:512]:
EventThread shut down