Re: Sample Cube Building Error during #4 Step Name: Build Dimension Dictionary

[Billy(Yiming) Liu]

The function initializeSerDe introduced by Hive 1.0+, it seems some old
hive libraries exist in the $hive_dependency.

2016-09-21 13:47 GMT+08:00 Sunwei :

> Hi all:
>
>   I'm using *apache-kylin-1.5.4-HBase1.x-bin* with* hadoop2.6.4 Hive1.2.1
> Hbase1.1.4*.
>
>   When i build the sample cube "kylin_sales_cube", i got a error at #4
> Step Name: Build Dimension Dictionary
>
> log output is:
>
> java.lang.NoSuchMethodError: org.apache.hadoop.hive.serde2.
> SerDeUtils.initializeSerDe(Lorg/apache/hadoop/hive/
> serde2/Deserializer;Lorg/apache/hadoop/conf/Configuration;Ljava/util/
> Properties;Ljava/util/Properties;)V
>
>  at org.apache.hive.hcatalog.mapreduce.InternalUtil.
> initializeDeserializer(InternalUtil.java:156)
>
>  at org.apache.hive.hcatalog.mapreduce.HCatRecordReader.
> createDeserializer(HCatRecordReader.java:127)
>
>  at org.apache.hive.hcatalog.mapreduce.HCatRecordReader.
> initialize(HCatRecordReader.java:92)
>
>  at org.apache.hive.hcatalog.data.transfer.impl.
> HCatInputFormatReader.read(HCatInputFormatReader.java:87)
>
>  at org.apache.kylin.source.hive.HiveTableReader.
> loadHCatRecordItr(HiveTableReader.java:174)
>
>  at org.apache.kylin.source.hive.HiveTableReader.next(
> HiveTableReader.java:99)
>
>  at org.apache.kylin.dict.TableColumnValueEnumerator.moveNext(
> TableColumnValueEnumerator.java:43)
>
>  at org.apache.kylin.dict.DictionaryGenerator$
> NumberDictBuilder.build(DictionaryGenerator.java:174)
>
>  at org.apache.kylin.dict.DictionaryGenerator.buildDictionary(
> DictionaryGenerator.java:81)
>
>  at org.apache.kylin.dict.DictionaryGenerator.buildDictionary(
> DictionaryGenerator.java:73)
>
>  at org.apache.kylin.dict.DictionaryManager.buildDictionary(
> DictionaryManager.java:321)
>
>  at org.apache.kylin.cube.CubeManager.buildDictionary(
> CubeManager.java:185)
>
>  at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.
> processSegment(DictionaryGeneratorCLI.java:50)
>
>  at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.
> processSegment(DictionaryGeneratorCLI.java:41)
>
>  at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(
> CreateDictionaryJob.java:56)
>
>  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>
>  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>
>  at org.apache.kylin.engine.mr.common.HadoopShellExecutable.
> doWork(HadoopShellExecutable.java:63)
>
>  at org.apache.kylin.job.execution.AbstractExecutable.
> execute(AbstractExecutable.java:112)
>
>  at org.apache.kylin.job.execution.DefaultChainedExecutable.
> doWork(DefaultChainedExecutable.java:57)
>
>  at org.apache.kylin.job.execution.AbstractExecutable.
> execute(AbstractExecutable.java:112)
>
>  at org.apache.kylin.job.impl.threadpool.DefaultScheduler$
> JobRunner.run(DefaultScheduler.java:136)
>
>  at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
>
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
>
>  at java.lang.Thread.run(Thread.java:745)
>
> How can i fix it?
>
> thanks ~
>



-- 
With Warm regards

Yiming Liu (刘一鸣)

Sample Cube Building Error during #4 Step Name: Build Dimension Dictionary

[Sunwei]

Hi all:

  I'm using apache-kylin-1.5.4-HBase1.x-bin with hadoop2.6.4 Hive1.2.1
Hbase1.1.4.

  When i build the sample cube "kylin_sales_cube", i got a error at #4 Step
Name: Build Dimension Dictionary

log output is:

java.lang.NoSuchMethodError:
org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(Lorg/apache/hadoop/
hive/serde2/Deserializer;Lorg/apache/hadoop/conf/Configuration;Ljava/util/Pr
operties;Ljava/util/Properties;)V

 at
org.apache.hive.hcatalog.mapreduce.InternalUtil.initializeDeserializer(Inter
nalUtil.java:156)

 at
org.apache.hive.hcatalog.mapreduce.HCatRecordReader.createDeserializer(HCatR
ecordReader.java:127)

 at
org.apache.hive.hcatalog.mapreduce.HCatRecordReader.initialize(HCatRecordRea
der.java:92)

 at
org.apache.hive.hcatalog.data.transfer.impl.HCatInputFormatReader.read(HCatI
nputFormatReader.java:87)

 at
org.apache.kylin.source.hive.HiveTableReader.loadHCatRecordItr(HiveTableRead
er.java:174)

 at
org.apache.kylin.source.hive.HiveTableReader.next(HiveTableReader.java:99)

 at
org.apache.kylin.dict.TableColumnValueEnumerator.moveNext(TableColumnValueEn
umerator.java:43)

 at
org.apache.kylin.dict.DictionaryGenerator$NumberDictBuilder.build(Dictionary
Generator.java:174)

 at
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerato
r.java:81)

 at
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerato
r.java:73)

 at
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.ja
va:321)

 at
org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:185)

 at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGe
neratorCLI.java:50)

 at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGe
neratorCLI.java:41)

 at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob
.java:56)

 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

 at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellEx
ecutable.java:63)

 at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable
.java:112)

 at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChaine
dExecutable.java:57)

 at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable
.java:112)

 at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultS
cheduler.java:136)

 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
42)

 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
17)

 at java.lang.Thread.run(Thread.java:745)

How can i fix it?

thanks ~