In 0.8 branch, Kylin start to read Hive table via the HCat APIs. The one HDFS file limitation is gone. However the small assumption still exists.
On Wed, May 27, 2015 at 4:19 PM, hongbin ma <[email protected]> wrote: > for now , kylin assumes dimension tables are relatively small, and there > exists only one hdfs file for that table. > > On Wed, May 27, 2015 at 4:17 PM, Luke Han <[email protected]> wrote: > > > Forward to mailing list for further support. > > > > Thanks. > > > > 在 2015年5月27日星期三 UTC+8下午4:11:09，donald fossouo写道： > >> > >> Hi i just start with kylin after build test cube and my own on HDP > >> environment : > >> > >> HDP 2.2 > >> > >> Hive 0.14 , hadoop 2.6 > >> > >> I got the following error in the 3rd step : > >> > >> > >> > >> java.lang.IllegalStateException: Expect 1 and only 1 non-zero file under > >> hdfs://my-name-node:8020/apps/hive/warehouse/dim_organisations, but > find 0 > >> at > >> org.apache.kylin.dict.lookup.HiveTable.findOnlyFile(HiveTable.java:123) > >> at > >> > org.apache.kylin.dict.lookup.HiveTable.computeHDFSLocation(HiveTable.java:107) > >> > >> at > >> > org.apache.kylin.dict.lookup.HiveTable.getHDFSLocation(HiveTable.java:83) > >> at > >> org.apache.kylin.dict.lookup.HiveTable.getFileTable(HiveTable.java:76) > >> at > >> org.apache.kylin.dict.lookup.HiveTable.getSignature(HiveTable.java:71) > >> at > >> > org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:164) > >> > >> at > >> org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:154) > >> at > >> > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:53) > >> > >> at > >> > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) > >> > >> at > >> > org.apache.kylin.job.hadoop.dict.CreateDictionaryJob.run(CreateDictionaryJob.java:53) > >> > >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > >> at > >> > org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) > >> > >> at > >> > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) > >> > >> at > >> > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50) > >> > >> at > >> > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) > >> > >> at > >> > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132) > >> > >> at > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > >> > >> at > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >> > >> at java.lang.Thread.run(Thread.java:745) > >> > >> result code:2 > >> > >> What is the cause of the problem? > >> > >> > > > -- > Regards, > > *Bin Mahone | 马洪宾* > Apache Kylin: http://kylin.io > Github: https://github.com/binmahone >
