Hi Bin,
Thanks for the reply.I have already mentioned that I have verified and the file
exists in HDFS.But the owner is Hive not Cloudera.
drwxrwxrwx - hive hive 0 2015-05-07 04:22
hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_dim_product
drwxrwxrwx - hive hive 0 2015-05-07 04:20
hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_fact
drwxrwxrwx - cloudera supergroup 0 2015-05-04 07:18
hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_cal_dt
drwxrwxrwx - cloudera supergroup 0 2015-05-04 07:18
hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_category_groupings
drwxrwxrwx - cloudera supergroup 0 2015-05-04 07:18
hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_sales
For the example bundled with Kylin the user is Cloudera. Is it some kind of
permission issue I am running into.Meanwhile i'll try to run it with hive table
(not external)
Regards
Abhijit DekaComputer ScientistAdobe SystemsBangalore
Ph-+91 80884 39067
On Friday, 8 May 2015 6:59 AM, hongbin ma <[email protected]> wrote:
for all the lookup tables, for example glaastest_dim_vendor in your case,
kylin asks hive the hdfs location of it, and will try to read the hdfs file
directly.
it seems hive returned the location:
hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_dim_vendor does ,
but the file does not exist. can you verify why hive returned such location(as
you described it's a external table in hbase?)
An alternative is that you convert all the lookup tables into non-external
tables
On Thu, May 7, 2015 at 4:40 PM, Abhijit Deka <[email protected]>
wrote:
Forgot to add the machine info.I am running it in CHD 5.1.0 virtualbox.Not a
cluster.
Regards
Abhijit DekaComputer ScientistAdobe SystemsBangalore
Ph-+91 80884 39067
On Thursday, 7 May 2015 1:36 AM, Abhijit Deka
<[email protected]> wrote:
Hi,
I am new to Kylin and i was trying out a simple cube.I did the below
steps.Please let me know if i have done anything wrong.
1.Done ETL in Pig.2.Stored the Fact and Dim tables in Hbase.3.Created external
table in Hive to query the data and also use in Kylin4.Created the cube as
instructed in documentation.5.Build the cube.
But the cube build failed in 3rd step.
Below are the logs
java.io.FileNotFoundException: File
hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_dim_vendor does
not exist.
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:654)
at
org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:102)
at
org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:712)
at
org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:708)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:708)
at org.apache.kylin.dict.lookup.HiveTable.findOnlyFile(HiveTable.java:116)
at
org.apache.kylin.dict.lookup.HiveTable.computeHDFSLocation(HiveTable.java:107)
at org.apache.kylin.dict.lookup.HiveTable.getHDFSLocation(HiveTable.java:83)
at org.apache.kylin.dict.lookup.HiveTable.getFileTable(HiveTable.java:76)
at org.apache.kylin.dict.lookup.HiveTable.getSignature(HiveTable.java:71)
at
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:164)
at org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:154)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:53)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
at
org.apache.kylin.job.hadoop.dict.CreateDictionaryJob.run(CreateDictionaryJob.java:53)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Now when I check for the file in HDFS the file is there but it looks like a
permission issue.(??)
drwxrwxrwx - hive hive 0 2015-05-07 04:22
hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_dim_product
drwxrwxrwx - hive hive 0 2015-05-07 04:20
hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_fact
drwxrwxrwx - cloudera supergroup 0 2015-05-04 07:18
hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_cal_dt
drwxrwxrwx - cloudera supergroup 0 2015-05-04 07:18
hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_category_groupings
drwxrwxrwx - cloudera supergroup 0 2015-05-04 07:18
hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_sales
How do i solve this issue.Thanks in advance.
Regards
Abhijit DekaComputer ScientistAdobe SystemsBangalore
--
Regards,
Bin Mahone | 马洪宾
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone
