This step is simple, and has no dependency on hbase, I couldn¹t think of
what special reason can cause it failureŠ
Looking at the information you provided, seems the generated cuboid files
are very small:
-bash-4.1# hdfs dfs -ls hdfs://
og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_by_
products/cuboid/*
Found 2 items
-rw-r--r-- 2 root hdfs 0 2015-06-15 04:12 hdfs://
og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_by_
products/cuboid/0d_cuboid/_SUCCESS
-rw-r--r-- 2 root hdfs 78 2015-06-15 04:12 hdfs://
og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_by_
products/cuboid/0d_cuboid/part-r-00000
...
If exclude the _SUCCESS file, there is only 1 file which is the
0-dimension; How many dimensions in your cube? If it has N dimensions,
there should be N {n}d_cuboid folders usually; You may need review and
ensure the cube is correct at first.
On 6/15/15, 5:49 PM, "Olivier Guennec" <[email protected]> wrote:
>Hi All,
>
>*1/ Environment*
>
>I have setup *Kylin 0.7.1 RC2* on *Docker 1.5.0* running* HDP 2.2.4.2*
>including *HBASE 0.98.4*.
>
>I can start Kylin, import tables and start a cube build process.
>
>But cube building process is failing at step 9 systematically.
>
>*HDFS, HBASE Region and Kylin are all crashing at the same time*.
>
>
>
>*2/ Details*
>
>
>*Here is the config of step 9*
>-conf /opt/kylin/kylin-0.7.1-incubating/conf/kylin_job_conf.xml -input
>/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_by_products/cuboid/*
>-output /tmp/sales_by_products/rowkey_stats -cubename sales_by_products
>-jobname Kylin_Region_Splits_Calculator_sales_by_products_Step
>
>
>*I can see successful cuboid files being created :*
>-bash-4.1# hdfs dfs -ls hdfs://
>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_by
>_products/cuboid/*
>Found 2 items
>-rw-r--r-- 2 root hdfs 0 2015-06-15 04:12 hdfs://
>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_by
>_products/cuboid/0d_cuboid/_SUCCESS
>-rw-r--r-- 2 root hdfs 78 2015-06-15 04:12 hdfs://
>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_by
>_products/cuboid/0d_cuboid/part-r-00000
>...
>
>*But output of MR job is empty*
>-bash-4.1# hdfs dfs -ls hdfs://og.mycorp.com:8020/tmp/sales_by_products/*
>Found 1 items
>drwxr-xr-x - root hdfs 0 2015-06-15 04:13 hdfs://
>og.mycorp.com:8020/tmp/sales_by_products/rowkey_stats/_temporary
>-bash-4.1.
>
>Could you think of anything special in step 9 that could cause this issue
>?
>
>Thanks for your help,
>
>Kind Regards,
>
>Olivier Guennec
>
>
>
>2015-06-15 10:43 GMT+01:00 Olivier Guennec <[email protected]>:
>
>> Hi All,
>>
>> *1/ Environment*
>>
>> I have setup *Kylin 0.7.1 RC2* on *Docker 1.5.0* running* HDP 2.2.4.2*
>> including *HBASE 0.98.4*.
>>
>> I can start Kylin, import tables and start a cube build process.
>>
>> But cube building process is failing at step 9 systematically.
>>
>> *HDFS, HBASE Region and Kylin are all crashing at the same time*.
>>
>> Please find Kylin logs attached.
>>
>>
>> *2/ Details*
>>
>>
>> *Here is the config of step 9*
>> -conf /opt/kylin/kylin-0.7.1-incubating/conf/kylin_job_conf.xml -input
>>
>>/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_by_products/cuboid/
>>*
>> -output /tmp/sales_by_products/rowkey_stats -cubename sales_by_products
>> -jobname Kylin_Region_Splits_Calculator_sales_by_products_Step
>>
>>
>> *I can see successful cuboid files being created :*
>> -bash-4.1# hdfs dfs -ls hdfs://
>>
>>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_b
>>y_products/cuboid/*
>> Found 2 items
>> -rw-r--r-- 2 root hdfs 0 2015-06-15 04:12 hdfs://
>>
>>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_b
>>y_products/cuboid/0d_cuboid/_SUCCESS
>> -rw-r--r-- 2 root hdfs 78 2015-06-15 04:12 hdfs://
>>
>>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_b
>>y_products/cuboid/0d_cuboid/part-r-00000
>> Found 2 items
>> -rw-r--r-- 2 root hdfs 0 2015-06-15 04:11 hdfs://
>>
>>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_b
>>y_products/cuboid/1d_cuboid/_SUCCESS
>> -rw-r--r-- 2 root hdfs 558 2015-06-15 04:11 hdfs://
>>
>>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_b
>>y_products/cuboid/1d_cuboid/part-r-00000
>> Found 2 items
>> -rw-r--r-- 2 root hdfs 0 2015-06-15 04:09 hdfs://
>>
>>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_b
>>y_products/cuboid/2d_cuboid/_SUCCESS
>> -rw-r--r-- 2 root hdfs 1758 2015-06-15 04:09 hdfs://
>>
>>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_b
>>y_products/cuboid/2d_cuboid/part-r-00000
>> Found 2 items
>> -rw-r--r-- 2 root hdfs 0 2015-06-15 04:08 hdfs://
>>
>>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_b
>>y_products/cuboid/3d_cuboid/_SUCCESS
>> -rw-r--r-- 2 root hdfs 1640 2015-06-15 04:08 hdfs://
>>
>>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_b
>>y_products/cuboid/3d_cuboid/part-r-00000
>> Found 2 items
>> -rw-r--r-- 2 root hdfs 0 2015-06-15 04:07 hdfs://
>>
>>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_b
>>y_products/cuboid/base_cuboid/_SUCCESS
>> -rw-r--r-- 2 root hdfs 515 2015-06-15 04:07 hdfs://
>>
>>og.mycorp.com:8020/tmp/kylin-7d8facf6-6588-4aeb-af9f-32e996985afd/sales_b
>>y_products/cuboid/base_cuboid/part-r-00000
>> -bash-4.1#
>>
>> *But output of MR job is empty*
>> -bash-4.1# hdfs dfs -ls
>>hdfs://og.mycorp.com:8020/tmp/sales_by_products/*
>> Found 1 items
>> drwxr-xr-x - root hdfs 0 2015-06-15 04:13 hdfs://
>> og.mycorp.com:8020/tmp/sales_by_products/rowkey_stats/_temporary
>> -bash-4.1.
>>
>> Could you think of anything special in step 9 that could cause this
>>issue ?
>>
>> Thanks for your help,
>>
>> Kind Regards,
>>
>> Olivier Guennec
>>
>>
