actually，I found the last step " convert to hfile" take too much time, more than 40 minutes for single region(use small, and result file about 5GB）
中国移动广东有限公司 网管中心 梁猛 13802880...@139.com From: ShaoFeng Shi Date: 2016-01-15 09:40 To: dev Subject: Re: beg suggestions to speed up the Kylin cube build The cube build performance is much determined by your Hadoop cluster's capacity. You can do some inspection with the MR job's statistics to analysis the potential bottlenecks. 2016-01-15 7:19 GMT+08:00 zhong zhang <zzaco...@gmail.com>: > Hi All, > > We are trying to build a nine-dimension cube: > eight mandatory dimensions and one hierarchy > dimension. The fact table is like 20G. Two lookup > tables are 1.3M and 357k separately. It takes like > 3 hours to go to 30% progress which is kind of slow. > > We'd like to know are there suggestions to speed up > the Kylin cube build. We got a suggestion from > a slide said that sort the dimension based on the > cardinality. Are there any other ways we can try? > > We also noticed that only half of the memory and > half of the CPU are used during the cube build. > Are there any ways to fully utilize the resource? > > Looking forward to hear from you. > > Best regards, > Zhong > -- Best regards, Shaofeng Shi
