Then it boils down to MR job tuning. You shall identify the slow MR job and take action from there. There are many ways to diagnose and tune MR job. Try google 'tune mapreduce job'.
On Fri, Apr 28, 2017 at 6:07 PM, suresh m <[email protected]> wrote: > Hi Thanks for your timely reply. > > I set kylin.cube.algorithm=layer , but still facing same issue. > > Please let me know any other tuning to improve performance. > > Regards, > > Suresh > > On Tue, Apr 18, 2017 at 7:23 PM, Li Yang <[email protected]> wrote: > > > > #24 Step Name: Build Cube > > > > It is in-mem cubing taking effect and seems it does not fit your cluster. > > Try set below in kylin.properties to use layer cubing instead. > > > > kylin.cube.algorithm=layer > > > > > > Cheers > > Yang > > > > On Tue, Apr 18, 2017 at 9:13 PM, Alberto Ramón < > [email protected]> > > wrote: > > > > > Is this your configuration? (check this properties): > > > > > > - > > > > > > yarn.nodemanager.resource.memory-mb =150/3 > > > - > > > > > > yarn.nodemanager.resource.cpu-vcores= 36/3 > > > > > > And Check the YARN UI, during build phase (you can see how many Mappers > > and > > > reduce, RAM and CPU are you using) > > > > > > 2017-04-18 13:45 GMT+01:00 suresh m <[email protected]>: > > > > > > > sorry it is for all 3 nodes. > > > > > > > > each node 55gb ram and 12 cpu are there. > > > > > > > > On Tue, Apr 18, 2017 at 6:09 PM, Alberto Ramón < > > > [email protected]> > > > > wrote: > > > > > > > > > Are these 150 GB and 36 CPUs configured in? > > > > > > > > > > - > > > > > > > > > > yarn.nodemanager.resource.memory-mb > > > > > - > > > > > > > > > > yarn.nodemanager.resource.cpu-vcores > > > > > > > > > > During Build process, you can open YARN UI how many map-reduce are > > you > > > > > using. > > > > > > > > > > 2017-04-18 13:21 GMT+01:00 suresh m <[email protected]>: > > > > > > > > > > > Hi, > > > > > > > > > > > > I can see 150gb ram and 36 cpu in our cluster. > > > > > > > > > > > > On Tue, Apr 18, 2017 at 12:48 PM, Alberto Ramón < > > > > > [email protected] > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Can you check how many ram and cpu is asigned to yarn? > > > > > > > > > > > > > > El 18 abr. 2017 7:40 a. m., "suresh m" <[email protected] > > > > > > > > escribió: > > > > > > > > > > > > > > > Hi This is the time consuming step in entire build process. > > > > > > > > > > > > > > > > #24 Step Name: Build Cube > > > > > > > > > > > > > > > > Please help me to get it resolved. > > > > > > > > > > > > > > > > Regards, > > > > > > > > Suresh > > > > > > > > > > > > > > > > On Mon, Apr 17, 2017 at 5:09 PM, 康凯森 <[email protected]> > > wrote: > > > > > > > > > > > > > > > > > what's the slowest step in your cube job? > > > > > > > > > please refer to http://kylin.apache.org/ > > > > > > docs20/howto/howto_optimize_ > > > > > > > > > build.html > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------ 原始邮件 ------------------ > > > > > > > > > 发件人: "suresh m";<[email protected]>; > > > > > > > > > 发送时间: 2017年4月17日(星期一) 晚上7:05 > > > > > > > > > 收件人: "dev"<[email protected]>; > > > > > > > > > > > > > > > > > > 主题: cube building taking 8 hours just for 6 thousands > records > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > > > > > One of my cube taking hours to build even though its having > > > less > > > > > > > > > records(6000), please provide me some tuning techniques to > > > > improve > > > > > > > > > performance in Kylin. > > > > > > > > > > > > > > > > > > Regards. > > > > > > > > > Suresh > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
