Currently there is no Spark version of the step "Convert Cuboid Data to HFile". The best shot is to tune the Hadoop MR job of converting to HFile.
I would suggest to start by checking parallel-ness of the job. See if there is enough mappers and reducers started. If not, consider cut the cuboid into smaller regions, which will increase the number of mappers and reducers. On Tue, May 8, 2018 at 2:56 AM, narendracs <[email protected]> wrote: > I am using spark engine for cube processing, step to Convert Cuboid Data to > HFile is taking most of time. > 14 M input record, 3 dimensions ( 1 UHC) , it took around 28 mins to build > the cube out of which 20 mins just for Convert Cuboid Data to HFile. I > noticed this step is running map reduce even though I have selected spark > as > engine type. > is there any way to make this step run on spark instead of MR ? > also there any configuration which can help to optimize this step? > > thanks > > -- > Sent from: http://apache-kylin.74782.x6.nabble.com/ >
