[
https://issues.apache.org/jira/browse/KYLIN-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI closed KYLIN-1094.
-------------------------------
Resolution: Won't Fix
Kylin 2.0 has the newly implemented by-layer spark cubing, so this task is not
needed.
> improve performance of spark cubing
> -----------------------------------
>
> Key: KYLIN-1094
> URL: https://issues.apache.org/jira/browse/KYLIN-1094
> Project: Kylin
> Issue Type: Sub-task
> Components: Spark Engine
> Affects Versions: v1.4.0
> Reporter: ZhouQianhao
> Assignee: Dong Li
> Fix For: Backlog
>
>
> POC result of spark cubing shows that, on a dataset of 150 million records,
> MR is about 100% faster than Spark, however we believe that Spark could be at
> least at same speed as MR, so optimization is needed here.
> We are asking Spark community for help now.
> the cluster info:
> vm: 8 nodes * (128G mem + 64 core)
> hadoop cluster: hdp 2.2.6
> spark running mode: yarn-client
> spark version: 1.5.1
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
