[
https://issues.apache.org/jira/browse/KYLIN-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962662#comment-16962662
]
hailin.huang commented on KYLIN-4213:
-------------------------------------
could we re-write it use scala not Java. I think the current spark engine using
java is complex and hard to read.
> The new build engine with Spark-SQL
> -----------------------------------
>
> Key: KYLIN-4213
> URL: https://issues.apache.org/jira/browse/KYLIN-4213
> Project: Kylin
> Issue Type: New Feature
> Components: Job Engine
> Affects Versions: Future
> Reporter: yiming.xu
> Assignee: yiming.xu
> Priority: Major
>
> 1. Use Spark-SQL to compute cuboid, build cuboid A, B, C , Sum(D) is sql
> "select A B C Sum(D) from table group by A, B, C".
> 2. To void many memory errors or other exceptions, we can auto set spark conf
> with build job.E.g use adaptive execution.
> 3. The snapshot table will save a table with parquet format.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
