[
https://issues.apache.org/jira/browse/KYLIN-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304732#comment-17304732
]
JiangYang commented on KYLIN-4939:
----------------------------------
We find many use cases should use cube-level lookup table snapshot instead of
segment-level ones. For segment-level ones, it will cause many issues. For
example for table *BATCH_VIEWS.xxx*, it changes everyday. Now there're around
2000 snapshots.
* For building a new snapshot, it will check the duplication to avoid
duplicated snapshots, which means it will check all of the 2000 snapshots. It
really costs time.
* For query & building, to cache the snapshot, if each takes around 6MB, the
total will need around 12GB memory.
After this change, we can not only speedup cubing but also reduce the burden of
memory cost.
> Transform lookup table snapshot from segment level to cube level
> ----------------------------------------------------------------
>
> Key: KYLIN-4939
> URL: https://issues.apache.org/jira/browse/KYLIN-4939
> Project: Kylin
> Issue Type: New Feature
> Affects Versions: v3.1.2
> Reporter: JiangYang
> Assignee: JiangYang
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
