[
https://issues.apache.org/jira/browse/KYLIN-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liyang updated KYLIN-781:
-------------------------
Summary: efficient in mem cubing like star cubing (was: efficient in mem
cubing)
> efficient in mem cubing like star cubing
> ----------------------------------------
>
> Key: KYLIN-781
> URL: https://issues.apache.org/jira/browse/KYLIN-781
> Project: Kylin
> Issue Type: Sub-task
> Components: Job Engine
> Reporter: liyang
> Assignee: liyang
> Fix For: v0.8.1
>
>
> Key idea
> - Assume base cuboid fit in mem (for a mapper split of course)
> - For child cuboid having common dim prefix, flow down records to child as
> soon as completed records can be fulfilled in child
> - Keep completed records in mem as long as possible, but also save them to
> disk in the background
> - When mem run short, pause flow down, remove completed records from mem, and
> resume flow from disk
> - Multiple flow downs at the same time, configurable thread number
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
