[
https://issues.apache.org/jira/browse/KYLIN-603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen closed KYLIN-603.
--------------------------
Resolution: Fixed
It's accomplished in the real-time OLAP feature, close this issue
> Add mem store for seconds data latency
> --------------------------------------
>
> Key: KYLIN-603
> URL: https://issues.apache.org/jira/browse/KYLIN-603
> Project: Kylin
> Issue Type: New Feature
> Reporter: liyang
> Assignee: liyang
> Priority: Major
> Fix For: Backlog
>
> Attachments: Kylin 0.8.x mem store.pdf
>
>
> For seconds level streaming data latency
> - Data is separated into Young and Aged.
> - Add mem store to hold young generation micro batch loaded from streaming
> (e.g. every 15 seconds). Mem store is good for high throughput and frequent
> access.
> - Aged generation is batch loaded from mem store to disk store daily or
> hourly.
> - Data is persisted in two copies, one as Inverted Index, one as Cube, for
> both young and aged.
> - Inverted Index is for analysis at raw record or low aggregation levels
> (cardinality > 10K), optimized for parallel random read.
> - Cube is for analysis at high aggregation levels (cardinality < 10K),
> optimized for sequential scan.
> - Hybrid bring aged from cube and young from inverted index when young cube
> is omitted.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
