[
https://issues.apache.org/jira/browse/KYLIN-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362513#comment-15362513
]
liyang commented on KYLIN-1844:
-------------------------------
I guess "A whole dimension is kept in memory." refers to dictionary?
Actually dictionary is not recommend for high cardinality columns.
> High cardinality dimensions in memory
> -------------------------------------
>
> Key: KYLIN-1844
> URL: https://issues.apache.org/jira/browse/KYLIN-1844
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
> Affects Versions: v1.2, v1.5.2
> Reporter: Abhilash L L
> Assignee: liyang
>
> A whole dimension is kept in memory.
> We should have a way to keep only certain number / size of total rows to be
> kept in memory. A LRU cache for rows in the dimension will help keep memory
> in check.
> Why not store all the dimensions data in hbase in a different table with a
> prefix of dimensionid, and all calls to the dimensions (get based on dim
> key), is mapped to hbase.
> This does mean it will cost more time on a miss.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
