[ 
https://issues.apache.org/jira/browse/KYLIN-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI closed KYLIN-1178.
-------------------------------

> Build dictionary in Hadoop cluster
> ----------------------------------
>
>                 Key: KYLIN-1178
>                 URL: https://issues.apache.org/jira/browse/KYLIN-1178
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Job Engine
>            Reporter: Shaofeng SHI
>            Assignee: Shaofeng SHI
>
> Kylin build dictionary in job engine node, usually this is okay. But if there 
> is some high cardinality dimentions, the JVM heap couldn't fit in all 
> distinct values, then job engine instance will crash with OOM error.
> Need to enhance on this, move the dictionary building to another process or a 
> hadoop node. Ideally only need modify "CreateDictionaryJob.java", move the 
> dictionary building to a mapper task.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to