[
https://issues.apache.org/jira/browse/KYLIN-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liyang updated KYLIN-1178:
--------------------------
Summary: Build dictionary in Hadoop cluster (was: Build dictionary out of
job engine)
> Build dictionary in Hadoop cluster
> ----------------------------------
>
> Key: KYLIN-1178
> URL: https://issues.apache.org/jira/browse/KYLIN-1178
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
> Reporter: Shaofeng SHI
> Assignee: Shaofeng SHI
>
> Kylin build dictionary in job engine node, usually this is okay. But if there
> is some high cardinality dimentions, the JVM heap couldn't fit in all
> distinct values, then job engine instance will crash with OOM error.
> Need to enhance on this, move the dictionary building to another process or a
> hadoop node. Ideally only need modify "CreateDictionaryJob.java", move the
> dictionary building to a mapper task.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
