Shaofeng SHI closed KYLIN-1178.

> Build dictionary in Hadoop cluster
> ----------------------------------
>                 Key: KYLIN-1178
>                 URL: https://issues.apache.org/jira/browse/KYLIN-1178
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Job Engine
>            Reporter: Shaofeng SHI
>            Assignee: Shaofeng SHI
> Kylin build dictionary in job engine node, usually this is okay. But if there 
> is some high cardinality dimentions, the JVM heap couldn't fit in all 
> distinct values, then job engine instance will crash with OOM error.
> Need to enhance on this, move the dictionary building to another process or a 
> hadoop node. Ideally only need modify "CreateDictionaryJob.java", move the 
> dictionary building to a mapper task.

This message was sent by Atlassian JIRA

Reply via email to