[
https://issues.apache.org/jira/browse/KYLIN-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15833450#comment-15833450
]
XIE FAN commented on KYLIN-2217:
--------------------------------
I will improve this once after I finish KYLIN-2374, thank you.
> Reducers build dictionaries locally
> -----------------------------------
>
> Key: KYLIN-2217
> URL: https://issues.apache.org/jira/browse/KYLIN-2217
> Project: Kylin
> Issue Type: Improvement
> Affects Versions: v1.5.4.1
> Reporter: XIE FAN
> Assignee: XIE FAN
> Fix For: v2.0.0
>
> Attachments: 0001-KYLIN-2217-Reducers-build-dictionaries-locally.patch
>
>
> In KYLIN-1851, we reduce the peek memory usage of the dictionary-building
> procedure by splitting a single Trie tree structure to Trie forest. But there
> still exist a bottleneck that all the dictionaries are built in Kylin client.
> In this issue, we want to use multi reducers to build different dictionaries
> locally and concurrently，which can further reduce the peek memory usage as
> well as speed up the dictionary-building procedure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
