[
https://issues.apache.org/jira/browse/KYLIN-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-2404.
---------------------------------
Resolution: Fixed
> Add "hive.merge.mapfiles" and "hive.merge.mapredfiles" to kylin_hive_conf.xml
> -----------------------------------------------------------------------------
>
> Key: KYLIN-2404
> URL: https://issues.apache.org/jira/browse/KYLIN-2404
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
> Reporter: Shaofeng SHI
> Assignee: Shaofeng SHI
> Priority: Minor
> Fix For: v2.0.0
>
>
> Since 1.5.3, Kylin uses a "redistribute" step to merge the small files to
> proper size after creating the intermediate hive table. While in some users'
> environment, hive merge small files is enabled by default, that will cause
> additional CPU and will impact on the cube building performance (in extreme
> case the files will be merged to 256MB, then only very small number of
> mappers be started in building).
> So Kylin should explicitly tell Hive to disable the merge small files feature
> when creating and redistributing the intermediate flat table. Adding
> "hive.merge.mapfiles" and "hive.merge.mapredfiles" to
> conf/kylin_hive_conf.xml with value "false" will solve this. The meaning of
> these two parameters can be found in
> https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
