[jira] [Commented] (KYLIN-2993) Add special mr config for base cuboid step
[ https://issues.apache.org/jira/browse/KYLIN-2993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307620#comment-16307620 ] Billy Liu commented on KYLIN-2993: -- Thanks [~kangkaisen] & [~liyang.g...@gmail.com]. To enable this feature, set kylin.engine.mr.base-cuboid-config-override.mapreduce.map.memory.mb= kylin.engine.mr.base-cuboid-config-override.mapreduce.reduce.memory.mb= kylin.engine.mr.base-cuboid-config-override.mapreduce.map.java.opts= kylin.engine.mr.base-cuboid-config-override.mapreduce.reduce.java.opts= > Add special mr config for base cuboid step > -- > > Key: KYLIN-2993 > URL: https://issues.apache.org/jira/browse/KYLIN-2993 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.1.0 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v2.3.0 > > Attachments: KYLIN-2993.patch > > > Refer to http://kylin.apache.org/blog/2016/08/01/count-distinct-in-kylin/, > currently, if user want to enlarge MR memory for global dict, they must use > kylin.engine.mr.config-override., which will enlarge the memory of all mr > job. In fact, we only need to enlarge the memory for "Build Base Cuboid", so > we could add a special mr config for base cuboid step. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2993) Add special mr config for base cuboid step
[ https://issues.apache.org/jira/browse/KYLIN-2993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290471#comment-16290471 ] kangkaisen commented on KYLIN-2993: --- OK, Thanks Liyang! > Add special mr config for base cuboid step > -- > > Key: KYLIN-2993 > URL: https://issues.apache.org/jira/browse/KYLIN-2993 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.1.0 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v2.3.0 > > Attachments: KYLIN-2993.patch > > > Refer to http://kylin.apache.org/blog/2016/08/01/count-distinct-in-kylin/, > currently, if user want to enlarge MR memory for global dict, they must use > kylin.engine.mr.config-override., which will enlarge the memory of all mr > job. In fact, we only need to enlarge the memory for "Build Base Cuboid", so > we could add a special mr config for base cuboid step. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2993) Add special mr config for base cuboid step
[ https://issues.apache.org/jira/browse/KYLIN-2993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290427#comment-16290427 ] liyang commented on KYLIN-2993: --- Merged. Thanks kaisen! https://github.com/apache/kylin/commit/ad3dd3f63d399640142bd2c97669080bc3e37560 > Add special mr config for base cuboid step > -- > > Key: KYLIN-2993 > URL: https://issues.apache.org/jira/browse/KYLIN-2993 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.1.0 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2993.patch > > > Refer to http://kylin.apache.org/blog/2016/08/01/count-distinct-in-kylin/, > currently, if user want to enlarge MR memory for global dict, they must use > kylin.engine.mr.config-override., which will enlarge the memory of all mr > job. In fact, we only need to enlarge the memory for "Build Base Cuboid", so > we could add a special mr config for base cuboid step. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
