[
https://issues.apache.org/jira/browse/KYLIN-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315482#comment-15315482
]
Dayue Gao commented on KYLIN-1706:
----------------------------------
This is one of the small but very helpful improvements I'd like to have in
Kylin. There were sometimes I want to adjust mapper/reducer heap size and
"mapreduce.task.io.sort.mb" for a specific cube to accelerate building. It's
become feasible now with this jira. Thank you to bring it!
In addition, I'd like to comment that, this jira is not sufficient to support
cube/project level queue isolation. To the best of my knowledge, there're still
two problems to deal with,
# this jira only allows user to override MR queue configuration in
kylin_job_conf.xml. However queue configuration also appears in
kylin_hive_conf.xml, and we should address it too.
# queue can have ACLs to specify who can submit application to it. Therefore,
Kylin needs to know not only which queue to submit job, but also which hadoop
user to impersonate to.
> Allow cube to override MR job configuration by properties
> ---------------------------------------------------------
>
> Key: KYLIN-1706
> URL: https://issues.apache.org/jira/browse/KYLIN-1706
> Project: Kylin
> Issue Type: Improvement
> Reporter: liyang
> Assignee: liyang
> Fix For: v1.5.3
>
>
> Currently cube can specify MR job configuration by a job_conf.xml file under
> conf/. This is still not sufficient for example, to specify 50+ different job
> queues, user will have to maintain 50+ different job_conf.xml files.
> By allowing config override from kylin properties, the 50+ job queue case
> will become a lot more easier.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
