[
https://issues.apache.org/jira/browse/TEZ-1233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303820#comment-14303820
]
Hitesh Shah commented on TEZ-1233:
----------------------------------
As part of the TezClient re-factor some of this support was dropped. There is
no api support currently to modify parameters per DAG. There are a couple of
reasons for this:
- the main one being distinguishing what parameters are AM/session specific
and which ones are only DAG specific. What configs can be overridden per DAG is
also something that needs to be looked at and restricted accordingly.
- also how this ties into container re-use across DAGs also needs to be
looked at.
My gut feeling is that we probably need to look at first support DAG-level
configs and verifying that they work in a clean manner before looking at
introducing per vertex configurations as vertex configs will overlay on top of
DAG-specific configs. Also, how are vertex specific configs meant to be passed
to Tez from the user? Does an API make sense or will it be a backdoor channel
via a config ( which will need dag specific config support in any case )?
> Allow configuration of framework parameters per vertex
> ------------------------------------------------------
>
> Key: TEZ-1233
> URL: https://issues.apache.org/jira/browse/TEZ-1233
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
>
> Currently, configuration properties specific in AMConfiguration are used to
> configure tasks - e.g. memory configs, process tree, etc. These should be
> configurable at a Vertex level.
> Also, tasks end up reading these configs from the dist cache each time -
> would be simpler to send over the wire as part of the TaskSpec.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
