[
https://issues.apache.org/jira/browse/TEZ-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14682356#comment-14682356
]
Siddharth Seth commented on TEZ-2710:
-------------------------------------
setConf per vertex is not propagated to tasks - so this won't work independent
of re-use.
For re-use, additional work will be required in the ContainerSignatureMatcher
to ensure the configs are matched - or we should propagate a new set of configs
to tasks, and override appropriate settings.
A better solution would be to allow Processors to request memory like the
Inputs / Outputs do. That's the eventual intent anyway. [~rajesh.balamohan] has
been talking about this for sometime (I think there's a jira open as well)
> TEZ_TASK_SCALE_MEMORY_RESERVE_FRACTION should be configurable at vertex level
> -----------------------------------------------------------------------------
>
> Key: TEZ-2710
> URL: https://issues.apache.org/jira/browse/TEZ-2710
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
>
> Was trying to set TezConfiguration.TEZ_TASK_SCALE_MEMORY_RESERVE_FRACTION
> used by WeightedScalingMemoryDistributor at vertex level. But it failed with
> "tez.task.scale.memory.reserve-fraction is set at the scope of VERTEX, but it
> is only valid in the scope of AM"
> Even otherwise [~sseth] was telling me that vertex.setConf() does not work
> as it is supposed to and not taken into account with container reuse.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
