[
https://issues.apache.org/jira/browse/TEZ-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17259184#comment-17259184
]
Attila Magyar edited comment on TEZ-4254 at 1/5/21, 8:08 PM:
-------------------------------------------------------------
[~Rajkumar Singh], got it, thanks for the explanation, so
conf.get(dep.getValue()) returns the substituted version not the placeholder
itself.
mapreduce.shuffle.ssl.enabled=true
tez.runtime.shuffle.ssl.enable=\{mapreduce.shuffle.ssl.enabled}
Here the second condition will be true because both tezValue and mrValue are
the same (we're unsetting after getting the tezValue).
It makes sense to me, but someone who is a committer in the TEZ project should
also review it. [~abstractdog] could you review this change please?
was (Author: amagyar):
[~Rajkumar Singh], got it, thanks for the explanation, so
conf.get(dep.getValue()) returns the substituted version not the placeholder
itself.
mapreduce.shuffle.ssl.enabled=truetez.runtime.shuffle.ssl.enable={mapreduce.shuffle.ssl.enabled}
Here the second condition will be true because both tezValue and mrValue are
the same (we're unsetting after getting the tezValue).
It makes sense to me, but someone who is a committer in the TEZ project should
also review it. [~abstractdog] could you review this change please?
> Don't unset the tez config if both mr and tez config have same value.
> ---------------------------------------------------------------------
>
> Key: TEZ-4254
> URL: https://issues.apache.org/jira/browse/TEZ-4254
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Rajkumar Singh
> Priority: Major
> Attachments: TEZ-4254.patch
>
>
> while translated mr config to the tez config we unset config here.
> https://github.com/apache/tez/blob/master/tez-mapreduce/src/main/java/org/apache/tez/mapreduce/hadoop/MRHelpers.java#L187
> There is no need to unset the config if both mr and tez have same value.
> we observed if equivalent of tez config for mapreduce.shuffle.ssl.enabled
> (tez.runtime.shuffle.ssl.enable) set to true, we unset the config, this
> propagate config
> tez.runtime.shuffle.ssl.enable={mapreduce.shuffle.ssl.enabled} at runtime and
> variable substitution does not work correctly in the absence of mrconfig
> runtime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
