[
https://issues.apache.org/jira/browse/TEZ-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18055810#comment-18055810
]
Raghav Aggarwal commented on TEZ-4685:
--------------------------------------
Thanks for the review [~abstractdog] .
I checked the TEZ-346 and my understanding is that the Tez AMÂ should read
configs from the TezClient only and not from tez-site.xml present/deployed on
the NodeManager. With that context, I believe the solution proposed in this PR
(for {_}dependent AM's{_}) is still valid. My reasoning is as follows:
{code:java}
Configuration conf = new TezConfiguration();
AMExtensions amExtensions = getFrameworkService(conf).getAMExtensions();
DAGProtos.ConfigurationProto confProto = amExtensions.loadConfigurationProto();
TezUtilsInternal.addUserSpecifiedTezConfiguration(conf,
confProto.getConfKeyValuesList()); {code}
The AM reads from the tez-conf.pb (in YARN flow) written by the client, which
contains modified session-level configurations. Meaning, Client configurations
(derived from confProto) take precedence and will override any values loaded
from tez-site.xml. The only side-effect is that the Tez AM will contain
additional configurations from the NodeManager's {{tez-site.xml}} that the
client did not specify. This ensures the AM has actual Tez service cluster
defaults,in contrast to falling back to the default values specified in the tez
code
> DagAppMaster is not picking tez-site.xml from classpath
> -------------------------------------------------------
>
> Key: TEZ-4685
> URL: https://issues.apache.org/jira/browse/TEZ-4685
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Raghav Aggarwal
> Assignee: Raghav Aggarwal
> Priority: Major
> Attachments: After.png, Before.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In DAGAppMaster#main()
> [https://github.com/apache/tez/blob/1fc00352ccae70d265649621aca22a5904ba771f/tez-dag/src/main/java/org/apache/tez/dag/app/DAGAppMaster.java#L2432]
> {code:java}
> Configuration conf = new Configuration(); {code}
> is not picking tez-site.xml from classpath. Even during debugging
> tez-site.xml is in classpath, its not getting picked up
> Moving to TezConfiguration is helping
> {code:java}
> Configuration conf = new TezConfiguration();{code}
> Please check the attached screenshots
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
