[
https://issues.apache.org/jira/browse/TEZ-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Seth updated TEZ-1534:
--------------------------------
Attachment: TEZ-1534.1.txt
Patch allows additional parameters to be filtered through - selective list from
properties used in hadoop-common, hdfs and yarn.
There's a unstable, semi-private conf property which can be used to change this
if something is missed out.
Based on an offline discussion with [~hitesh] - not removing HADOOP_CLASSPATH
from the AM/task classpath. If these configs exist, they'll be loaded and
overwritten by values from the client.
The main reason is in case new Configuration() instances are created by
hadoop-code - they'll have some configuration to work with.
[~hitesh] - please review.
> Make client side configs available to AM and tasks
> --------------------------------------------------
>
> Key: TEZ-1534
> URL: https://issues.apache.org/jira/browse/TEZ-1534
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Attachments: TEZ-1534.1.txt
>
>
> Configs from the client (specifically the ones provided to TezClient, along
> with YARN additions) should be shipped over to the cluster (AM and tasks),
> instead of AM/tasks depending on configs present on cluster nodes.
> These configs will primarily be used for Tez components like RPC servers,
> clients etc - and not by the Processor / Input / Output - which should be
> sending over fully configured payloads in any case.
> Tez should continue to run without core-site, hdfs-site, yarn-site etc in the
> classpath on cluster nodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
