[
https://issues.apache.org/jira/browse/HIVE-10066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-10066:
----------------------------------
Description:
>From [~hitesh]:
"Tez is a client-side only component ( no daemons, etc ) and therefore it is
meant to be installed on the gateway box ( or where its client libraries are
needed by any other services’ daemons). It does not have any cluster
dependencies both in terms of libraries/jars as well as configs. When it runs
on a worker node, everything was pre-packaged and made available to the worker
node via the distributed cache via the client code. Hence, its client-side
configs are also only needed on the same (client) node as where it is
installed. The only other install step needed is to have the tez tarball be
uploaded to HDFS and the config has an entry “tez.lib.uris” which points to the
HDFS path. "
We need a way to pass client jars and tez-site.xml to the LaunchMapper.
We should create a general purpose mechanism here which can supply additional
artifacts per job type.
was:
>From [~hitesh]:
"Tez is a client-side only component ( no daemons, etc ) and therefore it is
meant to be installed on the gateway box ( or where its client libraries are
needed by any other services’ daemons). It does not have any cluster
dependencies both in terms of libraries/jars as well as configs. When it runs
on a worker node, everything was pre-packaged and made available to the worker
node via the distributed cache via the client code. Hence, its client-side
configs are also only needed on the same (client) node as where it is
installed. The only other install step needed is to have the tez tarball be
uploaded to HDFS and the config has an entry “tez.lib.uris” which points to the
HDFS path. "
We need a way to pass client jars and tez-site.xml to the LaunchMapper.
We should create a general purpose mechanism here which would also include
sending hive-site.xml to LaunchMapper so that there is no duplication between
hive-site.xml and templeton.hive.properties in webhcat-site.xml.
> Hive on Tez job submission through WebHCat doesn't ship Tez artifacts
> ---------------------------------------------------------------------
>
> Key: HIVE-10066
> URL: https://issues.apache.org/jira/browse/HIVE-10066
> Project: Hive
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
>
> From [~hitesh]:
> "Tez is a client-side only component ( no daemons, etc ) and therefore it is
> meant to be installed on the gateway box ( or where its client libraries are
> needed by any other services’ daemons). It does not have any cluster
> dependencies both in terms of libraries/jars as well as configs. When it runs
> on a worker node, everything was pre-packaged and made available to the
> worker node via the distributed cache via the client code. Hence, its
> client-side configs are also only needed on the same (client) node as where
> it is installed. The only other install step needed is to have the tez
> tarball be uploaded to HDFS and the config has an entry “tez.lib.uris” which
> points to the HDFS path. "
> We need a way to pass client jars and tez-site.xml to the LaunchMapper.
> We should create a general purpose mechanism here which can supply additional
> artifacts per job type.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
