[
https://issues.apache.org/jira/browse/HIVE-7950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127440#comment-14127440
]
Josh Elser commented on HIVE-7950:
----------------------------------
Sure thing, [~gopalv].
I don't actually have to do any extra {{ADD JAR}} commands. The
AccumuloStorageHandler constructs a list of jars that need to be passed along
to the "execution engine" (via tmpjars in the Hadoop configuration). With the
'yarn' execution.engine, this works just fine -- the resources are localized
and added to the Map/Reduce containers and things are great.
When I try to run with 'tez', there are a few issues. The first is that, if
there is already a TezSessionState that was already open'ed (e.g. like what is
done when I just open the hive shell), it will have been started without those
extra 'tmpjars' resources from the StorageHandler and the query will fail
because we need those jars.
Sergey mentioned that Tez 0.5.0 had a new method that would allow more
resources to be added to an already started TezClient
({{TezClient#addAppMasterLocalFiles(Map<String, LocalResource>)}}).
Implementing this (in the hive-7950-tez-WIP.diff attachment), appears to have
successfully added the extra jars from the StorageHandler to the DAGAppMaster,
but the containers started to actually run the query are missing those extra
jars.
Does that make sense?
> StorageHandler resources aren't added to Tez Session if already Session is
> already Open
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-7950
> URL: https://issues.apache.org/jira/browse/HIVE-7950
> Project: Hive
> Issue Type: Bug
> Components: StorageHandler, Tez
> Reporter: Josh Elser
> Assignee: Josh Elser
> Fix For: 0.14.0
>
> Attachments: HIVE-7950-1.diff, hive-7950-tez-WIP.diff
>
>
> Was trying to run some queries using the AccumuloStorageHandler when using
> the Tez execution engine. Some things that classes which were added to
> tmpjars weren't making it into the container. When a Tez Session is already
> open, as is the normal case when simply using the `hive` command, the
> resources aren't added.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
