[
https://issues.apache.org/jira/browse/HIVE-574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Shao updated HIVE-574:
----------------------------
Attachment: HIVE-574.2.patch
This patch fixed a bug in ExecDriver.initialize where we were setting
"tmpjars", which were overwritten by ExecDriver.execute().
> there's a bunch of places where JavaUtils.getClassLoader() is called - all of
> these need to use the conf classloader instead.
I tried to change all of these to use conf classloader, but in lots of places
there are no shared conf objects, so I will leave it for now. I changed the
comment so it's clear that in hive (outside hadoop map-reduce jobs) we are
using thread classloader, while in hadoop map-reduce jobs we are using conf
classloaders.
> where are we invoking conf.setClassLoader? (addToClassPath should be calling
> this it seems)
I moved that logic outside of the addToClassPath function. All callers will do
that set.
> if conf.set/getClassLoader is supported in all versions 17 onwards - then we
> can just remove the -libjars business - since that doesn't help us any bit
> now (ExecDriver is not a Tool - so it doesn't even get the conf with the
> classpath set that a Tool would normally get by specifying the -libjars
> option).
Maybe in the future we should move ExecDriver to a tool? I think we should rely
more on hadoop to set the classpath for us, so maybe in the future we should
remove the special "if (local)" block in ExecDriver where we set the classpath
ourselves.
> Hive should use ClassLoader from hadoop Configuration
> -----------------------------------------------------
>
> Key: HIVE-574
> URL: https://issues.apache.org/jira/browse/HIVE-574
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.3.0, 0.3.1
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-574.1.patch, HIVE-574.2.patch
>
>
> See HIVE-338.
> Hive should always use the getClassByName method from hadoop Configuration,
> so that we choose the correct ClassLoader. Examples include all plug-in
> interfaces, including UDF/GenericUDF/UDAF, SerDe, and FileFormats. Basically
> the following code snippet shows the idea:
> {code}
> package org.apache.hadoop.conf;
> public class Configuration implements Iterable<Map.Entry<String,String>> {
> ...
> /**
> * Load a class by name.
> *
> * @param name the class name.
> * @return the class object.
> * @throws ClassNotFoundException if the class is not found.
> */
> public Class<?> getClassByName(String name) throws ClassNotFoundException {
> return Class.forName(name, true, classLoader);
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
