Hi All,
I am not having some hive classpath issue. I think this is a bug.
I wrote my own SerDe "com.stanley.MySerde", which is a simple json serializer;
It is generally the same with the built-in SerDe
"org.apache.hadoop.hive.serde2.DelimitedJSONSerDe".
Then I issued the command:add jar /path/to/myjar.jar; (I am sure this command
worked)create table t1.json_1 row format serde "com.stanley.MySerde" location
'/user/stanley/test-data-1/' as select * from t1.plain_table;
create table t1.json_2 row format serde
"org.apache.hadoop.hive.serde2.DelimitedJSONSerDe" location
'/user/stanley/test-data-2/' as select * from t1.plain_table;
The second command will succeed but the first one will fail with
ClassNotFoundException. But if I put myjar.jar to $HIVE_HOME/lib, both command
will succeed. I went through the code of the
org.apache.hadoop.hive.ql.plan.PlanUtils.java, seems it is using
Class.forname(clzname) to load the class, I think it should use the
Thread.contextClassLoader instead, am I right?There's a similar issue here:
https://issues.apache.org/jira/browse/HIVE-6495
Here's the exception trace: java.lang.ClassNotFoundException:
com.ebay.p13n.hive.bexbat.serde.JsonLazySimpleSerDe at
java.net.URLClassLoader$1.run(URLClassLoader.java:366) at
java.net.URLClassLoader$1.run(URLClassLoader.java:355) at
java.security.AccessController.doPrivileged(Native Method) at
java.net.URLClassLoader.findClass(URLClassLoader.java:354) at
java.lang.ClassLoader.loadClass(ClassLoader.java:425) at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at
java.lang.ClassLoader.loadClass(ClassLoader.java:358) at
java.lang.Class.forName0(Native Method) at
java.lang.Class.forName(Class.java:190) at
org.apache.hadoop.hive.ql.plan.PlanUtils.getTableDesc(PlanUtils.java:310) at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(SemanticAnalyzer.java:5874)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8278)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8169)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9001)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9267)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427) at
org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323) at
org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980) at
org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:906)
