[ https://issues.apache.org/jira/browse/HIVE-322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12679206#action_12679206 ]
coderplay edited comment on HIVE-322 at 3/7/09 7:28 AM: ------------------------------------------------------- The create function statement loads a function class staticly from $HIVE_AUX_JARS_PATH and hive.aux.jars.path http://issues.apache.org/jira/browse/HADOOP-4070 We ofen start a Hive thrift server, and it will up for quite a time. We cannot dynamically load another UDF in another jar unless we restart the server . It's not practical for us. This patch introduces a register statement for appending a jar path to hive on-the-fly. It works well both in CLI and Thrift server mode. usage: hive>register '/home/zhoumin/zhoumin.jar'; hive>CREATE TEMPORARY FUNCTION strlen AS 'hadoop.hive.udf.UdfStringLength'; hive>select strlen(time) from log_data; Nevertheless, this patch cannot pass current unit test of hive, because the MapTask run during unit test will serialize MapRedWork w/o having done a register command. It will cause the ClassLoader cannot find my UDF. see line 61 and line 63 in MapTask.java. was (Author: coderplay): The create function statement loads a function class staticly from $HIVE_AUX_JARS_PATH and hive.aux.jars.path http://issues.apache.org/jira/browse/HADOOP-4070 We ofen start a Hive thrift server, and it will up for quite a time. We cannot dynamically load another UDF in another jar unless we restart the server . It's not practical for us. This patch introduces a register statement for appending a jar path to hive on-the-fly. It works well both in CLI and Thrift server mode. usage: hive>register '/home/zhoumin/zhoumin.jar'; hive>CREATE TEMPORARY FUNCTION strlen AS 'taobao.hadoop.hive.udf.UdfStringLength'; hive>select strlen(time) from log_data; Nevertheless, this patch cannot pass current unit test of hive, because the MapTask run during unit test will serialize MapRedWork w/o having done a register command. It will cause the ClassLoader cannot find my UDF. see line 61 and line 63 in MapTask.java. > cannot create temporary udf dynamically, with a ClassNotFoundException > ----------------------------------------------------------------------- > > Key: HIVE-322 > URL: https://issues.apache.org/jira/browse/HIVE-322 > Project: Hadoop Hive > Issue Type: Bug > Components: Query Processor > Affects Versions: 0.3.0 > Reporter: Min Zhou > Priority: Blocker > Attachments: registerjars-v3.patch, registerjars1.patch, > registerjars2.patch > > > I found the ClassLoader cannot load my UDF when doing FunctionTask, because > the ClassLoader hasnot append its classpaths on-the-fly yet. > The ExecDriver' s addToClassPath(String[] newPaths) method is the only entry > for ClassLoader dynamically append its classhpaths (besides hadoop's > GenericOptionsParser). > But that function wasnot called before FunctionTask getting my UDF class by > class name. I think this is the reason why I came across that failure. > scenario description: > I set a peroperty in hive-site.xml to configure the classpath of my udf. > <property> > <name>hive.aux.jars.path</name> > <value>/home/hadoop/hdpsoft/hive-auxs/zhoumin.jar</value> > </property> > but failed to register it with a ClassNotFoundException when creating udf > through the sql command. > CREATE TEMPORARY FUNCTION strlen AS 'hadoop.hive.udf.UdfStringLength' > I'll make a patch soon. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.