Hi all,
I am writing a transform in Java with the help of wrappers in
contrib/mr. The code is as follows:
insert overwrite table raw_log_proc_test1
select transform (d.ll)
using 'java -cp imageshack.jar
com.imageshack.logsanalysis.HiveAccessLogLineStringMapper'
as (f1, f2)
from (select logline as ll from raw_log_test) d;
Here is the problem: java throws a NoClassDefFoundError unless I
include every jar with an "add file" and run the command as follows:
add file /usr/lib/hive/lib/hive_contrib.jar;
add file /home/hadoop/sandbox/imageshack.jar;
.... a bunch of other jars...
transform command string:
transform (d.ll) using 'java -cp
./imageshack.jar:./json.jar:./elephant-bird-1.0.jar:./commons-logging-1.0.4.jar
com.imageshack.log\
sanalysis.HiveAccessLogLineStringMapper'
I tried using "add jar" instead of "add files", and the command fails.
However, I looked into logs (INFO ExecMapper: conf classpath and INFO
ExecMapper: thread classpath ) and all the relevant jars are in the
classpath.
How can I run the transform...using command with java such that it
uses the correct classpath?...
Thanks,
--Leo
