Hello,
         I'm trying to submit a hadoop jar job via Oozie workflow. In
particular, trying to run it using the shell action by invoking the
/usr/bin/hadoop jar <jar_file> <MRClass>..


I tested this job separately from the command line and it works when I just
submit using "hadoop jar <jar_file> <class_name> " . But the same job fails
with the following error from within Oozie flow step with this ..

Exception in thread "main" java.lang.NoClassDefFoundError: Could not
initialize class org.apache.log4j.LogManager
        at org.apache.log4j.Logger.getLogger(Logger.java:117)
        at com.linkedin.camus.etl.kafka.CamusJob.<init>(CamusJob.java:128)
        at com.linkedin.camus.etl.kafka.CamusJob.<init>(CamusJob.java:124)
        at 
com.upsight.valhalla.kafka.dumper.KafkaDumper.main(KafkaDumper.java:19)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)



I'v made sure there's a log4j.xml and the LogManager impl in my uber jar.
I'd imagine others have tried submitting a  MR job with "hadoop jar "
runtime. Although, I'm also aware of the <map-reduce> action but it's too
cumbersome for me since i set a lot of params, counters in my job client
class (which implements the Tool abstract class).



Here's my workflow step config:

<action name="phShimDump">
        <shell xmlns="uri:oozie:shell-action:0.3">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <exec>/usr/bin/hadoop</exec>
            <argument>jar</argument>
            <argument>kafka-hdfs-etl/jar/kafka-hdfs-etl.jar</argument>

<argument>com.upsight.valhalla.kafka.dumper.KafkaDumper</argument>

<!--<argument>kafka-hdfs-etl/script/kafka-etl-console</argument>-->
            <argument>-P</argument>

<argument>kafka-hdfs-etl/script/oozie/${phShimDumpConfig}</argument>
            <argument>-D</argument>
            <argument>camus.job.name=ph_up_shim_data</argument>
            <argument>-D</argument>
            <argument>zookeeper.hosts=${zookeeperHosts}</argument>
            <argument>-D</argument>
            <argument>kafka.brokers=${kafkaBrokers}</argument>

<archive>/opt/kafka-hdfs-etl/kafka-hdfs-etl.tar.gz#kafka-hdfs-etl</archive>
            <capture-output/>
        </shell>
        <ok to="end" />
        <error to="kill" />
    </action>


thanks and regards,
-Gautam.

Reply via email to