I found an example of setting the classpath for map/reduce job using "-libjars" argument. I was able to resolve dependencies using this technique. Here is the link to the example, for your reference
http://pangool.net/executing_examples.html - Chetan On Wed, Oct 23, 2013 at 11:10 AM, Chetan <[email protected]> wrote: > Serega, thanks for quick response. > > I thought that executing M/R jobs using a driver program & Oozie Java > action is a common practice. I am relatively new with Oozie, so my > assumption could be wrong. I wonder if anyone has come across similar issue > in their project and had some creative solution. > > > On Wed, Oct 23, 2013 at 10:23 AM, Serega Sheypak <[email protected] > > wrote: > >> It's better to run map-reduce using built in map-reduce action. and don't >> try to invent your own. >> 1. Try to move to separate custom java actions your setup/cleanup code >> 2. use built in map-reduce. >> >> Oozie knows nothing about spawned MR job in your action. And you would >> have >> to reinvent oozie functionality in your current implementation. >> >> >> 2013/10/23 Chetan <[email protected]> >> >> > Hi, >> > >> > I am trying to execute Oozie Application, which has a single "Java >> Action" >> > step to execute Map/Reduce job using a driver program. I am using Java >> > Action instead of using Map-Reduce action to execute M/R job, since I >> want >> > to do some additional setup/cleanup using driver program prior to >> > submitting job. >> > >> > The M/R job that I am trying to submit has some dependencies on third >> party >> > jars. I have included these dependencies in my application/lib folder. >> I >> > have also added those to /user/oozie/share/lib folder as well. >> > >> > Following is the snippet from job.properties. >> > >> > oozie.libpath=${nameNode}/user/oozie/share/lib >> > oozie.use.system.libpath=true >> > >> > However, when I run this application, M/R job fails since its not able >> to >> > find some dependency classes. I feel that Oozie is adding all the >> > dependency libs to the classpath of Java action itself. However not to >> the >> > M/R job that is being submitted by the java action. >> > >> > Is there any way to get around this ? How could I make Oozie to set the >> > classpath for M/R job, either pointing to application/lib OR >> > /user/oozie/share/lib ? >> > >> > Thanks, Chetan >> > >> > > > > -- > - Chetan > http://about.me/chetan.kadam > -- - Chetan http://about.me/chetan.kadam
