Thank you. But where does pig copy the jar on the slave machines? Incase some of my older jars with conflicting classes already exists on the machines, is there a way for me to specify what classpath to use for the child jvms?
On Wed, Dec 14, 2011 at 6:59 PM, Dmitriy Ryaboy <[email protected]> wrote: > Yes, you do this by using register.... :) > Just figure out what jars are conflicting and get rid of the old one. > > On Wed, Dec 14, 2011 at 6:01 PM, Gayatri Rao <[email protected]> wrote: > > > Thanks so much for your input Dmitriy. It looks like thats what is > > happening. Is there a way I can set classpath on child jvms in pig? > > > > Thanks > > Gayatri > > > > On Wed, Dec 14, 2011 at 5:57 PM, Dmitriy Ryaboy <[email protected]> > > wrote: > > > > > Can you send the stack trace? > > > If it's not finding a *method* then it's likely you have a jar conflict > > > (several jars have different versions of the class and you are > > accidentally > > > using the wrong one) > > > > > > D > > > > > > On Wed, Dec 14, 2011 at 4:41 PM, Gayatri Rao <[email protected]> > > wrote: > > > > > > > It picksup the loader method but not the helper method. They are in > > > > spearate jars and I have registered both. > > > > I also checked my job.jar that was created in > > > > /user/gayatri/.staging/job2011xxxx/job.jar and all the classes both > the > > > > loader as well as the helper classes are present. > > > > > > > > On Wed, Dec 14, 2011 at 4:27 PM, Prashant Kommireddi < > > > [email protected] > > > > >wrote: > > > > > > > > > Does your script not find the loader or helper methods? Are they > > > packaged > > > > > together? If they are separate do you register both jars? > > > > > > > > > > Take a look at > http://pig.apache.org/docs/r0.9.1/basic.html#register > > . > > > > > > > > > > > > > > > On Wed, Dec 14, 2011 at 4:15 PM, Gayatri Rao <[email protected]> > > > > wrote: > > > > > > > > > > > Hi All, > > > > > > > > > > > > Can someone please point me to any documentation on how exactly > > > > REGISTER > > > > > > copies over the job.jar to the slave machines? > > > > > > I have some loader UDFs and some helper methods which are > > utilized > > > by > > > > > the > > > > > > loader UDFs. My job.jar has all the classes but my pig job fails > > > saying > > > > > > that its not able to find the helper methods used by the loader > > UDFs. > > > > > > I cant seem to understand why. Any insights into what might be > the > > > > > problem? > > > > > > > > > > > > Thanks > > > > > > Gayatri > > > > > > > > > > > > > > > > > > > > >
