Shawn... excellent!.. thank you. it worked. interestingly, I remember having to use the absolute path in local mode....
daniel On Fri, Jun 3, 2011 at 12:01 PM, Xiaomeng Wan <[email protected]> wrote: > Hi Daniel, > > We found it was the absolute path which didn't work. Try relative path > when register your travelocity-udfs.py. > > Shawn > > On Thu, Jun 2, 2011 at 10:49 PM, Daniel Eklund <[email protected]> wrote: > > I am tracing through the code, and on the backend, the pig implementation > of > > map PigBaseMap (or PigMapReduce) is deserializing the pig.mapPlan or > > pig.reducePlan. > > > > As a shot in the dark I installed jython.jar in the lib directories of > all > > the nodes in the cluster, but nothing happened. I am using the > jython-2.5.0 > > version. > > > > As I understand, the Register command should be shipping out the udfs (in > py > > form) into the distributed cache on the cluster. Is this true? Where > would > > I see this in HDFS? > > > > thanks, > > daniel > > > > On Thu, Jun 2, 2011 at 5:56 PM, Daniel Eklund <[email protected]> > wrote: > > > >> Does anyone have followup to this problem? > >> http://www.mail-archive.com/[email protected]/msg01502.html > >> > >> I am getting: > >> Caused by: java.io.IOException: Deserialization error: could not > >> instantiate 'org.apache.pig.scripting.jython.JythonFunction' with > arguments > >> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelev > >> ant]' > >> at > >> > org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55) > >> at > >> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.setup(PigMapReduce.java:299) > >> ... 8 more > >> Caused by: java.lang.RuntimeException: could not instantiate > >> 'org.apache.pig.scripting.jython.JythonFunction' with arguments > >> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelevant]' > >> at > >> > org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502) > >> at > >> > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.instantiateFunc(POUserFunc.java:109) > >> at > >> > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.readObject(POUserFunc.java:451) > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> at > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > >> at > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > >> at java.lang.reflect.Method.invoke(Method.java:597) > >> at > >> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974) > >> at > >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) > >> at > >> > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > >> > >> > > >
