I am tracing through the code, and on the backend, the pig implementation of map PigBaseMap (or PigMapReduce) is deserializing the pig.mapPlan or pig.reducePlan.
As a shot in the dark I installed jython.jar in the lib directories of all the nodes in the cluster, but nothing happened. I am using the jython-2.5.0 version. As I understand, the Register command should be shipping out the udfs (in py form) into the distributed cache on the cluster. Is this true? Where would I see this in HDFS? thanks, daniel On Thu, Jun 2, 2011 at 5:56 PM, Daniel Eklund <[email protected]> wrote: > Does anyone have followup to this problem? > http://www.mail-archive.com/[email protected]/msg01502.html > > I am getting: > Caused by: java.io.IOException: Deserialization error: could not > instantiate 'org.apache.pig.scripting.jython.JythonFunction' with arguments > '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelev > ant]' > at > org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.setup(PigMapReduce.java:299) > ... 8 more > Caused by: java.lang.RuntimeException: could not instantiate > 'org.apache.pig.scripting.jython.JythonFunction' with arguments > '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelevant]' > at > org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.instantiateFunc(POUserFunc.java:109) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.readObject(POUserFunc.java:451) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974) > at > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > >
