Thanks Shawn. Unfortunately I'm still getting the same error. I've tried adding the full and relative path to the python module in the CLASSPATH. The script initially did a 'Register' using a relative path to the module. I've also tried it with the full path (and different combinations of full and relative classpath). I also tried using /path/to/folder/with/module/*
Any other suggestions? On Mon, Aug 29, 2011 at 9:33 PM, Xiaomeng Wan <[email protected]> wrote: > you need to add your apachelogs.py into classpath too. > > Shawn > > On Mon, Aug 29, 2011 at 6:15 PM, Mark Roddy <[email protected]> wrote: >> I have a Jython UDF I've written that works fine in local mode but >> bombs out when I run it on my cluster. >> >> I'm running 0.8.0, and my stack trace and environment variables are below. >> >> java.io.IOException: Deserialization error: could not instantiate >> 'org.apache.pig.scripting.jython.JythonFunction' with arguments >> '[src/apachelogs.py, extract_apache_log]' >> at >> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55) >> at >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.setup(PigMapBase.java:151) >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322) >> at org.apache.hadoop.mapred.Child$4.run(Child.java:240) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:396) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115) >> >> >> export CLASSPATH=/usr/share/java/*:/home/mroddy/jython/jython.jar >> export HADOOP_HOME=/usr/lib/hadoop >> export >> PIG_CLASSPATH=/usr/lib/pig/contrib/piggybank/java/piggybank.jar:/usr/share/java/joda-time.jar:/home/mroddy/jython/jython.jar >> >
