That did it. Thanks Shawn!
On Tue, Aug 30, 2011 at 1:26 PM, Xiaomeng Wan <[email protected]> wrote: > Did you import any python modules in your udf? That wouldn't work. See > this thread. > > http://search-hadoop.com/m/4UsokeoB7C&subj=Re+python+udf+doesnt+work > > Shawn > > On Tue, Aug 30, 2011 at 9:03 AM, Mark Roddy <[email protected]> wrote: >> Thanks Shawn. Unfortunately I'm still getting the same error. I've >> tried adding the full and relative path to the python module in the >> CLASSPATH. The script initially did a 'Register' using a relative >> path to the module. I've also tried it with the full path (and >> different combinations of full and relative classpath). I also tried >> using /path/to/folder/with/module/* >> >> Any other suggestions? >> >> On Mon, Aug 29, 2011 at 9:33 PM, Xiaomeng Wan <[email protected]> wrote: >>> you need to add your apachelogs.py into classpath too. >>> >>> Shawn >>> >>> On Mon, Aug 29, 2011 at 6:15 PM, Mark Roddy <[email protected]> wrote: >>>> I have a Jython UDF I've written that works fine in local mode but >>>> bombs out when I run it on my cluster. >>>> >>>> I'm running 0.8.0, and my stack trace and environment variables are below. >>>> >>>> java.io.IOException: Deserialization error: could not instantiate >>>> 'org.apache.pig.scripting.jython.JythonFunction' with arguments >>>> '[src/apachelogs.py, extract_apache_log]' >>>> at >>>> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55) >>>> at >>>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.setup(PigMapBase.java:151) >>>> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) >>>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646) >>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322) >>>> at org.apache.hadoop.mapred.Child$4.run(Child.java:240) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at javax.security.auth.Subject.doAs(Subject.java:396) >>>> at >>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115) >>>> >>>> >>>> export CLASSPATH=/usr/share/java/*:/home/mroddy/jython/jython.jar >>>> export HADOOP_HOME=/usr/lib/hadoop >>>> export >>>> PIG_CLASSPATH=/usr/lib/pig/contrib/piggybank/java/piggybank.jar:/usr/share/java/joda-time.jar:/home/mroddy/jython/jython.jar >>>> >>> >> >
