Hi Russell, Check if your hadoop has MAPREDUCE-967. If not, pig's job.jar will be unjared on the tasks. With that, jython's static method's do not put /Lib in python sys path. You can use the instrumentation code i shared in your python udf file and find the sys.path on task nodes to confirm this. If /Lib is on sys.path, all imports will work.
One workaround is to put jython jar on all nodes of the cluster under hadoop lib. ~Aniket On Tue, Jul 24, 2012 at 5:21 PM, Russell Jurney <[email protected]>wrote: > Thanks, but that isn't my issue. I am unable to import any packages. > Trying to get the path right... > > Russell Jurney http://datasyndrome.com > > On Jul 24, 2012, at 10:00 AM, Chun Yang <[email protected]> > wrote: > > > Hi Russell, > > > > Are you able to import other modules beside email? If not, maybe this is > > related to your problem: https://issues.apache.org/jira/browse/PIG-2665 > > > > -Chun > > > > On 7/23/12 11:26 PM, "Russell Jurney" <[email protected]> wrote: > > > >> ls /me/jython2.5.2/Lib/ > >> > >> tons of class files... > >> email/ > >> > >> > >> This is in local mode, atm. I add this directory to my java classpath, > >> check. > >> > >> On Mon, Jul 23, 2012 at 11:10 PM, Aniket Mokashi <[email protected] > >wrote: > >> > >>> jar tf jython.jar | grep email > >>> > >>> Having jar in PIG_CLASSPATH would work if you have > >>> https://issues.apache.org/jira/browse/MAPREDUCE-967. > >>> > >>> You can use following to debug the sys.path on tasknodes- > >>> > >>> from java.lang import System > >>> print "python.home " > >>> print System.getProperties().getProperty("python.home") > >>> print "java.class.path " > >>> print System.getProperties().getProperty("java.class.path") > >>> print "install.root " > >>> print System.getProperties().getProperty("install.root") > >>> print "python.home " > >>> print System.getProperties().getProperty("python.home") > >>> > >>> ~Aniket > >>> > >>> On Mon, Jul 23, 2012 at 6:33 PM, Russell Jurney < > [email protected] > >>>> wrote: > >>> > >>>> No, how do I find which jar the email package is in? > >>>> > >>>> On Mon, Jul 23, 2012 at 6:02 PM, Norbert Burger < > >>> [email protected] > >>>>> wrote: > >>>> > >>>>> Have you registered the JAR in your Pig script (for local mode) and > >>>>> also added it to PIG_CLASSPATH (for remote mode, to get it into the > >>>>> distributed cache)? > >>>>> > >>>>> Norbert > >>>>> > >>>>> On Mon, Jul 23, 2012 at 8:33 PM, Russell Jurney > >>>>> <[email protected]> wrote: > >>>>>> The email package is a part of Jython, I believe: > >>>>>> http://www.jython.org/docs/library/email.html > >>>>>> > >>>>>> However, when I 'import email' in udfs.py, I get this error: > >>>>>> > >>>>>> 2012-07-23 17:32:51,027 [main] ERROR > >>> org.apache.pig.tools.grunt.Grunt - > >>>>>> ERROR 1121: Python Error. Traceback (most recent call last): > >>>>>> File "/Users/rjurney/Collecting-Data/src/pig/udfs.py", line 1, in > >>>>> <module> > >>>>>> import email > >>>>>> ImportError: No module named email > >>>>>> > >>>>>> > >>>>>> How do I import and use built-in packages in Jython? > >>>>>> > >>>>>> -- > >>>>>> Russell Jurney twitter.com/rjurney [email protected] > >>>>> datasyndrome.com > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Russell Jurney twitter.com/rjurney [email protected] > >>>> datasyndrome.com > >>>> > >>> > >>> > >>> > >>> -- > >>> "...:::Aniket:::... Quetzalco@tl" > >>> > >> > >> > >> > >> -- > >> Russell Jurney twitter.com/rjurney [email protected] > datasyndrome.com > > > -- "...:::Aniket:::... Quetzalco@tl"
