Root cause for this problem was https://issues.apache.org/jira/browse/MAPREDUCE-967 backward incompatibility.
Hadoop unpacks job.jar and puts jobCacheDir on classpath and hence jython is not able to find its jar (job.jar) on classpath (as its unpacked). With MAPREDUCE-967, hadoop puts job.jar on classpath and hence jython finds its jar (=job.jar) and updates the sys.path. Stan, I think if you put jython.jar on classpath for TT, it will work. (you have to restart TT). PIG-2010 is another possible choice. Thanks, Aniket On Mon, Mar 12, 2012 at 1:16 AM, Aniket Mokashi <[email protected]> wrote: > This looks like a bug to me. Jython cuts out jython.jar location from > classpath and appends Lib to it. But, in general on TT jython,jar is not > available and its "merged" into job.jar by pig. Hence, imports will always > fail. > > ~Aniket > > > On Mon, Mar 12, 2012 at 12:54 AM, Aniket Mokashi <[email protected]>wrote: > >> I spent some time debugging this. The reason is -- >> >> Sys.path on TT for jython is - ['__classpath__', '__pyclasspath__/'] >> >> And for client is ['', '/users/lib/Lib', >> '/users/lib/jython_simplejson.jar/Lib', '__classpath__', '__pyclasspath__/'] >> >> I am still figuring out why CLASSPATH (java.class.path property) on >> tasktracker doesn't have job.jar on it. Hints anyone? >> >> >> Thanks, >> >> Aniket >> >> On Tue, Oct 18, 2011 at 9:54 AM, Stan Rosenberg < >> [email protected]> wrote: >> >>> Hi Clay, >>> >>> I am running a very recent version (one that contains this patch) of >>> pig which was compiled from the trunk. >>> How can I examine the jar file to determine which jython modules have >>> been added? >>> >>> Thanks, >>> >>> stan >>> >>> On Tue, Oct 18, 2011 at 12:38 PM, Clay B. <[email protected]> wrote: >>> > Hi Stan, >>> > >>> > I believe you are hitting >>> https://issues.apache.org/jira/browse/PIG-1824 >>> > >>> > -Clay >>> > >>> > On Mon, 17 Oct 2011, Stan Rosenberg wrote: >>> > >>> >> Hi, >>> >> >>> >> What's a proper way to deploy python udfs? I've dropped the latest >>> >> version of jython.jar in $PIG_HOME/lib. >>> >> Things work in "local" mode, but when I run on a cluster, built-in >>> >> python modules cannot be found. E.g., urlparse cannot be located: >>> >> >>> >> ImportError: No module named urlparse >>> >> >>> >> at >>> org.python.core.PyException.fillInStackTrace(PyException.java:70) >>> >> at java.lang.Throwable.<init>(Throwable.java:181) >>> >> at java.lang.Exception.<init>(Exception.java:29) >>> >> at java.lang.RuntimeException.<init>(RuntimeException.java:32) >>> >> at org.python.core.PyException.<init>(PyException.java:46) >>> >> at org.python.core.PyException.<init>(PyException.java:43) >>> >> at org.python.core.PyException.<init>(PyException.java:61) >>> >> at org.python.core.Py.ImportError(Py.java:290) >>> >> at org.python.core.imp.import_first(imp.java:750) >>> >> at org.python.core.imp.import_name(imp.java:834) >>> >> at org.python.core.imp.importName(imp.java:884) >>> >> at >>> org.python.core.ImportFunction.__call__(__builtin__.java:1220) >>> >> at org.python.core.PyObject.__call__(PyObject.java:357) >>> >> at org.python.core.__builtin__.__import__(__builtin__.java:1173) >>> >> at org.python.core.imp.importFromAs(imp.java:978) >>> >> at org.python.core.imp.importFrom(imp.java:954) >>> >> at org.python.pycode._pyx3.f$0(udfs.py:40) >>> >> at org.python.pycode._pyx3.call_function(udfs.py) >>> >> at org.python.core.PyTableCode.call(PyTableCode.java:165) >>> >> at org.python.core.PyCode.call(PyCode.java:18) >>> >> at org.python.core.Py.runCode(Py.java:1261) >>> >> at >>> >> org.python.util.PythonInterpreter.execfile(PythonInterpreter.java:235) >>> >> at >>> >> >>> org.apache.pig.scripting.jython.JythonScriptEngine$Interpreter.execfile(JythonScriptEngine.java:176) >>> >> ... 15 more >>> >> >>> >> Thanks, >>> >> >>> >> stan >>> >> >>> > >>> >> >> >> >> -- >> "...:::Aniket:::... Quetzalco@tl" >> > > > > -- > "...:::Aniket:::... Quetzalco@tl" > -- "...:::Aniket:::... Quetzalco@tl"
