Hi,
My bashrc file contains $PIG_CLASSPATH that points to the absolute path of
jython.jar
Global classpath $CLASSPATH in bashrc also does similarly.
I have also put the UDFs in the classpath.
I have tried these queries
Register 'udf.py' USING jython AS udf
Register 'exact/path/to/udf.py' USING jython AS udf
Register 'exact/path/to/udf.py' USING jython.jar AS udf
Register 'exact/path/to/udf.py' USING 'exact/path/to/jython.jar' AS udf
Register 'exact/path/to/udf.py' USING exact/path/to/jython.jar AS udf
REGISTER 'udf.py' USING jython AS udf
REGISTER 'exact/path/to/udf.py' USING jython AS udf
REGISTER 'exact/path/to/udf.py' USING jython.jar AS udf
REGISTER 'exact/path/to/udf.py' USING 'exact/path/to/jython.jar' AS udf
REGISTER 'exact/path/to/udf.py' USING exact/path/to/jython.jar AS udf
All of them are giving the same errors.
Here's a very basic script that I'm trying to run:
-------
REGISTER 'exact/path/to/udf.py' USING jython AS udf;
records = LOAD 'path/to/data' AS (input_line:chararray);
schema_records = FOREACH records GENERATE udf.split_into_words(input_line);
DUMP schema_records;
-----------
Here's the python udf:
@outputSchema("x:{t:(field1:chararray, field1:chararray)}")
def split_into_words(input_line)
line = input_line.strip()
words = line.split()
return (words[0], words[1])
-----------
Am I missing something here?
-----Original Message-----
From: Jonathan Coveney [mailto:jcove...@gmail.com]
Sent: Friday, January 07, 2011 10:03 PM
To: user@pig.apache.org
Subject: Re: Python UDF Internal Error when Registering
You need to point exactly to jython.jar. so /path/to/jython.jar has to be
there. The same is true of the UDF (you have to point to the exact file).
2011/1/7 <deepak....@wipro.com>
> Oh Okay. I thought Jython comes bundled with PIG 0.8.
>
> Anyway, I downloaded Jython_Installer-x.xx.jar and unpacked it in the
> PIG Classpath. Also kept my UDF in classpath as suggested. But I am
> still getting the exact same error.
>
> Any more ideas?
>
> -----Original Message-----
> From: Jonathan Coveney [mailto:jcove...@gmail.com]
> Sent: Friday, January 07, 2011 8:04 PM
> To: user@pig.apache.org
> Subject: Re: Python UDF Internal Error when Registering
>
> Make sure that in the register, you use the absolute path. Also make
> sure that both the jython.jar and udf asre in your classpath. A good
> test is to do that, then try and register in local mode. If that
> works, then try on your cluster. Always making sure that your
> classpath has those details
>
> Sent via BlackBerry
>
> -----Original Message-----
> From: <deepak....@wipro.com>
> Date: Fri, 7 Jan 2011 19:57:16
> To: <user@pig.apache.org>
> Reply-To: user@pig.apache.org
> Subject: Python UDF Internal Error when Registering
>
> Hi,
>
> I get an error when I try to register my python udf. Why is this happening?
>
> grunt> Register 'udf.py' USING jython AS udf
> 2011-01-07 19:39:31,818 [main] ERROR org.apache.pig.tools.grunt.Grunt
> - ERROR 2998: Unhandled internal error.
> org/python/util/PythonInterpreter
>
> The Logs say the following:
>
> Pig Stack Trace
> ---------------
> ERROR 2998: Unhandled internal error.
> org/python/util/PythonInterpreter
>
> java.lang.NoClassDefFoundError: org/python/util/PythonInterpreter
> at
> org.apache.pig.scripting.jython.JythonScriptEngine$Interpreter.<clinit>(JythonScriptEngine.java:54)
> at
> org.apache.pig.scripting.jython.JythonScriptEngine.registerFunctions(JythonScriptEngine.java:105)
> at org.apache.pig.PigServer.registerCode(PigServer.java:507)
> at
> org.apache.pig.tools.grunt.GruntParser.processRegister(GruntParser.java:381)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:421)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> at org.apache.pig.Main.run(Main.java:510)
> at org.apache.pig.Main.main(Main.java:107)
> Caused by: java.lang.ClassNotFoundException:
> org.python.util.PythonInterpreter
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> ... 10 more
>
> ======================================================================
> ==========
> Any ideas?
>
> Please do not print this email unless it is absolutely necessary.
>
> The information contained in this electronic message and any
> attachments to this message are intended for the exclusive use of the
> addressee(s) and may contain proprietary, confidential or privileged
> information. If you are not the intended recipient, you should not
> disseminate, distribute or copy this e-mail. Please notify the sender
> immediately and destroy all copies of this message and any attachments.
>
> WARNING: Computer viruses can be transmitted via email. The recipient
> should check this email and any attachments for the presence of
> viruses. The company accepts no liability for any damage caused by any
> virus transmitted by this email.
>
> www.wipro.com
>
Please do not print this email unless it is absolutely necessary.
The information contained in this electronic message and any attachments to
this message are intended for the exclusive use of the addressee(s) and may
contain proprietary, confidential or privileged information. If you are not the
intended recipient, you should not disseminate, distribute or copy this e-mail.
Please notify the sender immediately and destroy all copies of this message and
any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient should
check this email and any attachments for the presence of viruses. The company
accepts no liability for any damage caused by any virus transmitted by this
email.
www.wipro.com
