Hi, - ERROR 2997: Unable to recreate exception from backed error: > java.io.IOException: Deserialization error: could not instantiate > 'org.apache.pig.scripting.jython.JythonFunction' with arguments
'[/transfer/marek/poc123-code/poc123-udf.py, parse_time_local]' This is a bug in Pig-0.10. Please refer to PIg-2761 and its related jiras: https://issues.apache.org/jira/browse/PIG-2761 https://issues.apache.org/jira/browse/PIG-2745 Thanks, Cheolsoo On Tue, Jun 19, 2012 at 4:28 PM, Chris Diehl <[email protected]> wrote: > Hi Marek, > > I wish I could offer a solution. I tried all sorts of permutations and to > no avail. I retreated back to our base install of Pig 0.8 where Jython UDFs > work for me. > Here's a list of things I've tried: > > 1) Setting JYTHON_HOME to the Jython path > 2) Setting PIG_CLASSPATH to <Jython path>/jython.jar > 3) Copying jython.jar to <Pig path>/lib > 4) Using Jython 2.5.0 and 2.5.2 > > I can't afford to burn any more time right now looking into this. I'm > hoping the problem can be uncovered. Jython UDF support is critical for my > productivity. > > Chris > > On Tue, Jun 19, 2012 at 2:40 PM, Marek Dudek <[email protected] > >wrote: > > > Chris, > > > > I can confirm the problem; just run into it myself :( > > > > In my case I was able to run my code succesfully on MapR cluster (hadoop > > 0.20.2), with Pig 0.9.1 and Jython 2.5.2. > > Now, I am getting same "Deserialization error" after upgrading to Pig > > 0.10.0 > > (see log snippets below). > > > > Any ideas/luck so far? Is there anything I can try/test to help? > > > > Marek > > > > > > > > ----------------- > > 2012-06-19 16:24:54,470 [main] INFO org.apache.pig.Main - Apache Pig > > version 0.10.0 (r1328203) compiled Apr 20 2012, 05:54:07 > > 2012-06-19 16:24:54,470 [main] INFO org.apache.pig.Main - Logging error > > messages to: /transfer/marek/poc123-code/pig_1340141094466.log > > (...) > > 2012-06-19 16:26:27,484 [main] ERROR > > org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed! > > 2012-06-19 16:26:27,488 [main] INFO > > org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics: > > > > HadoopVersion PigVersion UserId StartedAt > FinishedAt > > Features > > 0.20.2-dev 0.10.0 marek 2012-06-19 16:25:13 > 2012-06-19 > > 16:26:27 FILTER > > > > Failed! > > (...) > > > > 2012-06-19 16:26:27,488 [main] INFO > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLaunch > > er - Failed! > > 2012-06-19 16:26:27,544 [main] ERROR > org.apache.pig.tools.grunt.GruntParser > > - ERROR 2997: Unable to recreate exception from backed error: > > java.io.IOException: Deserialization error: could not instantiate > > 'org.apache.pig.scripting.jython.JythonFunction' with arguments > > '[/transfer/marek/poc123-code/poc123-udf.py, parse_time_local]' > > at > > > > > org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java: > > 55) > > at > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBa > > se.setup(PigGenericMapBase.java:177) > > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:654) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) > > at org.apache.hadoop.mapred.Child$4.run(Child.java:270) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at > > > > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja > > va:1109) > > at org.apache.hadoop.mapred.Child.main(Child.java:264) > > Caused by: java.lang.RuntimeException: could not instantiate > > 'org.apache.pig.scripting.jython.JythonFunction' with arguments > > '[/transfer/marek/poc123-code/poc123-udf.py, parse_time_local]' > > at > > > org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:553) > > at > > > > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperat > > ors.POUserFunc.instantiateFunc(POUserFunc.java:107) > > at > > > > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperat > > ors.POUserFunc.readObject(POUserFunc.java:439) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39 > > ) > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl > > .java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:969) > > at > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) > > at > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > > at > > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > > at > java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) > > at java.util.ArrayList.readObject(ArrayList.java:593) > > at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl > > .java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:969) > > at > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) > > at > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > > at > > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > > at > java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) > > at java.util.HashMap.readObject(HashMap.java:1030) > > at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl > > .java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:969) > > at > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) > > at > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > > at > > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > > at > > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) > > at > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) > > at > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > > at > > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > > at > > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) > > at > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) > > at > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > > at > > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > > at > java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) > > at java.util.ArrayList.readObject(ArrayList.java:593) > > at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl > > .java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:969) > > at > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) > > at > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > > at > > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > > at > > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) > > at > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) > > at > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > > at > > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > > at > java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) > > at java.util.HashMap.readObject(HashMap.java:1029) > > at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl > > .java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:969) > > at > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) > > at > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > > at > > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > > at > > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) > > at > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) > > at > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > > at > > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > > at > > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) > > at > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) > > at > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > > at > > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > > at > java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) > > at > > > > > org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java: > > 53) > > ... 9 more > > Caused by: java.lang.reflect.InvocationTargetException > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > > Method) > > at > > > > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAcces > > sorImpl.java:39) > > at > > > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstruc > > torAccessorImpl.java:27) > > at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > > at > > > org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:521) > > ... 77 more > > Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0: > > Could not initialize: /transfer/marek/poc123-code/poc123-udf.py > > at > > > > > org.apache.pig.scripting.jython.JythonFunction.<init>(JythonFunction.java:87 > > ) > > ... 82 more > > Caused by: java.lang.IllegalStateException: Could not initialize > > interpreter > > (from file system or classpath) with > > /transfer/marek/poc123-code/poc123-udf.py > > at > > > > > org.apache.pig.scripting.ScriptEngine.getScriptAsStream(ScriptEngine.java:14 > > 5) > > at > > > > > org.apache.pig.scripting.jython.JythonScriptEngine$Interpreter.init(JythonSc > > riptEngine.java:153) > > at > > > > > org.apache.pig.scripting.jython.JythonScriptEngine.getFunction(JythonScriptE > > ngine.java:328) > > at > > > > > org.apache.pig.scripting.jython.JythonFunction.<init>(JythonFunction.java:52 > > ) > > ... 82 more > > > > >
