[
https://issues.apache.org/jira/browse/PIG-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16841368#comment-16841368
]
Adam Szita commented on PIG-5387:
---------------------------------
So as far as I understand:
* in the testing part you are injecting an URLClassLoader if we're running
java11. Quite hacky but not worse than the current implementation which uses
reflection to invoke addURL on URLClassLoader;
* in Operator class I think using readObject like this will be fine, but
please add javadoc or comments to the readObject method that includes some
reference to the JVM bug. We don't want ppl to remove it by mistake :)
Having this in mind [^PIG-5387_2.patch] looks good to me.
> Test failures on JRE 11
> -----------------------
>
> Key: PIG-5387
> URL: https://issues.apache.org/jira/browse/PIG-5387
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Nandor Kollar
> Assignee: Nandor Kollar
> Priority: Major
> Attachments: PIG-5387_1.patch, PIG-5387_2.patch
>
>
> I tried to compile Pig with JDK 8 and execute the test with Java 11, and
> faced with several test failures. For example TestCommit#testCheckin2 failed
> with the following exception:
> {code}
> 2019-05-08 16:06:09,712 WARN [Thread-108] mapred.LocalJobRunner
> (LocalJobRunner.java:run(590)) - job_local1000317333_0003
> java.lang.Exception: java.io.IOException: Deserialization error: null
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:552)
> Caused by: java.io.IOException: Deserialization error: null
> at
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:62)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.setup(PigGenericMapBase.java:183)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.lang.NullPointerException
> at org.apache.pig.impl.plan.Operator.hashCode(Operator.java:106)
> at java.base/java.util.HashMap.hash(HashMap.java:339)
> at java.base/java.util.HashMap.readObject(HashMap.java:1461)
> at
> java.base/jdk.internal.reflect.GeneratedMethodAccessor12.invoke(Unknown
> Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1160)
> {code}
> It deserialization of one of the map plan failed, it appears we ran into
> [JDK-8201131|https://bugs.openjdk.java.net/browse/JDK-8201131]. I seems that
> the workaround in the issue report works, adding a readObject method to
> org.apache.pig.impl.plan.Operator:
> {code}
> private void readObject(ObjectInputStream in) throws
> ClassNotFoundException, IOException {
> in.defaultReadObject();
> }
> {code}
> solves the problem, however I'm not sure that this is the optimal solution.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
