[
https://issues.apache.org/jira/browse/PIG-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555970#comment-14555970
]
Hari Sekhon commented on PIG-4418:
----------------------------------
Jobs failures were intermittent and seemed non-deterministic so I tried running
re-running these jobs on mapreduce backend instead of tez and so far it's been
much more reliable successfully indexing billions of docs for many jobs without
any more failures with everything else remaining the same.
It may just be coincidence but I started having these problems around the time
of upgrade from HDP 2.2.0.0 to HDP 2.2.4.2.
I'm actually migrating this pipeline from Pig to Spark so I'm not intending to
do any more testing on this right now given it's working reliably when back on
mapreduce.
> NullPointerException in JVMReuseImpl
> ------------------------------------
>
> Key: PIG-4418
> URL: https://issues.apache.org/jira/browse/PIG-4418
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.14.0
> Reporter: Jeff Zhang
> Assignee: Rohini Palaniswamy
> Priority: Critical
> Fix For: 0.15.0
>
> Attachments: PIG-4418-1.patch, PIG-4418-2.patch
>
>
> {code}
> 2015-02-13 15:17:11,067 INFO [TezChild] task.TezTaskRunner: Encounted an
> error while executing task: attempt_1423730493153_0019_1_04_000002_0
> java.lang.NullPointerException
> at org.apache.pig.JVMReuseImpl.cleanupStaticData(JVMReuseImpl.java:46)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.close(PigProcessor.java:175)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.close(LogicalIOProcessorRuntimeTask.java:338)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:166)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
