Kuhu Shukla created TEZ-3587:
--------------------------------
Summary: Fetcher fetchInputs() can NPE on srcAttempt due to
missing entry in pathToAttemptMap
Key: TEZ-3587
URL: https://issues.apache.org/jira/browse/TEZ-3587
Project: Apache Tez
Issue Type: Sub-task
Reporter: Kuhu Shukla
Assignee: Kuhu Shukla
With Tez Shuffle Handler (or composite fetch on), unordered fetcher can fail to
populate the pathToAttemptsMap with the coorect value of "partition" member of
the PathPartition object. This can cause an NPE.
{code}
[main] examples.HashJoinExample (TezExampleBase.java:runDag(190)) - DAG
diagnostics: [Vertex failed, vertexName=joiner, vertexId=vertex_123_0001_2_02,
diagnostics=[Task failed, taskId=task_123_0001_2_02_000001,
diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
failure ) : java.lang.NullPointerException
at
org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:893)
at
org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:597)
at
org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:486)
at
org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:284)
at
org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:76)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
, errorMessage=Fetch failed:java.lang.NullPointerException
at
org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:893)
at
org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:597)
at
org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:486)
at
org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:284)
at
org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:76)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
