[
https://issues.apache.org/jira/browse/PIG-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohini Palaniswamy updated PIG-3937:
------------------------------------
Attachment: PIG-3937-1.patch
Gave a quick try. Seems to be a Tez bug with union vertex stuck on getting
input.
{code}
"main" prio=5 tid=7f9866000000 nid=0x10eacd000 waiting on condition [10eacb000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <7df02ce28> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
org.apache.tez.runtime.library.shuffle.common.impl.ShuffleManager.getNextInput(ShuffleManager.java:597)
at
org.apache.tez.runtime.library.common.readers.ShuffledUnorderedKVReader.moveToNextInput(ShuffledUnorderedKVReader.java:172)
at
org.apache.tez.runtime.library.common.readers.ShuffledUnorderedKVReader.next(ShuffledUnorderedKVReader.java:113)
at
org.apache.pig.backend.hadoop.executionengine.tez.operators.POShuffledValueInputTez.getNextTuple(POShuffledValueInputTez.java:113)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:300)
at
org.apache.pig.backend.hadoop.executionengine.tez.POStoreTez.getNextTuple(POStoreTez.java:113)
at
org.apache.pig.backend.hadoop.executionengine.tez.PigProcessor.runPipeline(PigProcessor.java:273)
at
org.apache.pig.backend.hadoop.executionengine.tez.PigProcessor.run(PigProcessor.java:172)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307)
at
org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:581)
{code}
> Use unsorted shuffle in Union
> -----------------------------
>
> Key: PIG-3937
> URL: https://issues.apache.org/jira/browse/PIG-3937
> Project: Pig
> Issue Type: Sub-task
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: tez-branch
>
> Attachments: PIG-3937-1.patch
>
>
> TEZ-661 provides unsorted shuffle
--
This message was sent by Atlassian JIRA
(v6.2#6252)
