[
https://issues.apache.org/jira/browse/PIG-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheolsoo Park updated PIG-3602:
-------------------------------
Attachment: unit_test.txt
[~rohini], not sure what's going on, but "ant test-tez" hangs after
TestCombiner with the patch.
I am attaching the thread dump that I took on my laptop. I see the following
stack trace, so it seems related to the shutdown hook-
{code}
[junit] "main" prio=5 tid=7fb11f800800 nid=0x1031f3000 in Object.wait()
[1031f2000]
[junit] java.lang.Thread.State: WAITING (on object monitor)
[junit] at java.lang.Object.wait(Native Method)
[junit] - waiting on <788320a30> (a
org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager$1)
[junit] at java.lang.Thread.join(Thread.java:1225)
[junit] - locked <788320a30> (a
org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager$1)
[junit] at java.lang.Thread.join(Thread.java:1278)
[junit] at
java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:79)
[junit] at
java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:24)
[junit] at java.lang.Shutdown.runHooks(Shutdown.java:79)
[junit] at java.lang.Shutdown.sequence(Shutdown.java:123)
[junit] at java.lang.Shutdown.exit(Shutdown.java:168)
[junit] - locked <7faf9fa58> (a java.lang.Class for java.lang.Shutdown)
[junit] at java.lang.Runtime.exit(Runtime.java:90)
[junit] at java.lang.System.exit(System.java:920)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:912)
{code}
> Tear down TezSessions when Pig exits
> ------------------------------------
>
> Key: PIG-3602
> URL: https://issues.apache.org/jira/browse/PIG-3602
> Project: Pig
> Issue Type: Sub-task
> Components: tez
> Affects Versions: tez-branch
> Reporter: Cheolsoo Park
> Assignee: Rohini Palaniswamy
> Fix For: tez-branch
>
> Attachments: PIG-3602-1.patch, unit_test.txt
>
>
> Currently, Pig reuses AMs via TezSession, but they are not shut down when Pig
> exits. There are two problems that I noticed with this-
> # Tez jobs are not marked as finished until TezSessions are expired after
> timeout. Since they occupy task slots, it blocks submitting jobs.
> # ant clean test-tez leaves orphan processes (DAGAppMaster).
> Ideally, TezSession should be kept alive while Pig runs but tore down when
> Pig exits.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
