[
https://issues.apache.org/jira/browse/TEZ-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246415#comment-14246415
]
Jeff Zhang commented on TEZ-1827:
---------------------------------
[~ashutoshc], This is caused by TEZ-1620 ( we would wait for all the apps to be
finished before shutting down MiniTezCluster). While in hive, it looks like you
didn't stop the TezSession before shutting down MiniTezCluster, so the Tez-AM
will be alive until it time-out. This is what I see in the logs, by default the
default interval is 5 minutes. Is it possible for hive to stop TezSession
before shutting down MiniTezCluster. If couldn't, we could provide a method
like forceStop which won't wait for apps to be finished. And you can use this
method to shutdown MiniTezCluster in MiniTezShim.
{code}
2014-12-14 23:20:18,660 INFO [AsyncDispatcher event handler] app.DAGAppMaster:
Waiting for next DAG to be submitted.
2014-12-14 23:20:18,804 INFO [IPC Server handler 0 on 59547] ipc.Server:
Served: getDAGStatus queueTime= 0 procesingTime= 0
2014-12-14 23:20:18,807 INFO [IPC Server handler 0 on 59547] ipc.Server:
Served: getDAGStatus queueTime= 1 procesingTime= 1
2014-12-14 23:20:18,935 INFO [AMRM Callback Handler Thread] util.RackResolver:
Resolved localhost to /default-rack
2014-12-14 23:20:19,120 INFO [DelayedContainerManager]
rm.YarnTaskSchedulerService: No taskRequests. Container's idle timeout delay
expired or is new. Releasing container,
containerId=container_1418627911923_0002_01_000007,
containerExpiryTime=1418628023935, idleTimeout=5000, taskRequestsCount=0,
heldContainers=4, delayedContainers=3, isNew=true
2014-12-14 23:20:19,120 INFO [DelayedContainerManager]
rm.YarnTaskSchedulerService: Releasing unused container:
container_1418627911923_0002_01_000007
2014-12-14 23:20:19,938 INFO [AMRM Callback Handler Thread]
rm.YarnTaskSchedulerService: Ignoring unknown container:
container_1418627911923_0002_01_000007
2014-12-14 23:25:33,912 INFO [Timer-0] app.DAGAppMaster: Session timed out,
lastDAGCompletionTime=1418628018660 ms, sessionTimeoutInterval=300000 ms
2014-12-14 23:25:33,912 INFO [Timer-0] rm.TaskSchedulerEventHandler:
TaskScheduler notified that it should unregister from RM
2014-12-14 23:25:33,912 INFO [Timer-0] app.DAGAppMaster: No current running
DAG, shutting down the AM
2014-12-14 23:25:33,912 INFO [Timer-0] app.DAGAppMaster:
DAGAppMasterShutdownHandler invoked
2014-12-14 23:25:33,912 INFO [Timer-0] app.DAGAppMaster: Handling DAGAppMaster
shutdown
2014-12-14 23:25:33,913 INFO [AMShutdownThread] app.DAGAppMaster: Sleeping for
5 seconds before shutting down
2014-12-14 23:25:38,914 INFO [AMShutdownThread] app.DAGAppMaster: Calling stop
for all the services
2014-12-14 23:25:38,915 INFO [AMShutdownThread] history.HistoryEventHandler:
Stopping HistoryEventHandler
{code}
> MiniTezCluster takes 10 minutes to shut down
> --------------------------------------------
>
> Key: TEZ-1827
> URL: https://issues.apache.org/jira/browse/TEZ-1827
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.5.2
> Reporter: Ashutosh Chauhan
> Priority: Minor
>
> Always reproducible
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
