[
https://issues.apache.org/jira/browse/TEZ-2475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14560220#comment-14560220
]
Jeff Zhang commented on TEZ-2475:
---------------------------------
[~sseth] Is it related to TEZ-1802 ? I see the following message at the end of
logs
{noformat}
2015-05-26 13:10:23,128 WARN [LocalContainerLauncher-SubTaskRunner]
token.Token (Token.java:getClassForIdentifier(121)) - Cannot find class for
token kind tez.job
2015-05-26 13:10:23,128 WARN [LocalContainerLauncher-SubTaskRunner]
token.Token (Token.java:getClassForIdentifier(121)) - Cannot find class for
token kind tez.job
Kind: tez.job, Service: application_1432638619418_0001, Ident: 1e 61 70 70
6c 69 63 61 74 69 6f 6e 5f 31 34 33 32 36 33 38 36 31 39 34 31 38 5f 30 30 30 31
2015-05-26 13:12:23,155 INFO [cascading shutdown hooks] flow.Flow
(BaseFlow.java:logInfo(1433)) - [20150526-131019-64BE78...] shutdown hook
calling stop on flow
2015-05-26 13:12:23,155 INFO [cascading shutdown hooks] flow.Flow
(BaseFlow.java:logInfo(1433)) - [20150526-131019-64BE78...] stopping all jobs
2015-05-26 13:12:23,156 INFO [cascading shutdown hooks] flow.Flow
(BaseFlow.java:logInfo(1433)) - [20150526-131019-64BE78...] stopping: (1/1)
...26-131019-64BE78F366.tcsv
{noformat}
> Tez local mode hanging in big testsuite
> ---------------------------------------
>
> Key: TEZ-2475
> URL: https://issues.apache.org/jira/browse/TEZ-2475
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.7.0, 0.6.1
> Reporter: André Kelpe
> Attachments: 2015-05-21_15-55-20_buildLog.log.gz
>
>
> we have a big test suite for lingual, our SQL layer for cascading. We are
> trying very hard to make it work correctly on Tez, but I am stuck:
> The setup is a huge suite of SQL based tests (6000+), which are being
> executed in order in local mode. At certain moments the whole process just
> stops. Nothing gets executed any longer. This is not all the time, but quite
> often. Note that it is not happening at the same line of code, more at
> random, which makes it quite complex to debug.
> What I am seeing, is these kind of stacktraces in the middle of the run:
> 2015-05-21 16:07:42,413 ERROR [TaskHeartbeatThread] task.TezTaskRunner
> (TezTaskRunner.java:reportError(333)) - TaskReporter reported error
> java.lang.InterruptedException
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2188)
> at
> org.apache.tez.runtime.task.TaskReporter$HeartbeatCallable.call(TaskReporter.java:187)
> at
> org.apache.tez.runtime.task.TaskReporter$HeartbeatCallable.call(TaskReporter.java:118)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> This looks like it could be related to the hang, but the hang is not
> happening immediately afterwards, but some time later.
> I have gone through quite a few JIRAs and saw that there were problems with
> locks and hanging threads before, which should be fixed, but it still happens.
> I have tried 0.6.1 and 0.7.0. Both show the same behaviour.
> This gist contains a thread dump of a hanging build:
> https://gist.github.com/fs111/1ee44469bf5cc31e5a52
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
