MOBIN created ZEPPELIN-4336:
-------------------------------
Summary: TimeoutLifecycleManagement has failed
Key: ZEPPELIN-4336
URL: https://issues.apache.org/jira/browse/ZEPPELIN-4336
Project: Zeppelin
Issue Type: Bug
Components: zeppelin-zengine
Affects Versions: 0.8.0
Reporter: MOBIN
Attachments: jstackLog.log
We have encountered a failure of TimeoutLifecycleManagement before, and
combined the relevant code according to ZEPPELIN-4133, but the problem still
appears.I upload full log
I found that the related process of TimeoutLifecycleManagement was blocked,and
Waiting for {color:#FF0000}0x00000002df8365c0{color} lock
{code:java}
"pool-3-thread-1" #33 prio=5 os_prio=0 tid=0x00007f86d970f800 nid=0x29ef
waiting for monitor entry [0x00007f84ff42c000]"pool-3-thread-1" #33 prio=5
os_prio=0 tid=0x00007f86d970f800 nid=0x29ef waiting for monitor entry
[0x00007f84ff42c000] java.lang.Thread.State: BLOCKED (on object monitor) at
org.apache.zeppelin.interpreter.ManagedInterpreterGroup.close(ManagedInterpreterGroup.java:89)
- waiting to lock <0x00000002df8365c0> (a
org.apache.zeppelin.interpreter.ManagedInterpreterGroup) at
org.apache.zeppelin.interpreter.lifecycle.TimeoutLifecycleManager.lambda$new$0(TimeoutLifecycleManager.java:50)
at
org.apache.zeppelin.interpreter.lifecycle.TimeoutLifecycleManager$$Lambda$64/1972143064.run(Unknown
Source) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
thread with 0x00000002df8365c0 lock is doing some close interpreter actions,
but it seems to be "blocked", causing 0x00000002df8365c0 lock to never be
released.
{code:java}
"qtp1659791576-43255" #43255 prio=5 os_prio=0 tid=0x00007f8170124800 nid=0x394d
runnable [0x00007f816e8e6000]"qtp1659791576-43255" #43255 prio=5 os_prio=0
tid=0x00007f8170124800 nid=0x394d runnable [0x00007f816e8e6000]
java.lang.Thread.State: RUNNABLE at
java.net.SocketInputStream.socketRead0(Native Method) at
java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at
java.net.SocketInputStream.read(SocketInputStream.java:170) at
java.net.SocketInputStream.read(SocketInputStream.java:141) at
java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at
java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at
java.io.BufferedInputStream.read(BufferedInputStream.java:345) - locked
<0x00000002df85e828> (a java.io.BufferedInputStream) at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) at
org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_shutdown(RemoteInterpreterService.java:394)
at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.shutdown(RemoteInterpreterService.java:382)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess$4.call(RemoteInterpreterManagedProcess.java:240)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess$4.call(RemoteInterpreterManagedProcess.java:237)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:135)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess.stop(RemoteInterpreterManagedProcess.java:237)
at
org.apache.zeppelin.interpreter.ManagedInterpreterGroup.close(ManagedInterpreterGroup.java:109)
- locked <0x00000002df8365c0> (a
org.apache.zeppelin.interpreter.ManagedInterpreterGroup) at
org.apache.zeppelin.interpreter.ManagedInterpreterGroup.close(ManagedInterpreterGroup.java:91)
- locked <0x00000002df8365c0> (a
org.apache.zeppelin.interpreter.ManagedInterpreterGroup) at
org.apache.zeppelin.interpreter.InterpreterSetting.close(InterpreterSetting.java:485)
at
org.apache.zeppelin.interpreter.InterpreterSettingManager.close(InterpreterSettingManager.java:979)
at
org.apache.zeppelin.rest.InterpreterRestApi.restartSetting(InterpreterRestApi.java:249)
at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76)
at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$$Lambda$132/1114616259.invoke(Unknown
Source) at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191)
at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415)
at org.glassfish.jersey.server.model.ResourceMethodInvoke
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
