ganguowei created ZEPPELIN-3193:
-----------------------------------
Summary: Zeppelin 0.7.3 Job run deadlock
Key: ZEPPELIN-3193
URL: https://issues.apache.org/jira/browse/ZEPPELIN-3193
Project: Zeppelin
Issue Type: Bug
Components: Interpreters
Affects Versions: 0.7.3
Reporter: ganguowei
Fix For: 0.7.4
Attachments: dump2.log
dump :
Found one Java-level deadlock:
=============================
"qtp2008362258-12861":
waiting to lock monitor 0x00007fc44400da28 (object 0x00000000c036c8e0, a
java.util.concurrent.ConcurrentHashMap),
which is held by "DefaultQuartzScheduler_Worker-5"
"DefaultQuartzScheduler_Worker-5":
waiting to lock monitor 0x00007fc458028438 (object 0x00000000c0bd0548, a
org.apache.zeppelin.notebook.Note),
which is held by "DefaultQuartzScheduler_Worker-2"
"DefaultQuartzScheduler_Worker-2":
waiting to lock monitor 0x00007fc44400da28 (object 0x00000000c036c8e0, a
java.util.concurrent.ConcurrentHashMap),
which is held by "DefaultQuartzScheduler_Worker-5"
Java stack information for the threads listed above:
===================================================
"qtp2008362258-12861":
at
org.apache.zeppelin.interpreter.InterpreterSettingManager.get(InterpreterSettingManager.java:981)
- waiting to lock <0x00000000c036c8e0> (a
java.util.concurrent.ConcurrentHashMap)
at
org.apache.zeppelin.interpreter.InterpreterSettingManager.getInterpreterSettings(InterpreterSettingManager.java:450)
at
org.apache.zeppelin.notebook.Notebook.getJobListByUnixTime(Notebook.java:852)
at
org.apache.zeppelin.socket.NotebookServer.unicastNoteJobInfo(NotebookServer.java:532)
at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:334)
at
org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(NotebookSocket.java:59)
at
org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextMessage(JettyListenerEventDriver.java:128)
at
org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69)
at
org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:65)
at
org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextFrame(JettyListenerEventDriver.java:122)
at
org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:161)
at
org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:309)
at
org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:214)
at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:220)
at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:258)
at
org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:632)
at
org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:480)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
"DefaultQuartzScheduler_Worker-5":
at org.apache.zeppelin.notebook.Note.stopDelayedPersistTimer(Note.java:789)
- waiting to lock <0x00000000c0bd0548> (a org.apache.zeppelin.notebook.Note)
at org.apache.zeppelin.notebook.Note.persist(Note.java:725)
at
org.apache.zeppelin.socket.NotebookServer$ParagraphListenerImpl.afterStatusChange(NotebookServer.java:2070)
at org.apache.zeppelin.scheduler.Job.setStatus(Job.java:149)
at org.apache.zeppelin.notebook.Paragraph.jobAbort(Paragraph.java:473)
at org.apache.zeppelin.scheduler.Job.abort(Job.java:240)
at
org.apache.zeppelin.interpreter.InterpreterSettingManager.stopJobAllInterpreter(InterpreterSettingManager.java:965)
at
org.apache.zeppelin.interpreter.InterpreterSettingManager.restart(InterpreterSettingManager.java:942)
- locked <0x00000000c036c8e0> (a java.util.concurrent.ConcurrentHashMap)
at
org.apache.zeppelin.interpreter.InterpreterSettingManager.restart(InterpreterSettingManager.java:956)
at org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebook.java:907)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
- locked <0x00000000c07232c8> (a java.lang.Object)
"DefaultQuartzScheduler_Worker-2":
at
org.apache.zeppelin.interpreter.InterpreterSettingManager.get(InterpreterSettingManager.java:981)
- waiting to lock <0x00000000c036c8e0> (a
java.util.concurrent.ConcurrentHashMap)
at
org.apache.zeppelin.interpreter.InterpreterSettingManager.getInterpreterSettings(InterpreterSettingManager.java:450)
at
org.apache.zeppelin.interpreter.InterpreterFactory.getInterpreter(InterpreterFactory.java:382)
at org.apache.zeppelin.notebook.Note.run(Note.java:616)
at org.apache.zeppelin.notebook.Note.runAll(Note.java:596)
at org.apache.zeppelin.notebook.Note.runAll(Note.java:587)
- locked <0x00000000c0bd0548> (a org.apache.zeppelin.notebook.Note)
at org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebook.java:885)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
- locked <0x00000000c0722830> (a java.lang.Object)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
