brucemen711 opened a new issue #4516:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4516
I set spark task (spark streaming) to 100 retries if job failed. But i got
task hanging when zookeeper OOM. The log is below:
`java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
at
org.apache.jute.BinaryOutputArchive.<init>(BinaryOutputArchive.java:33)
at
org.apache.jute.BinaryOutputArchive.getArchive(BinaryOutputArchive.java:38)
at org.apache.zookeeper.ClientCnxn$Packet.createBB(ClientCnxn.java:298)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:115)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
[WARN] 2021-01-21 03:04:12.958 org.apache.zookeeper.ClientCnxn:[1108] -
Client session timed out, have not heard from server in 27345ms for sessionid
0x1b7b1c3b7000d
[INFO] 2021-01-21 03:04:12.959
org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[164] -
develop mode is: false
[INFO] 2021-01-21 03:04:12.960 org.apache.zookeeper.ClientCnxn:[1156] -
Client session timed out, have not heard from server in 27345ms for sessionid
0x1b7b1c3b7000d, closing socket connection and attempting reconnect
[INFO] 2021-01-21 03:04:12.961
org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[182] - exec
local path: /tmp/dolphinscheduler/exec/process/3/7/283/870 cleared.
[INFO] 2021-01-21 03:04:13.060
org.apache.curator.framework.state.ConnectionStateManager:[251] - State change:
SUSPENDED
[INFO] 2021-01-21 03:04:13.061
org.apache.curator.framework.state.ConnectionStateManager:[251] - State change:
SUSPENDED`
**Which version of DolphinScheduler:**
-[1.3.4]
**Additional context**
Add any other context about the problem here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
