Bharath Vissapragada has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11317 )

Change subject: IMPALA-7483: abort stuck impalad/catalogd on JVM deadlock
......................................................................


Patch Set 2:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/11317/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/11317/2//COMMIT_MSG@22
PS2, Line 22: findDeadlockedThreads
Maybe worth injecting a deadlock locally and see if findDeadlockedThreads() 
works as expected too (if you have not done that already).


http://gerrit.cloudera.org:8080/#/c/11317/2/be/src/util/jni-util.cc
File be/src/util/jni-util.cc:

http://gerrit.cloudera.org:8080/#/c/11317/2/be/src/util/jni-util.cc@28
PS2, Line 28: 60
IMO, configuring it something like 300s is reasonable too since VM seems to be 
executing a safepoint for this. Also since we see deadlocks very rarely.


http://gerrit.cloudera.org:8080/#/c/11317/2/be/src/util/jni-util.cc@29
PS2, Line 29:     "Interval between JVM deadlock checks. If set to 0 or a 
negative value, deadlock "
nit: mark it (Advanced)?


http://gerrit.cloudera.org:8080/#/c/11317/2/fe/src/main/java/org/apache/impala/util/JvmPauseMonitor.java
File fe/src/main/java/org/apache/impala/util/JvmPauseMonitor.java:

http://gerrit.cloudera.org:8080/#/c/11317/2/fe/src/main/java/org/apache/impala/util/JvmPauseMonitor.java@176
PS2, Line 176:           return;
not related to your change but I just noticed this, can we log this as an error 
with exception trace so that we know if the pause monitor exits.


http://gerrit.cloudera.org:8080/#/c/11317/2/fe/src/main/java/org/apache/impala/util/JvmPauseMonitor.java@217
PS2, Line 217:         LOG.fatal("Aborting because of deadlocked threads in 
JVM.");
Should we dump the entire thread stacks (not just the deadlocked ones) and 
other JVM state, say JMX output at this point? (gives heap usage etc. if 
needed). Also any other stuff like /metrics etc?



--
To view, visit http://gerrit.cloudera.org:8080/11317
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If46f879e7ab9eeead0c6e9a09844e1356012898d
Gerrit-Change-Number: 11317
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <[email protected]>
Gerrit-Reviewer: Bharath Vissapragada <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Philip Zeyliger <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
Gerrit-Reviewer: Todd Lipcon <[email protected]>
Gerrit-Comment-Date: Fri, 24 Aug 2018 20:47:08 +0000
Gerrit-HasComments: Yes

Reply via email to