[ 
https://issues.apache.org/jira/browse/CASSANDRA-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198585#comment-17198585
 ] 

Yifan Cai commented on CASSANDRA-15214:
---------------------------------------

> where would that exception end up if it were rethrown

{{JVMStabilityInspector}} only re-throws {{OutOfMemoryError}}. Depending on the 
presence of those OOM-related JVM options, {{OnOutOfMemoryError}}, 
{{ExitOnOutOfMemoryError}} or {{HeapDumpOnOutOfMemoryError}}, the JVM exits and 
trigger a heap dump if it is a heap space OOM error. 

However, the call-sites indicate to not re-throw OOM error (e.g. 
[here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/InboundMessageHandler.java#L647-L659]),
 which I'd like to learn why we do not let the JVM to exit.

Netty by default just logs the exception, when {{exceptionCaught()}} is _not_ 
implemented in any of the handler in the inbound direction. For the outbound, 
client code handles exception by adding listener to {{ChannelFuture}} or 
{{ChannelPromise}}. We have the handling in both directions. 
Besides the inbound/outbound pathes, it looks like that Netty does do a lot of 
catch-{{Throwable}}-and-swallow things in its code base. So it is possible that 
errors from Netty internal are not bubbled up. For example, this 
[issue|https://github.com/netty/netty/issues/6096]. 

> OOMs caught and not rethrown
> ----------------------------
>
>                 Key: CASSANDRA-15214
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-15214
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Messaging/Client, Messaging/Internode
>            Reporter: Benedict Elliott Smith
>            Assignee: Yifan Cai
>            Priority: Normal
>             Fix For: 4.0, 4.0-rc
>
>         Attachments: oom-experiments.zip
>
>
> Netty (at least, and perhaps elsewhere in Executors) catches all exceptions, 
> so presently there is no way to ensure that an OOM reaches the JVM handler to 
> trigger a crash/heapdump.
> It may be that the simplest most consistent way to do this would be to have a 
> single thread spawned at startup that waits for any exceptions we must 
> propagate to the Runtime.
> We could probably submit a patch upstream to Netty, but for a guaranteed 
> future proof approach, it may be worth paying the cost of a single thread.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to