[
https://issues.apache.org/jira/browse/CASSANDRA-13006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118421#comment-16118421
]
Eric Evans commented on CASSANDRA-13006:
----------------------------------------
I think this behavior (invoking {{jmap}} on OOM) is a pretty serious violation
to the Element of Least-surprise. We already provide mechanisms for passing
arguments to the JVM, and TTBMK, all of them provide some means for dropping a
heap dump on out-of-memory.
It definitely caught me be surprise. We carried over
{{-XX:+HeapDumpOnOutOfMemoryError}} from our 2.2.x environment, only to have
Cassandra and the JVM racing to create a dump of the same name.
Additionally, something about all of this is buggy, because on more than one
occasion we've had Cassandra fork-bombing {{jmap}} processes
{noformat}
● cassandra-b.service - distributed storage system for structured data
Loaded: loaded (/lib/systemd/system/cassandra-b.service; static)
Active: active (running) since Sat 2017-08-05 22:32:07 UTC; 23h ago
Main PID: 25025 (java)
CGroup: /system.slice/cassandra-b.service
├─ 9213 jmap -histo 25025
├─ 9214 jmap -histo 25025
├─ 9284 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─ 9285 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─ 9388 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─ 9453 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─ 9519 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─ 9520 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─ 9733 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─ 9735 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─ 9736 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─14835 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─14836 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─14837 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─14839 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─14841 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─14844 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─18932 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─18933 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─18934 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─18935 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─18936 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─18937 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─18938 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─18939 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─18940 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─18942 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─18943 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─18944 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
├─18945 jmap
-dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025
[ ... ]
{noformat}
IMO, the sanest strategy here would be to leave the creation of heap dumps to
the JVM.
> Disable automatic heap dumps on OOM error
> -----------------------------------------
>
> Key: CASSANDRA-13006
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13006
> Project: Cassandra
> Issue Type: Bug
> Components: Configuration
> Reporter: anmols
> Assignee: Benjamin Lerer
> Priority: Minor
> Fix For: 3.0.9
>
> Attachments: 13006-3.0.9.txt
>
>
> With CASSANDRA-9861, a change was added to enable collecting heap dumps by
> default if the process encountered an OOM error. These heap dumps are stored
> in the Apache Cassandra home directory unless configured otherwise (see
> [Cassandra Support
> Document|https://support.datastax.com/hc/en-us/articles/204225959-Generating-and-Analyzing-Heap-Dumps]
> for this feature).
>
> The creation and storage of heap dumps aides debugging and investigative
> workflows, but is not be desirable for a production environment where these
> heap dumps may occupy a large amount of disk space and require manual
> intervention for cleanups.
>
> Managing heap dumps on out of memory errors and configuring the paths for
> these heap dumps are available as JVM options in JVM. The current behavior
> conflicts with the Boolean JVM flag HeapDumpOnOutOfMemoryError.
>
> A patch can be proposed here that would make the heap dump on OOM error honor
> the HeapDumpOnOutOfMemoryError flag. Users who would want to still generate
> heap dumps on OOM errors can set the -XX:+HeapDumpOnOutOfMemoryError JVM
> option.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
