[jira] [Updated] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[ https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-9861: -- Fix Version/s: (was: 2.2.x) 3.0.7 3.7 2.2.7 > When forcibly exiting due to OOM, we should produce a heap dump > --- > > Key: CASSANDRA-9861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9861 > Project: Cassandra > Issue Type: Bug > Components: Lifecycle >Reporter: Benedict >Assignee: Benjamin Lerer >Priority: Minor > Labels: lhf > Fix For: 2.2.7, 3.7, 3.0.7 > > Attachments: 9861-2.2-V2.txt, 9861-2.2.txt > > > CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to > lack of certainty about system state. However a side effect is that we never > produce heap dumps on OOM. We should ideally try to produce one forcibly > before exiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[ https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-9861: -- Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed into 2.2 at b189a7f6c955908d8b79d2b4104562a38ea62979 and merged into 3.0, 3.7 and trunk > When forcibly exiting due to OOM, we should produce a heap dump > --- > > Key: CASSANDRA-9861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9861 > Project: Cassandra > Issue Type: Bug > Components: Lifecycle >Reporter: Benedict >Assignee: Benjamin Lerer >Priority: Minor > Labels: lhf > Fix For: 2.2.x > > Attachments: 9861-2.2-V2.txt, 9861-2.2.txt > > > CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to > lack of certainty about system state. However a side effect is that we never > produce heap dumps on OOM. We should ideally try to produce one forcibly > before exiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[ https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-9861: - Status: Ready to Commit (was: Patch Available) > When forcibly exiting due to OOM, we should produce a heap dump > --- > > Key: CASSANDRA-9861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9861 > Project: Cassandra > Issue Type: Bug > Components: Lifecycle >Reporter: Benedict >Assignee: Benjamin Lerer >Priority: Minor > Labels: lhf > Fix For: 2.2.x > > Attachments: 9861-2.2-V2.txt, 9861-2.2.txt > > > CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to > lack of certainty about system state. However a side effect is that we never > produce heap dumps on OOM. We should ideally try to produce one forcibly > before exiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[ https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-9861: - Issue Type: Bug (was: Improvement) > When forcibly exiting due to OOM, we should produce a heap dump > --- > > Key: CASSANDRA-9861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9861 > Project: Cassandra > Issue Type: Bug > Components: Lifecycle >Reporter: Benedict >Assignee: Benjamin Lerer >Priority: Minor > Labels: lhf > Fix For: 2.2.x > > Attachments: 9861-2.2-V2.txt, 9861-2.2.txt > > > CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to > lack of certainty about system state. However a side effect is that we never > produce heap dumps on OOM. We should ideally try to produce one forcibly > before exiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[ https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-9861: -- Reviewer: Aleksey Yeschenko (was: Joshua McKenzie) > When forcibly exiting due to OOM, we should produce a heap dump > --- > > Key: CASSANDRA-9861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9861 > Project: Cassandra > Issue Type: Improvement > Components: Lifecycle >Reporter: Benedict >Assignee: Benjamin Lerer >Priority: Minor > Labels: lhf > Fix For: 2.2.x > > Attachments: 9861-2.2-V2.txt, 9861-2.2.txt > > > CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to > lack of certainty about system state. However a side effect is that we never > produce heap dumps on OOM. We should ideally try to produce one forcibly > before exiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[ https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-9861: -- Attachment: 9861-2.2-V2.txt The new patch try to use Sigar for retrieving the pid and default to parsing the JVM name if Sigar is not initialized (which is the case on my machine). It also try to resolve the path to jmap from JAVA_HOME and default to jmap if the file cannot be found. After the heap as been dumped an heap histogram will be writting to the log file. > When forcibly exiting due to OOM, we should produce a heap dump > --- > > Key: CASSANDRA-9861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9861 > Project: Cassandra > Issue Type: Improvement > Components: Lifecycle >Reporter: Benedict >Assignee: Benjamin Lerer >Priority: Minor > Labels: lhf > Fix For: 2.2.x > > Attachments: 9861-2.2-V2.txt, 9861-2.2.txt > > > CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to > lack of certainty about system state. However a side effect is that we never > produce heap dumps on OOM. We should ideally try to produce one forcibly > before exiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[ https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-9861: --- Reviewer: Joshua McKenzie > When forcibly exiting due to OOM, we should produce a heap dump > --- > > Key: CASSANDRA-9861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9861 > Project: Cassandra > Issue Type: Improvement > Components: Lifecycle >Reporter: Benedict >Assignee: Benjamin Lerer >Priority: Minor > Labels: lhf > Fix For: 2.2.x > > Attachments: 9861-2.2.txt > > > CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to > lack of certainty about system state. However a side effect is that we never > produce heap dumps on OOM. We should ideally try to produce one forcibly > before exiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[
https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-9861:
--
Attachment: 9861-2.2.txt
{quote}I guess the simplest way would be to add the
-XX:+HeapDumpOnOutOfMemoryError command line argument to our startup
scripts.{quote}
Sorry, I initially misunderstood the problem.
The patch use {{jmap}} to generate a heap dump when the
{{JVMStabilityInspector}} detects an OOMError.
One issue is to retrieve the process ID to be able to execute the {{jmap}}
call. The Java API will only provide a reliable way for retrieving the process
ID in Java 9. The current method used by the patch will work on the Oracle JVM
but might not work on all the JVMs. If the process ID cannot be retrieved no
dump will be generated.
||utests||dtests||
|[2.2|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-9861-2.2-testall/]|[2.2|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-9861-2.2-dtest/]|
|[3.0|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-9861-3.0-testall/]|[3.0|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-9861-3.0-dtest/]|
> When forcibly exiting due to OOM, we should produce a heap dump
> ---
>
> Key: CASSANDRA-9861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9861
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Benedict
>Assignee: Benjamin Lerer
>Priority: Minor
> Labels: lhf
> Fix For: 2.2.x
>
> Attachments: 9861-2.2.txt
>
>
> CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to
> lack of certainty about system state. However a side effect is that we never
> produce heap dumps on OOM. We should ideally try to produce one forcibly
> before exiting.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[ https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-9861: -- Status: Patch Available (was: Open) [~JoshuaMcKenzie] could you review? > When forcibly exiting due to OOM, we should produce a heap dump > --- > > Key: CASSANDRA-9861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9861 > Project: Cassandra > Issue Type: Improvement > Components: Lifecycle >Reporter: Benedict >Assignee: Benjamin Lerer >Priority: Minor > Labels: lhf > Fix For: 2.2.x > > Attachments: 9861-2.2.txt > > > CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to > lack of certainty about system state. However a side effect is that we never > produce heap dumps on OOM. We should ideally try to produce one forcibly > before exiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[ https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-9861: -- Component/s: Lifecycle > When forcibly exiting due to OOM, we should produce a heap dump > --- > > Key: CASSANDRA-9861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9861 > Project: Cassandra > Issue Type: Improvement > Components: Lifecycle >Reporter: Benedict >Assignee: Benjamin Lerer >Priority: Minor > Labels: lhf > Fix For: 2.2.x > > Attachments: 9861-2.2.txt > > > CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to > lack of certainty about system state. However a side effect is that we never > produce heap dumps on OOM. We should ideally try to produce one forcibly > before exiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[ https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-9861: -- Labels: lhf (was: ) > When forcibly exiting due to OOM, we should produce a heap dump > --- > > Key: CASSANDRA-9861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9861 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict >Priority: Minor > Labels: lhf > Fix For: 2.2.x > > > CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to > lack of certainty about system state. However a side effect is that we never > produce heap dumps on OOM. We should ideally try to produce one forcibly > before exiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
