[
https://issues.apache.org/jira/browse/CASSANDRA-13663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Otto updated CASSANDRA-13663:
--------------------------------------
Description:
Hello. My company runs a 5 node Cassandra cluster. For the last few weeks, we
have had a sporadic issue where one of the servers crashes without creating a
dump file and without any error messages in the logs. If one restarts the
service (which we have by now scripted to happen automatically), the servers
resumes work with no complaint.
Log files of the time of the last crash are attached, thou again they do not
log any crash happening.
Regarding out setup, we are running these servers on AMazon AWS, with 3 volumes
per server, one for the system, one for data and one for the commitlog. When a
crash happens, we can observe a sudden spike of read activity on the commitlog
volume. All of these have ample free space.
The servers are Ubuntu 16.04 servers and Cassandra is installed from the
apt-get packet for version 3.10.
It is worth noting that these crashes happen more often when nodetool is
running either repair job or a backup job, but this is by no means always the
case. As for frequency, we have had about 1-2 crashes per week for the last
month.
was:
Hello. My company runs a 5 node Cassandra cluster. For the last few weeks, we
have had a sporadic issue where one of the servers crashes without creating a
dump file and without any error messages in the logs. If one restarts the
service (which we have by now scripted to happen automatically), the servers
resumes work with no complaint.
Log files of the time of the last crash are attached, thou again they do not
log any crash happening.
Regarding out setup, we are running these servers on AMazon AWS, with 3 volumes
per server, one for the system, one for data and one for the commitlog. When a
crash happens, we can observe a sudden spike of read activity on the commitlog
volume.
The servers are Ubuntu 16.04 servers and Cassandra is installed from the
apt-get packet for version 3.10.
It is worth noting that these crashes happen more often when nodetool is
running either repair job or a backup job, but this is by no means always the
case. As for frequency, we have had about 1-2 crashes per week for the last
month.
> Cassandra 3.10 crashes without dump
> -----------------------------------
>
> Key: CASSANDRA-13663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13663
> Project: Cassandra
> Issue Type: Bug
> Reporter: Matthias Otto
> Priority: Minor
> Attachments: 2017-07-04 10_48_34-CloudWatch Management Console.png,
> cassandra debug.log, cassandra system.log
>
>
> Hello. My company runs a 5 node Cassandra cluster. For the last few weeks, we
> have had a sporadic issue where one of the servers crashes without creating a
> dump file and without any error messages in the logs. If one restarts the
> service (which we have by now scripted to happen automatically), the servers
> resumes work with no complaint.
> Log files of the time of the last crash are attached, thou again they do not
> log any crash happening.
> Regarding out setup, we are running these servers on AMazon AWS, with 3
> volumes per server, one for the system, one for data and one for the
> commitlog. When a crash happens, we can observe a sudden spike of read
> activity on the commitlog volume. All of these have ample free space.
> The servers are Ubuntu 16.04 servers and Cassandra is installed from the
> apt-get packet for version 3.10.
> It is worth noting that these crashes happen more often when nodetool is
> running either repair job or a backup job, but this is by no means always the
> case. As for frequency, we have had about 1-2 crashes per week for the last
> month.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
