[
https://issues.apache.org/jira/browse/CASSANDRA-6392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-6392.
---------------------------------------
Resolution: Duplicate
> Cassandra 2.0.1 leaking open file handles for deleted system hints
> ------------------------------------------------------------------
>
> Key: CASSANDRA-6392
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6392
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.0.1
> Cassandra datastax java driver 1.0.3 (cassandra-driver-core)
> Reporter: Prateek
>
> The system was running for a few days with a heavy write workload. The system
> crashed with too many open file handles. It seems the deleted system hint
> file handles are still open even though the file has been deleted on disk. We
> use a 3-node cluster with replication factor 3 on ec2 us-east-1 using
> NetworkTopologyStrategy. Default consistency level for write is ANY.
> This is blocking our production deployment and causing instability in
> production.
> Appreciate your quick response.
> Many thanks,
> Prateek
> See logs below:
> [(bloomreach-ami) ubuntu@ip-10-12-33-23 :/mnt/cassandra/log]# lsof -p 13568
> |wc -l
> 201499
> [(bloomreach-ami) ubuntu@ip-10-12-33-23 :/mnt/cassandra/log]# lsof -p 13568
> |grep deleted|wc -l
> 186261
> java 13568 ubuntu *676r REG 9,0 356 1610775839
> /mnt/cassandra/data/system/hints/system-hints-jb-848-Data.db (deleted)
> java 13568 ubuntu *677r REG 9,0 309 1610770742
> /mnt/cassandra/data/system/hints/system-hints-jb-845-Data.db (deleted)
> java 13568 ubuntu *678r REG 9,0 506 1610775703
> /mnt/cassandra/data/system/hints/system-hints-jb-800-Data.db (deleted)
> [(bloomreach-ami) ubuntu@ip-10-12-33-23 :/mnt/cassandra/log]# ls -l
> /mnt/cassandra/data/system/hints/system-hints-jb-800-Data.db
> ls: cannot access
> /mnt/cassandra/data/system/hints/system-hints-jb-800-Data.db: No such file or
> directory
--
This message was sent by Atlassian JIRA
(v6.1#6144)
