[jira] [Updated] (CASSANDRA-11714) shutdown script should wait more than 30 seconds before force kill
[ https://issues.apache.org/jira/browse/CASSANDRA-11714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna updated CASSANDRA-11714: - Complexity: Low Hanging Fruit > shutdown script should wait more than 30 seconds before force kill > -- > > Key: CASSANDRA-11714 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11714 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Wei Deng >Assignee: Wei Deng >Priority: Low > Labels: lhf > > I'm running with 3.0.6 and it appears that if I let it run for multiple days > (ingesting a never-ending streaming data from Kafka), then the next time I'm > restarting the JVM by "service cassandra restart", it will take 9 minutes to > replay the commit log segments (the commitlog_total_space_in_mb is using the > default 8GB and I observed close to 8GB commitlog segment files before the > restart). Here is a fragment of the system.log showing how long it takes: > {noformat} > INFO [main] 2016-05-04 19:06:54,356 CommitLog.java:168 - Replaying > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248187.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248191.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248195.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248198.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248201.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248205.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248209.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248213.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248217.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248221.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248225.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248230.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248234.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248238.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248242.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248246.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248250.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248254.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248258.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248262.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248266.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248270.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248274.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248278.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248282.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248286.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248290.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248294.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248299.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248303.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248307.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248311.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248315.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248319.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248323.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248327.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248331.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248335.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248340.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248344.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248348.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248352.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248356.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248360.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248364.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248368.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248372.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248376.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248380.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248384.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248388.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248392.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248396.log, >
[jira] [Updated] (CASSANDRA-11714) shutdown script should wait more than 30 seconds before force kill
[ https://issues.apache.org/jira/browse/CASSANDRA-11714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna updated CASSANDRA-11714: - Labels: lhf (was: ) > shutdown script should wait more than 30 seconds before force kill > -- > > Key: CASSANDRA-11714 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11714 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Wei Deng >Priority: Minor > Labels: lhf > > I'm running with 3.0.6 and it appears that if I let it run for multiple days > (ingesting a never-ending streaming data from Kafka), then the next time I'm > restarting the JVM by "service cassandra restart", it will take 9 minutes to > replay the commit log segments (the commitlog_total_space_in_mb is using the > default 8GB and I observed close to 8GB commitlog segment files before the > restart). Here is a fragment of the system.log showing how long it takes: > {noformat} > INFO [main] 2016-05-04 19:06:54,356 CommitLog.java:168 - Replaying > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248187.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248191.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248195.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248198.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248201.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248205.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248209.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248213.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248217.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248221.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248225.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248230.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248234.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248238.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248242.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248246.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248250.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248254.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248258.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248262.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248266.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248270.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248274.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248278.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248282.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248286.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248290.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248294.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248299.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248303.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248307.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248311.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248315.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248319.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248323.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248327.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248331.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248335.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248340.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248344.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248348.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248352.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248356.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248360.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248364.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248368.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248372.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248376.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248380.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248384.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248388.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248392.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248396.log, > /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248397.log, >