Re: Safely Disabling Compaction
The docs for the CLI say: - min_compaction_threshold: The minimum number of SSTables needed to start a minor compaction. Default is 4, setting to 0 disables minor compactions. If you are getting an error using 0 can you please raise a ticket on https://issues.apache.org/jira/browse/CASSANDRA (Also you can also set the compaction threshold on a single node via nodetool. ) Thanks - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 18/05/2012, at 3:09 PM, Vijay wrote: I would rather set the Keyspace setting min_compaction_threshold and max_compaction_threshold to be a higher number and once i am ready i will put the value back... This way i dont need to restart. Having said that why not set the compaction throughput to 1 (low enough to not have contention) and complete the stream? Regards, /VJ On Wed, May 16, 2012 at 2:43 PM, sj.climber sj.clim...@gmail.com wrote: Hi, In an effort to minimize IO contention, I'd like to disable compactions while I'm streaming SSTables to the cluster. When done streaming, I intend on forcing a major compaction through nodetool. Elsewhere in the forums, various folks suggest setting max_compaction_threshold = 0 to disable compaction. While this works sometimes (via 'update column family family with max_compaction_threshold=0'), I've observed a number of serious issues with this approach: 1) You can't create a column family with max_compaction_threshold = 0. The CLI reports that min_compaction_threshold must have a value = 2, and max_compaction_threshold can't be lower than it. Worse yet, trying to create a column family with max_compaction_threshold = 0 gets the cluster into a Schema Disagreement Exception (since the node on which you issue the migration command fails with a fatal error). 2) Cassandra will allow me to update an existing column family with max_compaction_threshold = 0. But if I restart the node, it will crash on startup. java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException ... org.apache.cassandra.config.CFMetaData.createCompactionStrategyInstance(CFMetaData.java:839) ... 14 more Caused by: java.lang.RuntimeException: The max_compaction_threshold cannot be smaller than the min. at org.apache.cassandra.db.ColumnFamilyStore.setMaximumCompactionThreshold(ColumnFamilyStore.java:1740) at org.apache. Is there another solution for more safely enabling/disabling compaction? Thanks! -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Safely-Disabling-Compaction-tp7562777.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
RE: Safely Disabling Compaction
To temporarily turn off compactions without schema update, use nodetool –h node_ip -p port setcompactionthreshold keyspace cfname minthreshold maxthreshold for every node and every column family you need. If nodetool throws same exception, do in 2 steps: 1. nodetool setcompactionthreshold keyspace cfname 0 32 (32 – use yours instead) 2. nodetool setcompactionthreshold keyspace cfname 0 0 To restore, set your normal values. Best regards / Pagarbiai Viktor Jevdokimov Senior Developer Email: viktor.jevdoki...@adform.commailto:viktor.jevdoki...@adform.com Phone: +370 5 212 3063, Fax +370 5 261 0453 J. Jasinskio 16C, LT-01112 Vilnius, Lithuania Follow us on Twitter: @adforminsiderhttp://twitter.com/#!/adforminsider What is Adform: watch this short videohttp://vimeo.com/adform/display [Adform News] http://www.adform.com Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies. From: Vijay [mailto:vijay2...@gmail.com] Sent: Friday, May 18, 2012 06:09 To: user@cassandra.apache.org Cc: cassandra-u...@incubator.apache.org Subject: Re: Safely Disabling Compaction I would rather set the Keyspace setting min_compaction_threshold and max_compaction_threshold to be a higher number and once i am ready i will put the value back... This way i dont need to restart. Having said that why not set the compaction throughput to 1 (low enough to not have contention) and complete the stream? Regards, /VJ On Wed, May 16, 2012 at 2:43 PM, sj.climber sj.clim...@gmail.commailto:sj.clim...@gmail.com wrote: Hi, In an effort to minimize IO contention, I'd like to disable compactions while I'm streaming SSTables to the cluster. When done streaming, I intend on forcing a major compaction through nodetool. Elsewhere in the forums, various folks suggest setting max_compaction_threshold = 0 to disable compaction. While this works sometimes (via 'update column family family with max_compaction_threshold=0'), I've observed a number of serious issues with this approach: 1) You can't create a column family with max_compaction_threshold = 0. The CLI reports that min_compaction_threshold must have a value = 2, and max_compaction_threshold can't be lower than it. Worse yet, trying to create a column family with max_compaction_threshold = 0 gets the cluster into a Schema Disagreement Exception (since the node on which you issue the migration command fails with a fatal error). 2) Cassandra will allow me to update an existing column family with max_compaction_threshold = 0. But if I restart the node, it will crash on startup. java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException ... org.apache.cassandra.config.CFMetaData.createCompactionStrategyInstance(CFMetaData.java:839) ... 14 more Caused by: java.lang.RuntimeException: The max_compaction_threshold cannot be smaller than the min. at org.apache.cassandra.db.ColumnFamilyStore.setMaximumCompactionThreshold(ColumnFamilyStore.java:1740) at org.apache. Is there another solution for more safely enabling/disabling compaction? Thanks! -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Safely-Disabling-Compaction-tp7562777.html Sent from the cassandra-u...@incubator.apache.orgmailto:cassandra-u...@incubator.apache.org mailing list archive at Nabble.com. inline: signature-logo29.png
Re: Safely Disabling Compaction
I would rather set the Keyspace setting min_compaction_threshold and max_compaction_threshold to be a higher number and once i am ready i will put the value back... This way i dont need to restart. Having said that why not set the compaction throughput to 1 (low enough to not have contention) and complete the stream? Regards, /VJ On Wed, May 16, 2012 at 2:43 PM, sj.climber sj.clim...@gmail.com wrote: Hi, In an effort to minimize IO contention, I'd like to disable compactions while I'm streaming SSTables to the cluster. When done streaming, I intend on forcing a major compaction through nodetool. Elsewhere in the forums, various folks suggest setting max_compaction_threshold = 0 to disable compaction. While this works sometimes (via 'update column family family with max_compaction_threshold=0'), I've observed a number of serious issues with this approach: 1) You can't create a column family with max_compaction_threshold = 0. The CLI reports that min_compaction_threshold must have a value = 2, and max_compaction_threshold can't be lower than it. Worse yet, trying to create a column family with max_compaction_threshold = 0 gets the cluster into a Schema Disagreement Exception (since the node on which you issue the migration command fails with a fatal error). 2) Cassandra will allow me to update an existing column family with max_compaction_threshold = 0. But if I restart the node, it will crash on startup. java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException ... org.apache.cassandra.config.CFMetaData.createCompactionStrategyInstance(CFMetaData.java:839) ... 14 more Caused by: java.lang.RuntimeException: The max_compaction_threshold cannot be smaller than the min. at org.apache.cassandra.db.ColumnFamilyStore.setMaximumCompactionThreshold(ColumnFamilyStore.java:1740) at org.apache. Is there another solution for more safely enabling/disabling compaction? Thanks! -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Safely-Disabling-Compaction-tp7562777.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
