[jira] [Updated] (CASSANDRA-3001) Make the compression algorithm and chunk length configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-3001: Attachment: 0001-Pluggable-algorithm-and-chunk-length.patch Make the compression algorithm and chunk length configurable Key: CASSANDRA-3001 URL: https://issues.apache.org/jira/browse/CASSANDRA-3001 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Labels: compression Fix For: 1.0 Attachments: 0001-Pluggable-algorithm-and-chunk-length.patch, 0002-Thrift-crap.patch, 0003-Add-deflate-compressor.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3001) Make the compression algorithm and chunk length configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-3001: Attachment: (was: 0001-Pluggable-algorithm-and-chunk-length.patch) Make the compression algorithm and chunk length configurable Key: CASSANDRA-3001 URL: https://issues.apache.org/jira/browse/CASSANDRA-3001 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Labels: compression Fix For: 1.0 Attachments: 0001-Pluggable-algorithm-and-chunk-length.patch, 0002-Thrift-crap.patch, 0003-Add-deflate-compressor.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3001) Make the compression algorithm and chunk length configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094401#comment-13094401 ] Sylvain Lebresne commented on CASSANDRA-3001: - I just updated the first patch to fix the tests compilation (the 'line 108' change). However, the `ant test-compression` failures are not of this patch doing. Trunk already fails and a git bisect indicates that those failure where introduced by CASSANDRA-2498 (we should probably wait for that to be fixed and check then that the tests do pass with this patch). Make the compression algorithm and chunk length configurable Key: CASSANDRA-3001 URL: https://issues.apache.org/jira/browse/CASSANDRA-3001 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Labels: compression Fix For: 1.0 Attachments: 0001-Pluggable-algorithm-and-chunk-length.patch, 0002-Thrift-crap.patch, 0003-Add-deflate-compressor.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3109) nodetool repair consuming large disk when it ran after increasing replication factor on live cluster
nodetool repair consuming large disk when it ran after increasing replication factor on live cluster Key: CASSANDRA-3109 URL: https://issues.apache.org/jira/browse/CASSANDRA-3109 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: Centos 5.5 Reporter: Sunil Kumar Hi all, I have two node cassandra cluster initial replication factor for this cluster is 1. When i changed replication factor to 2 on this live cluster.From cassandra wiki(http://wiki.apache.org/cassandra/Operations#Replication) it is recommended to run nodetool repair command on each node of the cluster. But when i run nodetool command on both node disk usage on the both node got increased to 50 times of the initial disk usage. Initially in my cluster data with one replication factor was 1 GB on each node now its 50GB on each node and still nodetool process is going on. Can any body help me in this problem or is it desired behavior of cassandra??? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3110) SSTables iterators are closed and before being used
SSTables iterators are closed and before being used --- Key: CASSANDRA-3110 URL: https://issues.apache.org/jira/browse/CASSANDRA-3110 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 3110.patch Seems there is misplaced finally blocks in CollationController: we close the sstable iterators and release the sstable references *before* having actually used said iterators/sstables. Note: this cause a lot of tests to fail in `ant test-compression` in particular. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3110) SSTables iterators are closed and before being used
[ https://issues.apache.org/jira/browse/CASSANDRA-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-3110: Attachment: 3110.patch SSTables iterators are closed and before being used --- Key: CASSANDRA-3110 URL: https://issues.apache.org/jira/browse/CASSANDRA-3110 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 3110.patch Seems there is misplaced finally blocks in CollationController: we close the sstable iterators and release the sstable references *before* having actually used said iterators/sstables. Note: this cause a lot of tests to fail in `ant test-compression` in particular. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3001) Make the compression algorithm and chunk length configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094416#comment-13094416 ] Pavel Yaskevich commented on CASSANDRA-3001: Let's wait. Make the compression algorithm and chunk length configurable Key: CASSANDRA-3001 URL: https://issues.apache.org/jira/browse/CASSANDRA-3001 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Labels: compression Fix For: 1.0 Attachments: 0001-Pluggable-algorithm-and-chunk-length.patch, 0002-Thrift-crap.patch, 0003-Add-deflate-compressor.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3111) Bug in ReversedType comparator
Bug in ReversedType comparator -- Key: CASSANDRA-3111 URL: https://issues.apache.org/jira/browse/CASSANDRA-3111 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.4 Reporter: Fabien Rousseau Priority: Minor Scenario : * create a cf with a reversed comparator * insert a few columns in a row * try to read data with : SliceRange(start='', finish='', reversed=true) ** no data is returned, but some columns are expected * try to read data with : SliceRange(start='', finish='', reversed=false) ** if not flushed on disk : no data is returned -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3111) Bug in ReversedType comparator
[ https://issues.apache.org/jira/browse/CASSANDRA-3111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabien Rousseau updated CASSANDRA-3111: --- Attachment: 001-CASSANDRA-3111.patch Bug in ReversedType comparator -- Key: CASSANDRA-3111 URL: https://issues.apache.org/jira/browse/CASSANDRA-3111 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.4 Reporter: Fabien Rousseau Priority: Minor Attachments: 001-CASSANDRA-3111.patch Scenario : * create a cf with a reversed comparator * insert a few columns in a row * try to read data with : SliceRange(start='', finish='', reversed=true) ** no data is returned, but some columns are expected * try to read data with : SliceRange(start='', finish='', reversed=false) ** if not flushed on disk : no data is returned -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1391) Allow Concurrent Schema Migrations
[ https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1391: --- Attachment: (was: CASSANDRA-1391.patch) Allow Concurrent Schema Migrations -- Key: CASSANDRA-1391 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Pavel Yaskevich Fix For: 1.0 CASSANDRA-1292 fixed multiple migrations started from the same node to properly queue themselves, but it is still possible for migrations initiated on different nodes to conflict and leave the cluster in a bad state. Since the system_add/drop/rename methods are accessible directly from the client API, they should be completely safe for concurrent use. It should be possible to allow for most types of concurrent migrations by converting the UUID schema ID into a VersionVectorClock (as provided by CASSANDRA-580). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1391) Allow Concurrent Schema Migrations
[ https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1391: --- Attachment: (was: CASSANDRA-1391.patch) Allow Concurrent Schema Migrations -- Key: CASSANDRA-1391 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Pavel Yaskevich Fix For: 1.0 CASSANDRA-1292 fixed multiple migrations started from the same node to properly queue themselves, but it is still possible for migrations initiated on different nodes to conflict and leave the cluster in a bad state. Since the system_add/drop/rename methods are accessible directly from the client API, they should be completely safe for concurrent use. It should be possible to allow for most types of concurrent migrations by converting the UUID schema ID into a VersionVectorClock (as provided by CASSANDRA-580). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1391) Allow Concurrent Schema Migrations
[ https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1391: --- Attachment: (was: CASSANDRA-1391.patch) Allow Concurrent Schema Migrations -- Key: CASSANDRA-1391 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Pavel Yaskevich Fix For: 1.0 CASSANDRA-1292 fixed multiple migrations started from the same node to properly queue themselves, but it is still possible for migrations initiated on different nodes to conflict and leave the cluster in a bad state. Since the system_add/drop/rename methods are accessible directly from the client API, they should be completely safe for concurrent use. It should be possible to allow for most types of concurrent migrations by converting the UUID schema ID into a VersionVectorClock (as provided by CASSANDRA-580). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1391) Allow Concurrent Schema Migrations
[ https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1391: --- Attachment: CASSANDRA-1391.patch Allow Concurrent Schema Migrations -- Key: CASSANDRA-1391 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-1391.patch CASSANDRA-1292 fixed multiple migrations started from the same node to properly queue themselves, but it is still possible for migrations initiated on different nodes to conflict and leave the cluster in a bad state. Since the system_add/drop/rename methods are accessible directly from the client API, they should be completely safe for concurrent use. It should be possible to allow for most types of concurrent migrations by converting the UUID schema ID into a VersionVectorClock (as provided by CASSANDRA-580). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1391) Allow Concurrent Schema Migrations
[
https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Yaskevich updated CASSANDRA-1391:
---
Attachment: CASSANDRA-1391.patch
updated patch to fix problem with avro map comparisons. This has a side effect
in a good sense - you don't need to include options that you don't want to
update when you do CLI `update {keyspace/column family}`.
Allow Concurrent Schema Migrations
--
Key: CASSANDRA-1391
URL: https://issues.apache.org/jira/browse/CASSANDRA-1391
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Stu Hood
Assignee: Pavel Yaskevich
Fix For: 1.0
Attachments: CASSANDRA-1391.patch
CASSANDRA-1292 fixed multiple migrations started from the same node to
properly queue themselves, but it is still possible for migrations initiated
on different nodes to conflict and leave the cluster in a bad state. Since
the system_add/drop/rename methods are accessible directly from the client
API, they should be completely safe for concurrent use.
It should be possible to allow for most types of concurrent migrations by
converting the UUID schema ID into a VersionVectorClock (as provided by
CASSANDRA-580).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1391) Allow Concurrent Schema Migrations
[ https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1391: --- Attachment: (was: CASSANDRA-1391.patch) Allow Concurrent Schema Migrations -- Key: CASSANDRA-1391 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-1391.patch CASSANDRA-1292 fixed multiple migrations started from the same node to properly queue themselves, but it is still possible for migrations initiated on different nodes to conflict and leave the cluster in a bad state. Since the system_add/drop/rename methods are accessible directly from the client API, they should be completely safe for concurrent use. It should be possible to allow for most types of concurrent migrations by converting the UUID schema ID into a VersionVectorClock (as provided by CASSANDRA-580). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3111) Bug in ReversedType comparator
[ https://issues.apache.org/jira/browse/CASSANDRA-3111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3111: -- Reviewer: slebresne Bug in ReversedType comparator -- Key: CASSANDRA-3111 URL: https://issues.apache.org/jira/browse/CASSANDRA-3111 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.4 Reporter: Fabien Rousseau Priority: Minor Attachments: 001-CASSANDRA-3111.patch Scenario : * create a cf with a reversed comparator * insert a few columns in a row * try to read data with : SliceRange(start='', finish='', reversed=true) ** no data is returned, but some columns are expected * try to read data with : SliceRange(start='', finish='', reversed=false) ** if not flushed on disk : no data is returned -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3110) SSTables iterators are closed and before being used
[ https://issues.apache.org/jira/browse/CASSANDRA-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094510#comment-13094510 ] Jonathan Ellis commented on CASSANDRA-3110: --- +1 SSTables iterators are closed and before being used --- Key: CASSANDRA-3110 URL: https://issues.apache.org/jira/browse/CASSANDRA-3110 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 3110.patch Seems there is misplaced finally blocks in CollationController: we close the sstable iterators and release the sstable references *before* having actually used said iterators/sstables. Note: this cause a lot of tests to fail in `ant test-compression` in particular. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3109) nodetool repair consuming large disk when it ran after increasing replication factor on live cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-3109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne resolved CASSANDRA-3109. - Resolution: Duplicate This is likely fixed by CASSANDRA-2816 (and possibly CASSANDRA-2280). nodetool repair consuming large disk when it ran after increasing replication factor on live cluster Key: CASSANDRA-3109 URL: https://issues.apache.org/jira/browse/CASSANDRA-3109 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: Centos 5.5 Reporter: Sunil Kumar Hi all, I have two node cassandra cluster initial replication factor for this cluster is 1. When i changed replication factor to 2 on this live cluster.From cassandra wiki(http://wiki.apache.org/cassandra/Operations#Replication) it is recommended to run nodetool repair command on each node of the cluster. But when i run nodetool command on both node disk usage on the both node got increased to 50 times of the initial disk usage. Initially in my cluster data with one replication factor was 1 GB on each node now its 50GB on each node and still nodetool process is going on. Can any body help me in this problem or is it desired behavior of cassandra??? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2433) Failed Streams Break Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2433: Attachment: 2433_v3.patch bq. Someone's probably going to want the JMX information but let's keep Stages for Verb-associated tasks Sounds good, updated patch add a new executor directly into AntiEntropy. bq. How about splitting onDead and onRestart in EndpointStateChange, then? Done. bq. I prefer using WrappedRunnable to a Callable I changed to use WrappedRunnable. However, we still need to have access to both the repair session and the future from the executor so the implementation returns a pair of those two objects. I'm only marginally convinced this is cleaner than the previous solution... Failed Streams Break Repair --- Key: CASSANDRA-2433 URL: https://issues.apache.org/jira/browse/CASSANDRA-2433 Project: Cassandra Issue Type: Bug Components: Core Reporter: Benjamin Coverston Assignee: Sylvain Lebresne Labels: repair Fix For: 0.8.5 Attachments: 0001-Put-repair-session-on-a-Stage-and-add-a-method-to-re-v4.patch, 0002-Register-in-gossip-to-handle-node-failures-v4.patch, 0003-Report-streaming-errors-back-to-repair-v4.patch, 0004-Reports-validation-compaction-errors-back-to-repair-v4.patch, 2433.patch, 2433_v2.patch, 2433_v3.patch Running repair in cases where a stream fails we are seeing multiple problems. 1. Although retry is initiated and completes, the old stream doesn't seem to clean itself up and repair hangs. 2. The temp files are left behind and multiple failures can end up filling up the data partition. These issues together are making repair very difficult for nearly everyone running repair on a non-trivial sized data set. This issue is also being worked on w.r.t CASSANDRA-2088, however that was moved to 0.8 for a few reasons. This ticket is to fix the immediate issues that we are seeing in 0.7. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163652 - in /cassandra/trunk: CHANGES.txt src/java/org/apache/cassandra/db/CollationController.java
Author: slebresne
Date: Wed Aug 31 14:50:35 2011
New Revision: 1163652
URL: http://svn.apache.org/viewvc?rev=1163652view=rev
Log:
Fix closing sstable iterators before using them
patch by slebresne; reviewed by jbellis for CASSANDRA-3110
Modified:
cassandra/trunk/CHANGES.txt
cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java
Modified: cassandra/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1163652r1=1163651r2=1163652view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Wed Aug 31 14:50:35 2011
@@ -50,6 +50,7 @@
* Expose gossip/FD info to JMX (CASSANDRA-2806)
* Fix streaming over SSL when compressed SSTable involved (CASSANDRA-3051)
* Add support for pluggable secondary index implementations (CASSANDRA-3078)
+ * Fix closing sstable iterators before using them (CASSANDRA-3110)
0.8.5
* fix NPE when encryption_options is unspecified (CASSANDRA-3007)
Modified:
cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java?rev=1163652r1=1163651r2=1163652view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java
Wed Aug 31 14:50:35 2011
@@ -116,6 +116,38 @@ public class CollationController
container.addColumn(iter.next());
}
}
+
+// we need to distinguish between there is no data at all for
this row (BF will let us rebuild that efficiently)
+// and there used to be data, but it's gone now (we should cache
the empty CF so we don't need to rebuild that slower)
+if (iterators.isEmpty())
+return null;
+
+// do a final collate. toCollate is boilerplate required to
provide a CloseableIterator
+final ColumnFamily c2 = container;
+CloseableIteratorIColumn toCollate = new
SimpleAbstractColumnIterator()
+{
+final IteratorIColumn iter = c2.iterator();
+
+protected IColumn computeNext()
+{
+return iter.hasNext() ? iter.next() : endOfData();
+}
+
+public ColumnFamily getColumnFamily()
+{
+return c2;
+}
+
+public DecoratedKey getKey()
+{
+return filter.key;
+}
+};
+ColumnFamily returnCF = container.cloneMeShallow();
+filter.collateColumns(returnCF,
Collections.singletonList(toCollate), cfs.metadata.comparator, gcBefore);
+
+// Caller is responsible for final removeDeletedCF. This is
important for cacheRow to work correctly:
+return returnCF;
}
finally
{
@@ -123,38 +155,6 @@ public class CollationController
for (IColumnIterator iter : iterators)
FileUtils.closeQuietly(iter);
}
-
-// we need to distinguish between there is no data at all for this
row (BF will let us rebuild that efficiently)
-// and there used to be data, but it's gone now (we should cache the
empty CF so we don't need to rebuild that slower)
-if (iterators.isEmpty())
-return null;
-
-// do a final collate. toCollate is boilerplate required to provide a
CloseableIterator
-final ColumnFamily c2 = container;
-CloseableIteratorIColumn toCollate = new
SimpleAbstractColumnIterator()
-{
-final IteratorIColumn iter = c2.iterator();
-
-protected IColumn computeNext()
-{
-return iter.hasNext() ? iter.next() : endOfData();
-}
-
-public ColumnFamily getColumnFamily()
-{
-return c2;
-}
-
-public DecoratedKey getKey()
-{
-return filter.key;
-}
-};
-ColumnFamily returnCF = container.cloneMeShallow();
-filter.collateColumns(returnCF, Collections.singletonList(toCollate),
cfs.metadata.comparator, gcBefore);
-
-// Caller is responsible for final removeDeletedCF. This is important
for cacheRow to work correctly:
-return returnCF;
}
/**
@@ -210,6 +210,16 @@ public class CollationController
sstablesIterated++;
}
}
+
+// we need to distinguish between there is no data at all for
this row (BF will let us rebuild that efficiently)
+// and there used to be data, but it's gone now (we should cache
the empty CF so we
[jira] [Updated] (CASSANDRA-2433) Failed Streams Break Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2433: Attachment: 2433_v3.patch (Sorry, I had attached the wrong version of v3, corrected now) Failed Streams Break Repair --- Key: CASSANDRA-2433 URL: https://issues.apache.org/jira/browse/CASSANDRA-2433 Project: Cassandra Issue Type: Bug Components: Core Reporter: Benjamin Coverston Assignee: Sylvain Lebresne Labels: repair Fix For: 0.8.5 Attachments: 0001-Put-repair-session-on-a-Stage-and-add-a-method-to-re-v4.patch, 0002-Register-in-gossip-to-handle-node-failures-v4.patch, 0003-Report-streaming-errors-back-to-repair-v4.patch, 0004-Reports-validation-compaction-errors-back-to-repair-v4.patch, 2433.patch, 2433_v2.patch, 2433_v3.patch Running repair in cases where a stream fails we are seeing multiple problems. 1. Although retry is initiated and completes, the old stream doesn't seem to clean itself up and repair hangs. 2. The temp files are left behind and multiple failures can end up filling up the data partition. These issues together are making repair very difficult for nearly everyone running repair on a non-trivial sized data set. This issue is also being worked on w.r.t CASSANDRA-2088, however that was moved to 0.8 for a few reasons. This ticket is to fix the immediate issues that we are seeing in 0.7. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2433) Failed Streams Break Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094591#comment-13094591 ] Jonathan Ellis commented on CASSANDRA-2433: --- bq. we still need to have access to both the repair session and the future from the executor so the implementation returns a pair of those two objects You can still use the RepairFuture approach, just use the FutureTask(Runnable, V) constructor Failed Streams Break Repair --- Key: CASSANDRA-2433 URL: https://issues.apache.org/jira/browse/CASSANDRA-2433 Project: Cassandra Issue Type: Bug Components: Core Reporter: Benjamin Coverston Assignee: Sylvain Lebresne Labels: repair Fix For: 0.8.5 Attachments: 0001-Put-repair-session-on-a-Stage-and-add-a-method-to-re-v4.patch, 0002-Register-in-gossip-to-handle-node-failures-v4.patch, 0003-Report-streaming-errors-back-to-repair-v4.patch, 0004-Reports-validation-compaction-errors-back-to-repair-v4.patch, 2433.patch, 2433_v2.patch, 2433_v3.patch Running repair in cases where a stream fails we are seeing multiple problems. 1. Although retry is initiated and completes, the old stream doesn't seem to clean itself up and repair hangs. 2. The temp files are left behind and multiple failures can end up filling up the data partition. These issues together are making repair very difficult for nearly everyone running repair on a non-trivial sized data set. This issue is also being worked on w.r.t CASSANDRA-2088, however that was moved to 0.8 for a few reasons. This ticket is to fix the immediate issues that we are seeing in 0.7. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3104) remove compaction_thread_priority setting
[ https://issues.apache.org/jira/browse/CASSANDRA-3104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094594#comment-13094594 ] Sylvain Lebresne commented on CASSANDRA-3104: - Forgot to +1. remove compaction_thread_priority setting - Key: CASSANDRA-3104 URL: https://issues.apache.org/jira/browse/CASSANDRA-3104 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 1.0 Attachments: 3104.txt compaction_throughput_mb_per_sec is a more effective throttle on compaction. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3106) getRangeToEndpointMap() method removed
[ https://issues.apache.org/jira/browse/CASSANDRA-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094593#comment-13094593 ] Jonathan Ellis commented on CASSANDRA-3106: --- Why isn't rpcaddress sufficient? getRangeToEndpointMap() method removed -- Key: CASSANDRA-3106 URL: https://issues.apache.org/jira/browse/CASSANDRA-3106 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 1.0 Reporter: Nick Bailey Assignee: Nick Bailey Priority: Minor Fix For: 1.0 Attachments: 0001-Add-getRangeToEndpointMap-method-back.patch When getRangeToRPCAddress was added, getRangeToEndpointMap was removed, however, both are useful. We should add it back. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163657 - in /cassandra/trunk: ./ conf/ src/java/org/apache/cassandra/config/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/db/compaction/ src/java/org/apache/cassandra/
Author: jbellis Date: Wed Aug 31 15:02:23 2011 New Revision: 1163657 URL: http://svn.apache.org/viewvc?rev=1163657view=rev Log: r/m compaction_thread_priority setting patch by jbellis; reviewed by slebresne for CASSANDRA-3104 Modified: cassandra/trunk/CHANGES.txt cassandra/trunk/NEWS.txt cassandra/trunk/conf/cassandra.yaml cassandra/trunk/src/java/org/apache/cassandra/config/Config.java cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java cassandra/trunk/src/java/org/apache/cassandra/db/HintedHandOffManager.java cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java cassandra/trunk/src/java/org/apache/cassandra/net/MessagingService.java Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1163657r1=1163656r2=1163657view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Wed Aug 31 15:02:23 2011 @@ -44,13 +44,14 @@ Thrift-Avro conversion methods (CASSANDRA-3032) * Add timeouts to client request schedulers (CASSANDRA-3079) * Cli to use hashes rather than array of hashes for strategy options (CASSANDRA-3081) - * LeveledCompactionStrategy (CASSANDRA-1608, 3085) + * LeveledCompactionStrategy (CASSANDRA-1608, 3085, 3110) * Improvements of the CLI `describe` command (CASSANDRA-2630) * reduce window where dropped CF sstables may not be deleted (CASSANDRA-2942) * Expose gossip/FD info to JMX (CASSANDRA-2806) * Fix streaming over SSL when compressed SSTable involved (CASSANDRA-3051) * Add support for pluggable secondary index implementations (CASSANDRA-3078) - * Fix closing sstable iterators before using them (CASSANDRA-3110) + * remove compaction_thread_priority setting (CASSANDRA-3104) + 0.8.5 * fix NPE when encryption_options is unspecified (CASSANDRA-3007) Modified: cassandra/trunk/NEWS.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/NEWS.txt?rev=1163657r1=1163656r2=1163657view=diff == --- cassandra/trunk/NEWS.txt (original) +++ cassandra/trunk/NEWS.txt Wed Aug 31 15:02:23 2011 @@ -5,6 +5,9 @@ Upgrading - - the BinaryMemtable bulk-load interface has been removed. Use the sstableloader tool instead. +- the compaction_thread_priority setting has been removed from + cassandra.yaml (use compaction_throughput_mb_per_sec to throttle + compaction instead) Features Modified: cassandra/trunk/conf/cassandra.yaml URL: http://svn.apache.org/viewvc/cassandra/trunk/conf/cassandra.yaml?rev=1163657r1=1163656r2=1163657view=diff == --- cassandra/trunk/conf/cassandra.yaml (original) +++ cassandra/trunk/conf/cassandra.yaml Wed Aug 31 15:02:23 2011 @@ -253,13 +253,6 @@ incremental_backups: false # is a data format change. snapshot_before_compaction: false -# change this to increase the compaction thread's priority. In java, 1 is the -# lowest priority and that is our default. -# -# Under Linux, this cannot be higher than 5 (normal thread priority) without -# running as root. -# compaction_thread_priority: 1 - # Add column indexes to a row after its contents reach this size. # Increase if your column values are large, or if you have a very large # number of columns. The competing causes are, Cassandra has to Modified: cassandra/trunk/src/java/org/apache/cassandra/config/Config.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/config/Config.java?rev=1163657r1=1163656r2=1163657view=diff == --- cassandra/trunk/src/java/org/apache/cassandra/config/Config.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/config/Config.java Wed Aug 31 15:02:23 2011 @@ -77,8 +77,7 @@ public class Config public Integer thrift_max_message_length_in_mb = 16; public Integer thrift_framed_transport_size_in_mb = 15; public Boolean snapshot_before_compaction = false; -public Integer compaction_thread_priority = Thread.MIN_PRIORITY; - + /* if the size of columns or super-columns are more than this, indexing will kick in */ public Integer column_index_size_in_kb = 64; public Integer in_memory_compaction_limit_in_mb = 256; Modified: cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java?rev=1163657r1=1163656r2=1163657view=diff == --- cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java Wed
[jira] [Commented] (CASSANDRA-3001) Make the compression algorithm and chunk length configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094609#comment-13094609 ] Sylvain Lebresne commented on CASSANDRA-3001: - All tests are passing now that CASSANDRA-3110 has been committed. Make the compression algorithm and chunk length configurable Key: CASSANDRA-3001 URL: https://issues.apache.org/jira/browse/CASSANDRA-3001 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Labels: compression Fix For: 1.0 Attachments: 0001-Pluggable-algorithm-and-chunk-length.patch, 0002-Thrift-crap.patch, 0003-Add-deflate-compressor.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3104) remove compaction_thread_priority setting
[ https://issues.apache.org/jira/browse/CASSANDRA-3104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094613#comment-13094613 ] Hudson commented on CASSANDRA-3104: --- Integrated in Cassandra #1060 (See [https://builds.apache.org/job/Cassandra/1060/]) r/m compaction_thread_priority setting patch by jbellis; reviewed by slebresne for CASSANDRA-3104 jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163657 Files : * /cassandra/trunk/CHANGES.txt * /cassandra/trunk/NEWS.txt * /cassandra/trunk/conf/cassandra.yaml * /cassandra/trunk/src/java/org/apache/cassandra/config/Config.java * /cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java * /cassandra/trunk/src/java/org/apache/cassandra/db/HintedHandOffManager.java * /cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java * /cassandra/trunk/src/java/org/apache/cassandra/net/MessagingService.java remove compaction_thread_priority setting - Key: CASSANDRA-3104 URL: https://issues.apache.org/jira/browse/CASSANDRA-3104 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 1.0 Attachments: 3104.txt compaction_throughput_mb_per_sec is a more effective throttle on compaction. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3110) SSTables iterators are closed and before being used
[ https://issues.apache.org/jira/browse/CASSANDRA-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094612#comment-13094612 ] Hudson commented on CASSANDRA-3110: --- Integrated in Cassandra #1060 (See [https://builds.apache.org/job/Cassandra/1060/]) Fix closing sstable iterators before using them patch by slebresne; reviewed by jbellis for CASSANDRA-3110 slebresne : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163652 Files : * /cassandra/trunk/CHANGES.txt * /cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java SSTables iterators are closed and before being used --- Key: CASSANDRA-3110 URL: https://issues.apache.org/jira/browse/CASSANDRA-3110 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 3110.patch Seems there is misplaced finally blocks in CollationController: we close the sstable iterators and release the sstable references *before* having actually used said iterators/sstables. Note: this cause a lot of tests to fail in `ant test-compression` in particular. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3106) getRangeToEndpointMap() method removed
[ https://issues.apache.org/jira/browse/CASSANDRA-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094621#comment-13094621 ] Nick Bailey commented on CASSANDRA-3106: The rpc address of a node doesn't really give any guarantee of uniqueness. getRangeToEndpointMap() method removed -- Key: CASSANDRA-3106 URL: https://issues.apache.org/jira/browse/CASSANDRA-3106 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 1.0 Reporter: Nick Bailey Assignee: Nick Bailey Priority: Minor Fix For: 1.0 Attachments: 0001-Add-getRangeToEndpointMap-method-back.patch When getRangeToRPCAddress was added, getRangeToEndpointMap was removed, however, both are useful. We should add it back. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2433) Failed Streams Break Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2433: Attachment: 2433_v4.patch You're right, don't know why I got carried away like that. v4 fixes this. Failed Streams Break Repair --- Key: CASSANDRA-2433 URL: https://issues.apache.org/jira/browse/CASSANDRA-2433 Project: Cassandra Issue Type: Bug Components: Core Reporter: Benjamin Coverston Assignee: Sylvain Lebresne Labels: repair Fix For: 0.8.5 Attachments: 0001-Put-repair-session-on-a-Stage-and-add-a-method-to-re-v4.patch, 0002-Register-in-gossip-to-handle-node-failures-v4.patch, 0003-Report-streaming-errors-back-to-repair-v4.patch, 0004-Reports-validation-compaction-errors-back-to-repair-v4.patch, 2433.patch, 2433_v2.patch, 2433_v3.patch, 2433_v4.patch Running repair in cases where a stream fails we are seeing multiple problems. 1. Although retry is initiated and completes, the old stream doesn't seem to clean itself up and repair hangs. 2. The temp files are left behind and multiple failures can end up filling up the data partition. These issues together are making repair very difficult for nearly everyone running repair on a non-trivial sized data set. This issue is also being worked on w.r.t CASSANDRA-2088, however that was moved to 0.8 for a few reasons. This ticket is to fix the immediate issues that we are seeing in 0.7. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3001) Make the compression algorithm and chunk length configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094628#comment-13094628 ] Pavel Yaskevich commented on CASSANDRA-3001: Great, I will test and commit in few hours. Make the compression algorithm and chunk length configurable Key: CASSANDRA-3001 URL: https://issues.apache.org/jira/browse/CASSANDRA-3001 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Labels: compression Fix For: 1.0 Attachments: 0001-Pluggable-algorithm-and-chunk-length.patch, 0002-Thrift-crap.patch, 0003-Add-deflate-compressor.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3087) Leveled compaction allows multiple simultaneous compaction Tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-3087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Coverston updated CASSANDRA-3087: -- Attachment: 3087-v2.txt Cleaned up a few of the imports. +1 Leveled compaction allows multiple simultaneous compaction Tasks Key: CASSANDRA-3087 URL: https://issues.apache.org/jira/browse/CASSANDRA-3087 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 1.0 Attachments: 3087-v2.txt, 3087.txt CASSANDRA-1608 attempts to restrict itself to one compaction task per CF (see discussion there for why this is necessary) by synchronizing LCS.getBackgroundTasks but this is not sufficient. Consider this sequence of events: 1. getBackgroundTasks returns a Task for compacting some L0 sstables. this Task is scheduled. 2. Another SSTable for this CF is flushed, so CompactionManager.submitBackground is called. getBT is not currently in-progress so the synchronization does not stop another Task from being returned and scheduled. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163677 - in /cassandra/branches/cassandra-0.8: ./ src/java/org/apache/cassandra/gms/ src/java/org/apache/cassandra/service/ test/unit/org/apache/cassandra/service/
Author: slebresne
Date: Wed Aug 31 16:06:09 2011
New Revision: 1163677
URL: http://svn.apache.org/viewvc?rev=1163677view=rev
Log:
Make repair report failure when a participating node dies
patch by slebresne; reviewed by jbellis for CASSANDRA-2433
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/FailureDetector.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/Gossiper.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/IEndpointStateChangeSubscriber.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/IFailureDetectionEventListener.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AntiEntropyService.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/MigrationManager.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageLoadBalancer.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/service/AntiEntropyServiceTestAbstract.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1163677r1=1163676r2=1163677view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Wed Aug 31 16:06:09 2011
@@ -46,6 +46,8 @@
* Add validation that Keyspace names are case-insensitively unique
(CASSANDRA-3066)
* catch invalid key_validation_class before instantiating UpdateColumnFamily
(CASSANDRA-3102)
* make Range and Bounds objects client-safe (CASSANDRA-3108)
+ * make repair report failure when a node participating dies (instead of
+ hanging forever) (CASSANDRA-2433)
0.8.4
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/FailureDetector.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/FailureDetector.java?rev=1163677r1=1163676r2=1163677view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/FailureDetector.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/FailureDetector.java
Wed Aug 31 16:06:09 2011
@@ -152,7 +152,7 @@ public class FailureDetector implements
{
for ( IFailureDetectionEventListener listener : fdEvntListeners_ )
{
-listener.convict(ep);
+listener.convict(ep, phi);
}
}
}
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/Gossiper.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/Gossiper.java?rev=1163677r1=1163676r2=1163677view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/Gossiper.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/Gossiper.java
Wed Aug 31 16:06:09 2011
@@ -244,7 +244,7 @@ public class Gossiper implements IFailur
*
* param @ endpoint end point that is convicted.
*/
-public void convict(InetAddress endpoint)
+public void convict(InetAddress endpoint, double phi)
{
EndpointState epState = endpointStateMap.get(endpoint);
if (epState.isAlive())
@@ -717,12 +717,11 @@ public class Gossiper implements IFailur
if (logger.isTraceEnabled())
logger.trace(Adding endpoint state for + ep);
endpointStateMap.put(ep, epState);
-if (epState.isAlive())
-{
-// the node restarted before we ever marked it down, so we'll
report it as dead briefly so maintenance like resetting the connection pool can
occur
-for (IEndpointStateChangeSubscriber subscriber : subscribers)
-subscriber.onDead(ep, epState);
-}
+
+// the node restarted: it is up to the subscriber to take whatever
action is necessary
+for (IEndpointStateChangeSubscriber subscriber : subscribers)
+subscriber.onRestart(ep, epState);
+
if (epState.getApplicationState(ApplicationState.STATUS) != null
!isDeadState(epState.getApplicationState(ApplicationState.STATUS).value))
markAlive(ep, epState);
else
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/IEndpointStateChangeSubscriber.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/IEndpointStateChangeSubscriber.java?rev=1163677r1=1163676r2=1163677view=diff
svn commit: r1163685 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/gms/ src/java/org/apache/cassandra/locator/ src/java/org/ap
Author: slebresne Date: Wed Aug 31 16:24:13 2011 New Revision: 1163685 URL: http://svn.apache.org/viewvc?rev=1163685view=rev Log: merge from 0.8 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/gms/FailureDetector.java cassandra/trunk/src/java/org/apache/cassandra/gms/Gossiper.java cassandra/trunk/src/java/org/apache/cassandra/gms/IEndpointStateChangeSubscriber.java cassandra/trunk/src/java/org/apache/cassandra/gms/IFailureDetectionEventListener.java cassandra/trunk/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java cassandra/trunk/src/java/org/apache/cassandra/service/AntiEntropyService.java cassandra/trunk/src/java/org/apache/cassandra/service/LoadBroadcaster.java cassandra/trunk/src/java/org/apache/cassandra/service/MigrationManager.java cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java cassandra/trunk/test/unit/org/apache/cassandra/service/AntiEntropyServiceTestAbstract.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 16:24:13 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7:1026516-1163266 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1163272 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1163272,1163677 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1163685r1=1163684r2=1163685view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Wed Aug 31 16:24:13 2011 @@ -96,6 +96,8 @@ * fix ip address String representation in the ring cache (CASSANDRA-3044) * fix ring cache compatibility when mixing pre-0.8.4 nodes with post- in the same cluster (CASSANDRA-3023) + * make repair report failure when a node participating dies (instead of + hanging forever) (CASSANDRA-2433) 0.8.4 * include files-to-be-streamed in StreamInSession.getSources (CASSANDRA-2972) Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 16:24:13 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 /cassandra/branches/cassandra-0.7/contrib:1026516-1163266 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1163272 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1163272,1163677 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 16:24:13 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1163266 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1163272 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1163272,1163677 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369
[jira] [Reopened] (CASSANDRA-2388) ColumnFamilyRecordReader fails for a given split because a host is down, even if records could reasonably be read from other replica.
[ https://issues.apache.org/jira/browse/CASSANDRA-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mck SembWever reopened CASSANDRA-2388: -- see last comment. (say if this should be a separate bug...) ColumnFamilyRecordReader fails for a given split because a host is down, even if records could reasonably be read from other replica. - Key: CASSANDRA-2388 URL: https://issues.apache.org/jira/browse/CASSANDRA-2388 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 0.6 Reporter: Eldon Stegall Assignee: Mck SembWever Labels: hadoop, inputformat Fix For: 0.7.9, 0.8.5 Attachments: 0002_On_TException_try_next_split.patch, CASSANDRA-2388-addition1.patch, CASSANDRA-2388-extended.patch, CASSANDRA-2388.patch, CASSANDRA-2388.patch, CASSANDRA-2388.patch, CASSANDRA-2388.patch ColumnFamilyRecordReader only tries the first location for a given split. We should try multiple locations for a given split. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2734) JDBC driver assumes schema is constant for the duration of the connection
[
https://issues.apache.org/jira/browse/CASSANDRA-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-2734:
--
Attachment: (was: 0001-include-schema-with-resultset.txt)
JDBC driver assumes schema is constant for the duration of the connection
-
Key: CASSANDRA-2734
URL: https://issues.apache.org/jira/browse/CASSANDRA-2734
Project: Cassandra
Issue Type: Sub-task
Components: API
Affects Versions: 0.8.0
Reporter: Cathy Daw
Assignee: Jonathan Ellis
Priority: Minor
Labels: cql
Fix For: 1.0
Attachments: 0001-include-schema-with-resultset.patch,
0002-ant-gen-thrift-java.txt, 0003-thrift-gen.txt
*The following statement fails when used with a Statement or
PreparedStatement*
{code}
res = stmt.executeQuery(SELECT bar FROM users);
res.next();
{code}
*Error Message*
{code}
[junit] Testcase: simpleSelect(com.datastax.cql.reproBugTest):Caused
an ERROR
[junit] null
[junit] java.lang.NullPointerException
[junit] at
org.apache.cassandra.cql.jdbc.ColumnDecoder.makeKeyColumn(ColumnDecoder.java:136)
[junit] at
org.apache.cassandra.cql.jdbc.CResultSet.next(CResultSet.java:388)
[junit] at
com.datastax.cql.reproBugTest.simpleSelect(reproBugTest.java:57)
[junit]
[junit]
[junit] Test com.datastax.cql.reproBugTest FAILED
{code}
*Here is a quick repro. Showing that res.next() works with other statements
but not select.*
_Also notice that ResultSet.getMetaData().getColumnCount() always returns
zero._
_I noticed in the existing driver tests similar test cases, so not sure the
issue._
*Steps to run script*
* you will need to drop this in your test directory
* change the package declaration
* ant test -Dtest.name=reproBugTest
{code}
package com.datastax.cql;
import java.sql.DriverManager;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import org.junit.Test;
public class reproBugTest {
@Test
public void simpleSelect() throws Exception {
Connection connection = null;
ResultSet res;
Statement stmt;
int colCount = 0;
try {
Class.forName(org.apache.cassandra.cql.jdbc.CassandraDriver);
// Check create keyspace
connection =
DriverManager.getConnection(jdbc:cassandra:root/root@127.0.0.1:9160/default);
stmt = connection.createStatement();
try {
System.out.println(Running DROP KS Statement);
res = stmt.executeQuery(DROP KEYSPACE ks1);
res.next();
System.out.println(Running CREATE KS Statement);
res = stmt.executeQuery(CREATE KEYSPACE ks1 with
strategy_class = 'org.apache.cassandra.locator.SimpleStrategy' and
strategy_options:replication_factor=1);
res.next();
} catch (SQLException e) {
if (e.getMessage().startsWith(Keyspace does not exist))
{
res = stmt.executeQuery(CREATE KEYSPACE ks1 with
strategy_class = 'org.apache.cassandra.locator.SimpleStrategy' and
strategy_options:replication_factor=1);
}
}
connection.close();
// Run Test
connection =
DriverManager.getConnection(jdbc:cassandra:root/root@127.0.0.1:9160/ks1);
stmt = connection.createStatement();
System.out.print(Running CREATE CF Statement);
res = stmt.executeQuery(CREATE COLUMNFAMILY users (KEY varchar
PRIMARY KEY, password varchar, gender varchar, session_token varchar, state
varchar, birth_year bigint));
colCount = res.getMetaData().getColumnCount();
System.out.println( -- Column Count: + colCount);
res.next();
System.out.print(Running INSERT Statement);
res = stmt.executeQuery(INSERT INTO users (KEY, password) VALUES
('user1', 'ch@nge'));
colCount = res.getMetaData().getColumnCount();
System.out.println( -- Column Count: + colCount);
res.next();
System.out.print(Running SELECT Statement);
res = stmt.executeQuery(SELECT bar FROM users);
colCount = res.getMetaData().getColumnCount();
System.out.println( -- Column Count: + colCount);
res.getRow();
res.next();
connection.close();
} catch (SQLException e) {
svn commit: r1163688 - in /cassandra/trunk: ./ src/java/org/apache/cassandra/db/compaction/
Author: jbellis
Date: Wed Aug 31 16:32:37 2011
New Revision: 1163688
URL: http://svn.apache.org/viewvc?rev=1163688view=rev
Log:
fix race that allowed multiple simultaneous leveled compaction tasks
patch by jbellis; reviewed by Ben Coverston for CASSANDRA-3087
Modified:
cassandra/trunk/CHANGES.txt
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/LeveledCompactionTask.java
Modified: cassandra/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1163688r1=1163687r2=1163688view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Wed Aug 31 16:32:37 2011
@@ -44,7 +44,7 @@
Thrift-Avro conversion methods (CASSANDRA-3032)
* Add timeouts to client request schedulers (CASSANDRA-3079)
* Cli to use hashes rather than array of hashes for strategy options
(CASSANDRA-3081)
- * LeveledCompactionStrategy (CASSANDRA-1608, 3085, 3110)
+ * LeveledCompactionStrategy (CASSANDRA-1608, 3085, 3110, 3087)
* Improvements of the CLI `describe` command (CASSANDRA-2630)
* reduce window where dropped CF sstables may not be deleted (CASSANDRA-2942)
* Expose gossip/FD info to JMX (CASSANDRA-2806)
Modified:
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java?rev=1163688r1=1163687r2=1163688view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
Wed Aug 31 16:32:37 2011
@@ -66,7 +66,17 @@ public class CompactionManager implement
public static final String MBEAN_OBJECT_NAME =
org.apache.cassandra.db:type=CompactionManager;
private static final Logger logger =
LoggerFactory.getLogger(CompactionManager.class);
public static final CompactionManager instance;
-// acquire as read to perform a compaction, and as write to prevent
compactions
+
+/**
+ * compactionLock has two purposes:
+ * - Compaction acquires its readLock so that multiple compactions can
happen simultaneously,
+ * but the KS/CF migtations acquire its writeLock, so they can be sure
no new SSTables will
+ * be created for a dropped CF posthumously. (Thus, compaction checks
CFS.isValid while the
+ * lock is acquired.)
+ * - Special compactions will acquire writelock instead of readlock to
make sure that all
+ * other compaction activity is quiesced and they can grab ALL the
sstables to do something.
+ * TODO this is too big a hammer -- we should only care about quiescing
all for the given CFS.
+ */
private final ReentrantReadWriteLock compactionLock = new
ReentrantReadWriteLock();
static
@@ -143,7 +153,6 @@ public class CompactionManager implement
{
public Object call() throws IOException
{
-// acquire the write lock to schedule all sstables
compactionLock.writeLock().lock();
try
{
Modified:
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java?rev=1163688r1=1163687r2=1163688view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
Wed Aug 31 16:32:37 2011
@@ -5,6 +5,7 @@ import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicReference;
import org.apache.commons.lang.StringUtils;
import org.slf4j.Logger;
@@ -12,7 +13,6 @@ import org.slf4j.LoggerFactory;
import org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor;
import org.apache.cassandra.db.ColumnFamilyStore;
-import org.apache.cassandra.db.DataTracker;
import org.apache.cassandra.io.sstable.SSTableReader;
import org.apache.cassandra.notifications.INotification;
import org.apache.cassandra.notifications.INotificationConsumer;
@@ -27,6 +27,7 @@ public class LeveledCompactionStrategy e
private LeveledManifest manifest;
private final String SSTABLE_SIZE_OPTION = sstable_size_in_mb;
private final int maxSSTableSize;
+private final AtomicReferenceLeveledCompactionTask task = new
[jira] [Created] (CASSANDRA-3112) Make repair fail when an unexpected error occurs
Make repair fail when an unexpected error occurs Key: CASSANDRA-3112 URL: https://issues.apache.org/jira/browse/CASSANDRA-3112 Project: Cassandra Issue Type: Bug Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Fix For: 1.1 CASSANDRA-2433 makes it so that nodetool repair will fail if a node participating to repair dies before completing his part of the repair. This handles most of the situation where repair was previously hanging, but repair can still hang if an unexpected error occurs during either the merkle tree creation (an on-disk corruption triggers an IOError say) or during streaming (though I'm not sure what could make streaming failed outside of 'one of the node died' (besides a bug)). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3112) Make repair fail when an unexpected error occurs
[ https://issues.apache.org/jira/browse/CASSANDRA-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-3112: Attachment: 0004-Reports-validation-compaction-errors-back-to-repair-v4.patch 0003-Report-streaming-errors-back-to-repair-v4.patch Attaching the 2 last patches from an initial version of CASSANDRA-2433 that were handling this. This will need to be rebased to current code though. Make repair fail when an unexpected error occurs Key: CASSANDRA-3112 URL: https://issues.apache.org/jira/browse/CASSANDRA-3112 Project: Cassandra Issue Type: Bug Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Labels: repair Fix For: 1.1 Attachments: 0003-Report-streaming-errors-back-to-repair-v4.patch, 0004-Reports-validation-compaction-errors-back-to-repair-v4.patch CASSANDRA-2433 makes it so that nodetool repair will fail if a node participating to repair dies before completing his part of the repair. This handles most of the situation where repair was previously hanging, but repair can still hang if an unexpected error occurs during either the merkle tree creation (an on-disk corruption triggers an IOError say) or during streaming (though I'm not sure what could make streaming failed outside of 'one of the node died' (besides a bug)). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163691 - in /cassandra/trunk/src/java/org/apache/cassandra: cache/AutoSavingCache.java db/compaction/CompactionInfo.java db/compaction/CompactionIterable.java db/compaction/CompactionMan
Author: jbellis
Date: Wed Aug 31 16:41:26 2011
New Revision: 1163691
URL: http://svn.apache.org/viewvc?rev=1163691view=rev
Log:
add CompactionInfo.id field
patch by Stu Hood; reviewed by jbellis for CASSANDRA-3107
Modified:
cassandra/trunk/src/java/org/apache/cassandra/cache/AutoSavingCache.java
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionInfo.java
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
cassandra/trunk/src/java/org/apache/cassandra/db/index/SecondaryIndexBuilder.java
Modified:
cassandra/trunk/src/java/org/apache/cassandra/cache/AutoSavingCache.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cache/AutoSavingCache.java?rev=1163691r1=1163690r2=1163691view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/cache/AutoSavingCache.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/cache/AutoSavingCache.java
Wed Aug 31 16:41:26 2011
@@ -216,7 +216,8 @@ public abstract class AutoSavingCacheK,
else
type = CompactionType.UNKNOWN;
-info = new CompactionInfo(ksname,
+info = new CompactionInfo(this.hashCode(),
+ ksname,
cfname,
type,
0,
Modified:
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionInfo.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionInfo.java?rev=1163691r1=1163690r2=1163691view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionInfo.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionInfo.java
Wed Aug 31 16:41:26 2011
@@ -23,16 +23,18 @@ import java.io.Serializable;
/** Implements serializable to allow structured info to be returned via JMX. */
public final class CompactionInfo implements Serializable
{
+private static final long serialVersionUID = 3695381572726744816L;
-
+private final int id;
private final String ksname;
private final String cfname;
private final CompactionType tasktype;
private final long bytesComplete;
private final long totalBytes;
-public CompactionInfo(String ksname, String cfname, CompactionType
tasktype, long bytesComplete, long totalBytes)
+public CompactionInfo(int id, String ksname, String cfname, CompactionType
tasktype, long bytesComplete, long totalBytes)
{
+this.id = id;
this.ksname = ksname;
this.cfname = cfname;
this.tasktype = tasktype;
@@ -43,7 +45,12 @@ public final class CompactionInfo implem
/** @return A copy of this CompactionInfo with updated progress. */
public CompactionInfo forProgress(long bytesComplete, long totalBytes)
{
-return new CompactionInfo(ksname, cfname, tasktype, bytesComplete,
totalBytes);
+return new CompactionInfo(id, ksname, cfname, tasktype, bytesComplete,
totalBytes);
+}
+
+public int getId()
+{
+return id;
}
public String getKeyspace()
@@ -74,7 +81,7 @@ public final class CompactionInfo implem
public String toString()
{
StringBuilder buff = new StringBuilder();
-buff.append(getTaskType()).append('@').append(hashCode());
+buff.append(getTaskType()).append('@').append(id);
buff.append('(').append(getKeyspace()).append(,
).append(getColumnFamily());
buff.append(,
).append(getBytesComplete()).append('/').append(getTotalBytes());
return buff.append(')').toString();
Modified:
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java?rev=1163691r1=1163690r2=1163691view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
Wed Aug 31 16:41:26 2011
@@ -88,7 +88,8 @@ implements IterableAbstractCompactedRow
public CompactionInfo getCompactionInfo()
{
-return new CompactionInfo(controller.getKeyspace(),
+return new CompactionInfo(this.hashCode(),
+ controller.getKeyspace(),
controller.getColumnFamily(),
type,
bytesRead,
Modified:
[jira] [Updated] (CASSANDRA-3106) getRangeToEndpointMap() method removed
[ https://issues.apache.org/jira/browse/CASSANDRA-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3106: -- Reviewer: brandon.williams getRangeToEndpointMap() method removed -- Key: CASSANDRA-3106 URL: https://issues.apache.org/jira/browse/CASSANDRA-3106 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 1.0 Reporter: Nick Bailey Assignee: Nick Bailey Priority: Minor Fix For: 1.0 Attachments: 0001-Add-getRangeToEndpointMap-method-back.patch When getRangeToRPCAddress was added, getRangeToEndpointMap was removed, however, both are useful. We should add it back. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3003) Trunk single-pass streaming doesn't handle large row correctly
[
https://issues.apache.org/jira/browse/CASSANDRA-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita updated CASSANDRA-3003:
--
Attachment: v3003-v4.txt
Added handling of counters inside SuperColumn.
Trunk single-pass streaming doesn't handle large row correctly
--
Key: CASSANDRA-3003
URL: https://issues.apache.org/jira/browse/CASSANDRA-3003
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0
Reporter: Sylvain Lebresne
Assignee: Yuki Morishita
Priority: Critical
Labels: streaming
Fix For: 1.0
Attachments: 3003-v1.txt, 3003-v2.txt, 3003-v3.txt, v3003-v4.txt
For normal column family, trunk streaming always buffer the whole row into
memory. In uses
{noformat}
ColumnFamily.serializer().deserializeColumns(in, cf, true, true);
{noformat}
on the input bytes.
We must avoid this for rows that don't fit in the inMemoryLimit.
Note that for regular column families, for a given row, there is actually no
need to even recreate the bloom filter of column index, nor to deserialize
the columns. It is enough to filter the key and row size to feed the index
writer, but then simply dump the rest on disk directly. This would make
streaming more efficient, avoid a lot of object creation and avoid the
pitfall of big rows.
Counters column family are unfortunately trickier, because each column needs
to be deserialized (to mark them as 'fromRemote'). However, we don't need to
do the double pass of LazilyCompactedRow for that. We can simply use a
SSTableIdentityIterator and deserialize/reserialize input as it comes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163695 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/db/marshal/ReversedType.java
Author: slebresne
Date: Wed Aug 31 16:55:19 2011
New Revision: 1163695
URL: http://svn.apache.org/viewvc?rev=1163695view=rev
Log:
Fix handling of the empty byte buffer by ReversedType
patch by frousseau; reviewed by slebresne for CASSANDRA-3111
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/marshal/ReversedType.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1163695r1=1163694r2=1163695view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Wed Aug 31 16:55:19 2011
@@ -48,6 +48,7 @@
* make Range and Bounds objects client-safe (CASSANDRA-3108)
* make repair report failure when a node participating dies (instead of
hanging forever) (CASSANDRA-2433)
+ * fix handling of the empty byte buffer by ReversedType (CASSANDRA-3111)
0.8.4
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/marshal/ReversedType.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/marshal/ReversedType.java?rev=1163695r1=1163694r2=1163695view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/marshal/ReversedType.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/marshal/ReversedType.java
Wed Aug 31 16:55:19 2011
@@ -59,6 +59,16 @@ public class ReversedTypeT extends Abs
public int compare(ByteBuffer o1, ByteBuffer o2)
{
+// An empty byte buffer is always smaller
+if (o1.remaining() == 0)
+{
+return o2.remaining() == 0 ? 0 : -1;
+}
+if (o2.remaining() == 0)
+{
+return 1;
+}
+
return -baseType.compare(o1, o2);
}
[jira] [Commented] (CASSANDRA-2034) Make Read Repair unnecessary when Hinted Handoff is enabled
[ https://issues.apache.org/jira/browse/CASSANDRA-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094684#comment-13094684 ] Jonathan Ellis commented on CASSANDRA-2034: --- what messages is RemoveTest waiting for that are causing the problem? Make Read Repair unnecessary when Hinted Handoff is enabled --- Key: CASSANDRA-2034 URL: https://issues.apache.org/jira/browse/CASSANDRA-2034 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Patricio Echague Fix For: 1.0 Attachments: 2034-formatting.txt, 2034-v16.txt, 2034-v17.txt, 2034-v18.txt, 2034-v19-rebased.txt, 2034-v19.txt, 2034-v20.txt, 2034-v21.txt, CASSANDRA-2034-trunk-v10.patch, CASSANDRA-2034-trunk-v11.patch, CASSANDRA-2034-trunk-v11.patch, CASSANDRA-2034-trunk-v12.patch, CASSANDRA-2034-trunk-v13.patch, CASSANDRA-2034-trunk-v14.patch, CASSANDRA-2034-trunk-v15.patch, CASSANDRA-2034-trunk-v2.patch, CASSANDRA-2034-trunk-v3.patch, CASSANDRA-2034-trunk-v4.patch, CASSANDRA-2034-trunk-v5.patch, CASSANDRA-2034-trunk-v6.patch, CASSANDRA-2034-trunk-v7.patch, CASSANDRA-2034-trunk-v8.patch, CASSANDRA-2034-trunk-v9.patch, CASSANDRA-2034-trunk.patch Original Estimate: 8h Remaining Estimate: 8h Currently, HH is purely an optimization -- if a machine goes down, enabling HH means RR/AES will have less work to do, but you can't disable RR entirely in most situations since HH doesn't kick in until the FailureDetector does. Let's add a scheduled task to the mutate path, such that we return to the client normally after ConsistencyLevel is achieved, but after RpcTimeout we check the responseHandler write acks and write local hints for any missing targets. This would making disabling RR when HH is enabled a much more reasonable option, which has a huge impact on read throughput. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2034) Make Read Repair unnecessary when Hinted Handoff is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094699#comment-13094699
]
Patricio Echague commented on CASSANDRA-2034:
-
I fixed the test.
If I'm not wrong,
{code}
for (InetAddress host : hosts)
{
Message msg = new Message(host,
StorageService.Verb.REPLICATION_FINISHED, new byte[0],
MessagingService.version_);
MessagingService.instance().sendRR(msg,
FBUtilities.getBroadcastAddress());
}
{code}
sends 5 messages but some of them (4) are caught by a local SinkManager
implementation(for testing purposes) and not processed.
And since you added a wait for the callbacks to be processed before exiting, I
had to add a force shutdown (for testing purposes) in order to make the test
complete successfully.
Make Read Repair unnecessary when Hinted Handoff is enabled
---
Key: CASSANDRA-2034
URL: https://issues.apache.org/jira/browse/CASSANDRA-2034
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Jonathan Ellis
Assignee: Patricio Echague
Fix For: 1.0
Attachments: 2034-formatting.txt, 2034-v16.txt, 2034-v17.txt,
2034-v18.txt, 2034-v19-rebased.txt, 2034-v19.txt, 2034-v20.txt, 2034-v21.txt,
CASSANDRA-2034-trunk-v10.patch, CASSANDRA-2034-trunk-v11.patch,
CASSANDRA-2034-trunk-v11.patch, CASSANDRA-2034-trunk-v12.patch,
CASSANDRA-2034-trunk-v13.patch, CASSANDRA-2034-trunk-v14.patch,
CASSANDRA-2034-trunk-v15.patch, CASSANDRA-2034-trunk-v2.patch,
CASSANDRA-2034-trunk-v3.patch, CASSANDRA-2034-trunk-v4.patch,
CASSANDRA-2034-trunk-v5.patch, CASSANDRA-2034-trunk-v6.patch,
CASSANDRA-2034-trunk-v7.patch, CASSANDRA-2034-trunk-v8.patch,
CASSANDRA-2034-trunk-v9.patch, CASSANDRA-2034-trunk.patch
Original Estimate: 8h
Remaining Estimate: 8h
Currently, HH is purely an optimization -- if a machine goes down, enabling
HH means RR/AES will have less work to do, but you can't disable RR entirely
in most situations since HH doesn't kick in until the FailureDetector does.
Let's add a scheduled task to the mutate path, such that we return to the
client normally after ConsistencyLevel is achieved, but after RpcTimeout we
check the responseHandler write acks and write local hints for any missing
targets.
This would making disabling RR when HH is enabled a much more reasonable
option, which has a huge impact on read throughput.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3111) Bug in ReversedType comparator
[ https://issues.apache.org/jira/browse/CASSANDRA-3111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094702#comment-13094702 ] Hudson commented on CASSANDRA-3111: --- Integrated in Cassandra-0.8 #306 (See [https://builds.apache.org/job/Cassandra-0.8/306/]) Fix handling of the empty byte buffer by ReversedType patch by frousseau; reviewed by slebresne for CASSANDRA-3111 slebresne : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163695 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/marshal/ReversedType.java Bug in ReversedType comparator -- Key: CASSANDRA-3111 URL: https://issues.apache.org/jira/browse/CASSANDRA-3111 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.4 Reporter: Fabien Rousseau Priority: Minor Fix For: 0.8.5 Attachments: 001-CASSANDRA-3111.patch Scenario : * create a cf with a reversed comparator * insert a few columns in a row * try to read data with : SliceRange(start='', finish='', reversed=true) ** no data is returned, but some columns are expected * try to read data with : SliceRange(start='', finish='', reversed=false) ** if not flushed on disk : no data is returned -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3113) Bundle sstableloader with the Debian package
Bundle sstableloader with the Debian package Key: CASSANDRA-3113 URL: https://issues.apache.org/jira/browse/CASSANDRA-3113 Project: Cassandra Issue Type: Improvement Affects Versions: 0.8.1 Reporter: Joaquin Casares Add sstableloader to /usr/bin/ along with the other Cassandra tools. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3107) CompactionInfo id is not stable
[ https://issues.apache.org/jira/browse/CASSANDRA-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094704#comment-13094704 ] Hudson commented on CASSANDRA-3107: --- Integrated in Cassandra #1061 (See [https://builds.apache.org/job/Cassandra/1061/]) add CompactionInfo.id field patch by Stu Hood; reviewed by jbellis for CASSANDRA-3107 jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163691 Files : * /cassandra/trunk/src/java/org/apache/cassandra/cache/AutoSavingCache.java * /cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionInfo.java * /cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java * /cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java * /cassandra/trunk/src/java/org/apache/cassandra/db/index/SecondaryIndexBuilder.java CompactionInfo id is not stable --- Key: CASSANDRA-3107 URL: https://issues.apache.org/jira/browse/CASSANDRA-3107 Project: Cassandra Issue Type: Improvement Reporter: Stu Hood Assignee: Stu Hood Priority: Minor Fix For: 1.0 Attachments: 0001-Stable-id-for-CompactionInfo.txt The original intention of displaying the hashcode of the CompactionInfo for a compaction was that it would be stable and unique for the lifetime of the compaction. But at some point in the refactoring, we moved to new-object-per-call, which means that the hashcode is not stable. Giving the CompactionInfo object an id field that can be generated from the hashcode of the actual compaction task would solve this problem. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3087) Leveled compaction allows multiple simultaneous compaction Tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-3087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094705#comment-13094705 ] Hudson commented on CASSANDRA-3087: --- Integrated in Cassandra #1061 (See [https://builds.apache.org/job/Cassandra/1061/]) fix race that allowed multiple simultaneous leveled compaction tasks patch by jbellis; reviewed by Ben Coverston for CASSANDRA-3087 jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163688 Files : * /cassandra/trunk/CHANGES.txt * /cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java * /cassandra/trunk/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java * /cassandra/trunk/src/java/org/apache/cassandra/db/compaction/LeveledCompactionTask.java Leveled compaction allows multiple simultaneous compaction Tasks Key: CASSANDRA-3087 URL: https://issues.apache.org/jira/browse/CASSANDRA-3087 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 1.0 Attachments: 3087-v2.txt, 3087.txt CASSANDRA-1608 attempts to restrict itself to one compaction task per CF (see discussion there for why this is necessary) by synchronizing LCS.getBackgroundTasks but this is not sufficient. Consider this sequence of events: 1. getBackgroundTasks returns a Task for compacting some L0 sstables. this Task is scheduled. 2. Another SSTable for this CF is flushed, so CompactionManager.submitBackground is called. getBT is not currently in-progress so the synchronization does not stop another Task from being returned and scheduled. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3096) Test RoundRobinScheduler timeouts
[ https://issues.apache.org/jira/browse/CASSANDRA-3096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094728#comment-13094728 ] Ryan King commented on CASSANDRA-3096: -- looks good, +1 Test RoundRobinScheduler timeouts - Key: CASSANDRA-3096 URL: https://issues.apache.org/jira/browse/CASSANDRA-3096 Project: Cassandra Issue Type: Bug Components: API Reporter: Stu Hood Assignee: Stu Hood Fix For: 1.0 Attachments: 0001-Properly-throw-timeouts-decrement-the-count-of-waiters.txt CASSANDRA-3079 was very hasty, and introduced two bugs that would: 1) cause the scheduler to busywait after a timeout, 2) never actually throw timeouts. This calls for a test. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163731 - in /cassandra/trunk: src/java/org/apache/cassandra/gms/Gossiper.java src/java/org/apache/cassandra/service/StorageService.java test/unit/org/apache/cassandra/service/RemoveTest.
Author: jbellis
Date: Wed Aug 31 18:17:06 2011
New Revision: 1163731
URL: http://svn.apache.org/viewvc?rev=1163731view=rev
Log:
add delay parameter to removeToken so RemoveTest doesn't have to wait for full
RING_DELAY
patch by jbellis
Modified:
cassandra/trunk/src/java/org/apache/cassandra/gms/Gossiper.java
cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
cassandra/trunk/test/unit/org/apache/cassandra/service/RemoveTest.java
Modified: cassandra/trunk/src/java/org/apache/cassandra/gms/Gossiper.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/gms/Gossiper.java?rev=1163731r1=1163730r2=1163731view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/gms/Gossiper.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/gms/Gossiper.java Wed Aug 31
18:17:06 2011
@@ -362,8 +362,9 @@ public class Gossiper implements IFailur
* @param endpoint - the endpoint being removed
* @param token - the token being removed
* @param mytoken - my own token for replication coordination
+ * @param delay
*/
-public void advertiseRemoving(InetAddress endpoint, Token token, Token
mytoken)
+public void advertiseRemoving(InetAddress endpoint, Token token, Token
mytoken, int delay)
{
EndpointState epState = endpointStateMap.get(endpoint);
// remember this node's generation
@@ -372,7 +373,7 @@ public class Gossiper implements IFailur
logger.info(Sleeping for + StorageService.RING_DELAY + ms to
ensure + endpoint + does not change);
try
{
-Thread.sleep(StorageService.RING_DELAY);
+Thread.sleep(delay);
}
catch (InterruptedException e)
{
Modified:
cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java?rev=1163731r1=1163730r2=1163731view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
Wed Aug 31 18:17:06 2011
@@ -27,6 +27,7 @@ import java.net.UnknownHostException;
import java.nio.ByteBuffer;
import java.util.*;
import java.util.concurrent.*;
+import javax.lang.model.type.TypeKind;
import javax.management.MBeanServer;
import javax.management.ObjectName;
@@ -2107,6 +2108,11 @@ public class StorageService implements I
*/
public void removeToken(String tokenString)
{
+removeToken(tokenString, RING_DELAY);
+}
+
+public void removeToken(String tokenString, int delay)
+{
InetAddress myAddress = FBUtilities.getBroadcastAddress();
Token localToken = tokenMetadata_.getToken(myAddress);
Token token = partitioner.getTokenFactory().fromString(tokenString);
@@ -2153,7 +2159,7 @@ public class StorageService implements I
calculatePendingRanges();
// the gossiper will handle spoofing this node's state to
REMOVING_TOKEN for us
// we add our own token so other nodes to let us know when they're done
-Gossiper.instance.advertiseRemoving(endpoint, token, localToken);
+Gossiper.instance.advertiseRemoving(endpoint, token, localToken,
delay);
// kick off streaming commands
restoreReplicaCount(endpoint, myAddress);
Modified: cassandra/trunk/test/unit/org/apache/cassandra/service/RemoveTest.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/test/unit/org/apache/cassandra/service/RemoveTest.java?rev=1163731r1=1163730r2=1163731view=diff
==
--- cassandra/trunk/test/unit/org/apache/cassandra/service/RemoveTest.java
(original)
+++ cassandra/trunk/test/unit/org/apache/cassandra/service/RemoveTest.java Wed
Aug 31 18:17:06 2011
@@ -124,7 +124,7 @@ public class RemoveTest extends CleanupH
{
try
{
-ss.removeToken(token);
+ss.removeToken(token, 0);
}
catch (Exception e)
{
[jira] [Commented] (CASSANDRA-3003) Trunk single-pass streaming doesn't handle large row correctly
[
https://issues.apache.org/jira/browse/CASSANDRA-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094753#comment-13094753
]
Sylvain Lebresne commented on CASSANDRA-3003:
-
I think this is a little bit sad to deserialize all the columns in the
non-counter case. We do need to do it right now because of the computation of
the max timestamp, but maybe we could have the other side send use the max
timestamp as part of the stream header (but I agree, it's a bit more
complicated).
For the record, the handling of counter columns amounts to the initial
proposition of Stu of moving the cleanup to the reads (though the solution is
slightly different). So the we'll cleanup on each read before the sstable is
compacted remark does hold here, but I don't see a better solution right now
and the those sstables will likely be compacted quickly argument probably
make this ok anyway.
Other comments:
* we need to use Integer.MIN_VALUE as the value for expireBefore when
deserializing the columns, otherwise the expired columns will be converted to
DeletedColumns, which will change there serialized size (and thus screw up the
data size and column index)
* for markDeltaAsDeleted, we must check if the length is already negative and
leave it so if it is, otherwise if a streamed sstable get re-streamed to
another node before it was compacted, we could end up not cleaning the delta
correctly.
* it would be nice in SSTW.appendFromStream() to assert the sanity of our
little deserialize-reserialize dance and assert what we did write the number of
bytes that we wrote in the header.
* the patch change a clearAllDelta to a markDeltaAsDeleted in CounterColumnTest
which is bogus (and the test does fail with that change).
* I would markDeltaAsDeleted to markForClearingDelta as this describe what the
function does better
* nitpick: there is a few space at end of lines in some comments (I know I
know, I'm picky).
Trunk single-pass streaming doesn't handle large row correctly
--
Key: CASSANDRA-3003
URL: https://issues.apache.org/jira/browse/CASSANDRA-3003
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0
Reporter: Sylvain Lebresne
Assignee: Yuki Morishita
Priority: Critical
Labels: streaming
Fix For: 1.0
Attachments: 3003-v1.txt, 3003-v2.txt, 3003-v3.txt, v3003-v4.txt
For normal column family, trunk streaming always buffer the whole row into
memory. In uses
{noformat}
ColumnFamily.serializer().deserializeColumns(in, cf, true, true);
{noformat}
on the input bytes.
We must avoid this for rows that don't fit in the inMemoryLimit.
Note that for regular column families, for a given row, there is actually no
need to even recreate the bloom filter of column index, nor to deserialize
the columns. It is enough to filter the key and row size to feed the index
writer, but then simply dump the rest on disk directly. This would make
streaming more efficient, avoid a lot of object creation and avoid the
pitfall of big rows.
Counters column family are unfortunately trickier, because each column needs
to be deserialized (to mark them as 'fromRemote'). However, we don't need to
do the double pass of LazilyCompactedRow for that. We can simply use a
SSTableIdentityIterator and deserialize/reserialize input as it comes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2434) node bootstrapping can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul cannon updated CASSANDRA-2434: --- Attachment: 2434.patch.txt This still needs some testing, but I'm putting it up now in case anyone has some time to take a look and make sure my approach is sane. Adds an optional -n argument to nodetool join to allow bootstrapping from closest live node (n == non-strict). Also recognizes an optional property cassandra.join_strict which can be set to false when a bootstrap is triggered by cassandra.join_ring. node bootstrapping can violate consistency -- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: paul cannon Fix For: 1.1 Attachments: 2434.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163753 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/cql/ src/java/org/apache/cassandra/dht/ src/java/org/apache
Author: jbellis Date: Wed Aug 31 19:26:21 2011 New Revision: 1163753 URL: http://svn.apache.org/viewvc?rev=1163753view=rev Log: merge from 0.8 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/cql/QueryProcessor.java cassandra/trunk/src/java/org/apache/cassandra/dht/Bounds.java cassandra/trunk/src/java/org/apache/cassandra/dht/Range.java cassandra/trunk/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java cassandra/trunk/src/java/org/apache/cassandra/service/ClientState.java cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 19:26:21 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7:1026516-1163266 +/cassandra/branches/cassandra-0.7:1026516-1163280 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1163272,1163677,1163695 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1163752 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1163753r1=1163752r2=1163753view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Wed Aug 31 19:26:21 2011 @@ -99,6 +99,10 @@ * make repair report failure when a node participating dies (instead of hanging forever) (CASSANDRA-2433) * fix handling of the empty byte buffer by ReversedType (CASSANDRA-3111) + * Add validation that Keyspace names are case-insensitively unique (CASSANDRA-3066) + * catch invalid key_validation_class before instantiating UpdateColumnFamily (CASSANDRA-3102) + * make Range and Bounds objects client-safe (CASSANDRA-3108) + * optionally skip log4j configuration (CASSANDRA-3061) 0.8.4 * include files-to-be-streamed in StreamInSession.getSources (CASSANDRA-2972) Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 19:26:21 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1163266 +/cassandra/branches/cassandra-0.7/contrib:1026516-1163280 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1163272,1163677,1163695 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1163752 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 19:26:21 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1163266 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1163280 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1163272,1163677,1163695 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1163752
buildbot failure in ASF Buildbot on cassandra-trunk
The Buildbot has detected a new failure on builder cassandra-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1583 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: scheduler Build Source Stamp: [branch cassandra/trunk] 1163753 Blamelist: jbellis BUILD FAILED: failed compile sincerely, -The Buildbot
[jira] [Reopened] (CASSANDRA-3076) AssertionError in new GCInspector log
[
https://issues.apache.org/jira/browse/CASSANDRA-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reopened CASSANDRA-3076:
---
Still seeing this post-patch in trunk:
{noformat}
[junit] ERROR 14:23:17,606 Fatal exception in thread
Thread[ScheduledTasks:1,5,main]
[junit] java.lang.ArithmeticException: / by zero
[junit] at
org.apache.cassandra.service.GCInspector.logGCResults(GCInspector.java:119)
[junit] at
org.apache.cassandra.service.GCInspector.access$000(GCInspector.java:41)
[junit] at
org.apache.cassandra.service.GCInspector$1.run(GCInspector.java:85)
[junit] at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
[junit] at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
[junit] at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
[junit] at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
[junit] at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
[junit] at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
[junit] at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[junit] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[junit] at java.lang.Thread.run(Thread.java:680)
{noformat}
AssertionError in new GCInspector log
-
Key: CASSANDRA-3076
URL: https://issues.apache.org/jira/browse/CASSANDRA-3076
Project: Cassandra
Issue Type: Bug
Environment: Lion OSX
Reporter: T Jake Luciani
Assignee: T Jake Luciani
Priority: Minor
Fix For: 0.7.9, 0.8.5
Attachments: 3076.txt
Small regression from CASSANDRA-2868
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163759 - /cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java
Author: jbellis
Date: Wed Aug 31 19:31:39 2011
New Revision: 1163759
URL: http://svn.apache.org/viewvc?rev=1163759view=rev
Log:
fix merge
Modified:
cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java
Modified:
cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java?rev=1163759r1=1163758r2=1163759view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java
Wed Aug 31 19:31:39 2011
@@ -725,7 +725,7 @@ public class ThriftValidation
// keyspace names must be unique case-insensitively because the
keyspace name becomes the directory
// where we store CF sstables. Names that differ only in case would
thus cause problems on
// case-insensitive filesystems (NTFS, most installations of HFS+).
-for (String ksName : DatabaseDescriptor.getTables())
+for (String ksName : Schema.instance.getTables())
{
if (ksName.equalsIgnoreCase(newKsName))
throw new InvalidRequestException(Keyspace names must be
case-insensitively unique);
[jira] [Commented] (CASSANDRA-2434) node bootstrapping can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094818#comment-13094818 ] Nick Bailey commented on CASSANDRA-2434: Seems to me like the option to stream from the closest replica might just add more confusion without really gaining anything. The node that is leaving the replica set will never be in another datacenter. It could be on a different rack, but if you are following best practices and alternating racks then it is likely either on the same rack or there is only one copy on that rack and the best case possible is streaming from another rack anyway. node bootstrapping can violate consistency -- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: paul cannon Fix For: 1.1 Attachments: 2434.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build on builder cassandra-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1584 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: scheduler Build Source Stamp: [branch cassandra/trunk] 1163759 Blamelist: jbellis Build succeeded! sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-3061) Optionally skip log4j configuration
[ https://issues.apache.org/jira/browse/CASSANDRA-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094823#comment-13094823 ] Hudson commented on CASSANDRA-3061: --- Integrated in Cassandra-0.8 #307 (See [https://builds.apache.org/job/Cassandra-0.8/307/]) Optionally skip log4j configuration Patch by tjake; reviewed by Aaron Morton for CASSANDRA-3061 jake : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163736 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java Optionally skip log4j configuration --- Key: CASSANDRA-3061 URL: https://issues.apache.org/jira/browse/CASSANDRA-3061 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8.4 Reporter: Aaron Morton Assignee: T Jake Luciani Priority: Minor Fix For: 0.8.5 Attachments: 0001-3061.patch, 3061_v2.txt from this thread http://groups.google.com/group/brisk-users/browse_thread/thread/3a18f4679673bea8 When brisk accesses cassandra classes inside of a Hadoop Task JVM the AbstractCassandraDaemon uses a log4j PropertyConfigurator to setup cassandra logging. This closes all the existing appenders, including the TaskLogAppender for the hadoop task. They are not opened again because they are not in the config. log4j has Logger Repositories to handle multiple configs in the same process, but there is a bit of suck involved in making a RepositorySelector. Two examples... http://www.mail-archive.com/log4j-dev@jakarta.apache.org/msg02972.html http://docs.redhat.com/docs/en-US/JBoss_Enterprise_Application_Platform/4.2/html/Getting_Started_Guide/logging.log4j.reposelect.html Basically all the selector has access to thread local storage, and it looks like normally people get the class loader from the current thread. A thread will inherit it's class loader from the thread that created it, unless otherwise specified. We have code in the same thread the uses hadoop and cassandra classes, so this could be a dead end. As a work around i've added cassandra.log4j.configure JVM param and made the AbstractCassandraServer skip the log4j config if it's false. My job completes and I can see the cassandra code logging an extra message I put in into the Hadoop task log file... 2011-08-19 15:56:06,442 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Metrics system not started: Cannot locate configuration: tried hadoop-metrics2-maptask.properties, hadoop-metrics2.properties 2011-08-19 15:56:06,776 INFO org.apache.cassandra.service.AbstractCassandraDaemon: Logging initialized externally 2011-08-19 15:56:07,332 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 0 The param has to be passed to the task JVM, so need to modify Haddop mapred-site.xml as follows property namemapred.child.java.opts/name value-Xmx256m -Dcassandra.log4j.configure=false/value description Tune your mapred jvm arguments for best performance. Also see documentation from jvm vendor. /description /property It's not pretty but it works. In my extra log4j logging I can see the second reset() call is gone. Change the to Hadoop TaskLogAppender also stops the NPE but there may also be some lost log messages https://issues.apache.org/jira/browse/HADOOP-7556 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163762 - /cassandra/trunk/NEWS.txt
Author: jbellis Date: Wed Aug 31 19:41:51 2011 New Revision: 1163762 URL: http://svn.apache.org/viewvc?rev=1163762view=rev Log: update NEWS for #2034 Modified: cassandra/trunk/NEWS.txt Modified: cassandra/trunk/NEWS.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/NEWS.txt?rev=1163762r1=1163761r2=1163762view=diff == --- cassandra/trunk/NEWS.txt (original) +++ cassandra/trunk/NEWS.txt Wed Aug 31 19:41:51 2011 @@ -19,6 +19,12 @@ Features Leveled compaction means you only need to keep a few MB of space free for compaction instead of (in the worst case) 50%. - Windows Service (cassandra.bat install to enable) +- Hinted Handoff has two major improvements: +- Hint replay is much more efficient thanks to a change in the data model +- Hints are created for all replicas that do not ack a write. (Formerly, + only replicas known to be down when the write started were hinted.) + This means that running with read repair completely off is much more + viable than before. Other -
[jira] [Updated] (CASSANDRA-3076) AssertionError in new GCInspector log
[ https://issues.apache.org/jira/browse/CASSANDRA-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-3076: -- Attachment: v1-0001-CASSANDRA-3076-avoid-div-by-zero.txt AssertionError in new GCInspector log - Key: CASSANDRA-3076 URL: https://issues.apache.org/jira/browse/CASSANDRA-3076 Project: Cassandra Issue Type: Bug Environment: Lion OSX Reporter: T Jake Luciani Assignee: T Jake Luciani Priority: Minor Fix For: 0.7.9, 0.8.5 Attachments: 3076.txt, v1-0001-CASSANDRA-3076-avoid-div-by-zero.txt Small regression from CASSANDRA-2868 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3076) AssertionError in new GCInspector log
[ https://issues.apache.org/jira/browse/CASSANDRA-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094844#comment-13094844 ] Jonathan Ellis commented on CASSANDRA-3076: --- +1 new patch AssertionError in new GCInspector log - Key: CASSANDRA-3076 URL: https://issues.apache.org/jira/browse/CASSANDRA-3076 Project: Cassandra Issue Type: Bug Environment: Lion OSX Reporter: T Jake Luciani Assignee: T Jake Luciani Priority: Minor Fix For: 0.7.9, 0.8.5 Attachments: 3076.txt, v1-0001-CASSANDRA-3076-avoid-div-by-zero.txt Small regression from CASSANDRA-2868 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163772 - in /cassandra/trunk/src/java/org/apache/cassandra/service: StorageService.java StorageServiceMBean.java
Author: brandonwilliams
Date: Wed Aug 31 19:57:17 2011
New Revision: 1163772
URL: http://svn.apache.org/viewvc?rev=1163772view=rev
Log:
Restore getRangeToEndpointMap.
Patch by Nick Bailey, reviewed by brandonwilliams for CASSANDRA-3106
Modified:
cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
cassandra/trunk/src/java/org/apache/cassandra/service/StorageServiceMBean.java
Modified:
cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java?rev=1163772r1=1163771r2=1163772view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
Wed Aug 31 19:57:17 2011
@@ -161,7 +161,7 @@ public class StorageService implements I
private IPartitioner partitioner = DatabaseDescriptor.getPartitioner();
public VersionedValue.VersionedValueFactory valueFactory = new
VersionedValue.VersionedValueFactory(partitioner);
-
+
public static final StorageService instance = new StorageService();
public static IPartitioner getPartitioner()
@@ -249,7 +249,7 @@ public class StorageService implements I
MessagingService.instance().registerVerbHandlers(Verb.GOSSIP_DIGEST_SYN, new
GossipDigestSynVerbHandler());
MessagingService.instance().registerVerbHandlers(Verb.GOSSIP_DIGEST_ACK, new
GossipDigestAckVerbHandler());
MessagingService.instance().registerVerbHandlers(Verb.GOSSIP_DIGEST_ACK2, new
GossipDigestAck2VerbHandler());
-
+
MessagingService.instance().registerVerbHandlers(Verb.DEFINITIONS_UPDATE, new
DefinitionsUpdateVerbHandler());
MessagingService.instance().registerVerbHandlers(Verb.TRUNCATE, new
TruncateVerbHandler());
MessagingService.instance().registerVerbHandlers(Verb.SCHEMA_CHECK,
new SchemaCheckVerbHandler());
@@ -325,10 +325,10 @@ public class StorageService implements I
try { Thread.sleep(1000L); } catch (InterruptedException e) {}
StageManager.shutdownNow();
}
-
-public boolean isInitialized()
-{
-return initialized;
+
+public boolean isInitialized()
+{
+return initialized;
}
public synchronized void initClient() throws IOException,
ConfigurationException
@@ -351,7 +351,7 @@ public class StorageService implements I
Gossiper.instance.register(this);
Gossiper.instance.start((int)(System.currentTimeMillis() / 1000)); //
needed for node-ring gathering.
MessagingService.instance().listen(FBUtilities.getLocalAddress());
-
+
// sleep a while to allow gossip to warm up (the other nodes need to
know about this one before they can reply).
try
{
@@ -597,6 +597,27 @@ public class StorageService implements I
}
/**
+ * for a keyspace, return the ranges and corresponding listen addresses.
+ * @param keyspace
+ * @return
+ */
+public MapRange, ListString getRangeToEndpointMap(String keyspace)
+{
+// some people just want to get a visual representation of things.
Allow null and set it to the first
+// non-system table.
+if (keyspace == null)
+keyspace = Schema.instance.getNonSystemTables().get(0);
+
+/* All the ranges for the tokens */
+MapRange, ListString map = new HashMapRange, ListString();
+for (Map.EntryRange,ListInetAddress entry :
getRangeToAddressMap(keyspace).entrySet())
+{
+map.put(entry.getKey(), stringify(entry.getValue()));
+}
+return map;
+}
+
+/**
* for a keyspace, return the ranges and corresponding RPC addresses for a
given keyspace.
* @param keyspace
* @return
@@ -686,7 +707,7 @@ public class StorageService implements I
* the token associated with an endpoint, and knowing its operation mode.
Nodes can start in either bootstrap or
* normal mode, and from bootstrap mode can change mode to normal. A node
in bootstrap mode needs to have
* pendingranges set in TokenMetadata; a node in normal mode should
instead be part of the token ring.
- *
+ *
* Normal progression of ApplicationState.STATUS values for a node should
be like this:
* STATUS_BOOTSTRAPPING,token
* if bootstrapping. stays this way until all files are received.
@@ -1091,7 +1112,7 @@ public class StorageService implements I
* @param ranges the ranges to find sources for
* @return multimap of addresses to ranges the address is responsible for
*/
-private MultimapInetAddress, Range getNewSourceRanges(String table,
SetRange ranges)
+private MultimapInetAddress, Range getNewSourceRanges(String table,
SetRange ranges)
{
InetAddress
[jira] [Commented] (CASSANDRA-2434) node bootstrapping can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094848#comment-13094848 ] Jonathan Ellis commented on CASSANDRA-2434: --- bq. what about the case where the node leaving the replica set is dead Good point. We do need something to make that possible. node bootstrapping can violate consistency -- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: paul cannon Fix For: 1.1 Attachments: 2434.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2434) node bootstrapping can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094845#comment-13094845 ] paul cannon commented on CASSANDRA-2434: Judging by the irc channel and user list, assuming people will follow best practices seems a bit of a dead end. Plus, what about the case where the node leaving the replica set is dead? You still want the option to allow choosing another to stream from. And we probably shouldn't default to choosing another without explicit permission, because of the consistency violation stuff. node bootstrapping can violate consistency -- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: paul cannon Fix For: 1.1 Attachments: 2434.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2434) node bootstrapping can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094846#comment-13094846 ] Jonathan Ellis commented on CASSANDRA-2434: --- bq. The node that is leaving the replica set will never be in another datacenter I think that's only strictly true for NTS, but I'm fine leaving it out. Not worth adding complexity for ONTS at this point. node bootstrapping can violate consistency -- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: paul cannon Fix For: 1.1 Attachments: 2434.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2434) node bootstrapping can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094850#comment-13094850 ] Nick Bailey commented on CASSANDRA-2434: Yeah I was assuming ONTS is basically deprecated at this point. Didn't think about the dead case though. I suppose just a 'force' type of option and a warning indicating the possible consistency issues works. node bootstrapping can violate consistency -- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: paul cannon Fix For: 1.1 Attachments: 2434.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2434) node bootstrapping can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094853#comment-13094853 ] Jonathan Ellis commented on CASSANDRA-2434: --- As long as we need to handle the dead case I don't see any harm in having a slightly more generally-useful use closest option instead of force to pick random live replica option. node bootstrapping can violate consistency -- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: paul cannon Fix For: 1.1 Attachments: 2434.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163782 - /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/GCInspector.java
Author: jake Date: Wed Aug 31 20:04:51 2011 New Revision: 1163782 URL: http://svn.apache.org/viewvc?rev=1163782view=rev Log: Fix div by zero error take two patch by tjake; reviewed by jbellis for CASSANDRA-3076 Modified: cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/GCInspector.java Modified: cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/GCInspector.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/GCInspector.java?rev=1163782r1=1163781r2=1163782view=diff == --- cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/GCInspector.java (original) +++ cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/GCInspector.java Wed Aug 31 20:04:51 2011 @@ -103,10 +103,12 @@ public class GCInspector Long previousCount = gccounts.get(gc.getName()); Long count = gc.getCollectionCount(); -if (count == 0) -continue; + if (previousCount == null) -previousCount = 0L; +previousCount = 0L; +if (count == previousCount) +continue; + gccounts.put(gc.getName(), count); MemoryUsage mu = membean.getHeapMemoryUsage();
svn commit: r1163783 - in /cassandra/branches/cassandra-0.8: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/service/
Author: jake Date: Wed Aug 31 20:05:50 2011 New Revision: 1163783 URL: http://svn.apache.org/viewvc?rev=1163783view=rev Log: merge from 0.7 Modified: cassandra/branches/cassandra-0.8/ (props changed) cassandra/branches/cassandra-0.8/contrib/ (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/GCInspector.java Propchange: cassandra/branches/cassandra-0.8/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 20:05:50 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7:1026516-1163280 +/cassandra/branches/cassandra-0.7:1026516-1163782 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/branches/cassandra-0.8:1090934-1125013,1125041 /cassandra/branches/cassandra-0.8.0:1125021-1130369 Propchange: cassandra/branches/cassandra-0.8/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 20:05:50 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1163280 +/cassandra/branches/cassandra-0.7/contrib:1026516-1163782 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125041 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 20:05:50 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1163280 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1163782 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125041 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 20:05:50 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1163280 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1163782 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654 /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1090934-1125013,1125041 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1125021-1130369 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 20:05:50 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516-1163280
[jira] [Resolved] (CASSANDRA-3076) AssertionError in new GCInspector log
[ https://issues.apache.org/jira/browse/CASSANDRA-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani resolved CASSANDRA-3076. --- Resolution: Fixed committed AssertionError in new GCInspector log - Key: CASSANDRA-3076 URL: https://issues.apache.org/jira/browse/CASSANDRA-3076 Project: Cassandra Issue Type: Bug Environment: Lion OSX Reporter: T Jake Luciani Assignee: T Jake Luciani Priority: Minor Fix For: 0.7.9, 0.8.5 Attachments: 3076.txt, v1-0001-CASSANDRA-3076-avoid-div-by-zero.txt Small regression from CASSANDRA-2868 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163784 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/service/
Author: jake Date: Wed Aug 31 20:06:43 2011 New Revision: 1163784 URL: http://svn.apache.org/viewvc?rev=1163784view=rev Log: merge from 0.8 Modified: cassandra/trunk/ (props changed) cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/service/GCInspector.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 20:06:43 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7:1026516-1163280 +/cassandra/branches/cassandra-0.7:1026516-1163782 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1163752 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1163783 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 20:06:43 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1163280 +/cassandra/branches/cassandra-0.7/contrib:1026516-1163782 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1163752 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1163783 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 20:06:43 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1163280 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1163782 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1163752 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1163783 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Aug 31 20:06:43 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1163280 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1163782 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1090934-1125013,1125019-1163752 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1090934-1125013,1125019-1163783
[jira] [Commented] (CASSANDRA-2434) node bootstrapping can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094864#comment-13094864 ] Nick Bailey commented on CASSANDRA-2434: Well, I imagine the 'force' option would pick the nearest live node. By 'force' I mean the option should be posed to the user as We can't guarantee consistency in your cluster after this bootstrap since a node is down, if you would like to do this anyway, specify option X. Just saying you can either bootstrap or bootstrap from the closest node doesn't convey the implications as well I don't think. Maybe we are on the same page and arguing over wording though. node bootstrapping can violate consistency -- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: paul cannon Fix For: 1.1 Attachments: 2434.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163795 - in /cassandra/branches/cassandra-0.7: CHANGES.txt debian/changelog
Author: eevans Date: Wed Aug 31 20:20:36 2011 New Revision: 1163795 URL: http://svn.apache.org/viewvc?rev=1163795view=rev Log: update changelogs for 0.7.9 re-roll Modified: cassandra/branches/cassandra-0.7/CHANGES.txt cassandra/branches/cassandra-0.7/debian/changelog Modified: cassandra/branches/cassandra-0.7/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1163795r1=1163794r2=1163795view=diff == --- cassandra/branches/cassandra-0.7/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.7/CHANGES.txt Wed Aug 31 20:20:36 2011 @@ -1,7 +1,3 @@ -0.7.10 - * fix UnavailableException with writes at CL.EACH_QUORM (CASSANDRA-3084) - - 0.7.9 * don't include tmp files as sstable when create cfs (CASSANDRA-2929) * log Java classpath on startup (CASSANDRA-2895) @@ -14,6 +10,7 @@ * remove gossip state when a new IP takes over a token (CASSANDRA-3071) * work around native memory leak in com.sun.management.GarbageCollectorMXBean (CASSANDRA-2868, 3076) + * fix UnavailableException with writes at CL.EACH_QUORM (CASSANDRA-3084) 0.7.8 Modified: cassandra/branches/cassandra-0.7/debian/changelog URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/debian/changelog?rev=1163795r1=1163794r2=1163795view=diff == --- cassandra/branches/cassandra-0.7/debian/changelog (original) +++ cassandra/branches/cassandra-0.7/debian/changelog Wed Aug 31 20:20:36 2011 @@ -2,7 +2,7 @@ cassandra (0.7.9) unstable; urgency=low * New stable point release - -- Eric Evans eev...@apache.org Tue, 23 Aug 2011 14:53:39 -0500 + -- Eric Evans eev...@apache.org Wed, 31 Aug 2011 15:17:38 -0500 cassandra (0.7.8) unstable; urgency=low
svn commit: r1163801 [1/4] - in /cassandra/trunk: ./ interface/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/avro/ src/java/org/apache/cassandra/cli/ src/java/org/apache/cassandra/config
Author: xedin Date: Wed Aug 31 20:30:20 2011 New Revision: 1163801 URL: http://svn.apache.org/viewvc?rev=1163801view=rev Log: Make the compression algorithm and chunk length configurable patch by Sylvain Lebresne; reviewed by Pavel Yaskevich for CASSANDRA-3001 Added: cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressionParameters.java cassandra/trunk/src/java/org/apache/cassandra/io/compress/DeflateCompressor.java cassandra/trunk/src/java/org/apache/cassandra/io/compress/ICompressor.java cassandra/trunk/src/java/org/apache/cassandra/io/compress/SnappyCompressor.java Modified: cassandra/trunk/CHANGES.txt cassandra/trunk/interface/cassandra.thrift cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/AuthenticationException.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/AuthenticationRequest.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/AuthorizationException.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CfDef.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnDef.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnOrSuperColumn.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnParent.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnPath.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CounterColumn.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CounterSuperColumn.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CqlResult.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CqlRow.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Deletion.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/IndexClause.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/IndexExpression.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/KeyCount.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/KeyRange.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/KeySlice.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/KsDef.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Mutation.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SchemaDisagreementException.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SlicePredicate.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SliceRange.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/TimedOutException.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/TokenRange.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/UnavailableException.java cassandra/trunk/src/avro/internode.genavro cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java cassandra/trunk/src/java/org/apache/cassandra/config/CFMetaData.java cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java cassandra/trunk/src/resources/org/apache/cassandra/cli/CliHelp.yaml cassandra/trunk/test/unit/org/apache/cassandra/SchemaLoader.java cassandra/trunk/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java cassandra/trunk/tools/stress/src/org/apache/cassandra/stress/Session.java Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1163801r1=1163800r2=1163801view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Wed Aug 31 20:30:20 2011 @@ -53,7 +53,7 @@ * remove compaction_thread_priority setting (CASSANDRA-3104) * generate hints for replicas that timeout, not just replicas that are known to be down before starting
svn commit: r1163801 [4/4] - in /cassandra/trunk: ./ interface/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/avro/ src/java/org/apache/cassandra/cli/ src/java/org/apache/cassandra/config
Added:
cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressionParameters.java?rev=1163801view=auto
==
---
cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
(added)
+++
cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
Wed Aug 31 20:30:20 2011
@@ -0,0 +1,196 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.cassandra.io.compress;
+
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.apache.commons.lang.builder.EqualsBuilder;
+import org.apache.commons.lang.builder.HashCodeBuilder;
+
+import org.apache.cassandra.config.ConfigurationException;
+
+public class CompressionParameters
+{
+public final static int DEFAULT_CHUNK_LENGTH = 65536;
+public static final String CHUNK_LENGTH_PARAMETER = chunk_length_kb;
+
+public final Class? extends ICompressor compressorClass;
+public final MapString, String compressionOptions;
+
+public final transient ICompressor compressor;
+public final transient int chunkLength;
+
+public CompressionParameters(CharSequence compressorClassName, Map?
extends CharSequence, ? extends CharSequence options) throws
ConfigurationException
+{
+this(compressorClassName, copyOptions(options), -1);
+}
+
+public CompressionParameters(CharSequence compressorClassName, MapString,
String options, int chunkLength) throws ConfigurationException
+{
+this(createCompressor(parseCompressorClass(compressorClassName),
options), options, chunkLength 0 ? getChunkLength(options) : chunkLength);
+validateChunkLength();
+}
+
+public CompressionParameters(ICompressor compressor)
+{
+this(compressor, null, DEFAULT_CHUNK_LENGTH);
+}
+
+public CompressionParameters(ICompressor compressor, MapString, String
compressionOptions, int chunkLength)
+{
+this.compressorClass = compressor == null ? null :
compressor.getClass();
+this.compressionOptions = compressor == null ? null :
(compressionOptions == null ? Collections.String, StringemptyMap() :
compressionOptions);
+this.chunkLength = chunkLength;
+this.compressor = compressor;
+}
+
+private static Class? extends ICompressor
parseCompressorClass(CharSequence cc) throws ConfigurationException
+{
+if (cc == null)
+return null;
+
+String className = cc.toString();
+className = className.contains(.) ? className :
org.apache.cassandra.io.compress. + className;
+try
+{
+return (Class? extends ICompressor)Class.forName(className);
+}
+catch (Exception e)
+{
+throw new ConfigurationException(Could not create Compression for
type + cc.toString(), e);
+}
+}
+
+private static ICompressor createCompressor(Class? extends ICompressor
compressorClass, MapString, String compressionOptions) throws
ConfigurationException
+{
+if (compressorClass == null)
+return null;
+
+try
+{
+Method method = compressorClass.getMethod(create, Map.class);
+return (ICompressor)method.invoke(null, compressionOptions);
+}
+catch (NoSuchMethodException e)
+{
+throw new ConfigurationException(create method not found, e);
+}
+catch (SecurityException e)
+{
+throw new ConfigurationException(Access forbiden, e);
+}
+catch (IllegalAccessException e)
+{
+throw new ConfigurationException(Cannot access method create in
+ compressorClass.getName(), e);
+}
+catch (InvocationTargetException e)
+{
+throw new ConfigurationException(compressorClass.getSimpleName() +
.create() throwed an error, e);
+}
+catch (ExceptionInInitializerError e)
+
[jira] [Resolved] (CASSANDRA-3001) Make the compression algorithm and chunk length configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich resolved CASSANDRA-3001. Resolution: Fixed Committed. Make the compression algorithm and chunk length configurable Key: CASSANDRA-3001 URL: https://issues.apache.org/jira/browse/CASSANDRA-3001 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Labels: compression Fix For: 1.0 Attachments: 0001-Pluggable-algorithm-and-chunk-length.patch, 0002-Thrift-crap.patch, 0003-Add-deflate-compressor.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3114) After Choosing EC2Snitch you can't migrate off w/o a full cluster restart
After Choosing EC2Snitch you can't migrate off w/o a full cluster restart - Key: CASSANDRA-3114 URL: https://issues.apache.org/jira/browse/CASSANDRA-3114 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.4, 0.7.8 Reporter: Benjamin Coverston Once you choose the Ec2Snitch the gossip messages will trigger this exception if you try to move (for example) to the property file snitch: ERROR [pool-2-thread-11] 2011-08-30 16:38:06,935 Cassandra.java (line 3041) Internal error processing get_slice java.lang.NullPointerException at org.apache.cassandra.locator.Ec2Snitch.getDatacenter(Ec2Snitch.java:84) at org.apache.cassandra.locator.DynamicEndpointSnitch.getDatacenter(DynamicEndpointSnitch.java:122) at org.apache.cassandra.service.DatacenterReadCallback.assureSufficientLiveNodes(DatacenterReadCallback.java:77) at org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:516) at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:480) at org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:109) at org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:263) at org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:345) at org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:306) at org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:3033) at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889) at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:187) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3106) getRangeToEndpointMap() method removed
[ https://issues.apache.org/jira/browse/CASSANDRA-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094889#comment-13094889 ] Hudson commented on CASSANDRA-3106: --- Integrated in Cassandra #1063 (See [https://builds.apache.org/job/Cassandra/1063/]) Restore getRangeToEndpointMap. Patch by Nick Bailey, reviewed by brandonwilliams for CASSANDRA-3106 brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163772 Files : * /cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java * /cassandra/trunk/src/java/org/apache/cassandra/service/StorageServiceMBean.java getRangeToEndpointMap() method removed -- Key: CASSANDRA-3106 URL: https://issues.apache.org/jira/browse/CASSANDRA-3106 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 1.0 Reporter: Nick Bailey Assignee: Nick Bailey Priority: Minor Fix For: 1.0 Attachments: 0001-Add-getRangeToEndpointMap-method-back.patch When getRangeToRPCAddress was added, getRangeToEndpointMap was removed, however, both are useful. We should add it back. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3001) Make the compression algorithm and chunk length configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094890#comment-13094890 ] Hudson commented on CASSANDRA-3001: --- Integrated in Cassandra #1063 (See [https://builds.apache.org/job/Cassandra/1063/]) Make the compression algorithm and chunk length configurable patch by Sylvain Lebresne; reviewed by Pavel Yaskevich for CASSANDRA-3001 xedin : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163801 Files : * /cassandra/trunk/CHANGES.txt * /cassandra/trunk/interface/cassandra.thrift * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/AuthenticationException.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/AuthenticationRequest.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/AuthorizationException.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CfDef.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnDef.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnOrSuperColumn.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnParent.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnPath.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CounterColumn.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CounterSuperColumn.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CqlResult.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CqlRow.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Deletion.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/IndexClause.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/IndexExpression.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/KeyCount.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/KeyRange.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/KeySlice.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/KsDef.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Mutation.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SchemaDisagreementException.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SlicePredicate.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SliceRange.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/TimedOutException.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/TokenRange.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/UnavailableException.java * /cassandra/trunk/src/avro/internode.genavro * /cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java * /cassandra/trunk/src/java/org/apache/cassandra/config/CFMetaData.java * /cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java * /cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java * /cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java * /cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressionParameters.java * /cassandra/trunk/src/java/org/apache/cassandra/io/compress/DeflateCompressor.java * /cassandra/trunk/src/java/org/apache/cassandra/io/compress/ICompressor.java * /cassandra/trunk/src/java/org/apache/cassandra/io/compress/SnappyCompressor.java * /cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java * /cassandra/trunk/src/resources/org/apache/cassandra/cli/CliHelp.yaml * /cassandra/trunk/test/unit/org/apache/cassandra/SchemaLoader.java * /cassandra/trunk/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java * /cassandra/trunk/tools/stress/src/org/apache/cassandra/stress/Session.java Make the compression algorithm and chunk length configurable Key: CASSANDRA-3001 URL: https://issues.apache.org/jira/browse/CASSANDRA-3001 Project: Cassandra
[jira] [Created] (CASSANDRA-3115) LongCompactionSpeedTest running longing starting with builds on Aug31
LongCompactionSpeedTest running longing starting with builds on Aug31
-
Key: CASSANDRA-3115
URL: https://issues.apache.org/jira/browse/CASSANDRA-3115
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.5
Environment: Cassandra-0.8 branch, nightly builds.
MacOS and Debian
Reporter: Cathy Daw
Priority: Minor
The Long tests started consistently timing out as this build of cassandra:
[https://jenkins.qa.datastax.com/job/CassandraLong/131/console]
The regression server shows pretty consistent run times for this test, and then
consistent timeouts from this point forward.
{code}
[junit] Testsuite:
org.apache.cassandra.db.compaction.LongCompactionSpeedTest
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 111.379 sec
[junit]
[junit] - Standard Output ---
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 1637 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: 6144 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 2379 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=50: 15690 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=50 colsper=1: 20953 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=1000 colsper=5: 5672 ms
{code}
*Single node local run: Build 1056 / on Aug 30 / Macbook Pro w/ 8 GB ram (all
apps shutdown)*
{panel}
+Run 1: Fresh install with no log or lib dir+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 850 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *3004 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 767 ms
+Run 2: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 826 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *3030 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 776 ms
+Run 3: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 830 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *2964 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 635 ms
+Run 4: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 931 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *2987 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 910 ms
{panel}
*Singled node local run: Build 1062 / on Aug 31 / Macbook pro w/ 8GB ram (all
apps shutdown)*
{panel}
+Run 1: Fresh restart with no log or lib dir+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 802 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *17649 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 713 ms
+Run 2: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 832 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *16875 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 868 ms
+Run 3: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 809 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *16818 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 807 ms
+Run 4: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 834 ms
[junit]
[jira] [Updated] (CASSANDRA-1391) Allow Concurrent Schema Migrations
[ https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1391: --- Attachment: CASSANDRA-1391.patch rebased. Allow Concurrent Schema Migrations -- Key: CASSANDRA-1391 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-1391.patch CASSANDRA-1292 fixed multiple migrations started from the same node to properly queue themselves, but it is still possible for migrations initiated on different nodes to conflict and leave the cluster in a bad state. Since the system_add/drop/rename methods are accessible directly from the client API, they should be completely safe for concurrent use. It should be possible to allow for most types of concurrent migrations by converting the UUID schema ID into a VersionVectorClock (as provided by CASSANDRA-580). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3115) LongCompactionSpeedTest running longer starting with builds on Aug31
[
https://issues.apache.org/jira/browse/CASSANDRA-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cathy Daw updated CASSANDRA-3115:
-
Summary: LongCompactionSpeedTest running longer starting with builds on
Aug31 (was: LongCompactionSpeedTest running longing starting with builds on
Aug31)
LongCompactionSpeedTest running longer starting with builds on Aug31
Key: CASSANDRA-3115
URL: https://issues.apache.org/jira/browse/CASSANDRA-3115
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.5
Environment: Cassandra-0.8 branch, nightly builds.
MacOS and Debian
Reporter: Cathy Daw
Priority: Minor
The Long tests started consistently timing out as this build of cassandra:
[https://jenkins.qa.datastax.com/job/CassandraLong/131/console]
The regression server shows pretty consistent run times for this test, and
then consistent timeouts from this point forward.
{code}
[junit] Testsuite:
org.apache.cassandra.db.compaction.LongCompactionSpeedTest
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 111.379 sec
[junit]
[junit] - Standard Output ---
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 1637 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: 6144 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 2379 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=50: 15690 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=50 colsper=1: 20953 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=1000 colsper=5: 5672 ms
{code}
*Single node local run: Build 1056 / on Aug 30 / Macbook Pro w/ 8 GB ram
(all apps shutdown)*
{panel}
+Run 1: Fresh install with no log or lib dir+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 850 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *3004 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 767 ms
+Run 2: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 826 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *3030 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 776 ms
+Run 3: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 830 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *2964 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 635 ms
+Run 4: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 931 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *2987 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 910 ms
{panel}
*Singled node local run: Build 1062 / on Aug 31 / Macbook pro w/ 8GB ram (all
apps shutdown)*
{panel}
+Run 1: Fresh restart with no log or lib dir+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 802 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *17649 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 713 ms
+Run 2: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 832 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *16875 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 868 ms
+Run 3: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 809 ms
[junit]
svn commit: r1163817 - /cassandra/trunk/NEWS.txt
Author: xedin
Date: Wed Aug 31 21:12:31 2011
New Revision: 1163817
URL: http://svn.apache.org/viewvc?rev=1163817view=rev
Log:
NEWS.txt updated to reflect latest compression developments
Modified:
cassandra/trunk/NEWS.txt
Modified: cassandra/trunk/NEWS.txt
URL:
http://svn.apache.org/viewvc/cassandra/trunk/NEWS.txt?rev=1163817r1=1163816r2=1163817view=diff
==
--- cassandra/trunk/NEWS.txt (original)
+++ cassandra/trunk/NEWS.txt Wed Aug 31 21:12:31 2011
@@ -12,7 +12,13 @@ Upgrading
Features
- cassandra.bat install will install Cassandra as a Windows Service
-- SSTable compression can be enabled by setting options ???
+- SSTable compression can be enabled by setting compressionParameters
+ attribute on CFMetaData (set compressionParameters to null to disable
compression).
+ SnappyCompressor and DeflateCompressor are currently
+ available as compression class and chunk_length_kb as compression
+ option. Stress tool supports -I option to provide compressor and
+ CLI `{create/update} column family` commands support compression
+ and compression_options parameters.
- Compressed SSTable blocks are checksummed to protect against bitrot
- New LevelDB-inspired compaction algorithm can be enabled by setting the
Columnfamily compaction_strategy=LeveledCompactionStrategy option.
[jira] [Commented] (CASSANDRA-3114) After Choosing EC2Snitch you can't migrate off w/o a full cluster restart
[ https://issues.apache.org/jira/browse/CASSANDRA-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094904#comment-13094904 ] Brandon Williams commented on CASSANDRA-3114: - I'm not sure there's a good solution here. We could make PFEPS inject the local nodes dc/rack info into gossip similar to what I suggested in CASSANDRA-1974, but you'd still have to name things with the ec2snitch conventions for things to not break, and it would be very PFEPS-specific; other snitches are out of the question. Ultimately I'm inclined to say you need to choose your snitch like you choose your partitioner: very carefully. After Choosing EC2Snitch you can't migrate off w/o a full cluster restart - Key: CASSANDRA-3114 URL: https://issues.apache.org/jira/browse/CASSANDRA-3114 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.8, 0.8.4 Reporter: Benjamin Coverston Once you choose the Ec2Snitch the gossip messages will trigger this exception if you try to move (for example) to the property file snitch: ERROR [pool-2-thread-11] 2011-08-30 16:38:06,935 Cassandra.java (line 3041) Internal error processing get_slice java.lang.NullPointerException at org.apache.cassandra.locator.Ec2Snitch.getDatacenter(Ec2Snitch.java:84) at org.apache.cassandra.locator.DynamicEndpointSnitch.getDatacenter(DynamicEndpointSnitch.java:122) at org.apache.cassandra.service.DatacenterReadCallback.assureSufficientLiveNodes(DatacenterReadCallback.java:77) at org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:516) at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:480) at org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:109) at org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:263) at org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:345) at org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:306) at org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:3033) at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889) at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:187) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2434) node bootstrapping can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Bailey updated CASSANDRA-2434: --- Reviewer: nickmbailey node bootstrapping can violate consistency -- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: paul cannon Fix For: 1.1 Attachments: 2434.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3115) LongCompactionSpeedTest running longer starting with builds on Aug31
[
https://issues.apache.org/jira/browse/CASSANDRA-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams reassigned CASSANDRA-3115:
---
Assignee: Brandon Williams
LongCompactionSpeedTest running longer starting with builds on Aug31
Key: CASSANDRA-3115
URL: https://issues.apache.org/jira/browse/CASSANDRA-3115
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.5
Environment: Cassandra-0.8 branch, nightly builds.
MacOS and Debian
Reporter: Cathy Daw
Assignee: Brandon Williams
Priority: Minor
The Long tests started consistently timing out as this build of cassandra:
[https://jenkins.qa.datastax.com/job/CassandraLong/131/console]
The regression server shows pretty consistent run times for this test, and
then consistent timeouts from this point forward.
{code}
[junit] Testsuite:
org.apache.cassandra.db.compaction.LongCompactionSpeedTest
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 111.379 sec
[junit]
[junit] - Standard Output ---
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 1637 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: 6144 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 2379 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=50: 15690 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=50 colsper=1: 20953 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=1000 colsper=5: 5672 ms
{code}
*Single node local run: Build 1056 / on Aug 30 / Macbook Pro w/ 8 GB ram
(all apps shutdown)*
{panel}
+Run 1: Fresh install with no log or lib dir+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 850 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *3004 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 767 ms
+Run 2: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 826 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *3030 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 776 ms
+Run 3: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 830 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *2964 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 635 ms
+Run 4: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 931 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *2987 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 910 ms
{panel}
*Singled node local run: Build 1062 / on Aug 31 / Macbook pro w/ 8GB ram (all
apps shutdown)*
{panel}
+Run 1: Fresh restart with no log or lib dir+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 802 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *17649 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 713 ms
+Run 2: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 832 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20 colsper=1: *16875 ms*
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=100 rowsper=800 colsper=5: 868 ms
+Run 3: Invoke test without restarting the server+
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=1 colsper=20: 809 ms
[junit] org.apache.cassandra.db.compaction.LongCompactionSpeedTest:
sstables=2 rowsper=20
[jira] [Updated] (CASSANDRA-3056) Able to set path location of HeapDump in cassandra-env
[ https://issues.apache.org/jira/browse/CASSANDRA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Evans updated CASSANDRA-3056: -- Reviewer: urandom Able to set path location of HeapDump in cassandra-env -- Key: CASSANDRA-3056 URL: https://issues.apache.org/jira/browse/CASSANDRA-3056 Project: Cassandra Issue Type: Improvement Affects Versions: 0.7.8, 0.8.4 Reporter: David Talbott Priority: Minor Labels: lhf Attachments: CASSANDRA-3056-1.txt, CASSANDRA-3056-2.txt, CASSANDRA-3056-3.txt We should be able to designate the path location to put any perf dumps that are performed. By Default with this not set the perf dump can occur on the root disk and fill the drive. Should be able to solve this by simply inserting JVM_OPTS=$JVM_OPTS -XX:HeapDumpPath=path to dir into cassandra-env.sh as a default option available and set. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2500) Ruby dbi client (for CQL) that conforms to AR:ConnectionAdapter
[ https://issues.apache.org/jira/browse/CASSANDRA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094987#comment-13094987 ] Robert Jackson commented on CASSANDRA-2500: --- I have been using the repo from GitHub[1] quite a bit locally and things work quite well at this point. At this point the tests pass, and a simple comparison against the fauna/cassandra gem shows that the performance is at least on par with the fauna client(in most cases it is better)[2]. Also, note that it is possible to require both cassandra-cql and fauna/cassandra at the same time without conflict. [1] https://github.com/kreynolds/cassandra-cql [2] https://gist.github.com/1185026 Ruby dbi client (for CQL) that conforms to AR:ConnectionAdapter --- Key: CASSANDRA-2500 URL: https://issues.apache.org/jira/browse/CASSANDRA-2500 Project: Cassandra Issue Type: Task Components: API Reporter: Jon Hermes Assignee: Kelley Reynolds Labels: cql Fix For: 0.8.5 Attachments: 2500.txt, genthriftrb.txt, rbcql-0.0.0.tgz Create a ruby driver for CQL. Lacking something standard (such as py-dbapi), going with something common instead -- RoR ActiveRecord Connection Adapter (http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/AbstractAdapter.html). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163873 - /cassandra/branches/cassandra-0.8/conf/cassandra-env.sh
Author: eevans
Date: Wed Aug 31 23:57:37 2011
New Revision: 1163873
URL: http://svn.apache.org/viewvc?rev=1163873view=rev
Log:
set -XX:HeapDumpPath based on env
Patch by satish babu krishnamoorthy and eevans for CASSANDRA-3056
Modified:
cassandra/branches/cassandra-0.8/conf/cassandra-env.sh
Modified: cassandra/branches/cassandra-0.8/conf/cassandra-env.sh
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/conf/cassandra-env.sh?rev=1163873r1=1163872r2=1163873view=diff
==
--- cassandra/branches/cassandra-0.8/conf/cassandra-env.sh (original)
+++ cassandra/branches/cassandra-0.8/conf/cassandra-env.sh Wed Aug 31 23:57:37
2011
@@ -117,7 +117,12 @@ JVM_OPTS=$JVM_OPTS -XX:ThreadPriorityPo
JVM_OPTS=$JVM_OPTS -Xms${MAX_HEAP_SIZE}
JVM_OPTS=$JVM_OPTS -Xmx${MAX_HEAP_SIZE}
JVM_OPTS=$JVM_OPTS -Xmn${HEAP_NEWSIZE}
-JVM_OPTS=$JVM_OPTS -XX:+HeapDumpOnOutOfMemoryError
+JVM_OPTS=$JVM_OPTS -XX:+HeapDumpOnOutOfMemoryError
+
+# set jvm HeapDumpPath with CASSANDRA_HEAPDUMP_DIR
+if [ x$CASSANDRA_HEAPDUMP_DIR != x ]; then
+JVM_OPTS=$JVM_OPTS
-XX:HeapDumpPath=$CASSANDRA_HEAPDUMP_DIR/cassandra-`date +%s`-pid$$.hprof
+fi
if [ `uname` = Linux ] ; then
# reduce the per-thread stack size to minimize the impact of Thrift
[jira] [Issue Comment Edited] (CASSANDRA-3080) Add throttling for internode streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095009#comment-13095009 ] Stu Hood edited comment on CASSANDRA-3080 at 9/1/11 12:07 AM: -- Attaching an example TRACE log from a stream throttling session (grep 'Stream'). The throttling is fairly bursty because we send 10 MB chunks, but since these are sub-100ms bursts, I'm not sure its worth worrying about. was (Author: stuhood): Attaching an example log from a stream throttling session (grep 'Stream'). The throttling is fairly bursty because we send 10 MB chunks, but since these are sub-second 100ms bursts, I'm not sure its worth worrying about. Add throttling for internode streaming -- Key: CASSANDRA-3080 URL: https://issues.apache.org/jira/browse/CASSANDRA-3080 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Fix For: 1.0 Attachments: 0001-Extract-throttling-into-a-Throttle-class.txt, 0002-Remove-orphaned-throttling-path.txt, 0003-Add-stream_throughput_outbound_megabits_per_sec-settin.txt, 0004-Add-outbound-stream-throttling.txt, stream-throttling-example.log Cassandra does (mostly) sequential reads from disk to send data to other nodes, which means that it is easily possible to stream upwards of 100 MB/s per source node. To avoid affecting service, we should add streaming throttling across all streams in the outbound direction, preferably configurable from JMX, and with `nodetool netstats` integration. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3080) Add throttling for internode streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-3080: Attachment: stream-throttling-example.log Attaching an example log from a stream throttling session (grep 'Stream'). The throttling is fairly bursty because we send 10 MB chunks, but since these are sub-second 100ms bursts, I'm not sure its worth worrying about. Add throttling for internode streaming -- Key: CASSANDRA-3080 URL: https://issues.apache.org/jira/browse/CASSANDRA-3080 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Fix For: 1.0 Attachments: 0001-Extract-throttling-into-a-Throttle-class.txt, 0002-Remove-orphaned-throttling-path.txt, 0003-Add-stream_throughput_outbound_megabits_per_sec-settin.txt, 0004-Add-outbound-stream-throttling.txt, stream-throttling-example.log Cassandra does (mostly) sequential reads from disk to send data to other nodes, which means that it is easily possible to stream upwards of 100 MB/s per source node. To avoid affecting service, we should add streaming throttling across all streams in the outbound direction, preferably configurable from JMX, and with `nodetool netstats` integration. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3070) counter repair
[ https://issues.apache.org/jira/browse/CASSANDRA-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095012#comment-13095012 ] ivan commented on CASSANDRA-3070: - Hi Sylvain, our sstables contain sensitive information so i can't provide them. Sorry. I reloaded sstables in our test environment and catched a new ouput log (). In this new log there is two new debug message: 1. rows containing CF resolve string (message printed at the begining of resolve method in src/java/org/apache/cassandra/db/ColumnFamily.java) 2. rows containing CF addAll string (message printed at the begining of addAll method in src/java/org/apache/cassandra/db/ColumnFamily.java) We have a backup of sstables with these counters so I can do any test on them. We have a 6 node cluster using RF=3. When we experienced problems with some counters I started to debug this problem. Using LOCAL_QUORUM CL we get the same answer from all servers but using ONE CL we get a lower number from 2 servers of 6. The results from the 2 server was lower with 3 than other server. I found the following: - server (10.20.255.55) notices when there is a digest mismatch (using LOCAL_QUORUM) - server (10.20.255.55) sends a repair (rowmutation) message to related servers - server (10.20.255.53) receives this mutation (which contains the same total() received by client) - when mutation is handled by Memtable.put() ColumnFamily.resolve() produces a different result (data contained in Memtable have a delta and the right counter value is not applied instead of this deltha) I don't know the resolved value is correct or not (I suspect it's not beacuse total() value seems to be wrong), because I don't know in details how counter works. Regards, ivan counter repair -- Key: CASSANDRA-3070 URL: https://issues.apache.org/jira/browse/CASSANDRA-3070 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: ivan Assignee: Sylvain Lebresne Attachments: counter_local_quroum_maybeschedulerepairs.txt, counter_local_quroum_maybeschedulerepairs_2.txt Hi! We have some counters out of sync but repair doesn't sync values. We tried nodetool repair. We use LOCAL_QUORUM for read. A repair row mutation is sent to other nodes while reading a bad row but counters wasn't repaired by mutation. Output of two nodes were uploaded. (Some new debug messages were added.) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3116) Compactions can (seriously )delay schema migrations
Compactions can (seriously )delay schema migrations --- Key: CASSANDRA-3116 URL: https://issues.apache.org/jira/browse/CASSANDRA-3116 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Eric Evans A compaction lock is acquired when dropping keyspaces or column families which will cause the schema migration to block if a compaction is in progress. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3117) StorageServiceMBean is missing a getCompactionThroughputMbPerSec() method
StorageServiceMBean is missing a getCompactionThroughputMbPerSec() method - Key: CASSANDRA-3117 URL: https://issues.apache.org/jira/browse/CASSANDRA-3117 Project: Cassandra Issue Type: Bug Reporter: Eric Evans Priority: Minor Without a getter, you can assign a new value but not query the existing one (which is strange). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3116) Compactions can (seriously )delay schema migrations
[ https://issues.apache.org/jira/browse/CASSANDRA-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3116: -- Fix Version/s: 1.1 agreed that this sucks. would like to switch to some kind of test-and-set safety for compaction vs migration instead. not immediately obvious to me how to do this. Compactions can (seriously )delay schema migrations --- Key: CASSANDRA-3116 URL: https://issues.apache.org/jira/browse/CASSANDRA-3116 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Eric Evans Fix For: 1.1 A compaction lock is acquired when dropping keyspaces or column families which will cause the schema migration to block if a compaction is in progress. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1163898 - in /cassandra/trunk: CHANGES.txt src/java/org/apache/cassandra/scheduler/RoundRobinScheduler.java src/java/org/apache/cassandra/scheduler/WeightedQueue.java
Author: jbellis
Date: Thu Sep 1 03:14:12 2011
New Revision: 1163898
URL: http://svn.apache.org/viewvc?rev=1163898view=rev
Log:
Properly throw timeouts, decrement the count of waiters on timeout, fix
off-by-one in taskCount
patch by Stu Hood; reviewed by Ryan King for CASSANDRA-3096
Modified:
cassandra/trunk/CHANGES.txt
cassandra/trunk/src/java/org/apache/cassandra/scheduler/RoundRobinScheduler.java
cassandra/trunk/src/java/org/apache/cassandra/scheduler/WeightedQueue.java
Modified: cassandra/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1163898r1=1163897r2=1163898view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Thu Sep 1 03:14:12 2011
@@ -42,7 +42,7 @@
* Add install command to cassandra.bat (CASSANDRA-292)
* clean up KSMetadata, CFMetadata from unnecessary
Thrift-Avro conversion methods (CASSANDRA-3032)
- * Add timeouts to client request schedulers (CASSANDRA-3079)
+ * Add timeouts to client request schedulers (CASSANDRA-3079, 3096)
* Cli to use hashes rather than array of hashes for strategy options
(CASSANDRA-3081)
* LeveledCompactionStrategy (CASSANDRA-1608, 3085, 3110, 3087)
* Improvements of the CLI `describe` command (CASSANDRA-2630)
Modified:
cassandra/trunk/src/java/org/apache/cassandra/scheduler/RoundRobinScheduler.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/scheduler/RoundRobinScheduler.java?rev=1163898r1=1163897r2=1163898view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/scheduler/RoundRobinScheduler.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/scheduler/RoundRobinScheduler.java
Thu Sep 1 03:14:12 2011
@@ -43,7 +43,6 @@ public class RoundRobinScheduler impleme
//Map of queue id to weighted queue
private final NonBlockingHashMapString, WeightedQueue queues;
-private static boolean started = false;
private final Semaphore taskCount;
@@ -56,12 +55,12 @@ public class RoundRobinScheduler impleme
public RoundRobinScheduler(RequestSchedulerOptions options)
{
-assert !started;
-
defaultWeight = options.default_weight;
weights = options.weights;
-taskCount = new Semaphore(options.throttle_limit);
+// the task count is acquired for the first time _after_ releasing a
thread, so we pre-decrement
+taskCount = new Semaphore(options.throttle_limit - 1);
+
queues = new NonBlockingHashMapString, WeightedQueue();
Runnable runnable = new Runnable()
{
@@ -75,7 +74,6 @@ public class RoundRobinScheduler impleme
};
Thread scheduler = new Thread(runnable, REQUEST-SCHEDULER);
scheduler.start();
-started = true;
logger.info(Started the RoundRobin Request Scheduler);
}
@@ -86,7 +84,21 @@ public class RoundRobinScheduler impleme
try
{
queueSize.release();
-weightedQueue.put(t, timeoutMS);
+try
+{
+weightedQueue.put(t, timeoutMS);
+// the scheduler will release us when a slot is available
+}
+catch (TimeoutException e)
+{
+queueSize.acquireUninterruptibly();
+throw e;
+}
+catch (InterruptedException e)
+{
+queueSize.acquireUninterruptibly();
+throw e;
+}
}
catch (InterruptedException e)
{
Modified:
cassandra/trunk/src/java/org/apache/cassandra/scheduler/WeightedQueue.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/scheduler/WeightedQueue.java?rev=1163898r1=1163897r2=1163898view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/scheduler/WeightedQueue.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/scheduler/WeightedQueue.java
Thu Sep 1 03:14:12 2011
@@ -60,7 +60,8 @@ class WeightedQueue implements WeightedQ
public void put(Thread t, long timeoutMS) throws InterruptedException,
TimeoutException
{
-queue.offer(new WeightedQueue.Entry(t), timeoutMS,
TimeUnit.MILLISECONDS);
+if (!queue.offer(new WeightedQueue.Entry(t), timeoutMS,
TimeUnit.MILLISECONDS))
+throw new TimeoutException(Failed to acquire request scheduler
slot for ' + key + ');
}
public Thread poll()
[jira] [Commented] (CASSANDRA-1391) Allow Concurrent Schema Migrations
[ https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095098#comment-13095098 ] Jonathan Ellis commented on CASSANDRA-1391: --- what is the isIsolated mode stuff for? Allow Concurrent Schema Migrations -- Key: CASSANDRA-1391 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-1391.patch CASSANDRA-1292 fixed multiple migrations started from the same node to properly queue themselves, but it is still possible for migrations initiated on different nodes to conflict and leave the cluster in a bad state. Since the system_add/drop/rename methods are accessible directly from the client API, they should be completely safe for concurrent use. It should be possible to allow for most types of concurrent migrations by converting the UUID schema ID into a VersionVectorClock (as provided by CASSANDRA-580). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2961) Expire dead gossip states based on time
[ https://issues.apache.org/jira/browse/CASSANDRA-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2961: -- Fix Version/s: 1.0 Assignee: (was: Patricio Echague) Expire dead gossip states based on time --- Key: CASSANDRA-2961 URL: https://issues.apache.org/jira/browse/CASSANDRA-2961 Project: Cassandra Issue Type: Improvement Reporter: Brandon Williams Fix For: 1.0 Currently dead states are held until aVeryLongTime, 3 days. The problem is that if a node reboots within this period, it begins a new 3 days and will repopulate the ring with the dead state. While mostly harmless, perpetuating the state forever is at least wasting a small amount of bandwidth. Instead, we can expire states based on a ttl, which will require that the cluster be loosely time synced; within the quarantine period of 60s. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
