[jira] [Updated] (CASSANDRA-1608) Redesigned Compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Coverston updated CASSANDRA-1608: -- Attachment: 1608-v5.txt Added range filters to reads. Redesigned Compaction - Key: CASSANDRA-1608 URL: https://issues.apache.org/jira/browse/CASSANDRA-1608 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Attachments: 0001-leveldb-style-compaction.patch, 1608-v2.txt, 1608-v3.txt, 1608-v4.txt, 1608-v5.txt After seeing the I/O issues in CASSANDRA-1470, I've been doing some more thinking on this subject that I wanted to lay out. I propose we redo the concept of how compaction works in Cassandra. At the moment, compaction is kicked off based on a write access pattern, not read access pattern. In most cases, you want the opposite. You want to be able to track how well each SSTable is performing in the system. If we were to keep statistics in-memory of each SSTable, prioritize them based on most accessed, and bloom filter hit/miss ratios, we could intelligently group sstables that are being read most often and schedule them for compaction. We could also schedule lower priority maintenance on SSTable's not often accessed. I also propose we limit the size of each SSTable to a fix sized, that gives us the ability to better utilize our bloom filters in a predictable manner. At the moment after a certain size, the bloom filters become less reliable. This would also allow us to group data most accessed. Currently the size of an SSTable can grow to a point where large portions of the data might not actually be accessed as often. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1137891 - /cassandra/branches/cassandra-0.8/debian/changelog
Author: slebresne Date: Tue Jun 21 07:38:17 2011 New Revision: 1137891 URL: http://svn.apache.org/viewvc?rev=1137891view=rev Log: Update debian package changelog data for reroll Modified: cassandra/branches/cassandra-0.8/debian/changelog Modified: cassandra/branches/cassandra-0.8/debian/changelog URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/debian/changelog?rev=1137891r1=1137890r2=1137891view=diff == --- cassandra/branches/cassandra-0.8/debian/changelog (original) +++ cassandra/branches/cassandra-0.8/debian/changelog Tue Jun 21 07:38:17 2011 @@ -2,7 +2,7 @@ cassandra (0.8.1) unstable; urgency=low * New release - -- Sylvain Lebresne slebre...@apache.org Thu, 16 Jun 2011 09:37:27 +0200 + -- Sylvain Lebresne slebre...@apache.org Thu, 21 Jun 2011 09:37:27 +0200 cassandra (0.8.0) unstable; urgency=low
[jira] [Updated] (CASSANDRA-2788) Add startup option renew the NodeId (for counters)
[ https://issues.apache.org/jira/browse/CASSANDRA-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2788: Fix Version/s: (was: 0.8.2) 0.8.1 Add startup option renew the NodeId (for counters) -- Key: CASSANDRA-2788 URL: https://issues.apache.org/jira/browse/CASSANDRA-2788 Project: Cassandra Issue Type: Improvement Affects Versions: 0.8.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Labels: counters Fix For: 0.8.1 Attachments: 0001-Option-to-renew-the-NodeId-on-startup.patch If an sstable of a counter column family is corrupted, the only safe solution a user have right now is to: # Remove the NodeId System table to force the node to regenerate a new NodeId (and thus stop incrementing on it's previous, corrupted, subcount) # Remove all the sstables for that column family on that node (this is important because otherwise the node will never get repaired for it's previous subcount) This is far from being ideal, but I think this is the price we pay for avoiding the read-before-write. In any case, the first step (remove the NodeId system table) happens to remove the list of the old NodeId this node has, which could prevent us for merging the other potential previous nodeId. This is ok but sub-optimal. This ticket proposes to add a new startup flag to make the node renew it's NodeId, thus replacing this first. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2781) regression: exposing cache size through MBean
[ https://issues.apache.org/jira/browse/CASSANDRA-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2781: Fix Version/s: (was: 0.8.2) 0.8.1 regression: exposing cache size through MBean - Key: CASSANDRA-2781 URL: https://issues.apache.org/jira/browse/CASSANDRA-2781 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 beta 1 Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Fix For: 0.8.1 Attachments: 2781-v1.txt Looks like it was part of CASSANDRA-1969. A method called size, as opposed to getSize, won't be exposed through jmx. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2530) Additional AbstractType data type definitions to enrich CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2530: Fix Version/s: (was: 0.8.2) 0.8.1 Additional AbstractType data type definitions to enrich CQL --- Key: CASSANDRA-2530 URL: https://issues.apache.org/jira/browse/CASSANDRA-2530 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8.0 beta 2 Reporter: Rick Shaw Assignee: Rick Shaw Priority: Trivial Labels: cql Fix For: 0.8.1 Attachments: rebase-for-new-abstracttypes-and cql-stuff-v1.txt Provide 5 additional Datatypes: ByteType, DateType, BooleanType, FloatType, DoubleType. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2787) java agent option missing in cassandra.bat file
[ https://issues.apache.org/jira/browse/CASSANDRA-2787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2787: Fix Version/s: (was: 0.8.2) 0.8.1 java agent option missing in cassandra.bat file --- Key: CASSANDRA-2787 URL: https://issues.apache.org/jira/browse/CASSANDRA-2787 Project: Cassandra Issue Type: Bug Components: Packaging Affects Versions: 0.8.0 Reporter: rene kochen Assignee: rene kochen Priority: Minor Fix For: 0.8.1 This option must be included in cassandra.bat: -javaagent:%CASSANDRA_HOME%\lib\jamm-0.2.2.jar Otherwise you see the following warnings in cassandra log: WARN 12:02:32,478 MemoryMeter uninitialized (jamm not specified as java agent); assuming liveRatio of 10.0. Usually this means cassandra-env.sh disabled jamm because you are using a buggy JRE; upgrade to the Sun JRE instead -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2797) Repair hangs if a neighbor has nothing to send
[ https://issues.apache.org/jira/browse/CASSANDRA-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2797: Fix Version/s: (was: 0.8.2) 0.8.1 Repair hangs if a neighbor has nothing to send --- Key: CASSANDRA-2797 URL: https://issues.apache.org/jira/browse/CASSANDRA-2797 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Labels: repair, streaming Fix For: 0.8.1 Attachments: 0001-Always-initiate-streaming-transfer-to-notify-remote-.patch This is actually a streaming problem. If a StreamOutSession has nothing to transfer (i.e, no sstables have the requested ranges), it will not even initiate the transfer and simply close the session right away. The problem is that if the session was initiated by a remote end (through a StreamRequestMessage), the remote end will never be notified and never run his callback. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2801) Tombstone are not purged when the row is in only one sstable
[ https://issues.apache.org/jira/browse/CASSANDRA-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2801: Attachment: 0001-Don-t-echo-row-when-purge-is-possible.patch Tombstone are not purged when the row is in only one sstable Key: CASSANDRA-2801 URL: https://issues.apache.org/jira/browse/CASSANDRA-2801 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.1 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 0.8.1 Attachments: 0001-Don-t-echo-row-when-purge-is-possible.patch We messed up the refactor of compactionController. It echoes rows if they are present in only one sstable, even if they could be purged. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2801) Tombstone are not purged when the row is in only one sstable
Tombstone are not purged when the row is in only one sstable Key: CASSANDRA-2801 URL: https://issues.apache.org/jira/browse/CASSANDRA-2801 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.1 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 0.8.1 Attachments: 0001-Don-t-echo-row-when-purge-is-possible.patch We messed up the refactor of compactionController. It echoes rows if they are present in only one sstable, even if they could be purged. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2786) After a minor compaction, deleted key-slices are visible again
[ https://issues.apache.org/jira/browse/CASSANDRA-2786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2786: Attachment: 0001-Fix-wrong-purge-of-deleted-cf.patch We were wrongfully skipping deleted rows with no columns during compaction. This indeed don't affect 0.7 since this was due to a refactor of PrecompactedRow in 0.8. Patch attached with a unit test to catch the error. After a minor compaction, deleted key-slices are visible again -- Key: CASSANDRA-2786 URL: https://issues.apache.org/jira/browse/CASSANDRA-2786 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.0 Environment: Reproduced on single Cassandra node (CentOS 5.5) Reproduced on single Cassandra node (Windows Server 2008) Reporter: rene kochen Attachments: 0001-Fix-wrong-purge-of-deleted-cf.patch, CassandraIssue.zip, CassandraIssueJava.zip After a minor compaction, deleted key-slices are visible again. Steps to reproduce: 1) Insert a row named test. 2) Insert 50 rows. During this step, row test is included in a major compaction: file-1, file-2, file-3 and file-4 compacted to file-5 (includes test). 3) Delete row named test. 4) Insert 50 rows. During this step, row test is included in a minor compaction: file-6, file-7, file-8 and file-9 compacted to file-10 (should include tombstoned test). After step 4, row test is live again. Test environment: Single node with empty database. Standard configured super-column-family (I see this behavior with several gc_grace settings (big and small values): create column family Customers with column_type = 'Super' and comparator = 'BytesType; In Cassandra 0.7.6 I observe the expected behavior, i.e. after step 4, the row is still deleted. I've included a .NET program to reproduce the problem. I will add a Java version later on. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2786) After a minor compaction, deleted key-slices are visible again
[ https://issues.apache.org/jira/browse/CASSANDRA-2786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052514#comment-13052514 ] Jonathan Ellis commented on CASSANDRA-2786: --- +1 (can we make the testing constructor package-local?) After a minor compaction, deleted key-slices are visible again -- Key: CASSANDRA-2786 URL: https://issues.apache.org/jira/browse/CASSANDRA-2786 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.0 Environment: Reproduced on single Cassandra node (CentOS 5.5) Reproduced on single Cassandra node (Windows Server 2008) Reporter: rene kochen Attachments: 0001-Fix-wrong-purge-of-deleted-cf.patch, CassandraIssue.zip, CassandraIssueJava.zip After a minor compaction, deleted key-slices are visible again. Steps to reproduce: 1) Insert a row named test. 2) Insert 50 rows. During this step, row test is included in a major compaction: file-1, file-2, file-3 and file-4 compacted to file-5 (includes test). 3) Delete row named test. 4) Insert 50 rows. During this step, row test is included in a minor compaction: file-6, file-7, file-8 and file-9 compacted to file-10 (should include tombstoned test). After step 4, row test is live again. Test environment: Single node with empty database. Standard configured super-column-family (I see this behavior with several gc_grace settings (big and small values): create column family Customers with column_type = 'Super' and comparator = 'BytesType; In Cassandra 0.7.6 I observe the expected behavior, i.e. after step 4, the row is still deleted. I've included a .NET program to reproduce the problem. I will add a Java version later on. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2801) Tombstone are not purged when the row is in only one sstable
[ https://issues.apache.org/jira/browse/CASSANDRA-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052516#comment-13052516 ] Jonathan Ellis commented on CASSANDRA-2801: --- +1 Tombstone are not purged when the row is in only one sstable Key: CASSANDRA-2801 URL: https://issues.apache.org/jira/browse/CASSANDRA-2801 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.1 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 0.8.1 Attachments: 0001-Don-t-echo-row-when-purge-is-possible.patch We messed up the refactor of compactionController. It echoes rows if they are present in only one sstable, even if they could be purged. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1966) Option to control how many items are read on cache load
[
https://issues.apache.org/jira/browse/CASSANDRA-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052519#comment-13052519
]
Chris Burroughs commented on CASSANDRA-1966:
{noformat}
* Beware that, unlike in {@link #keySet()}, obtaining the set is emNOT/em
* a constant-time operation. Because of the asynchronous nature of the page
* replacement policy, determining the retention ordering requires a traversal
* of the keys.
{noformat}
I used the special case on -1 to only call descendingKeySet when needed and
otherwise use keySet. I can special case on Integer.MAX_VALUE instead, but we
will still need the conditional branch.
Option to control how many items are read on cache load
---
Key: CASSANDRA-1966
URL: https://issues.apache.org/jira/browse/CASSANDRA-1966
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Chris Burroughs
Assignee: Chris Burroughs
Priority: Minor
Attachments: 1966-v1.txt
CASSANDRA-1417 added an option to save the key and/or row cache keys which is
cool. However, for a row large cache it can take a long time to read all of
the rows. For example I have a 400,000 item row cache, and loading that on
restart takes a little under an hour.
In addition to configuring the size of the row cache, and how often it should
be saved to disk, I propose an option to control how many items are loaded on
startup (or alternately only saving n items out of the full row cache to
begin with).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1137982 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/db/compaction/CompactionController.java
Author: slebresne
Date: Tue Jun 21 12:48:28 2011
New Revision: 1137982
URL: http://svn.apache.org/viewvc?rev=1137982view=rev
Log:
Purge tombstones even if the row is in only one sstable
patch by slebresne; reviewed by jbellis for CASSANDRA-2801
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionController.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1137982r1=1137981r2=1137982view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Tue Jun 21 12:48:28 2011
@@ -68,6 +68,7 @@
* Add startup flag to renew counter node id (CASSANDRA-2788)
* add jamm agent to cassandra.bat (CASSANDRA-2787)
* fix repair hanging if a neighbor has nothing to send (CASSANDRA-2797)
+ * purge tombstone even if row is in only one sstable (CASSANDRA-2801)
0.8.0-final
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionController.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionController.java?rev=1137982r1=1137981r2=1137982view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionController.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionController.java
Tue Jun 21 12:48:28 2011
@@ -106,14 +106,13 @@ public class CompactionController
/**
* @return an AbstractCompactedRow implementation to write the merged rows
in question.
*
- * If there is a single source row, the data is from a current-version
sstable,
- * and we aren't forcing deserialization for scrub,
- * write it unchanged. Otherwise, we deserialize, purge tombstones, and
- * reserialize in the latest version.
+ * If there is a single source row, the data is from a current-version
sstable, we don't
+ * need to purge and we aren't forcing deserialization for scrub, write it
unchanged.
+ * Otherwise, we deserialize, purge tombstones, and reserialize in the
latest version.
*/
public AbstractCompactedRow getCompactedRow(ListSSTableIdentityIterator
rows)
{
-if (rows.size() == 1 !needDeserialize())
+if (rows.size() == 1 !needDeserialize()
!shouldPurge(rows.get(0).getKey()))
return new EchoedRow(rows.get(0));
long rowSize = 0;
svn commit: r1137984 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java test/unit/org/apache/cassandra/db/compaction/CompactionsTest.j
Author: slebresne
Date: Tue Jun 21 12:51:34 2011
New Revision: 1137984
URL: http://svn.apache.org/viewvc?rev=1137984view=rev
Log:
Fix wrong purge of deleted cf during compaction
patch by slebresne; reviewed by jbellis for CASSANDRA-2786
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1137984r1=1137983r2=1137984view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Tue Jun 21 12:51:34 2011
@@ -69,6 +69,7 @@
* add jamm agent to cassandra.bat (CASSANDRA-2787)
* fix repair hanging if a neighbor has nothing to send (CASSANDRA-2797)
* purge tombstone even if row is in only one sstable (CASSANDRA-2801)
+ * Fix wrong purge of deleted cf during compaction (CASSANDRA-2786)
0.8.0-final
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java?rev=1137984r1=1137983r2=1137984view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
Tue Jun 21 12:51:34 2011
@@ -45,16 +45,20 @@ public class PrecompactedRow extends Abs
private static Logger logger =
LoggerFactory.getLogger(PrecompactedRow.class);
private final ColumnFamily compactedCf;
+private final int gcBefore;
+// For testing purposes
public PrecompactedRow(DecoratedKey key, ColumnFamily compacted)
{
super(key);
this.compactedCf = compacted;
+this.gcBefore = Integer.MAX_VALUE;
}
public PrecompactedRow(CompactionController controller,
ListSSTableIdentityIterator rows)
{
super(rows.get(0).getKey());
+this.gcBefore = controller.gcBefore;
ColumnFamily cf = null;
for (SSTableIdentityIterator row : rows)
@@ -120,7 +124,7 @@ public class PrecompactedRow extends Abs
public boolean isEmpty()
{
-return compactedCf == null || compactedCf.getColumnCount() == 0;
+return compactedCf == null ||
ColumnFamilyStore.removeDeletedCF(compactedCf, gcBefore) == null;
}
public int columnCount()
Modified:
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java?rev=1137984r1=1137983r2=1137984view=diff
==
---
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java
(original)
+++
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java
Tue Jun 21 12:51:34 2011
@@ -23,6 +23,7 @@ import java.net.InetAddress;
import java.nio.ByteBuffer;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;
+import java.util.Collection;
import java.util.List;
import java.util.ArrayList;
import java.util.Set;
@@ -34,11 +35,10 @@ import org.junit.Test;
import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.CleanupHelper;
-import org.apache.cassandra.db.Table;
-import org.apache.cassandra.db.ColumnFamilyStore;
-import org.apache.cassandra.db.DecoratedKey;
-import org.apache.cassandra.db.RowMutation;
+import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.filter.QueryFilter;
import org.apache.cassandra.db.filter.QueryPath;
+import org.apache.cassandra.io.sstable.*;
import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.Pair;
@@ -157,6 +157,7 @@ public class CompactionsTest extends Cle
buckets = CompactionManager.getBuckets(pairs, 10); // notice the min
is 10
assertEquals(1, buckets.size());
}
+
@Test
public void testEchoedRow() throws IOException, ExecutionException,
InterruptedException
{
@@ -187,4 +188,51 @@ public class CompactionsTest extends Cle
// Now assert we do have the two keys
assertEquals(4, Util.getRangeSlice(store).size());
}
+
+@Test
+public void testDontPurgeAccidentaly() throws IOException,
ExecutionException, InterruptedException
+{
+// This
[jira] [Commented] (CASSANDRA-2786) After a minor compaction, deleted key-slices are visible again
[ https://issues.apache.org/jira/browse/CASSANDRA-2786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052570#comment-13052570 ] Sylvain Lebresne commented on CASSANDRA-2786: - Committed, thanks. I did not made the testing constructor package-local because it is used in the AntiEntropyTests which are not on the same package. But I agree it's not the cleanest thing ever. After a minor compaction, deleted key-slices are visible again -- Key: CASSANDRA-2786 URL: https://issues.apache.org/jira/browse/CASSANDRA-2786 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.0 Environment: Reproduced on single Cassandra node (CentOS 5.5) Reproduced on single Cassandra node (Windows Server 2008) Reporter: rene kochen Fix For: 0.8.1 Attachments: 0001-Fix-wrong-purge-of-deleted-cf.patch, CassandraIssue.zip, CassandraIssueJava.zip After a minor compaction, deleted key-slices are visible again. Steps to reproduce: 1) Insert a row named test. 2) Insert 50 rows. During this step, row test is included in a major compaction: file-1, file-2, file-3 and file-4 compacted to file-5 (includes test). 3) Delete row named test. 4) Insert 50 rows. During this step, row test is included in a minor compaction: file-6, file-7, file-8 and file-9 compacted to file-10 (should include tombstoned test). After step 4, row test is live again. Test environment: Single node with empty database. Standard configured super-column-family (I see this behavior with several gc_grace settings (big and small values): create column family Customers with column_type = 'Super' and comparator = 'BytesType; In Cassandra 0.7.6 I observe the expected behavior, i.e. after step 4, the row is still deleted. I've included a .NET program to reproduce the problem. I will add a Java version later on. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2786) After a minor compaction, deleted key-slices are visible again
[ https://issues.apache.org/jira/browse/CASSANDRA-2786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne resolved CASSANDRA-2786. - Resolution: Fixed Fix Version/s: 0.8.1 Reviewer: jbellis Assignee: Sylvain Lebresne After a minor compaction, deleted key-slices are visible again -- Key: CASSANDRA-2786 URL: https://issues.apache.org/jira/browse/CASSANDRA-2786 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.0 Environment: Reproduced on single Cassandra node (CentOS 5.5) Reproduced on single Cassandra node (Windows Server 2008) Reporter: rene kochen Assignee: Sylvain Lebresne Fix For: 0.8.1 Attachments: 0001-Fix-wrong-purge-of-deleted-cf.patch, CassandraIssue.zip, CassandraIssueJava.zip After a minor compaction, deleted key-slices are visible again. Steps to reproduce: 1) Insert a row named test. 2) Insert 50 rows. During this step, row test is included in a major compaction: file-1, file-2, file-3 and file-4 compacted to file-5 (includes test). 3) Delete row named test. 4) Insert 50 rows. During this step, row test is included in a minor compaction: file-6, file-7, file-8 and file-9 compacted to file-10 (should include tombstoned test). After step 4, row test is live again. Test environment: Single node with empty database. Standard configured super-column-family (I see this behavior with several gc_grace settings (big and small values): create column family Customers with column_type = 'Super' and comparator = 'BytesType; In Cassandra 0.7.6 I observe the expected behavior, i.e. after step 4, the row is still deleted. I've included a .NET program to reproduce the problem. I will add a Java version later on. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2792) Bootstrapping node stalls. Bootstrapper thinks it is still streaming some sstables. The source nodes do not. Caused by IllegalStateException on source nodes.
[ https://issues.apache.org/jira/browse/CASSANDRA-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2792: Attachment: 0001-Make-StreamOutSession-threadSafe.patch bq. the source nodes should report to the bootstrapping node that there has been a problem, and/or the bootstrapping node should timeout and report the the issue. This will be addressed by CASSANDRA-2433 (or almost). This won't go into 0.7 however and since this change network protocol it's yet unclear it will even go into 0.8 (though my personnal opinion is that this should go in asap). bq. Secondly, there is an issue with what is causing IllegalStateException. Attaching a patch that hopefully fixes that. It's a bit of a guess because I don't know how to reproduce but I suppose that since the currentFile field is not volatile in streamOutSession, changing it may not propagate to all threads (at least that explication fits with the exception message). The patch also use a thread safe map for pendingFiles since those are modified in differnt thread, even though I have stronger doubt that this could be a problem (but it doesn't cost us anything to be on the safe side). Bootstrapping node stalls. Bootstrapper thinks it is still streaming some sstables. The source nodes do not. Caused by IllegalStateException on source nodes. - Key: CASSANDRA-2792 URL: https://issues.apache.org/jira/browse/CASSANDRA-2792 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.6 Environment: Ubuntu Reporter: Dominic Williams Assignee: Sylvain Lebresne Fix For: 0.7.7 Attachments: 0001-Make-StreamOutSession-threadSafe.patch Original Estimate: 4h Remaining Estimate: 4h I am bootstrapping a node into a 4 node cluster with RF3 (1 node is currently down due to sstable issues, but the cluster is running without issues). There are two keyspaces FightMyMonster and FMM_Studio. The first keyspace successfully streams and the whole operation is probably at 99% when it stalls on some sstables in the much smaller FMM_Studio keyspace. Netstats on the bootstrapping node reports it is still streaming: Mode: Bootstrapping Not sending any streams. Streaming from: /192.168.1.4 FMM_Studio: /var/opt/cassandra/data/FMM_Studio/PartsData-f-101-Data.db sections=1 progress=0/76453 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/PartsData-f-103-Data.db sections=1 progress=0/90475 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/PartsData-f-102-Data.db sections=1 progress=0/4304182 - 0% Streaming from: /192.168.1.3 FMM_Studio: /var/opt/cassandra/data/FMM_Studio/PartsData-f-158-Data.db sections=2 progress=0/146990 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/AuthorClasses-f-81-Data.db sections=1 progress=0/3992 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/Studio-f-70-Data.db sections=1 progress=0/1776 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/PartsData-f-159-Data.db sections=2 progress=0/136829 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/PartsData-f-157-Data.db sections=2 progress=0/5779597 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/AuthorClasses-f-82-Data.db sections=1 progress=0/161 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/Studio-f-71-Data.db sections=1 progress=0/135 - 0% Pool NameActive Pending Completed Commandsn/a 0334 Responses n/a 0 421957 However, running netstats on the source nodes reports they are not streaming: Mode: Normal Nothing streaming to /192.168.1.9 Not receiving any streams. Pool NameActive Pending Completed Commandsn/a 01949476 Responses n/a 11778768 Examination of the logs on the source nodes show an IllegalStateException that has likely interrupted/broken the streaming process. 17 22:27:05,924 StreamOut.java (line 126) Beginning transfer to /192.168.1.9 INFO [StreamStage:1] 2011-06-17 22:27:05,925 StreamOut.java (line 100) Flushing memtables for FMM_Studio... INFO [StreamStage:1] 2011-06-17 22:27:06,004 StreamOut.java (line 173) Stream context metadata [/var/opt/cassandra/data/FMM_Studio/Classes-f-107-Data.db sections=1 progress=0/1585378 - 0%, /var/opt/cas sandra/data/FMM_Studio/PartsData-f-100-Data.db sections=1 progress=0/76453 - 0%, /var/opt/cassandra/data/FMM_Studio/PartsData-f-98-Data.db sections=1 progress=0/4309514 - 0%, /var/opt/cassandra/data/FMM
[jira] [Commented] (CASSANDRA-2798) Repair Fails 0.8
[
https://issues.apache.org/jira/browse/CASSANDRA-2798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052613#comment-13052613
]
David Arena commented on CASSANDRA-2798:
Ok. so i cant exactly hand out the inserting script but i can give you an
indication of the data format.. our scripts for testing are complex classes,
whereby building objects.. However this is EXACTLY an output of what actually
is written to cassandra ( CF test1 ) with a little obfuscation..
UUID4 key:
f9bb44f2844241df971e0975005c87dc
DATA FORMAT:
{'i': '[{pr: [XYZ, 0.47], va: [[XZ, 0.19]], pu: 1307998855,
iu: http://devel.test.com/test2730;, it: TESTERtype 0: TESTERobject
2730, pi: {XYZ: 0.31, XZ: 0.47}, id: 0!2730}]', 'cu': 'XYZ',
'cd': '1308668648'}
Try loading 100,000 of these with random uuid4 keys...
Furthermore, i have retried the test.. precisely again and again.. including a
flush before killing node3.. Still im left with..
Before...
10.0.1.150 Up Normal 2.61 GB 33.33% 0
10.0.1.152 Up Normal 2.61 GB 33.33%
56713727820156410577229101238628035242
10.0.1.154 Up Normal 2.61 GB 33.33%
113427455640312821154458202477256070485
After Killing Node and Restart...
10.0.1.150 Up Normal 2.61 GB 33.33% 0
10.0.1.152 Up Normal 2.61 GB 33.33%
56713727820156410577229101238628035242
10.0.1.154 Up Normal 61.69 KB33.33%
113427455640312821154458202477256070485
After Running Repair...
10.0.1.150 Up Normal 4.76 GB 33.33% 0
10.0.1.152 Up Normal 5.41 GB 33.33%
56713727820156410577229101238628035242
10.0.1.154 Up Normal 8.87 GB 33.33%
113427455640312821154458202477256070485
After Running Flush Compact on ALL nodes...
10.0.1.150 Up Normal 4.76 GB 33.33% 0
10.0.1.152 Up Normal 5.41 GB 33.33%
56713727820156410577229101238628035242
10.0.1.154 Up Normal 4.86 GB 33.33%
113427455640312821154458202477256070485
This does not occur in 0.7.6.. in fact it works perfectly..
Repair Fails 0.8
Key: CASSANDRA-2798
URL: https://issues.apache.org/jira/browse/CASSANDRA-2798
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.0
Reporter: David Arena
Assignee: Sylvain Lebresne
I am seeing a fatal problem in the new 0.8
Im running a 3 node cluster with a replication_factor of 3..
On Node 3.. If i
# kill -9 cassandra-pid
# rm -rf All data logs
# start cassandra
# nodetool -h node-3-ip repair
The whole cluster become duplicated..
* e.g Before
node 1 - 2.65GB
node 2 - 2.65GB
node 3 - 2.65GB
* e.g After
node 1 - 5.3GB
node 2 - 5.3GB
node 3 - 7.95GB
- nodetool repair, never ends (96 hours +), however there is no streams
running, nor any cpu or disk activity..
- Manually killing the repair and restarting does not help.. Restarting the
server/cassandra does not help..
- nodetool flush,compact,cleanup all complete, but do not help...
This is not occuring in 0.7.6.. I have come to the conclusion this is a Major
0.8 issue
Running: CentOS 5.6, JDK 1.6.0_26
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2792) Bootstrapping node stalls. Bootstrapper thinks it is still streaming some sstables. The source nodes do not. Caused by IllegalStateException on source nodes.
[ https://issues.apache.org/jira/browse/CASSANDRA-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052621#comment-13052621 ] Jonathan Ellis commented on CASSANDRA-2792: --- +1 on the threadsafety changes Bootstrapping node stalls. Bootstrapper thinks it is still streaming some sstables. The source nodes do not. Caused by IllegalStateException on source nodes. - Key: CASSANDRA-2792 URL: https://issues.apache.org/jira/browse/CASSANDRA-2792 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.6 Environment: Ubuntu Reporter: Dominic Williams Assignee: Sylvain Lebresne Fix For: 0.7.7 Attachments: 0001-Make-StreamOutSession-threadSafe.patch Original Estimate: 4h Remaining Estimate: 4h I am bootstrapping a node into a 4 node cluster with RF3 (1 node is currently down due to sstable issues, but the cluster is running without issues). There are two keyspaces FightMyMonster and FMM_Studio. The first keyspace successfully streams and the whole operation is probably at 99% when it stalls on some sstables in the much smaller FMM_Studio keyspace. Netstats on the bootstrapping node reports it is still streaming: Mode: Bootstrapping Not sending any streams. Streaming from: /192.168.1.4 FMM_Studio: /var/opt/cassandra/data/FMM_Studio/PartsData-f-101-Data.db sections=1 progress=0/76453 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/PartsData-f-103-Data.db sections=1 progress=0/90475 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/PartsData-f-102-Data.db sections=1 progress=0/4304182 - 0% Streaming from: /192.168.1.3 FMM_Studio: /var/opt/cassandra/data/FMM_Studio/PartsData-f-158-Data.db sections=2 progress=0/146990 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/AuthorClasses-f-81-Data.db sections=1 progress=0/3992 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/Studio-f-70-Data.db sections=1 progress=0/1776 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/PartsData-f-159-Data.db sections=2 progress=0/136829 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/PartsData-f-157-Data.db sections=2 progress=0/5779597 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/AuthorClasses-f-82-Data.db sections=1 progress=0/161 - 0% FMM_Studio: /var/opt/cassandra/data/FMM_Studio/Studio-f-71-Data.db sections=1 progress=0/135 - 0% Pool NameActive Pending Completed Commandsn/a 0334 Responses n/a 0 421957 However, running netstats on the source nodes reports they are not streaming: Mode: Normal Nothing streaming to /192.168.1.9 Not receiving any streams. Pool NameActive Pending Completed Commandsn/a 01949476 Responses n/a 11778768 Examination of the logs on the source nodes show an IllegalStateException that has likely interrupted/broken the streaming process. 17 22:27:05,924 StreamOut.java (line 126) Beginning transfer to /192.168.1.9 INFO [StreamStage:1] 2011-06-17 22:27:05,925 StreamOut.java (line 100) Flushing memtables for FMM_Studio... INFO [StreamStage:1] 2011-06-17 22:27:06,004 StreamOut.java (line 173) Stream context metadata [/var/opt/cassandra/data/FMM_Studio/Classes-f-107-Data.db sections=1 progress=0/1585378 - 0%, /var/opt/cas sandra/data/FMM_Studio/PartsData-f-100-Data.db sections=1 progress=0/76453 - 0%, /var/opt/cassandra/data/FMM_Studio/PartsData-f-98-Data.db sections=1 progress=0/4309514 - 0%, /var/opt/cassandra/data/FMM _Studio/PartsData-f-99-Data.db sections=1 progress=0/90475 - 0%], 11 sstables. INFO [StreamStage:1] 2011-06-17 22:27:06,005 StreamOutSession.java (line 174) Streaming to /192.168.1.9 INFO [StreamStage:1] 2011-06-17 22:27:06,006 StreamOut.java (line 126) Beginning transfer to /192.168.1.9 INFO [StreamStage:1] 2011-06-17 22:27:06,007 StreamOut.java (line 100) Flushing memtables for FightMyMonster... INFO [StreamStage:1] 2011-06-17 22:27:06,007 ColumnFamilyStore.java (line 1065) Enqueuing flush of Memtable-MonsterMarket_1@1054909557(338 bytes, 24 operations) INFO [StreamStage:1] 2011-06-17 22:27:06,007 ColumnFamilyStore.java (line 1065) Enqueuing flush of Memtable-UserFights@239934867(1124836 bytes, 965 operations) INFO [FlushWriter:409] 2011-06-17 22:27:06,007 Memtable.java (line 157) Writing Memtable-MonsterMarket_1@1054909557(338 bytes, 24 operations) INFO [StreamStage:1] 2011-06-17 22:27:06,007 ColumnFamilyStore.java (line 1065)
[jira] [Issue Comment Edited] (CASSANDRA-2798) Repair Fails 0.8
[
https://issues.apache.org/jira/browse/CASSANDRA-2798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052613#comment-13052613
]
David Arena edited comment on CASSANDRA-2798 at 6/21/11 3:37 PM:
-
Ok. so i cant exactly hand out the inserting script but i can give you an
indication of the data format.. our scripts for testing are complex classes,
whereby building objects.. However this is EXACTLY an output of what actually
is written to cassandra ( CF test1 ) with a little obfuscation..
UUID4 key:
f9bb44f2844241df971e0975005c87dc
DATA FORMAT:
{'i': '[{pr: [XYZ, 0.47], va: [[XZ, 0.19]], pu: 1307998855,
iu: http://devel.test.com/test2730;, it: TESTERtype 0: TESTERobject
2730, pi: {XYZ: 0.31, XZ: 0.47}, id: 0!2730}]', 'cu': 'XYZ',
'cd': '1308668648'}
For the CF test 2, the data format looks like this..
UUID4 key:
f9bb44f2844241df971e0975005c87dc
DATA FORMAT:
('0!2243', {'rt': '1308221914', 'ri': '1308218344', 'pu': '1308218344'}),
('1!2342', {'pu': '1308080741'}), ('2!1731', {'pu': '1308618693'}), ('3!3772',
{'pu': '1308338296'})..
There can be up to 100 fields per key in CF test2..
basically for every insert in CF test1, there is a corresponding insert in CF
test2(with roughly 50-100 fields)
Try loading 100,000 of random uuid4 keys into CF test1 with the corresponding
keys/fields in CF test2...
Furthermore, i have retried the test.. precisely again and again.. including a
flush before killing node3.. Still im not able to succeed..
Before...
10.0.1.150 Up Normal 2.61 GB 33.33% 0
10.0.1.152 Up Normal 2.61 GB 33.33%
56713727820156410577229101238628035242
10.0.1.154 Up Normal 2.61 GB 33.33%
113427455640312821154458202477256070485
After Killing Node and Restart...
10.0.1.150 Up Normal 2.61 GB 33.33% 0
10.0.1.152 Up Normal 2.61 GB 33.33%
56713727820156410577229101238628035242
10.0.1.154 Up Normal 61.69 KB33.33%
113427455640312821154458202477256070485
After Running Repair...
10.0.1.150 Up Normal 4.76 GB 33.33% 0
10.0.1.152 Up Normal 5.41 GB 33.33%
56713727820156410577229101238628035242
10.0.1.154 Up Normal 8.87 GB 33.33%
113427455640312821154458202477256070485
After Running Flush Compact on ALL nodes...
10.0.1.150 Up Normal 4.76 GB 33.33% 0
10.0.1.152 Up Normal 5.41 GB 33.33%
56713727820156410577229101238628035242
10.0.1.154 Up Normal 4.86 GB 33.33%
113427455640312821154458202477256070485
This does not occur in 0.7.6.. in fact it works perfectly..
was (Author: arenstar):
Ok. so i cant exactly hand out the inserting script but i can give you an
indication of the data format.. our scripts for testing are complex classes,
whereby building objects.. However this is EXACTLY an output of what actually
is written to cassandra ( CF test1 ) with a little obfuscation..
UUID4 key:
f9bb44f2844241df971e0975005c87dc
DATA FORMAT:
{'i': '[{pr: [XYZ, 0.47], va: [[XZ, 0.19]], pu: 1307998855,
iu: http://devel.test.com/test2730;, it: TESTERtype 0: TESTERobject
2730, pi: {XYZ: 0.31, XZ: 0.47}, id: 0!2730}]', 'cu': 'XYZ',
'cd': '1308668648'}
Try loading 100,000 of these with random uuid4 keys...
Furthermore, i have retried the test.. precisely again and again.. including a
flush before killing node3.. Still im left with..
Before...
10.0.1.150 Up Normal 2.61 GB 33.33% 0
10.0.1.152 Up Normal 2.61 GB 33.33%
56713727820156410577229101238628035242
10.0.1.154 Up Normal 2.61 GB 33.33%
113427455640312821154458202477256070485
After Killing Node and Restart...
10.0.1.150 Up Normal 2.61 GB 33.33% 0
10.0.1.152 Up Normal 2.61 GB 33.33%
56713727820156410577229101238628035242
10.0.1.154 Up Normal 61.69 KB33.33%
113427455640312821154458202477256070485
After Running Repair...
10.0.1.150 Up Normal 4.76 GB 33.33% 0
10.0.1.152 Up Normal 5.41 GB 33.33%
56713727820156410577229101238628035242
10.0.1.154 Up Normal 8.87 GB 33.33%
113427455640312821154458202477256070485
After Running Flush Compact on ALL nodes...
10.0.1.150 Up Normal 4.76 GB 33.33% 0
10.0.1.152 Up Normal 5.41 GB 33.33%
56713727820156410577229101238628035242
10.0.1.154 Up Normal 4.86
[jira] [Commented] (CASSANDRA-2634) .NET driver for CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-2634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052647#comment-13052647 ] Jonathan Ellis commented on CASSANDRA-2634: --- Michal, are you working on this? .NET driver for CQL --- Key: CASSANDRA-2634 URL: https://issues.apache.org/jira/browse/CASSANDRA-2634 Project: Cassandra Issue Type: Sub-task Components: API Affects Versions: 0.8 beta 1 Reporter: Michal Augustýn Priority: Minor The goal is to create ADO.NET driver for Cassandra, using CQL. So we have to implement [IDbConnection|http://msdn.microsoft.com/en-us/library/system.data.idbconnection.aspx] interface and all related interfaces (i.e. [IDbCommand|http://msdn.microsoft.com/en-us/library/system.data.idbcommand.aspx]). We must ensure that the connection is pooled. The implementation will be probably similar to CASSANDRA-1710. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2802) Enable encryption for data across the DC only.
Enable encryption for data across the DC only. -- Key: CASSANDRA-2802 URL: https://issues.apache.org/jira/browse/CASSANDRA-2802 Project: Cassandra Issue Type: New Feature Components: Contrib Affects Versions: 0.7.7 Environment: JVM Reporter: Vijay Assignee: Upayavira Priority: Trivial Fix For: 0.8.1 Make DC level Encryption option 1) Modify EncryptionOptions to add inter_dc option. 2) Modify OutboundTCPConnection.connect() to check if it is in the same DC and if the encryption option is enabled. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2802) Enable encryption for data across the DC only.
[ https://issues.apache.org/jira/browse/CASSANDRA-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vijay reassigned CASSANDRA-2802: Assignee: Vijay (was: Upayavira) Enable encryption for data across the DC only. -- Key: CASSANDRA-2802 URL: https://issues.apache.org/jira/browse/CASSANDRA-2802 Project: Cassandra Issue Type: New Feature Components: Contrib Affects Versions: 0.7.7 Environment: JVM Reporter: Vijay Assignee: Vijay Priority: Trivial Fix For: 0.8.1 Make DC level Encryption option 1) Modify EncryptionOptions to add inter_dc option. 2) Modify OutboundTCPConnection.connect() to check if it is in the same DC and if the encryption option is enabled. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2336) Extract SSTable.Builder/IndexWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2336: -- Reviewer: jbellis (was: slebresne) extracted IW and Builder - Rebuilder. Can you rebase 03? Extract SSTable.Builder/IndexWriter --- Key: CASSANDRA-2336 URL: https://issues.apache.org/jira/browse/CASSANDRA-2336 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Minor Fix For: 1.0 Attachments: 0001-CASSANDRA-2336-Extract-IndexWriter.txt, 0002-CASSANDRA-2336-Extract-Builder.txt, 0003-CASSANDRA-2336-Move-statistics-writing-into-IndexWrite.txt The Builder and IndexWriter classes in SSTableWriter are static, and independently useful. Additionally, we need the ability to subclass them for CASSANDRA-674 and CASSANDRA-2319. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2336) Extract SSTable.Builder/IndexWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052655#comment-13052655 ] Jonathan Ellis edited comment on CASSANDRA-2336 at 6/21/11 4:47 PM: extracted IW and Builder - Rebuilder in r1138084. Can you rebase 03? was (Author: jbellis): extracted IW and Builder - Rebuilder. Can you rebase 03? Extract SSTable.Builder/IndexWriter --- Key: CASSANDRA-2336 URL: https://issues.apache.org/jira/browse/CASSANDRA-2336 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Minor Fix For: 1.0 Attachments: 0001-CASSANDRA-2336-Extract-IndexWriter.txt, 0002-CASSANDRA-2336-Extract-Builder.txt, 0003-CASSANDRA-2336-Move-statistics-writing-into-IndexWrite.txt The Builder and IndexWriter classes in SSTableWriter are static, and independently useful. Additionally, we need the ability to subclass them for CASSANDRA-674 and CASSANDRA-2319. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-47) SSTable compression
[ https://issues.apache.org/jira/browse/CASSANDRA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-47: Priority: Major (was: Minor) Assignee: Pavel Yaskevich SSTable compression --- Key: CASSANDRA-47 URL: https://issues.apache.org/jira/browse/CASSANDRA-47 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Labels: compression Fix For: 1.0 We should be able to do SSTable compression which would trade CPU for I/O (almost always a good trade). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2164) debian build dep on ant-optional is missing
[ https://issues.apache.org/jira/browse/CASSANDRA-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052670#comment-13052670 ] paul cannon commented on CASSANDRA-2164: +1 this from me, fwiw debian build dep on ant-optional is missing --- Key: CASSANDRA-2164 URL: https://issues.apache.org/jira/browse/CASSANDRA-2164 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.1 Reporter: Peter Schuller Assignee: Peter Schuller Priority: Minor Attachments: 2164.txt Without the ant-optional package installed in Debian, builds fail (on lenny) with: Could not create type regexpmapper due to No supported regular expression matcher found: java.lang.ClassNotFoundException: org.apache.tools.ant.util.regexp.Jdk14RegexpMatcher The attached patch makes it build. Tested on lenny and squeeze. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2803) Cassandra deb should depend on libjna-java
Cassandra deb should depend on libjna-java -- Key: CASSANDRA-2803 URL: https://issues.apache.org/jira/browse/CASSANDRA-2803 Project: Cassandra Issue Type: Improvement Components: Packaging Reporter: paul cannon Priority: Minor Cassandra debs (0.7, 0.8, trunk) currently include a Recommends: for libjna-java, the package that includes the JNA jar. The original reason for the Recommends: instead of Depends: was that it's technically possible to run without JNA. However, since (a) I know of no reason not to use JNA, and (b) the Cassandra RPMs already require JNA, let us change this Recommends: to Depends: for all future debs. I don't believe this affects the licensing issues which stopped us from bundling JNA with cassandra directly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2164) debian build dep on ant-optional is missing
[ https://issues.apache.org/jira/browse/CASSANDRA-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2164: -- Reviewer: thepaul (was: urandom) Affects Version/s: (was: 0.7.1) 0.7.0 Fix Version/s: 0.7.7 debian build dep on ant-optional is missing --- Key: CASSANDRA-2164 URL: https://issues.apache.org/jira/browse/CASSANDRA-2164 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.0 Reporter: Peter Schuller Assignee: Peter Schuller Priority: Minor Fix For: 0.7.7 Attachments: 2164.txt Without the ant-optional package installed in Debian, builds fail (on lenny) with: Could not create type regexpmapper due to No supported regular expression matcher found: java.lang.ClassNotFoundException: org.apache.tools.ant.util.regexp.Jdk14RegexpMatcher The attached patch makes it build. Tested on lenny and squeeze. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1138102 - /cassandra/branches/cassandra-0.7/debian/control
Author: jbellis Date: Tue Jun 21 17:23:54 2011 New Revision: 1138102 URL: http://svn.apache.org/viewvc?rev=1138102view=rev Log: add ant-optional debian Build-Depends patch by Peter Schuller; reviewed by Paul Cannon for CASSANDRA-2164 Modified: cassandra/branches/cassandra-0.7/debian/control Modified: cassandra/branches/cassandra-0.7/debian/control URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/debian/control?rev=1138102r1=1138101r2=1138102view=diff == --- cassandra/branches/cassandra-0.7/debian/control (original) +++ cassandra/branches/cassandra-0.7/debian/control Tue Jun 21 17:23:54 2011 @@ -2,7 +2,7 @@ Source: cassandra Section: misc Priority: extra Maintainer: Eric Evans eev...@apache.org -Build-Depends: debhelper (= 5), openjdk-6-jdk (= 6b11) | java6-sdk, ant (= 1.7) +Build-Depends: debhelper (= 5), openjdk-6-jdk (= 6b11) | java6-sdk, ant (= 1.7), ant-optional (= 1.7) Homepage: http://cassandra.apache.org Vcs-Svn: https://svn.apache.org/repos/asf/cassandra/trunk Vcs-Browser: http://svn.apache.org/viewvc/cassandra/trunk
[jira] [Commented] (CASSANDRA-2164) debian build dep on ant-optional is missing
[ https://issues.apache.org/jira/browse/CASSANDRA-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052686#comment-13052686 ] paul cannon commented on CASSANDRA-2164: +1 again, but this time With Authority! The extra dependency is both necessary and satisfiable on all debian/ubuntu releases since 2008 (Lenny/Hardy or later), and it's doubtful that Cassandra builds, runs correctly, and is properly supportable on older releases anyway. debian build dep on ant-optional is missing --- Key: CASSANDRA-2164 URL: https://issues.apache.org/jira/browse/CASSANDRA-2164 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.0 Reporter: Peter Schuller Assignee: Peter Schuller Priority: Minor Fix For: 0.7.7, 0.8.2 Attachments: 2164.txt Without the ant-optional package installed in Debian, builds fail (on lenny) with: Could not create type regexpmapper due to No supported regular expression matcher found: java.lang.ClassNotFoundException: org.apache.tools.ant.util.regexp.Jdk14RegexpMatcher The attached patch makes it build. Tested on lenny and squeeze. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2803) Cassandra deb should depend on libjna-java
[ https://issues.apache.org/jira/browse/CASSANDRA-2803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul cannon updated CASSANDRA-2803: --- Attachment: 2803.txt Cassandra deb should depend on libjna-java -- Key: CASSANDRA-2803 URL: https://issues.apache.org/jira/browse/CASSANDRA-2803 Project: Cassandra Issue Type: Improvement Components: Packaging Reporter: paul cannon Priority: Minor Attachments: 2803.txt Cassandra debs (0.7, 0.8, trunk) currently include a Recommends: for libjna-java, the package that includes the JNA jar. The original reason for the Recommends: instead of Depends: was that it's technically possible to run without JNA. However, since (a) I know of no reason not to use JNA, and (b) the Cassandra RPMs already require JNA, let us change this Recommends: to Depends: for all future debs. I don't believe this affects the licensing issues which stopped us from bundling JNA with cassandra directly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2786) After a minor compaction, deleted key-slices are visible again
[ https://issues.apache.org/jira/browse/CASSANDRA-2786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052700#comment-13052700 ] Hudson commented on CASSANDRA-2786: --- Integrated in Cassandra-0.8 #182 (See [https://builds.apache.org/job/Cassandra-0.8/182/]) Fix wrong purge of deleted cf during compaction patch by slebresne; reviewed by jbellis for CASSANDRA-2786 slebresne : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1137984 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java After a minor compaction, deleted key-slices are visible again -- Key: CASSANDRA-2786 URL: https://issues.apache.org/jira/browse/CASSANDRA-2786 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.0 Environment: Reproduced on single Cassandra node (CentOS 5.5) Reproduced on single Cassandra node (Windows Server 2008) Reporter: rene kochen Assignee: Sylvain Lebresne Fix For: 0.8.1 Attachments: 0001-Fix-wrong-purge-of-deleted-cf.patch, CassandraIssue.zip, CassandraIssueJava.zip After a minor compaction, deleted key-slices are visible again. Steps to reproduce: 1) Insert a row named test. 2) Insert 50 rows. During this step, row test is included in a major compaction: file-1, file-2, file-3 and file-4 compacted to file-5 (includes test). 3) Delete row named test. 4) Insert 50 rows. During this step, row test is included in a minor compaction: file-6, file-7, file-8 and file-9 compacted to file-10 (should include tombstoned test). After step 4, row test is live again. Test environment: Single node with empty database. Standard configured super-column-family (I see this behavior with several gc_grace settings (big and small values): create column family Customers with column_type = 'Super' and comparator = 'BytesType; In Cassandra 0.7.6 I observe the expected behavior, i.e. after step 4, the row is still deleted. I've included a .NET program to reproduce the problem. I will add a Java version later on. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2801) Tombstone are not purged when the row is in only one sstable
[ https://issues.apache.org/jira/browse/CASSANDRA-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052701#comment-13052701 ] Hudson commented on CASSANDRA-2801: --- Integrated in Cassandra-0.8 #182 (See [https://builds.apache.org/job/Cassandra-0.8/182/]) Purge tombstones even if the row is in only one sstable patch by slebresne; reviewed by jbellis for CASSANDRA-2801 slebresne : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1137982 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionController.java Tombstone are not purged when the row is in only one sstable Key: CASSANDRA-2801 URL: https://issues.apache.org/jira/browse/CASSANDRA-2801 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.1 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 0.8.1 Attachments: 0001-Don-t-echo-row-when-purge-is-possible.patch We messed up the refactor of compactionController. It echoes rows if they are present in only one sstable, even if they could be purged. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2634) .NET driver for CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-2634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052708#comment-13052708 ] Michal Augustýn commented on CASSANDRA-2634: I'm very sorry - I'm now very busy at work and so I don't have time for this. I could probably implement that in late summer. .NET driver for CQL --- Key: CASSANDRA-2634 URL: https://issues.apache.org/jira/browse/CASSANDRA-2634 Project: Cassandra Issue Type: Sub-task Components: API Affects Versions: 0.8 beta 1 Reporter: Michal Augustýn Priority: Minor The goal is to create ADO.NET driver for Cassandra, using CQL. So we have to implement [IDbConnection|http://msdn.microsoft.com/en-us/library/system.data.idbconnection.aspx] interface and all related interfaces (i.e. [IDbCommand|http://msdn.microsoft.com/en-us/library/system.data.idbcommand.aspx]). We must ensure that the connection is pooled. The implementation will be probably similar to CASSANDRA-1710. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2798) Repair Fails 0.8
[ https://issues.apache.org/jira/browse/CASSANDRA-2798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052714#comment-13052714 ] Sylvain Lebresne commented on CASSANDRA-2798: - Alright, that was my mistake, I was testing on 0.8 current branch, thinking there was no reason this would have been fixed since 0.8.0 but it may actually be that it is fixed. On 0.8.0, I'm indeed able to reproduce that scenario. However, if after the all of this I restart the nodes and redo a compact after the restart, everything goes back to normal load. Which make me think that it's possibly CASSANDRA-2765 that prevents the compaction to actually happen. Restarting allows it to happen again. Can you validate (or deny) that a restart followed by a compact fixes this ? Repair Fails 0.8 Key: CASSANDRA-2798 URL: https://issues.apache.org/jira/browse/CASSANDRA-2798 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.0 Reporter: David Arena Assignee: Sylvain Lebresne I am seeing a fatal problem in the new 0.8 Im running a 3 node cluster with a replication_factor of 3.. On Node 3.. If i # kill -9 cassandra-pid # rm -rf All data logs # start cassandra # nodetool -h node-3-ip repair The whole cluster become duplicated.. * e.g Before node 1 - 2.65GB node 2 - 2.65GB node 3 - 2.65GB * e.g After node 1 - 5.3GB node 2 - 5.3GB node 3 - 7.95GB - nodetool repair, never ends (96 hours +), however there is no streams running, nor any cpu or disk activity.. - Manually killing the repair and restarting does not help.. Restarting the server/cassandra does not help.. - nodetool flush,compact,cleanup all complete, but do not help... This is not occuring in 0.7.6.. I have come to the conclusion this is a Major 0.8 issue Running: CentOS 5.6, JDK 1.6.0_26 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1138113 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/compaction/ test/unit/org/apache/cassandra/db/compaction
Author: slebresne Date: Tue Jun 21 18:06:55 2011 New Revision: 1138113 URL: http://svn.apache.org/viewvc?rev=1138113view=rev Log: Merge from 0.8 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java cassandra/trunk/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 18:06:55 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7:1026516-1133874,1135638 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1137774 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1137774,1137984 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1138113r1=1138112r2=1138113view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Tue Jun 21 18:06:55 2011 @@ -84,6 +84,7 @@ * make stress.jar executable (CASSANDRA-2744) * add daemon mode to java stress (CASSANDRA-2267) * expose the DC and rack of a node through JMX and nodetool ring (CASSANDRA-2531) + * Fix wrong purge of deleted cf during compaction (CASSANDRA-2786) 0.8.0-final Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 18:06:55 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 /cassandra/branches/cassandra-0.7/contrib:1026516-1133874,1135638 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1137774 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1137774,1137984 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 18:06:55 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1133874,1135638 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1137774 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1137774,1137984 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 18:06:55 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1133874,1135638
svn commit: r1138114 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/compaction/
Author: slebresne Date: Tue Jun 21 18:07:58 2011 New Revision: 1138114 URL: http://svn.apache.org/viewvc?rev=1138114view=rev Log: merge from 0.8 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionController.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 18:07:58 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7:1026516-1133874,1135638 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1137774,1137984 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1137774,1137982,1137984 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1138114r1=1138113r2=1138114view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Tue Jun 21 18:07:58 2011 @@ -85,6 +85,7 @@ * add daemon mode to java stress (CASSANDRA-2267) * expose the DC and rack of a node through JMX and nodetool ring (CASSANDRA-2531) * Fix wrong purge of deleted cf during compaction (CASSANDRA-2786) + * purge tombstone even if row is in only one sstable (CASSANDRA-2801) 0.8.0-final Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 18:07:58 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 /cassandra/branches/cassandra-0.7/contrib:1026516-1133874,1135638 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1137774,1137984 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1137774,1137982,1137984 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 18:07:58 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1133874,1135638 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1137774,1137984 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1137774,1137982,1137984 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 18:07:58 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1133874,1135638
[jira] [Commented] (CASSANDRA-47) SSTable compression
[ https://issues.apache.org/jira/browse/CASSANDRA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052734#comment-13052734 ] Ryan King commented on CASSANDRA-47: I think this is going to be obsoleted by CASSANDRA-674. SSTable compression --- Key: CASSANDRA-47 URL: https://issues.apache.org/jira/browse/CASSANDRA-47 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Labels: compression Fix For: 1.0 We should be able to do SSTable compression which would trade CPU for I/O (almost always a good trade). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-1717) Cassandra cannot detect corrupt-but-readable column data
[ https://issues.apache.org/jira/browse/CASSANDRA-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-1717: - Assignee: Pavel Yaskevich Should probably do this (either column-level or block-level) at the same time as CASSANDRA-47. Cassandra cannot detect corrupt-but-readable column data Key: CASSANDRA-1717 URL: https://issues.apache.org/jira/browse/CASSANDRA-1717 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: checksums.txt Most corruptions of on-disk data due to bitrot render the column (or row) unreadable, so the data can be replaced by read repair or anti-entropy. But if the corruption keeps column data readable we do not detect it, and if it corrupts to a higher timestamp value can even resist being overwritten by newer values. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1717) Cassandra cannot detect corrupt-but-readable column data
[ https://issues.apache.org/jira/browse/CASSANDRA-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052765#comment-13052765 ] Ryan King commented on CASSANDRA-1717: -- I know I'm starting to sound like a broken record, but CASSANDRA-674 is going to include checksums. And its almost ready for reviewing. Cassandra cannot detect corrupt-but-readable column data Key: CASSANDRA-1717 URL: https://issues.apache.org/jira/browse/CASSANDRA-1717 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: checksums.txt Most corruptions of on-disk data due to bitrot render the column (or row) unreadable, so the data can be replaced by read repair or anti-entropy. But if the corruption keeps column data readable we do not detect it, and if it corrupts to a higher timestamp value can even resist being overwritten by newer values. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1138148 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/streaming/StreamOutSession.java
Author: slebresne
Date: Tue Jun 21 19:47:36 2011
New Revision: 1138148
URL: http://svn.apache.org/viewvc?rev=1138148view=rev
Log:
Improve thread safety in StreamOutSession
patch by slebresne; reviewed by jbellis for CASSANDRA-2792
Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/streaming/StreamOutSession.java
Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1138148r1=1138147r2=1138148view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Tue Jun 21 19:47:36 2011
@@ -18,7 +18,8 @@
* fix nodetool ring use with Ec2Snitch (CASSANDRA-2733)
* fix removing columns and subcolumns that are supressed by a row or
supercolumn tombstone during replica resolution (CASSANDRA-2590)
- * use threadsafe collections for StreamInSession (CASSANDRA-2766)
+ * use threadsafe collections for StreamInSession and StreamOutSession
+ (CASSANDRA-2766, CASSANDRA-2792)
0.7.6
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/streaming/StreamOutSession.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/streaming/StreamOutSession.java?rev=1138148r1=1138147r2=1138148view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/streaming/StreamOutSession.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/streaming/StreamOutSession.java
Tue Jun 21 19:47:36 2011
@@ -64,12 +64,12 @@ public class StreamOutSession
return streams.get(new PairInetAddress, Long(host, sessionId));
}
-private final MapString, PendingFile files = new LinkedHashMapString,
PendingFile();
+private final MapString, PendingFile files = new
NonBlockingHashMapString, PendingFile();
public final String table;
private final PairInetAddress, Long context;
private final Runnable callback;
-private String currentFile;
+private volatile String currentFile;
private StreamOutSession(String table, PairInetAddress, Long context,
Runnable callback)
{
svn commit: r1138149 - in /cassandra/branches/cassandra-0.8: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/streaming/
Author: slebresne Date: Tue Jun 21 19:50:53 2011 New Revision: 1138149 URL: http://svn.apache.org/viewvc?rev=1138149view=rev Log: Merge from 0.7 Modified: cassandra/branches/cassandra-0.8/ (props changed) cassandra/branches/cassandra-0.8/CHANGES.txt cassandra/branches/cassandra-0.8/contrib/ (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/streaming/StreamOutSession.java Propchange: cassandra/branches/cassandra-0.8/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 19:50:53 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7:1026516-1133874,1135638 +/cassandra/branches/cassandra-0.7:1026516-1133874,1135638,1138148 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/branches/cassandra-0.8:1090934-1125013,1125041 /cassandra/branches/cassandra-0.8.0:1125021-1130369 Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1138149r1=1138148r2=1138149view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Tue Jun 21 19:50:53 2011 @@ -1,3 +1,6 @@ +0,8.2 + * improve thread-safety in StreamOutSession (CASSANDRA-2792) + 0.8.1 * CQL: - support for insert, delete in BATCH (CASSANDRA-2537) Propchange: cassandra/branches/cassandra-0.8/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 19:50:53 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1133874,1135638 +/cassandra/branches/cassandra-0.7/contrib:1026516-1133874,1135638,1138148 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125041 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 19:50:53 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1133874,1135638 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1133874,1135638,1138148 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125041 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 19:50:53 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1133874,1135638 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1133874,1135638,1138148 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654 /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1090934-1125013,1125041
svn commit: r1138150 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/streaming/
Author: slebresne Date: Tue Jun 21 19:52:55 2011 New Revision: 1138150 URL: http://svn.apache.org/viewvc?rev=1138150view=rev Log: merge from 0.8 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/streaming/StreamOutSession.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 19:52:55 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7:1026516-1133874,1135638 +/cassandra/branches/cassandra-0.7:1026516-1133874,1135638,1138148 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1137774,1137982,1137984 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1137774,1137982,1137984,1138149 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1138150r1=1138149r2=1138150view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Tue Jun 21 19:52:55 2011 @@ -10,10 +10,7 @@ 0.8.2 - * fix cache mbean getSize (CASSANDRA-2781) - * Add Date, Float, Double, and Boolean types (CASSANDRA-2530) - * fix repair hanging if a neighbor has nothing to send (CASSANDRA-2797) - * add jamm agent to cassandra.bat (CASSANDRA-2787) + * improve thread-safety in StreamOutSession (CASSANDRA-2792) 0.8.1 @@ -84,6 +81,10 @@ * make stress.jar executable (CASSANDRA-2744) * add daemon mode to java stress (CASSANDRA-2267) * expose the DC and rack of a node through JMX and nodetool ring (CASSANDRA-2531) + * fix cache mbean getSize (CASSANDRA-2781) + * Add Date, Float, Double, and Boolean types (CASSANDRA-2530) + * fix repair hanging if a neighbor has nothing to send (CASSANDRA-2797) + * add jamm agent to cassandra.bat (CASSANDRA-2787) * Fix wrong purge of deleted cf during compaction (CASSANDRA-2786) * purge tombstone even if row is in only one sstable (CASSANDRA-2801) Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 19:52:55 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1133874,1135638 +/cassandra/branches/cassandra-0.7/contrib:1026516-1133874,1135638,1138148 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1137774,1137982,1137984 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1137774,1137982,1137984,1138149 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Jun 21 19:52:55 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1133874,1135638 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1133874,1135638,1138148 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1137774,1137982,1137984 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1137774,1137982,1137984,1138149
[jira] [Updated] (CASSANDRA-1966) Option to control how many items are read on cache load
[ https://issues.apache.org/jira/browse/CASSANDRA-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Burroughs updated CASSANDRA-1966: --- Attachment: 1966-v2.txt - double -- int for thrift/avro - -1 -- max int for special value (but it's still a special value) - CLI support I think I got everything from http://wiki.apache.org/cassandra/ConfigurationNotes down, but admit that I'm nervous at the size of the patch. Max int is slightly confusing in jconsole, but I don't have a strong opinion between that and -1. Option to control how many items are read on cache load --- Key: CASSANDRA-1966 URL: https://issues.apache.org/jira/browse/CASSANDRA-1966 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Attachments: 1966-v1.txt, 1966-v2.txt CASSANDRA-1417 added an option to save the key and/or row cache keys which is cool. However, for a row large cache it can take a long time to read all of the rows. For example I have a 400,000 item row cache, and loading that on restart takes a little under an hour. In addition to configuring the size of the row cache, and how often it should be saved to disk, I propose an option to control how many items are loaded on startup (or alternately only saving n items out of the full row cache to begin with). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1138152 - /cassandra/branches/cassandra-0.8/CHANGES.txt
Author: slebresne Date: Tue Jun 21 19:54:04 2011 New Revision: 1138152 URL: http://svn.apache.org/viewvc?rev=1138152view=rev Log: Fix typo in changelog Modified: cassandra/branches/cassandra-0.8/CHANGES.txt Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1138152r1=1138151r2=1138152view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Tue Jun 21 19:54:04 2011 @@ -1,4 +1,4 @@ -0,8.2 +0.8.2 * improve thread-safety in StreamOutSession (CASSANDRA-2792) 0.8.1
[jira] [Issue Comment Edited] (CASSANDRA-1966) Option to control how many items are read on cache load
[ https://issues.apache.org/jira/browse/CASSANDRA-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052789#comment-13052789 ] Chris Burroughs edited comment on CASSANDRA-1966 at 6/21/11 7:54 PM: - - double -- int for thrift/avro - -1 -- max int for special value (but it's still a special value) - CLI support - Schema migration should work now, only jmx changes worked with v1. I think I got everything from http://wiki.apache.org/cassandra/ConfigurationNotes down, but admit that I'm nervous at the size of the patch. Max int is slightly confusing in jconsole, but I don't have a strong opinion between that and -1. was (Author: cburroughs): - double -- int for thrift/avro - -1 -- max int for special value (but it's still a special value) - CLI support I think I got everything from http://wiki.apache.org/cassandra/ConfigurationNotes down, but admit that I'm nervous at the size of the patch. Max int is slightly confusing in jconsole, but I don't have a strong opinion between that and -1. Option to control how many items are read on cache load --- Key: CASSANDRA-1966 URL: https://issues.apache.org/jira/browse/CASSANDRA-1966 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Attachments: 1966-v1.txt, 1966-v2.txt CASSANDRA-1417 added an option to save the key and/or row cache keys which is cool. However, for a row large cache it can take a long time to read all of the rows. For example I have a 400,000 item row cache, and loading that on restart takes a little under an hour. In addition to configuring the size of the row cache, and how often it should be saved to disk, I propose an option to control how many items are loaded on startup (or alternately only saving n items out of the full row cache to begin with). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2804) expose dropped messages, exceptions over JMX
expose dropped messages, exceptions over JMX Key: CASSANDRA-2804 URL: https://issues.apache.org/jira/browse/CASSANDRA-2804 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.8.2 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2405) should expose 'time since last successful repair' for easier aes monitoring
[ https://issues.apache.org/jira/browse/CASSANDRA-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052815#comment-13052815 ] Jonathan Ellis commented on CASSANDRA-2405: --- Nick and Tyler pointed out that we want similar information for basically everything that runs on CompactionManager. Maybe we could make the DetailedInfo cf take an arbitrary map from CompactionInfo? (Rename to BackgroundIODetails?) should expose 'time since last successful repair' for easier aes monitoring --- Key: CASSANDRA-2405 URL: https://issues.apache.org/jira/browse/CASSANDRA-2405 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Peter Schuller Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.8.2 Attachments: CASSANDRA-2405-v2.patch, CASSANDRA-2405-v3.patch, CASSANDRA-2405-v4.patch, CASSANDRA-2405.patch The practical implementation issues of actually ensuring repair runs is somewhat of an undocumented/untreated issue. One hopefully low hanging fruit would be to at least expose the time since last successful repair for a particular column family, to make it easier to write a correct script to monitor for lack of repair in a non-buggy fashion. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2805) Clean up mbeans that return Internal Cassandra types
Clean up mbeans that return Internal Cassandra types Key: CASSANDRA-2805 URL: https://issues.apache.org/jira/browse/CASSANDRA-2805 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.1 Reporter: Nick Bailey Priority: Minor Fix For: 0.8.2 We need to clean up wherever we return internal cassandra objects over jmx. Namely CompactionInfo objects as well as Tokens. There may be a few other examples. This is bad for two reasons 1. You have to load the cassandra jar when querying these mbeans, which sucks. 2. Stuff breaks between versions when things are moved. For example, CASSANDRA-1610 moves the compaction related classes around. Any code querying those jmx mbeans in 0.8.0 is now broken in 0.8.1. For things like CompactionInfo we should just expose more mbean methods or serialize to something standard like json. I'd like to target this for 0.8.2. Since we've already broken compatibility between 0.8.0 and 0.8.1, I'd say just fix this everywhere now. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2405) should expose 'time since last successful repair' for easier aes monitoring
[ https://issues.apache.org/jira/browse/CASSANDRA-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052818#comment-13052818 ] Pavel Yaskevich commented on CASSANDRA-2405: Separate task? should expose 'time since last successful repair' for easier aes monitoring --- Key: CASSANDRA-2405 URL: https://issues.apache.org/jira/browse/CASSANDRA-2405 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Peter Schuller Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.8.2 Attachments: CASSANDRA-2405-v2.patch, CASSANDRA-2405-v3.patch, CASSANDRA-2405-v4.patch, CASSANDRA-2405.patch The practical implementation issues of actually ensuring repair runs is somewhat of an undocumented/untreated issue. One hopefully low hanging fruit would be to at least expose the time since last successful repair for a particular column family, to make it easier to write a correct script to monitor for lack of repair in a non-buggy fashion. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2806) Expose gossip/FD info to JMX
Expose gossip/FD info to JMX Key: CASSANDRA-2806 URL: https://issues.apache.org/jira/browse/CASSANDRA-2806 Project: Cassandra Issue Type: Improvement Reporter: Brandon Williams Assignee: Brandon Williams Priority: Minor Fix For: 0.8.2 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2805) Clean up mbeans that return Internal Cassandra types
[ https://issues.apache.org/jira/browse/CASSANDRA-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Bailey reassigned CASSANDRA-2805: -- Assignee: Nick Bailey Clean up mbeans that return Internal Cassandra types Key: CASSANDRA-2805 URL: https://issues.apache.org/jira/browse/CASSANDRA-2805 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.1 Reporter: Nick Bailey Assignee: Nick Bailey Priority: Minor Fix For: 0.8.2 We need to clean up wherever we return internal cassandra objects over jmx. Namely CompactionInfo objects as well as Tokens. There may be a few other examples. This is bad for two reasons 1. You have to load the cassandra jar when querying these mbeans, which sucks. 2. Stuff breaks between versions when things are moved. For example, CASSANDRA-1610 moves the compaction related classes around. Any code querying those jmx mbeans in 0.8.0 is now broken in 0.8.2. (assuming those moves stay in the 0.8 branch) For things like CompactionInfo we should just expose more mbean methods or serialize to something standard like json. I'd like to target this for 0.8.2. Since we've already broken compatibility between 0.8.0 and 0.8.1, I'd say just fix this everywhere now. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of ThriftExamples by ChrisLarsen
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The ThriftExamples page has been changed by ChrisLarsen:
http://wiki.apache.org/cassandra/ThriftExamples?action=diffrev1=87rev2=88
Comment:
Added a CPP example for Thrift
## page was copied from ClientExamples
-
'''If you are new to Cassandra and just want to get started, take a look at
the available clients [[ClientOptions06|ClientOptions (v0.6)]] instead.'''
TableOfContents()
-
-
- This page shows examples of using the low-level
[[http://incubator.apache.org/thrift/|Thrift]] interface, primarily intended
for client library developers.
+ This page shows examples of using the low-level
[[http://incubator.apache.org/thrift/|Thrift]] interface, primarily intended
for client library developers.
-
-
To generate the bindings for a particular language, first find out if Thrift
supports that language. If it does, you can run thrift --gen XYZ
interface/cassandra.thrift for whatever XYZ you fancy.
@@ -556, +551 @@
}
}
}}}
+ == C++ ==
+ This is for Cassandra version 0.7 or later
+
+ {{{#!c
+ #include Thrift.h
+ #include transport/TSocket.h
+ #include transport/TTransport.h
+ #include transport/TBufferTransports.h
+ #include protocol/TProtocol.h
+ #include protocol/TBinaryProtocol.h
+ #include Cassandra.h
+
+ using namespace apache::thrift;
+ using namespace apache::thrift::transport;
+ using namespace apache::thrift::protocol;
+ using namespace org::apache::cassandra;
+
+ int main(int argc, char *argv[]){
+ try{
+ boost::shared_ptrTTransport socket = boost::shared_ptrTSocket(new
TSocket(127.0.0.1, 9160));
+ boost::shared_ptrTTransport tr =
boost::shared_ptrTFramedTransport(new TFramedTransport (socket));
+ boost::shared_ptrTProtocol p = boost::shared_ptrTBinaryProtocol(new
TBinaryProtocol(tr));
+ CassandraClient cass(p);
+ tr-open();
+
+ cass.set_keyspace(Keyspace1);
+
+ string key = 1;
+ ColumnParent cparent;
+ cparent.column_family = Standard1;
+ Column c;
+ c.name = name;
+ c.value = John Smith;
+
+ // have to go through all of this just to get the timestamp in ms
+ struct timeval td;
+ gettimeofday(td, NULL);
+ int64_t ms = td.tv_sec;
+ ms = ms * 1000;
+ int64_t usec = td.tv_usec;
+ usec = usec / 1000;
+ ms += usec;
+ c.timestamp = ms;
+
+ // insert the name column
+ cass.insert(key, cparent, c, ConsistencyLevel::ONE);
+
+ // insert another column, age
+ c.name = age;
+ c.value = 42;
+ cass.insert(key, cparent, c, ConsistencyLevel::ONE);
+
+ // get a single cell
+ ColumnPath cp;
+ cp.__isset.column = true; // this must be set of you'll get an
error re: Padraig O'Sullivan
+ cp.column = name;
+ cp.column_family = Standard1;
+ cp.super_column = ;
+ ColumnOrSuperColumn sc;
+
+ cass.get(sc, key, cp, ConsistencyLevel::ONE);
+ printf(Column [%s] Value [%s] TS [%lld]\n,
+ sc.column.name.c_str(), sc.column.value.c_str(), sc.column.timestamp);
+
+ // get the entire row for a key
+ SliceRange sr;
+ sr.start = ;
+ sr.finish = ;
+
+ SlicePredicate sp;
+ sp.slice_range = sr;
+ sp.__isset.slice_range = true; // set __isset for the columns instead if
you use them
+
+ KeyRange range;
+ range.start_key = key;
+ range.end_key = ;
+ range.__isset.start_key = true;
+ range.__isset.end_key = true;
+
+ vectorKeySlice results;
+ cass.get_range_slices(results, cparent, sp, range, ConsistencyLevel::ONE);
+ for(size_t i=0; iresults.size(); i++){
+ printf(Key: %s\n, results[i].key.c_str());
+ for(size_t x=0; xresults[i].columns.size(); x++){
+ printf(Column: %s Value: %s\n,
results[i].columns[x].column.name.c_str(),
+ results[i].columns[x].column.value.c_str());
+ }
+ }
+
+ tr-close();
+ }catch(TTransportException te){
+ printf(Exception: %s [%d]\n, te.what(), te.getType());
+ }catch(InvalidRequestException ire){
+ printf(Exception: %s [%s]\n, ire.what(), ire.why.c_str());
+ }catch(NotFoundException nfe){
+ printf(Exception: %s\n, nfe.what());
+ }
+ printf(Done!!!\n);
+ return;
+ }
+ }}}
== Notes ==
The Cassandra.Client object is always sending its request to the same
Cassandra node in the cluster. The server then determines if and where the
request should be routed to (Server-based routing). DNS Round Robin or a
Cassandra.Client object pool connected to several servers in the cluster can be
used to get higher throughput and availability.
[jira] [Commented] (CASSANDRA-2735) Timestamp Based Compaction Strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052834#comment-13052834 ] Alan Liang commented on CASSANDRA-2735: --- This compaction strategy is useful for time series data. Eg. you capture counts for each minute, hour, day. Ordering and compacting the sstables by column timestamp allows you to expire sstables more effectively compared to the size tiered approach in trunk. This is because the size tiered approach could combine an old sstable with a new sstable, which renders the sstable to look like it is quite new. You would not be able to expire the old data in this case. Timestamp Based Compaction Strategy --- Key: CASSANDRA-2735 URL: https://issues.apache.org/jira/browse/CASSANDRA-2735 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Alan Liang Assignee: Alan Liang Priority: Minor Labels: compaction Attachments: 0004-timestamp-bucketed-compaction-strategy.patch Compaction strategy implementation based on max timestamp ordering of the sstables while satisfying max sstable size, min and max compaction thresholds. It also handles expiration of sstables based on a timestamp. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2735) Timestamp Based Compaction Strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Liang updated CASSANDRA-2735: -- Attachment: 0001-timestamp-bucketed-compaction-strategy.patch Highlights of this patch: - Introduce a timestamp compaction strategy - Introduce Expiration Task - option to delete or move to expired folder - Tests for timestamp bucketing strategy This patch depends on https://issues.apache.org/jira/browse/CASSANDRA-2753 to be committed. Timestamp Based Compaction Strategy --- Key: CASSANDRA-2735 URL: https://issues.apache.org/jira/browse/CASSANDRA-2735 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Alan Liang Assignee: Alan Liang Priority: Minor Labels: compaction Attachments: 0001-timestamp-bucketed-compaction-strategy.patch Compaction strategy implementation based on max timestamp ordering of the sstables while satisfying max sstable size, min and max compaction thresholds. It also handles expiration of sstables based on a timestamp. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2735) Timestamp Based Compaction Strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Liang updated CASSANDRA-2735: -- Attachment: (was: 0004-timestamp-bucketed-compaction-strategy.patch) Timestamp Based Compaction Strategy --- Key: CASSANDRA-2735 URL: https://issues.apache.org/jira/browse/CASSANDRA-2735 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Alan Liang Assignee: Alan Liang Priority: Minor Labels: compaction Attachments: 0001-timestamp-bucketed-compaction-strategy.patch Compaction strategy implementation based on max timestamp ordering of the sstables while satisfying max sstable size, min and max compaction thresholds. It also handles expiration of sstables based on a timestamp. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2807) ColumnFamilyInputFormat configuration should support multiple initial addresses
ColumnFamilyInputFormat configuration should support multiple initial addresses
---
Key: CASSANDRA-2807
URL: https://issues.apache.org/jira/browse/CASSANDRA-2807
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Affects Versions: 0.8.0
Reporter: Greg Katz
The {{ColumnFamilyInputFormat}} class only allows a single initial node to be
specified through the cassandra.thrift.address configuration property. The
configuration should support a list of nodes in order to account for the
possibility that the initial node becomes unavailable.
By contrast, the {{RingCache}} class used by the {{ColumnFamilyRecordWriter}}
reads the exact same {{cassandra.thrift.address}} property but splits its value
on commas to allow multiple initial nodes to be specified.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2405) should expose 'time since last successful repair' for easier aes monitoring
[ https://issues.apache.org/jira/browse/CASSANDRA-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052870#comment-13052870 ] Pavel Yaskevich commented on CASSANDRA-2405: v3 does not have a detailed info CF so it can be committed... should expose 'time since last successful repair' for easier aes monitoring --- Key: CASSANDRA-2405 URL: https://issues.apache.org/jira/browse/CASSANDRA-2405 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Peter Schuller Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.8.2 Attachments: CASSANDRA-2405-v2.patch, CASSANDRA-2405-v3.patch, CASSANDRA-2405-v4.patch, CASSANDRA-2405.patch The practical implementation issues of actually ensuring repair runs is somewhat of an undocumented/untreated issue. One hopefully low hanging fruit would be to at least expose the time since last successful repair for a particular column family, to make it easier to write a correct script to monitor for lack of repair in a non-buggy fashion. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2500) Ruby dbi client (for CQL) that conforms to AR:ConnectionAdapter
[ https://issues.apache.org/jira/browse/CASSANDRA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-2500: --- Reviewer: xedin (was: rwjblue) Ruby dbi client (for CQL) that conforms to AR:ConnectionAdapter --- Key: CASSANDRA-2500 URL: https://issues.apache.org/jira/browse/CASSANDRA-2500 Project: Cassandra Issue Type: Task Reporter: Jon Hermes Assignee: Jon Hermes Attachments: 2500.txt, genthriftrb.txt, rbcql-0.0.0.tgz Create a ruby driver for CQL. Lacking something standard (such as py-dbapi), going with something common instead -- RoR ActiveRecord Connection Adapter (http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/AbstractAdapter.html). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Trivial Update of ThriftExamples by ChrisLarsen
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The ThriftExamples page has been changed by ChrisLarsen:
http://wiki.apache.org/cassandra/ThriftExamples?action=diffrev1=88rev2=89
Comment:
Just added C++ code highlighting tag to the C++ example
To create Cassandra.php and cassandra_types.php you must use thrift -gen php
interface/cassandra.thrift of the cassandra package the rest of these files
come from the main thrift package.
- {{{
+ {{{#
?php
$GLOBALS['THRIFT_ROOT'] = '/usr/share/php/Thrift';
require_once $GLOBALS['THRIFT_ROOT'].'/packages/cassandra/Cassandra.php';
@@ -554, +554 @@
== C++ ==
This is for Cassandra version 0.7 or later
- {{{#!c
+ {{{#!cplusplus
#include Thrift.h
#include transport/TSocket.h
#include transport/TTransport.h
@@ -636, +636 @@
printf(Key: %s\n, results[i].key.c_str());
for(size_t x=0; xresults[i].columns.size(); x++){
printf(Column: %s Value: %s\n,
results[i].columns[x].column.name.c_str(),
- results[i].columns[x].column.value.c_str());
+ results[i].columns[x].column.value.c_str());
}
}
[jira] [Commented] (CASSANDRA-2336) Extract SSTable.Builder/IndexWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13052980#comment-13052980 ] Stu Hood commented on CASSANDRA-2336: - So it looks like the reason this didn't apply cleanly was because it applies atop 2576 and 2629. If those patches look alright, would it be alright if we backed out r1138084 and applied them in that order? I'll confirm that they apply. Extract SSTable.Builder/IndexWriter --- Key: CASSANDRA-2336 URL: https://issues.apache.org/jira/browse/CASSANDRA-2336 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Minor Fix For: 1.0 Attachments: 0001-CASSANDRA-2336-Extract-IndexWriter.txt, 0002-CASSANDRA-2336-Extract-Builder.txt, 0003-CASSANDRA-2336-Move-statistics-writing-into-IndexWrite.txt The Builder and IndexWriter classes in SSTableWriter are static, and independently useful. Additionally, we need the ability to subclass them for CASSANDRA-674 and CASSANDRA-2319. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2336) Extract SSTable.Builder/IndexWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2336: Attachment: (was: 0003-CASSANDRA-2336-Move-statistics-writing-into-IndexWrite.txt) Extract SSTable.Builder/IndexWriter --- Key: CASSANDRA-2336 URL: https://issues.apache.org/jira/browse/CASSANDRA-2336 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Minor Fix For: 1.0 Attachments: 0001-CASSANDRA-2336-Extract-IndexWriter.txt, 0002-CASSANDRA-2336-Extract-Builder.txt, 0003-CASSANDRA-2336-Move-statistics-writing-into-IndexWrite.txt The Builder and IndexWriter classes in SSTableWriter are static, and independently useful. Additionally, we need the ability to subclass them for CASSANDRA-674 and CASSANDRA-2319. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2336) Extract SSTable.Builder/IndexWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2336: Attachment: (was: 0002-CASSANDRA-2336-Extract-Builder.txt) Extract SSTable.Builder/IndexWriter --- Key: CASSANDRA-2336 URL: https://issues.apache.org/jira/browse/CASSANDRA-2336 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Minor Fix For: 1.0 Attachments: 0001-CASSANDRA-2336-Extract-IndexWriter.txt, 0002-CASSANDRA-2336-Extract-Builder.txt, 0003-CASSANDRA-2336-Move-statistics-writing-into-IndexWrite.txt The Builder and IndexWriter classes in SSTableWriter are static, and independently useful. Additionally, we need the ability to subclass them for CASSANDRA-674 and CASSANDRA-2319. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2336) Extract SSTable.Builder/IndexWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2336: Attachment: 0003-CASSANDRA-2336-Move-statistics-writing-into-IndexWrite.txt 0002-CASSANDRA-2336-Extract-Builder.txt 0001-CASSANDRA-2336-Extract-IndexWriter.txt Rebased: applies atop #2468, #2576, #2629 with r1138084 reverted. Extract SSTable.Builder/IndexWriter --- Key: CASSANDRA-2336 URL: https://issues.apache.org/jira/browse/CASSANDRA-2336 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Minor Fix For: 1.0 Attachments: 0001-CASSANDRA-2336-Extract-IndexWriter.txt, 0002-CASSANDRA-2336-Extract-Builder.txt, 0003-CASSANDRA-2336-Move-statistics-writing-into-IndexWrite.txt The Builder and IndexWriter classes in SSTableWriter are static, and independently useful. Additionally, we need the ability to subclass them for CASSANDRA-674 and CASSANDRA-2319. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2336) Extract SSTable.Builder/IndexWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2336: Attachment: (was: 0001-CASSANDRA-2336-Extract-IndexWriter.txt) Extract SSTable.Builder/IndexWriter --- Key: CASSANDRA-2336 URL: https://issues.apache.org/jira/browse/CASSANDRA-2336 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Minor Fix For: 1.0 Attachments: 0001-CASSANDRA-2336-Extract-IndexWriter.txt, 0002-CASSANDRA-2336-Extract-Builder.txt, 0003-CASSANDRA-2336-Move-statistics-writing-into-IndexWrite.txt The Builder and IndexWriter classes in SSTableWriter are static, and independently useful. Additionally, we need the ability to subclass them for CASSANDRA-674 and CASSANDRA-2319. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2468) Clean up after failed compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2468: Attachment: (was: 0001-CASSANDRA-2468-clean-up-temp-files-after-failed-compac-v08-4.txt) Clean up after failed compaction Key: CASSANDRA-2468 URL: https://issues.apache.org/jira/browse/CASSANDRA-2468 Project: Cassandra Issue Type: Bug Components: Core Reporter: Jonathan Ellis Assignee: Aaron Morton Priority: Minor Fix For: 0.7.7 Attachments: 0001-CASSANDRA-2468-clean-up-temp-files-after-failed-compac.txt, 0001-clean-up-temp-files-after-failed-compaction-v08-2.patch, 0001-clean-up-temp-files-after-failed-compaction-v08-3.patch, 0001-clean-up-temp-files-after-failed-compaction-v08.patch, 0001-cleanup-temp-files-after-failed-compaction-v07.patch (Started in CASSANDRA-2088.) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2468) Clean up after failed compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2468: Attachment: 0001-CASSANDRA-2468-clean-up-temp-files-after-failed-compac.txt Rebased for trunk (assuming r1138084 is reverted). Clean up after failed compaction Key: CASSANDRA-2468 URL: https://issues.apache.org/jira/browse/CASSANDRA-2468 Project: Cassandra Issue Type: Bug Components: Core Reporter: Jonathan Ellis Assignee: Aaron Morton Priority: Minor Fix For: 0.7.7 Attachments: 0001-CASSANDRA-2468-clean-up-temp-files-after-failed-compac.txt, 0001-clean-up-temp-files-after-failed-compaction-v08-2.patch, 0001-clean-up-temp-files-after-failed-compaction-v08-3.patch, 0001-clean-up-temp-files-after-failed-compaction-v08.patch, 0001-cleanup-temp-files-after-failed-compaction-v07.patch (Started in CASSANDRA-2088.) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2576) Rewrite into new file post streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2576: Attachment: 0002-CASSANDRA-2576-Rebuild-into-a-new-file-to-minimize-mag.txt 0001-CASSANDRA-2576-Don-t-depend-on-a-byte-for-byte-match-f.txt Rebased for trunk: applies atop #2468 (assuming r1138084 is reverted). Rewrite into new file post streaming Key: CASSANDRA-2576 URL: https://issues.apache.org/jira/browse/CASSANDRA-2576 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Fix For: 1.0 Attachments: 0001-CASSANDRA-2576-Don-t-depend-on-a-byte-for-byte-match-f.txt, 0002-CASSANDRA-2576-Rebuild-into-a-new-file-to-minimize-mag.txt Commutative/counter column families use a separate path to rebuild sstables post streaming, and that path currently rewrites the data within the streamed file. While this is great for space efficiency, it means a duplicated code path for writing sstables, which makes it more difficult to make changes like #674. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2576) Rewrite into new file post streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2576: Attachment: (was: 0001-CASSANDRA-2576-Don-t-depend-on-a-byte-for-byte-match-f.txt) Rewrite into new file post streaming Key: CASSANDRA-2576 URL: https://issues.apache.org/jira/browse/CASSANDRA-2576 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Fix For: 1.0 Attachments: 0001-CASSANDRA-2576-Don-t-depend-on-a-byte-for-byte-match-f.txt, 0002-CASSANDRA-2576-Rebuild-into-a-new-file-to-minimize-mag.txt Commutative/counter column families use a separate path to rebuild sstables post streaming, and that path currently rewrites the data within the streamed file. While this is great for space efficiency, it means a duplicated code path for writing sstables, which makes it more difficult to make changes like #674. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2576) Rewrite into new file post streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2576: Attachment: (was: 0002-CASSANDRA-2576-Rebuild-into-a-new-file-to-minimize-mag.txt) Rewrite into new file post streaming Key: CASSANDRA-2576 URL: https://issues.apache.org/jira/browse/CASSANDRA-2576 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Fix For: 1.0 Attachments: 0001-CASSANDRA-2576-Don-t-depend-on-a-byte-for-byte-match-f.txt, 0002-CASSANDRA-2576-Rebuild-into-a-new-file-to-minimize-mag.txt Commutative/counter column families use a separate path to rebuild sstables post streaming, and that path currently rewrites the data within the streamed file. While this is great for space efficiency, it means a duplicated code path for writing sstables, which makes it more difficult to make changes like #674. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2629) Move key reads into SSTableIterators
[ https://issues.apache.org/jira/browse/CASSANDRA-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2629: Attachment: (was: 0002-CASSANDRA-2629-Remove-the-retry-with-key-from-index-st.txt) Move key reads into SSTableIterators Key: CASSANDRA-2629 URL: https://issues.apache.org/jira/browse/CASSANDRA-2629 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Fix For: 1.0 All SSTableIterators have a constructor that assumes the key and length has already been parsed. Moving this logic inside the iterator will improve symmetry and allow the file format to change without iterator consumers knowing it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2629) Move key reads into SSTableIterators
[ https://issues.apache.org/jira/browse/CASSANDRA-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2629: Attachment: (was: 0001-CASSANDRA-2629-Move-key-and-row-size-reading-into-the-.txt) Move key reads into SSTableIterators Key: CASSANDRA-2629 URL: https://issues.apache.org/jira/browse/CASSANDRA-2629 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Fix For: 1.0 All SSTableIterators have a constructor that assumes the key and length has already been parsed. Moving this logic inside the iterator will improve symmetry and allow the file format to change without iterator consumers knowing it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2629) Move key reads into SSTableIterators
[ https://issues.apache.org/jira/browse/CASSANDRA-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2629: Attachment: 0002-CASSANDRA-2629-Remove-the-retry-with-key-from-index-st.txt 0001-CASSANDRA-2629-Move-key-and-row-size-reading-into-the-.txt Rebased for trunk: applies atop #2468 and #2576 (assuming r1138084 is reverted). Move key reads into SSTableIterators Key: CASSANDRA-2629 URL: https://issues.apache.org/jira/browse/CASSANDRA-2629 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Fix For: 1.0 Attachments: 0001-CASSANDRA-2629-Move-key-and-row-size-reading-into-the-.txt, 0002-CASSANDRA-2629-Remove-the-retry-with-key-from-index-st.txt All SSTableIterators have a constructor that assumes the key and length has already been parsed. Moving this logic inside the iterator will improve symmetry and allow the file format to change without iterator consumers knowing it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1138282 - in /cassandra/trunk/src/java/org/apache/cassandra: db/compaction/CompactionManager.java io/sstable/IndexWriter.java io/sstable/Rebuilder.java io/sstable/SSTableWriter.java
Author: jbellis
Date: Wed Jun 22 02:32:30 2011
New Revision: 1138282
URL: http://svn.apache.org/viewvc?rev=1138282view=rev
Log:
revert r1138084
Removed:
cassandra/trunk/src/java/org/apache/cassandra/io/sstable/IndexWriter.java
cassandra/trunk/src/java/org/apache/cassandra/io/sstable/Rebuilder.java
Modified:
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
Modified:
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java?rev=1138282r1=1138281r2=1138282view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
Wed Jun 22 02:32:30 2011
@@ -832,7 +832,7 @@ public class CompactionManager implement
public FutureSSTableReader submitSSTableBuild(final Descriptor desc,
OperationType type)
{
// invalid descriptions due to missing or dropped CFS are handled by
SSTW and StreamInSession.
-final Rebuilder builder = SSTableWriter.createBuilder(desc, type);
+final SSTableWriter.Builder builder =
SSTableWriter.createBuilder(desc, type);
CallableSSTableReader callable = new CallableSSTableReader()
{
public SSTableReader call() throws IOException
Modified:
cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java?rev=1138282r1=1138281r2=1138282view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
Wed Jun 22 02:32:30 2011
@@ -19,9 +19,7 @@
package org.apache.cassandra.io.sstable;
-import java.io.File;
-import java.io.IOError;
-import java.io.IOException;
+import java.io.*;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.Collections;
@@ -29,6 +27,10 @@ import java.util.HashSet;
import java.util.Set;
import com.google.common.collect.Sets;
+
+import org.apache.cassandra.db.commitlog.ReplayPosition;
+import org.apache.cassandra.db.compaction.*;
+import org.apache.cassandra.utils.ByteBufferUtil;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -37,10 +39,7 @@ import org.apache.cassandra.config.Datab
import org.apache.cassandra.db.ColumnFamily;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.db.DecoratedKey;
-import org.apache.cassandra.db.commitlog.ReplayPosition;
-import org.apache.cassandra.db.compaction.AbstractCompactedRow;
-import org.apache.cassandra.db.compaction.CompactionController;
-import org.apache.cassandra.db.compaction.PrecompactedRow;
+import org.apache.cassandra.db.Table;
import org.apache.cassandra.dht.IPartitioner;
import org.apache.cassandra.io.util.BufferedRandomAccessFile;
import org.apache.cassandra.io.util.FileMark;
@@ -48,7 +47,7 @@ import org.apache.cassandra.io.util.File
import org.apache.cassandra.io.util.SegmentedFile;
import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.streaming.OperationType;
-import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.BloomFilter;
import org.apache.cassandra.utils.EstimatedHistogram;
import org.apache.cassandra.utils.FBUtilities;
@@ -233,7 +232,7 @@ public class SSTableWriter extends SSTab
return dataFile.getFilePointer();
}
-public static Rebuilder createBuilder(Descriptor desc, OperationType type)
+public static Builder createBuilder(Descriptor desc, OperationType type)
{
if (!desc.isLatestVersion)
// TODO: streaming between different versions will fail: need
support for
@@ -241,7 +240,83 @@ public class SSTableWriter extends SSTab
throw new RuntimeException(String.format(Cannot recover SSTable
with version %s (current version %s).,
desc.version,
Descriptor.CURRENT_VERSION));
-return new Rebuilder(desc, type);
+return new Builder(desc, type);
+}
+
+/**
+ * Removes the given SSTable from temporary status and opens it,
rebuilding the
+ * bloom filter and row index from the data file.
+ */
+public static class Builder implements CompactionInfo.Holder
+{
+private final Descriptor desc;
+private final OperationType type;
+private final ColumnFamilyStore cfs;
+private RowIndexer indexer;
+
+public Builder(Descriptor desc,
