date:20101215

ReadResponseResolver check digests against each other
-

 Key: CASSANDRA-1865
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1865
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0 rc 2
Reporter: Sylvain Lebresne


When ReadResponseResolver receives a digest query, if it contains multiple 
digests (which will be the case on a initial read), 
it doesn't compare the digests together and only compare all the data receives 
to one of the digest.

Provided an initial read ask data to only 1 node, the rest being digest, there 
seems to be a good change for missing an 
inconsistency.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1865) ReadResponseResolver check digests against each other


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971687#action_12971687
 ] 

Sylvain Lebresne commented on CASSANDRA-1865:
-

This patch is extracted from the patch attached to #1072 (and hence, all credit 
goes to the author). But unless I'm missing something obvious it seems we want 
to fix that asap.

 ReadResponseResolver check digests against each other
 -

 Key: CASSANDRA-1865
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1865
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0 rc 2
Reporter: Sylvain Lebresne
 Attachments: 
 0001-Compare-digests-together-in-read-response-resolver.patch


 When ReadResponseResolver receives a digest query, if it contains multiple 
 digests (which will be the case on a initial read), 
 it doesn't compare the digests together and only compare all the data 
 receives to one of the digest.
 Provided an initial read ask data to only 1 node, the rest being digest, 
 there seems to be a good change for missing an 
 inconsistency.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (CASSANDRA-1865) ReadResponseResolver check digests against each other


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-1865:


Attachment: 0001-Compare-digests-together-in-read-response-resolver.patch

 ReadResponseResolver check digests against each other
 -

 Key: CASSANDRA-1865
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1865
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0 rc 2
Reporter: Sylvain Lebresne
 Attachments: 
 0001-Compare-digests-together-in-read-response-resolver.patch


 When ReadResponseResolver receives a digest query, if it contains multiple 
 digests (which will be the case on a initial read), 
 it doesn't compare the digests together and only compare all the data 
 receives to one of the digest.
 Provided an initial read ask data to only 1 node, the rest being digest, 
 there seems to be a good change for missing an 
 inconsistency.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Resolved: (CASSANDRA-1865) ReadResponseResolver check digests against each other


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-1865.
---

Resolution: Duplicate

CASSANDRA-1830 already has a patch for this, can you review?

 ReadResponseResolver check digests against each other
 -

 Key: CASSANDRA-1865
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1865
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0 rc 2
Reporter: Sylvain Lebresne
 Attachments: 
 0001-Compare-digests-together-in-read-response-resolver.patch


 When ReadResponseResolver receives a digest query, if it contains multiple 
 digests (which will be the case on a initial read), 
 it doesn't compare the digests together and only compare all the data 
 receives to one of the digest.
 Provided an initial read ask data to only 1 node, the rest being digest, 
 there seems to be a good change for missing an 
 inconsistency.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1865) ReadResponseResolver check digests against each other


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971699#action_12971699
 ] 

Sylvain Lebresne commented on CASSANDRA-1865:
-

Oups. Going to review 1830 right away.

 ReadResponseResolver check digests against each other
 -

 Key: CASSANDRA-1865
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1865
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0 rc 2
Reporter: Sylvain Lebresne
 Attachments: 
 0001-Compare-digests-together-in-read-response-resolver.patch


 When ReadResponseResolver receives a digest query, if it contains multiple 
 digests (which will be the case on a initial read), 
 it doesn't compare the digests together and only compare all the data 
 receives to one of the digest.
 Provided an initial read ask data to only 1 node, the rest being digest, 
 there seems to be a good change for missing an 
 inconsistency.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

svn commit: r1049580 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/service/StorageProxy.java

Author: jbellis
Date: Wed Dec 15 15:10:21 2010
New Revision: 1049580

URL: http://svn.apache.org/viewvc?rev=1049580view=rev
Log:
change exceptionfor readrequests duringbootstrap from InvalidRequest to 
Unavailable
patch by Nate McCall; reviewed by jbellis for CASSANDRA-1862

Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java

Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1049580r1=1049579r2=1049580view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Wed Dec 15 15:10:21 2010
@@ -9,6 +9,8 @@ dev
column families (CASSANDRA-1835)
  * unregister index MBeans when index is dropped (CASSANDRA-1843)
  * make ByteBufferUtil.clone thread-safe (CASSANDRA-1847)
+ * change exception for read requests during bootstrap from 
+   InvalidRequest to Unavailable (CASSANDRA-1862)
 
 
 0.7.0-rc2

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java?rev=1049580r1=1049579r2=1049580view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java
 Wed Dec 15 15:10:21 2010
@@ -215,7 +215,7 @@ public class StorageProxy implements Sto
 throws IOException, UnavailableException, TimeoutException, 
InvalidRequestException
 {
 if (StorageService.instance.isBootstrapMode())
-throw new InvalidRequestException(This node cannot accept reads 
until it has bootstrapped);
+throw new UnavailableException();
 long startTime = System.nanoTime();
 
 ListRow rows;

[jira] Resolved: (CASSANDRA-1862) StorageProxy throws an InvalidRequestException in readProtocol during bootstrap


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-1862.
---

Resolution: Fixed
  Reviewer: jbellis
  Assignee: Nate McCall

committed, thanks!

 StorageProxy throws an InvalidRequestException in readProtocol during 
 bootstrap
 ---

 Key: CASSANDRA-1862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1862
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0 rc 2
Reporter: Nate McCall
Assignee: Nate McCall
Priority: Minor
 Fix For: 0.7.0

 Attachments: 1862.txt


 Though the error message provides details, IRE is supposed to signify poorly 
 formed API requests. In the context of a client request, an 
 UnavailableException is more appropriate. This would allow the client to take 
 action like removing the node from its host list. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1837) Deleted columns are resurrected after a flush


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971701#action_12971701
 ] 

Jonathan Ellis commented on CASSANDRA-1837:
---

+1

 Deleted columns are resurrected after a flush
 -

 Key: CASSANDRA-1837
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1837
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0 rc 1
Reporter: Brandon Williams
Assignee: Gary Dusbabek
Priority: Blocker
 Fix For: 0.7.0

 Attachments: 1837-0.6-unit-test.diff, 
 v1-0001-unit-tests-that-show-bug.txt, 
 v1-0002-copy-deletion-time-to-reduced-column-families.txt, v2-0002.txt


 Easily reproduced with the cli:
 {noformat}
 [defa...@unknown] create keyspace testks;
 2785d67c-02df-11e0-ac09-e700f669bcfc
 [defa...@unknown] use testks;
 Authenticated to keyspace: testks
 [defa...@testks] create column family testcf;
 2fbad20d-02df-11e0-ac09-e700f669bcfc
 [defa...@testks] set testcf['test']['foo'] = 'foo';
 Value inserted.
 [defa...@testks] set testcf['test']['bar'] = 'bar';
 Value inserted.
 [defa...@testks] list testcf;
 Using default limit of 100
 ---
 RowKey: test
 = (column=626172, value=626172, timestamp=129182186912)
 = (column=666f6f, value=666f6f, timestamp=129182185732)
 1 Row Returned.
 [defa...@testks] del testcf['test'];
 row removed.
 [defa...@testks] list testcf;
 Using default limit of 100
 ---
 RowKey: test
 1 Row Returned.
 {noformat}
 Now flush testks and look again:
 {noformat}
 [defa...@testks] list testcf;
 Using default limit of 100
 ---
 RowKey: test
 = (column=626172, value=626172, timestamp=129182186912)
 = (column=666f6f, value=666f6f, timestamp=129182185732)
 1 Row Returned.
 {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (CASSANDRA-1830) ReadResponseResolver might miss an inconsistency


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-1830:
--

Affects Version/s: 0.7.0
   0.6.9

 ReadResponseResolver might miss an inconsistency
 

 Key: CASSANDRA-1830
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1830
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.6.9, 0.7.0
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 0.7.0

 Attachments: 1830.txt


 Rather than comparing the digests of all the digest requests to one another, 
 the last one seen wins and is compared to the digest of each version seen 
 from a data request.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

svn commit: r1049583 - in /cassandra/trunk/src/java/org/apache/cassandra/db: RowIteratorFactory.java columniterator/IColumnIterator.java

Author: gdusbabek
Date: Wed Dec 15 15:17:03 2010
New Revision: 1049583

URL: http://svn.apache.org/viewvc?rev=1049583view=rev
Log:
track row deletions when merging cols to form a row. patch by gdusbabek and 
jbellis. CASSANDRA-1837

Modified:
cassandra/trunk/src/java/org/apache/cassandra/db/RowIteratorFactory.java

cassandra/trunk/src/java/org/apache/cassandra/db/columniterator/IColumnIterator.java

Modified: 
cassandra/trunk/src/java/org/apache/cassandra/db/RowIteratorFactory.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/RowIteratorFactory.java?rev=1049583r1=1049582r2=1049583view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/db/RowIteratorFactory.java 
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/db/RowIteratorFactory.java 
Wed Dec 15 15:17:03 2010
@@ -18,6 +18,8 @@
 package org.apache.cassandra.db;
 
 import java.io.Closeable;
+import java.io.IOError;
+import java.io.IOException;
 import java.util.ArrayList;
 import java.util.Collection;
 import java.util.Comparator;
@@ -128,8 +130,7 @@ public class RowIteratorFactory
 ComparatorIColumn colComparator = 
filter.filter.getColumnComparator(comparator);
 IteratorIColumn colCollated = 
IteratorUtils.collatedIterator(colComparator, colIters);
 
-ColumnFamily returnCF = null;
-
+ColumnFamily returnCF;
 // First check if this row is in the rowCache. If it is we can 
skip the rest
 ColumnFamily cached = cfs.getRawCachedRow(key);
 if (cached != null)
@@ -137,23 +138,34 @@ public class RowIteratorFactory
 QueryFilter keyFilter = new QueryFilter(key, filter.path, 
filter.filter);
 returnCF = cfs.filterColumnFamily(cached, keyFilter, 
gcBefore);
 }
-else
+else if (colCollated.hasNext())
 {
-returnCF = firstMemtable.getColumnFamily(key);
+returnCF = firstMemtable.getColumnFamily(key);
 // TODO this is a little subtle: the Memtable 
ColumnIterator has to be a shallow clone of the source CF,
 // with deletion times set correctly, so we can use it as 
the base CF to add query results to.
 // (for sstable ColumnIterators we do not care if it is a 
shallow clone or not.)
 returnCF = returnCF == null ? 
ColumnFamily.create(firstMemtable.getTableName(), filter.getColumnFamilyName())
-: returnCF.cloneMeShallow();
-
-if (colCollated.hasNext())
-{
-filter.collectCollatedColumns(returnCF, colCollated, 
gcBefore);
-}
-else
+: returnCF.cloneMeShallow();
+long lastDeletedAt = Long.MIN_VALUE;
+for (IColumnIterator columns : colIters)
 {
-returnCF = null;
+columns.hasNext(); // force cf initializtion
+try
+{
+if (columns.getColumnFamily().isMarkedForDelete())
+lastDeletedAt = Math.max(lastDeletedAt, 
columns.getColumnFamily().getMarkedForDeleteAt());
+}
+catch (IOException e)
+{
+throw new IOError(e);
+}
 }
+returnCF.markedForDeleteAt.set(lastDeletedAt);
+filter.collectCollatedColumns(returnCF, colCollated, 
gcBefore);
+}
+else
+{
+returnCF = null;
 }
 
 Row rv = new Row(key, returnCF);

Modified: 
cassandra/trunk/src/java/org/apache/cassandra/db/columniterator/IColumnIterator.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/columniterator/IColumnIterator.java?rev=1049583r1=1049582r2=1049583view=diff
==
--- 
cassandra/trunk/src/java/org/apache/cassandra/db/columniterator/IColumnIterator.java
 (original)
+++ 
cassandra/trunk/src/java/org/apache/cassandra/db/columniterator/IColumnIterator.java
 Wed Dec 15 15:17:03 2010
@@ -31,8 +31,9 @@ import org.apache.cassandra.db.ColumnFam
 public interface IColumnIterator extends IteratorIColumn
 {
 /**
- *  returns the CF of the column being iterated.  Do not modify the 
returned CF; clone first.
- *  The CF is only guaranteed to be available after a call to next() or 
hasNext().
+ * returns the CF of the column being iterated.  Do not

svn commit: r1049582 - /cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java

Author: gdusbabek
Date: Wed Dec 15 15:16:46 2010
New Revision: 1049582

URL: http://svn.apache.org/viewvc?rev=1049582view=rev
Log:
unit tests that show bug. patch by gdusbabek, reviewed by jbellis. 
CASSANDRA-1837

Modified:
cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java

Modified: 
cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java?rev=1049582r1=1049581r2=1049582view=diff
==
--- 
cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java 
(original)
+++ 
cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java 
Wed Dec 15 15:16:46 2010
@@ -21,12 +21,17 @@ package org.apache.cassandra.db;
 import java.io.IOException;
 import java.nio.ByteBuffer;
 import java.util.Arrays;
+import java.util.Collection;
 import java.util.LinkedList;
 import java.util.List;
 import java.util.Random;
 import java.util.concurrent.ExecutionException;
 import java.util.concurrent.TimeUnit;
 
+import org.apache.cassandra.db.marshal.LongType;
+import org.apache.cassandra.thrift.SlicePredicate;
+import org.apache.cassandra.thrift.SliceRange;
+import org.apache.commons.lang.ArrayUtils;
 import org.apache.commons.lang.StringUtils;
 import org.junit.Test;
 
@@ -48,6 +53,8 @@ import org.apache.cassandra.utils.FBUtil
 import org.apache.cassandra.utils.WrappedRunnable;
 
 import static junit.framework.Assert.assertEquals;
+import static org.apache.cassandra.Util.column;
+import static org.apache.cassandra.Util.getBytes;
 import static org.junit.Assert.assertNull;
 import org.apache.cassandra.utils.ByteBufferUtil;
 
@@ -328,6 +335,170 @@ public class ColumnFamilyStoreTest exten
 String key = new 
String(rows.get(0).key.key.array(),rows.get(0).key.key.position(),rows.get(0).key.key.remaining());
 
 assert k1.equals( key );
 }
+
+@Test
+public void testDeleteSuperRowSticksAfterFlush() throws Throwable
+{
+String tableName = Keyspace1;
+String cfName= Super1;
+ByteBuffer scfName = ByteBufferUtil.bytes(SuperDuper);
+Table table = Table.open(tableName);
+ColumnFamilyStore cfs = table.getColumnFamilyStore(cfName);
+DecoratedKey key = Util.dk(flush-resurrection);
+
+// create an isolated sstable.
+putColsSuper(cfs, key, scfName, 
+new Column(getBytes(1), ByteBufferUtil.bytes(val1), 1),
+new Column(getBytes(2), ByteBufferUtil.bytes(val2), 1),
+new Column(getBytes(3), ByteBufferUtil.bytes(val3), 1));
+cfs.forceBlockingFlush();
+
+// insert, don't flush.
+putColsSuper(cfs, key, scfName, 
+new Column(getBytes(4), ByteBufferUtil.bytes(val4), 1),
+new Column(getBytes(5), ByteBufferUtil.bytes(val5), 1),
+new Column(getBytes(6), ByteBufferUtil.bytes(val6), 1));
+
+// verify insert.
+final SlicePredicate sp = new SlicePredicate();
+sp.setSlice_range(new SliceRange());
+sp.getSlice_range().setCount(100);
+sp.getSlice_range().setStart(ArrayUtils.EMPTY_BYTE_ARRAY);
+sp.getSlice_range().setFinish(ArrayUtils.EMPTY_BYTE_ARRAY);
+
+assertRowAndColCount(1, 6, scfName, false, cfs.getRangeSlice(scfName, 
Util.range(f, g), 100, QueryFilter.getFilter(sp, cfs.getComparator(;
+
+// deeleet.
+RowMutation rm = new RowMutation(table.name, key.key);
+rm.delete(new QueryPath(cfName, scfName), 2);
+rm.apply();
+
+// verify delete.
+assertRowAndColCount(1, 0, scfName, false, cfs.getRangeSlice(scfName, 
Util.range(f, g), 100, QueryFilter.getFilter(sp, cfs.getComparator(;
+
+// flush
+cfs.forceBlockingFlush();
+
+// re-verify delete.
+assertRowAndColCount(1, 0, scfName, false, cfs.getRangeSlice(scfName, 
Util.range(f, g), 100, QueryFilter.getFilter(sp, cfs.getComparator(;
+
+// late insert.
+putColsSuper(cfs, key, scfName, 
+new Column(getBytes(4), ByteBufferUtil.bytes(val4), 1L),
+new Column(getBytes(7), ByteBufferUtil.bytes(val7), 1L));
+
+// re-verify delete.
+assertRowAndColCount(1, 0, scfName, false, cfs.getRangeSlice(scfName, 
Util.range(f, g), 100, QueryFilter.getFilter(sp, cfs.getComparator(;
+
+// make sure new writes are recognized.
+putColsSuper(cfs, key, scfName, 
+new Column(getBytes(3), ByteBufferUtil.bytes(val3), 3),
+new Column(getBytes(8), ByteBufferUtil.bytes(val8), 3),
+new Column(getBytes(9), ByteBufferUtil.bytes(val9), 3));
+assertRowAndColCount(1, 3, scfName, false,

buildbot failure in ASF Buildbot on cassandra-trunk

2010-12-15 Thread buildbot

The Buildbot has detected a new failure of cassandra-trunk on ASF Buildbot.
Full details are available at:
 http://ci.apache.org/builders/cassandra-trunk/builds/860

Buildbot URL: http://ci.apache.org/

Buildslave for this Build: isis_ubuntu

Build Reason: 
Build Source Stamp: [branch cassandra/trunk] 1049582
Blamelist: gdusbabek

BUILD FAILED: failed compile

sincerely,
 -The Buildbot

svn commit: r1049586 - /cassandra/trunk/CHANGES.txt

Author: gdusbabek
Date: Wed Dec 15 15:19:43 2010
New Revision: 1049586

URL: http://svn.apache.org/viewvc?rev=1049586view=rev
Log:
changes for CASSANDRA-1837

Modified:
cassandra/trunk/CHANGES.txt

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1049586r1=1049585r2=1049586view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Wed Dec 15 15:19:43 2010
@@ -13,6 +13,8 @@
column families (CASSANDRA-1835)
  * unregister index MBeans when index is dropped (CASSANDRA-1843)
  * make ByteBufferUtil.clone thread-safe (CASSANDRA-1847)
+ * deleted columns resurrected after a flush on slice read path 
+   (CASSANDRA-1837)
 
 
 0.7.0-rc2

buildbot success in ASF Buildbot on cassandra-trunk

2010-12-15 Thread buildbot

The Buildbot has detected a restored build of cassandra-trunk on ASF Buildbot.
Full details are available at:
 http://ci.apache.org/builders/cassandra-trunk/builds/861

Buildbot URL: http://ci.apache.org/

Buildslave for this Build: isis_ubuntu

Build Reason: 
Build Source Stamp: [branch cassandra/trunk] 1049583
Blamelist: gdusbabek

Build succeeded!

sincerely,
 -The Buildbot

svn commit: r1049588 - in /cassandra/branches/cassandra-0.7: ./ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/db/columniterator/ test/unit/org/apache/cassandra/db/

Author: gdusbabek
Date: Wed Dec 15 15:22:27 2010
New Revision: 1049588

URL: http://svn.apache.org/viewvc?rev=1049588view=rev
Log:
track row deletions when merging cols to form a row. patch by gdusbabek and 
jbellis. CASSANDRA-1837

Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/RowIteratorFactory.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/IColumnIterator.java

cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java

Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1049588r1=1049587r2=1049588view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Wed Dec 15 15:22:27 2010
@@ -11,6 +11,8 @@ dev
  * make ByteBufferUtil.clone thread-safe (CASSANDRA-1847)
  * change exception for read requests during bootstrap from 
InvalidRequest to Unavailable (CASSANDRA-1862)
+ * deleted columns resurrected after a flush on slice read path 
+   (CASSANDRA-1837)
 
 
 0.7.0-rc2

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/RowIteratorFactory.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/RowIteratorFactory.java?rev=1049588r1=1049587r2=1049588view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/RowIteratorFactory.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/RowIteratorFactory.java
 Wed Dec 15 15:22:27 2010
@@ -18,6 +18,8 @@
 package org.apache.cassandra.db;
 
 import java.io.Closeable;
+import java.io.IOError;
+import java.io.IOException;
 import java.util.ArrayList;
 import java.util.Collection;
 import java.util.Comparator;
@@ -128,8 +130,7 @@ public class RowIteratorFactory
 ComparatorIColumn colComparator = 
filter.filter.getColumnComparator(comparator);
 IteratorIColumn colCollated = 
IteratorUtils.collatedIterator(colComparator, colIters);
 
-ColumnFamily returnCF = null;
-
+ColumnFamily returnCF;
 // First check if this row is in the rowCache. If it is we can 
skip the rest
 ColumnFamily cached = cfs.getRawCachedRow(key);
 if (cached != null)
@@ -137,23 +138,34 @@ public class RowIteratorFactory
 QueryFilter keyFilter = new QueryFilter(key, filter.path, 
filter.filter);
 returnCF = cfs.filterColumnFamily(cached, keyFilter, 
gcBefore);
 }
-else
+else if (colCollated.hasNext())
 {
-returnCF = firstMemtable.getColumnFamily(key);
+returnCF = firstMemtable.getColumnFamily(key);
 // TODO this is a little subtle: the Memtable 
ColumnIterator has to be a shallow clone of the source CF,
 // with deletion times set correctly, so we can use it as 
the base CF to add query results to.
 // (for sstable ColumnIterators we do not care if it is a 
shallow clone or not.)
 returnCF = returnCF == null ? 
ColumnFamily.create(firstMemtable.getTableName(), filter.getColumnFamilyName())
-: returnCF.cloneMeShallow();
-
-if (colCollated.hasNext())
-{
-filter.collectCollatedColumns(returnCF, colCollated, 
gcBefore);
-}
-else
+: returnCF.cloneMeShallow();
+long lastDeletedAt = Long.MIN_VALUE;
+for (IColumnIterator columns : colIters)
 {
-returnCF = null;
+columns.hasNext(); // force cf initializtion
+try
+{
+if (columns.getColumnFamily().isMarkedForDelete())
+lastDeletedAt = Math.max(lastDeletedAt, 
columns.getColumnFamily().getMarkedForDeleteAt());
+}
+catch (IOException e)
+{
+throw new IOError(e);
+}
 }
+returnCF.markedForDeleteAt.set(lastDeletedAt);
+filter.collectCollatedColumns(returnCF, colCollated, 
gcBefore);
+}
+else
+{
+returnCF = null;
 }
 
 Row rv = new Row(key, returnCF);

svn commit: r1049589 - in /cassandra/trunk: ./ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/cli/ src/java/org/apache/cassandra/service/

Author: gdusbabek
Date: Wed Dec 15 15:24:14 2010
New Revision: 1049589

URL: http://svn.apache.org/viewvc?rev=1049589view=rev
Log:
merge from 0.7

Modified:
cassandra/trunk/   (props changed)
cassandra/trunk/CHANGES.txt

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)
cassandra/trunk/src/java/org/apache/cassandra/cli/CliUserHelp.java
cassandra/trunk/src/java/org/apache/cassandra/service/StorageProxy.java

Propchange: cassandra/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 15:24:14 2010
@@ -1,5 +1,5 @@
 /cassandra/branches/cassandra-0.6:922689-1044118
-/cassandra/branches/cassandra-0.7:1026517-1049323
+/cassandra/branches/cassandra-0.7:1026517-1049588
 /incubator/cassandra/branches/cassandra-0.3:774578-796573
 /incubator/cassandra/branches/cassandra-0.4:810145-834239,834349-834350
 /incubator/cassandra/branches/cassandra-0.5:72-915439

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1049589r1=1049588r2=1049589view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Wed Dec 15 15:24:14 2010
@@ -13,6 +13,8 @@
column families (CASSANDRA-1835)
  * unregister index MBeans when index is dropped (CASSANDRA-1843)
  * make ByteBufferUtil.clone thread-safe (CASSANDRA-1847)
+ * change exception for read requests during bootstrap from 
+   InvalidRequest to Unavailable (CASSANDRA-1862)
  * deleted columns resurrected after a flush on slice read path 
(CASSANDRA-1837)
 

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 15:24:14 2010
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1044118
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026517-1049323
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026517-1049588
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573
 
/incubator/cassandra/branches/cassandra-0.4/interface/gen-java/org/apache/cassandra/service/Cassandra.java:810145-834239,834349-834350
 
/incubator/cassandra/branches/cassandra-0.5/interface/gen-java/org/apache/cassandra/service/Cassandra.java:72-903502

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 15:24:14 2010
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1044118
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026517-1049323
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026517-1049588
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/column_t.java:774578-792198
 
/incubator/cassandra/branches/cassandra-0.4/interface/gen-java/org/apache/cassandra/service/Column.java:810145-834239,834349-834350
 
/incubator/cassandra/branches/cassandra-0.5/interface/gen-java/org/apache/cassandra/service/Column.java:72-903502

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 15:24:14 2010
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1044118
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026517-1049323
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026517-1049588

[jira] Commented: (CASSANDRA-1862) StorageProxy throws an InvalidRequestException in readProtocol during bootstrap


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971705#action_12971705
 ] 

Hudson commented on CASSANDRA-1862:
---

Integrated in Cassandra-0.7 #83 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.7/83/])
change exceptionfor readrequests duringbootstrap from InvalidRequest to 
Unavailable
patch by Nate McCall; reviewed by jbellis for CASSANDRA-1862


 StorageProxy throws an InvalidRequestException in readProtocol during 
 bootstrap
 ---

 Key: CASSANDRA-1862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1862
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0 rc 2
Reporter: Nate McCall
Assignee: Nate McCall
Priority: Minor
 Fix For: 0.7.0

 Attachments: 1862.txt


 Though the error message provides details, IRE is supposed to signify poorly 
 formed API requests. In the context of a client request, an 
 UnavailableException is more appropriate. This would allow the client to take 
 action like removing the node from its host list. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1837) Deleted columns are resurrected after a flush


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971707#action_12971707
 ] 

Hudson commented on CASSANDRA-1837:
---

Integrated in Cassandra-0.7 #84 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.7/84/])
track row deletions when merging cols to form a row. patch by gdusbabek and 
jbellis. CASSANDRA-1837


 Deleted columns are resurrected after a flush
 -

 Key: CASSANDRA-1837
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1837
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0 rc 1
Reporter: Brandon Williams
Assignee: Gary Dusbabek
Priority: Blocker
 Fix For: 0.7.0

 Attachments: 1837-0.6-unit-test.diff, 
 v1-0001-unit-tests-that-show-bug.txt, 
 v1-0002-copy-deletion-time-to-reduced-column-families.txt, v2-0002.txt


 Easily reproduced with the cli:
 {noformat}
 [defa...@unknown] create keyspace testks;
 2785d67c-02df-11e0-ac09-e700f669bcfc
 [defa...@unknown] use testks;
 Authenticated to keyspace: testks
 [defa...@testks] create column family testcf;
 2fbad20d-02df-11e0-ac09-e700f669bcfc
 [defa...@testks] set testcf['test']['foo'] = 'foo';
 Value inserted.
 [defa...@testks] set testcf['test']['bar'] = 'bar';
 Value inserted.
 [defa...@testks] list testcf;
 Using default limit of 100
 ---
 RowKey: test
 = (column=626172, value=626172, timestamp=129182186912)
 = (column=666f6f, value=666f6f, timestamp=129182185732)
 1 Row Returned.
 [defa...@testks] del testcf['test'];
 row removed.
 [defa...@testks] list testcf;
 Using default limit of 100
 ---
 RowKey: test
 1 Row Returned.
 {noformat}
 Now flush testks and look again:
 {noformat}
 [defa...@testks] list testcf;
 Using default limit of 100
 ---
 RowKey: test
 = (column=626172, value=626172, timestamp=129182186912)
 = (column=666f6f, value=666f6f, timestamp=129182185732)
 1 Row Returned.
 {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (CASSANDRA-1830) ReadResponseResolver might miss an inconsistency


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-1830:


Attachment: 1830_0.7.txt

+1 on v6 version (very minor comment: the import of ColumnFamily in 
DigestMismatchException.java is unnecessary)

Also attaching a port to 0.7.

 ReadResponseResolver might miss an inconsistency
 

 Key: CASSANDRA-1830
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1830
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.6.9, 0.7.0
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 0.7.0

 Attachments: 1830.txt, 1830_0.7.txt


 Rather than comparing the digests of all the digest requests to one another, 
 the last one seen wins and is compared to the digest of each version seen 
 from a data request.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (CASSANDRA-1830) ReadResponseResolver might miss an inconsistency


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-1830:
--

Affects Version/s: (was: 0.7.0)
   (was: 0.6.9)
   0.6
Fix Version/s: 0.6.9

 ReadResponseResolver might miss an inconsistency
 

 Key: CASSANDRA-1830
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1830
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.6
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 0.6.9, 0.7.0

 Attachments: 1830.txt, 1830_0.7.txt


 Rather than comparing the digests of all the digest requests to one another, 
 the last one seen wins and is compared to the digest of each version seen 
 from a data request.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1837) Deleted columns are resurrected after a flush


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971720#action_12971720
 ] 

Hudson commented on CASSANDRA-1837:
---

Integrated in Cassandra #631 (See 
[https://hudson.apache.org/hudson/job/Cassandra/631/])
changes for CASSANDRA-1837
track row deletions when merging cols to form a row. patch by gdusbabek and 
jbellis. CASSANDRA-1837
unit tests that show bug. patch by gdusbabek, reviewed by jbellis. 
CASSANDRA-1837


 Deleted columns are resurrected after a flush
 -

 Key: CASSANDRA-1837
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1837
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0 rc 1
Reporter: Brandon Williams
Assignee: Gary Dusbabek
Priority: Blocker
 Fix For: 0.7.0

 Attachments: 1837-0.6-unit-test.diff, 
 v1-0001-unit-tests-that-show-bug.txt, 
 v1-0002-copy-deletion-time-to-reduced-column-families.txt, v2-0002.txt


 Easily reproduced with the cli:
 {noformat}
 [defa...@unknown] create keyspace testks;
 2785d67c-02df-11e0-ac09-e700f669bcfc
 [defa...@unknown] use testks;
 Authenticated to keyspace: testks
 [defa...@testks] create column family testcf;
 2fbad20d-02df-11e0-ac09-e700f669bcfc
 [defa...@testks] set testcf['test']['foo'] = 'foo';
 Value inserted.
 [defa...@testks] set testcf['test']['bar'] = 'bar';
 Value inserted.
 [defa...@testks] list testcf;
 Using default limit of 100
 ---
 RowKey: test
 = (column=626172, value=626172, timestamp=129182186912)
 = (column=666f6f, value=666f6f, timestamp=129182185732)
 1 Row Returned.
 [defa...@testks] del testcf['test'];
 row removed.
 [defa...@testks] list testcf;
 Using default limit of 100
 ---
 RowKey: test
 1 Row Returned.
 {noformat}
 Now flush testks and look again:
 {noformat}
 [defa...@testks] list testcf;
 Using default limit of 100
 ---
 RowKey: test
 = (column=626172, value=626172, timestamp=129182186912)
 = (column=666f6f, value=666f6f, timestamp=129182185732)
 1 Row Returned.
 {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

svn commit: r1049628 - in /cassandra/branches/cassandra-0.6: CHANGES.txt src/java/org/apache/cassandra/service/DigestMismatchException.java src/java/org/apache/cassandra/service/ReadResponseResolver.j

Author: jbellis
Date: Wed Dec 15 16:09:31 2010
New Revision: 1049628

URL: http://svn.apache.org/viewvc?rev=1049628view=rev
Log:
ReadResponseResolver check digests against each other
patch by Randall Leeds and jbellis; reviewed by slebrense for CASSANDRA-1830

Modified:
cassandra/branches/cassandra-0.6/CHANGES.txt

cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/DigestMismatchException.java

cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/ReadResponseResolver.java

Modified: cassandra/branches/cassandra-0.6/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.6/CHANGES.txt?rev=1049628r1=1049627r2=1049628view=diff
==
--- cassandra/branches/cassandra-0.6/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.6/CHANGES.txt Wed Dec 15 16:09:31 2010
@@ -18,6 +18,7 @@
(CASSANDRA-1811)
  * correct ordering of drain operations so CL.recover is no longer necessary
(CASSANDRA-1408)
+ * ReadResponseResolver check digests against each other (CASSANDRA-1830)
 
 
 0.6.8

Modified: 
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/DigestMismatchException.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/DigestMismatchException.java?rev=1049628r1=1049627r2=1049628view=diff
==
--- 
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/DigestMismatchException.java
 (original)
+++ 
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/DigestMismatchException.java
 Wed Dec 15 16:09:31 2010
@@ -18,10 +18,15 @@
 
 package org.apache.cassandra.service;
 
+import org.apache.cassandra.utils.FBUtilities;
+
 public class DigestMismatchException extends Exception
 {
-   public DigestMismatchException(String message)
-   {
-   super(message);
-   }
+public DigestMismatchException(String key, byte[] digest1, byte[] digest2)
+{
+super(String.format(Mismatch for key %s (%s vs %s),
+key,
+FBUtilities.bytesToHex(digest1),
+FBUtilities.bytesToHex(digest2)));
+}
 }

Modified: 
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/ReadResponseResolver.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/ReadResponseResolver.java?rev=1049628r1=1049627r2=1049628view=diff
==
--- 
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/ReadResponseResolver.java
 (original)
+++ 
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/ReadResponseResolver.java
 Wed Dec 15 16:09:31 2010
@@ -74,9 +74,8 @@ public class ReadResponseResolver implem
ListColumnFamily versions = new 
ArrayListColumnFamily(responses.size());
ListInetAddress endPoints = new 
ArrayListInetAddress(responses.size());
String key = null;
-   byte[] digest = new byte[0];
-   boolean isDigestQuery = false;
-
+   byte[] digest = null;
+
 /*
 * Populate the list of rows from each of the messages
 * Check to see if there is a digest query. If a digest 
@@ -90,8 +89,16 @@ public class ReadResponseResolver implem
 continue; // arrived after quorum already achieved
 if (result.isDigestQuery())
 {
-digest = result.digest();
-isDigestQuery = true;
+if (digest == null)
+{
+digest = result.digest();
+}
+else
+{
+byte[] digest2 = result.digest();
+if (!Arrays.equals(digest, digest2))
+throw new DigestMismatchException(key, digest, 
digest2);
+}
 }
 else
 {
@@ -103,16 +110,13 @@ public class ReadResponseResolver implem
 
// If there was a digest query compare it with all the data 
digests
// If there is a mismatch then throw an exception so that read 
repair can happen.
-if (isDigestQuery)
+if (digest != null)
 {
 for (ColumnFamily cf : versions)
 {
-if (!Arrays.equals(ColumnFamily.digest(cf), digest))
-{
-/* Wrap the key as the context in this exception */
-String s = String.format(Mismatch for key %s (%s vs %s), 
key, FBUtilities.bytesToHex(ColumnFamily.digest(cf)), 
FBUtilities.bytesToHex(digest));
-throw new DigestMismatchException(s);
-

[jira] Commented: (CASSANDRA-1072) Increment counters

[
https://issues.apache.org/jira/browse/CASSANDRA-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971712#action_12971712
]

Sylvain Lebresne commented on CASSANDRA-1072:
-

Thanks for the rebasing. A few comments/remarks:
# add and batch_add should throw an InvalidRequestException if CL != CL.ONE.
Right now this is checked by an assert in StorageProxy which is far too
late.
# Why doesn't the code really stuff the partitionedCounter inside the column
value ? Keeping the value and the partionedCounter aside seems wasteful. It
wastes space (the information is stored twice) and it wastes cpu (the
total is recomputed each time the context is modified, in particular after
each call to reconcile, while in practice is would be enough to compute it
just before returning the final value to the client). I admit the cpu part
is not so clear as if a counter doesn't change, we avoid recomputing the
value on each read by keeping it aside. I still think this would be a win
to no keep the pre-computed value. And anyway, I find this juggling
between the value and the partionedCounter adds clutter and is error-prone.
# That's not really a remark on the patch itself, but the patch includes the
fix for CASSANDRA-1830, so I think we'd better commit the #1830 first and
rebase this afterwards.

A few other more minor remarks:
# Renaming repair_on_write to replicate_on_write would make me happy, in
that I do hope we can later improve this mechanism so that it helps
establish the usual consistency guarantees. In which case I believe
replicate_on_write will be a more precise and less confusing
name./nitpicking
# About the repair on write, I would actually be in favor of renaming it
further in replicate_counter_on_write (or repair_counter_on_write :))
and/or disallowing it for non counter CFs. Even though there is nothing
intrinsically wrong with doing repair_on_write on non counter CFs, I think
it would be a disservice to users to let them think it could be useful
(unless it does have a use for non counter CFs, but I really don't see any).
# A few lines in ColumnSerializer do not follow the coding style.

Arguably those are either trivially fixable and/or more a matter of opinion
than anything. The exception being maybe the point on getting rid of keeping
both value and partionedCounter. I do believe it would be a nice change but I
admit this is not crucial and I would be OK with committing as is and keep
this as a future refactoring in the interest of having things move forward
(note however that changing this will break the on disk format, but I think we
agreed that we'll probably break it anyway to introduce uuids). Anyway, I
would still love to get your opinions on the issue :)

Increment counters
--

Key: CASSANDRA-1072
URL: https://issues.apache.org/jira/browse/CASSANDRA-1072
Project: Cassandra
Issue Type: Sub-task
Components: Core
Reporter: Johan Oskarsson
Assignee: Kelvin Kakugawa
Attachments: CASSANDRA-1072.121310.patch, CASSANDRA-1072.patch,
increment_test.py, Partitionedcountersdesigndoc.pdf

Break out the increment counters out of CASSANDRA-580. Classes are shared
between the two features but without the plain version vector code the
changeset becomes smaller and more manageable.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (CASSANDRA-1862) StorageProxy throws an InvalidRequestException in readProtocol during bootstrap

2010-12-15 Thread Nate McCall (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nate McCall updated CASSANDRA-1862:
---

Attachment: 1862_0.6.txt

Backport to 0.6

 StorageProxy throws an InvalidRequestException in readProtocol during 
 bootstrap
 ---

 Key: CASSANDRA-1862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1862
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0 rc 2
Reporter: Nate McCall
Assignee: Nate McCall
Priority: Minor
 Fix For: 0.7.0

 Attachments: 1862.txt, 1862_0.6.txt


 Though the error message provides details, IRE is supposed to signify poorly 
 formed API requests. In the context of a client request, an 
 UnavailableException is more appropriate. This would allow the client to take 
 action like removing the node from its host list. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Reopened: (CASSANDRA-1862) StorageProxy throws an InvalidRequestException in readProtocol during bootstrap

2010-12-15 Thread Nate McCall (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nate McCall reopened CASSANDRA-1862:



 StorageProxy throws an InvalidRequestException in readProtocol during 
 bootstrap
 ---

 Key: CASSANDRA-1862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1862
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0 rc 2
Reporter: Nate McCall
Assignee: Nate McCall
Priority: Minor
 Fix For: 0.7.0

 Attachments: 1862.txt, 1862_0.6.txt


 Though the error message provides details, IRE is supposed to signify poorly 
 formed API requests. In the context of a client request, an 
 UnavailableException is more appropriate. This would allow the client to take 
 action like removing the node from its host list. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

svn commit: r1049625 - in /cassandra/branches/cassandra-0.7: ./ interface/thrift/gen-java/org/apache/cassandra/thrift/

Author: jbellis
Date: Wed Dec 15 16:07:29 2010
New Revision: 1049625

URL: http://svn.apache.org/viewvc?rev=1049625view=rev
Log:
merge from 0.6

Modified:
cassandra/branches/cassandra-0.7/   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)

Propchange: cassandra/branches/cassandra-0.7/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 16:07:29 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6:922689-1044118
+/cassandra/branches/cassandra-0.6:922689-1049622
 /cassandra/branches/cassandra-0.7:1035666
 /cassandra/trunk:1026516-1026734,1028929
 /incubator/cassandra/branches/cassandra-0.3:774578-796573

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 16:07:29 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1044118
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1049622
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1035666
 
/cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1026734,1028929
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 16:07:29 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1044118
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1049622
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1035666
 
/cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1026734,1028929
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/column_t.java:774578-792198

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 16:07:29 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1044118
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1049622
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1035666
 
/cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516-1026734,1028929
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/InvalidRequestException.java:774578-796573

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 16:07:29 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:922689-1044118
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:922689-1049622
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1035666
 
/cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1026516-1026734,1028929
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/NotFoundException.java:774578-796573

Propchange:

svn commit: r1049636 - in /cassandra/trunk: ./ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/service/

Author: jbellis
Date: Wed Dec 15 16:29:49 2010
New Revision: 1049636

URL: http://svn.apache.org/viewvc?rev=1049636view=rev
Log:
merge from 0.7

Modified:
cassandra/trunk/   (props changed)
cassandra/trunk/CHANGES.txt

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)

cassandra/trunk/src/java/org/apache/cassandra/service/DigestMismatchException.java

cassandra/trunk/src/java/org/apache/cassandra/service/ReadResponseResolver.java

Propchange: cassandra/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 16:29:49 2010
@@ -1,5 +1,5 @@
-/cassandra/branches/cassandra-0.6:922689-1044118
-/cassandra/branches/cassandra-0.7:1026517-1049588
+/cassandra/branches/cassandra-0.6:922689-1049628
+/cassandra/branches/cassandra-0.7:1026517-1049631
 /incubator/cassandra/branches/cassandra-0.3:774578-796573
 /incubator/cassandra/branches/cassandra-0.4:810145-834239,834349-834350
 /incubator/cassandra/branches/cassandra-0.5:72-915439

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1049636r1=1049635r2=1049636view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Wed Dec 15 16:29:49 2010
@@ -17,6 +17,7 @@
InvalidRequest to Unavailable (CASSANDRA-1862)
  * deleted columns resurrected after a flush on slice read path 
(CASSANDRA-1837)
+ * ReadResponseResolver check digests against each other (CASSANDRA-1830)
 
 
 0.7.0-rc2

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 16:29:49 2010
@@ -1,5 +1,5 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1044118
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026517-1049588
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1049628
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026517-1049631
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573
 
/incubator/cassandra/branches/cassandra-0.4/interface/gen-java/org/apache/cassandra/service/Cassandra.java:810145-834239,834349-834350
 
/incubator/cassandra/branches/cassandra-0.5/interface/gen-java/org/apache/cassandra/service/Cassandra.java:72-903502

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 16:29:49 2010
@@ -1,5 +1,5 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1044118
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026517-1049588
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1049628
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026517-1049631
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/column_t.java:774578-792198
 
/incubator/cassandra/branches/cassandra-0.4/interface/gen-java/org/apache/cassandra/service/Column.java:810145-834239,834349-834350
 
/incubator/cassandra/branches/cassandra-0.5/interface/gen-java/org/apache/cassandra/service/Column.java:72-903502

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 16:29:49 2010
@@ -1,5 +1,5 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1044118
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026517-1049588

[jira] Updated: (CASSANDRA-982) read repair on quorum consistencylevel


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-982:
-

Assignee: (was: Matthew F. Dennis)

 read repair on quorum consistencylevel
 --

 Key: CASSANDRA-982
 URL: https://issues.apache.org/jira/browse/CASSANDRA-982
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Priority: Minor
 Fix For: 0.7.1

 Attachments: 
 0001-better-digest-checking-for-ReadResponseResolver.patch, 
 982-resolve-digests-v2.txt


 CASSANDRA-930 made read repair fuzzy optional, but this only helps with 
 ConsistencyLevel.ONE:
 - Quorum reads always send requests to all nodes
 - only the first Quorum's worth of responses get compared
 So what we'd like to do two changes:
 - only send read requests to the closest R live nodes
 - if read repair is enabled, also compare results from the other nodes in the 
 background

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

svn commit: r1049631 - in /cassandra/branches/cassandra-0.7: ./ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/service/

Author: jbellis
Date: Wed Dec 15 16:13:30 2010
New Revision: 1049631

URL: http://svn.apache.org/viewvc?rev=1049631view=rev
Log:
merge from 0.6

Modified:
cassandra/branches/cassandra-0.7/   (props changed)
cassandra/branches/cassandra-0.7/CHANGES.txt

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/DigestMismatchException.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/ReadResponseResolver.java

Propchange: cassandra/branches/cassandra-0.7/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 16:13:30 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6:922689-1049622
+/cassandra/branches/cassandra-0.6:922689-1049628
 /cassandra/branches/cassandra-0.7:1035666
 /cassandra/trunk:1026516-1026734,1028929
 /incubator/cassandra/branches/cassandra-0.3:774578-796573

Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1049631r1=1049630r2=1049631view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Wed Dec 15 16:13:30 2010
@@ -13,6 +13,7 @@ dev
InvalidRequest to Unavailable (CASSANDRA-1862)
  * deleted columns resurrected after a flush on slice read path 
(CASSANDRA-1837)
+ * ReadResponseResolver check digests against each other (CASSANDRA-1830)
 
 
 0.7.0-rc2

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 16:13:30 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1049622
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1049628
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1035666
 
/cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1026734,1028929
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 16:13:30 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1049622
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1049628
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1035666
 
/cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1026734,1028929
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/column_t.java:774578-792198

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 16:13:30 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1049622
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1049628
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1035666
 
/cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516-1026734,1028929
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/InvalidRequestException.java:774578-796573

Propchange:

[jira] Commented: (CASSANDRA-1824) Schema only fully propagates from seeds


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971721#action_12971721
 ] 

Hudson commented on CASSANDRA-1824:
---

Integrated in Cassandra #631 (See 
[https://hudson.apache.org/hudson/job/Cassandra/631/])


 Schema only fully propagates from seeds
 ---

 Key: CASSANDRA-1824
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1824
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7 beta 1
Reporter: Brandon Williams
Assignee: Gary Dusbabek
 Fix For: 0.7.0

 Attachments: v2-0001-always-put-schema-state-in-local-gossip.txt


 If you have nodes X, Y, and Z, and Y already has some schema, but X and Z do 
 not, and X is the seed node for the cluster, X will pick up the schema from 
 Y, but it will never propagate to Z.  If X has the schema, it will propagate 
 to both Y and Z.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1830) ReadResponseResolver might miss an inconsistency


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971732#action_12971732
 ] 

Hudson commented on CASSANDRA-1830:
---

Integrated in Cassandra-0.6 #23 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.6/23/])
ReadResponseResolver check digests against each other
patch by Randall Leeds and jbellis; reviewed by slebrense for CASSANDRA-1830


 ReadResponseResolver might miss an inconsistency
 

 Key: CASSANDRA-1830
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1830
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.6
Reporter: Jonathan Ellis
Assignee: Randall Leeds
Priority: Minor
 Fix For: 0.6.9, 0.7.0

 Attachments: 1830.txt, 1830_0.7.txt


 Rather than comparing the digests of all the digest requests to one another, 
 the last one seen wins and is compared to the digest of each version seen 
 from a data request.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1862) StorageProxy throws an InvalidRequestException in readProtocol during bootstrap


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971736#action_12971736
 ] 

Jonathan Ellis commented on CASSANDRA-1862:
---

I'm a little leery of breaking 0.6 compatibility for something fairly minor.  
Thoughts?

 StorageProxy throws an InvalidRequestException in readProtocol during 
 bootstrap
 ---

 Key: CASSANDRA-1862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1862
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0 rc 2
Reporter: Nate McCall
Assignee: Nate McCall
Priority: Minor
 Fix For: 0.7.0

 Attachments: 1862.txt, 1862_0.6.txt


 Though the error message provides details, IRE is supposed to signify poorly 
 formed API requests. In the context of a client request, an 
 UnavailableException is more appropriate. This would allow the client to take 
 action like removing the node from its host list. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1470) use direct io for compaction

2010-12-15 Thread Hiram Chirino (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971713#action_12971713
 ] 

Hiram Chirino commented on CASSANDRA-1470:
--

BTW.. if you guys want to avoid the JNA dependency so you can distribute native 
libraries out of the box with Cassandra you might want to consider using 
[HawtJNI|http://hawtjni.fusesource.org/documentation/index.html].

You still use a coding style similar to JNA.  You basically just define the 
Java interface and add annotations.  Instead of using a FFI like JNA, HawtJNI 
code generates an actual JNI library which your native methods bind so you get 
slightly better performance.

Here's an example of a project using it: 
[example|https://github.com/chirino/jansi-native]

If you guys are interested, let me know and I can help port you from JNA to 
HawtJNI




 use direct io for compaction
 

 Key: CASSANDRA-1470
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1470
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 0.7.1

 Attachments: 1470-v2.txt, 1470.txt, CASSANDRA-1470-for-0.6.patch, 
 CASSANDRA-1470-v10-for-0.7.patch, CASSANDRA-1470-v11-for-0.7.patch, 
 CASSANDRA-1470-v12-0.7.patch, CASSANDRA-1470-v13-0.7.patch, 
 CASSANDRA-1470-v14-0.7.patch, CASSANDRA-1470-v2.patch, 
 CASSANDRA-1470-v3-0.7-with-LastErrorException-support.patch, 
 CASSANDRA-1470-v4-for-0.7.patch, CASSANDRA-1470-v5-for-0.7.patch, 
 CASSANDRA-1470-v6-for-0.7.patch, CASSANDRA-1470-v7-for-0.7.patch, 
 CASSANDRA-1470-v8-for-0.7.patch, CASSANDRA-1470-v9-for-0.7.patch, 
 CASSANDRA-1470.patch, 
 use.DirectIORandomAccessFile.for.commitlog.against.1022235.patch


 When compaction scans through a group of sstables, it forces the data in the 
 os buffer cache being used for hot reads, which can have a dramatic negative 
 effect on performance.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1072) Increment counters

[
https://issues.apache.org/jira/browse/CASSANDRA-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971745#action_12971745
]

Kelvin Kakugawa commented on CASSANDRA-1072:

Yes, I agree that modifying the value / partitioned counter implementation is
interesting. I thought about it and my original rationale isn't as
strong--optimizing for reads. We could do a lazy optimization that keeps the
calculated total, in memory, after it's been read the first time (invalidating,
after an update).

I think the rest of the comments are reasonable, as well. Although, I've fixed
1830 in this patch months ago. (I should have broken it out into a separate
patch.) And, I'd prefer replicate_on_write, because it *may* be more general.
Although, like you, I don't see a use for it beyond counters, atm.

I'd be happy to fix the above after commit, though. As long as we modify the
value / partitioned counter implementation before 0.7.0 is final.

Increment counters
--

Break out the increment counters out of CASSANDRA-580. Classes are shared
between the two features but without the plain version vector code the
changeset becomes smaller and more manageable.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1862) StorageProxy throws an InvalidRequestException in readProtocol during bootstrap

2010-12-15 Thread Nate McCall (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971751#action_12971751
 ] 

Nate McCall commented on CASSANDRA-1862:


I think this is extremely minor (the exception declaration of the method did 
not change) and brings us in line with the API as specified in 
cassandra.thrift: 

IRE:
Invalid request could mean keyspace or column family does not exist, required 
parameters are missing, or a parameter is malformed...

vs. UE:
Not all the replicas required could be created and/or read.

 StorageProxy throws an InvalidRequestException in readProtocol during 
 bootstrap
 ---

 Key: CASSANDRA-1862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1862
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0 rc 2
Reporter: Nate McCall
Assignee: Nate McCall
Priority: Minor
 Fix For: 0.7.0

 Attachments: 1862.txt, 1862_0.6.txt


 Though the error message provides details, IRE is supposed to signify poorly 
 formed API requests. In the context of a client request, an 
 UnavailableException is more appropriate. This would allow the client to take 
 action like removing the node from its host list. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1072) Increment counters


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971752#action_12971752
 ] 

Jonathan Ellis commented on CASSANDRA-1072:
---

We can't put this in a stable release series until we can commit to not 
breaking backwards compatibility as we fix things.  We should commit to trunk, 
then once everything is stablilized we can consider backporting to 0.7.x.  (Not 
to mention 0.7.0 is deep into the rc series, it's too late to add major 
features even if it were at no-known-problems.)

 Increment counters
 --

 Key: CASSANDRA-1072
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1072
 Project: Cassandra
  Issue Type: Sub-task
  Components: Core
Reporter: Johan Oskarsson
Assignee: Kelvin Kakugawa
 Attachments: CASSANDRA-1072.121310.patch, CASSANDRA-1072.patch, 
 increment_test.py, Partitionedcountersdesigndoc.pdf


 Break out the increment counters out of CASSANDRA-580. Classes are shared 
 between the two features but without the plain version vector code the 
 changeset becomes smaller and more manageable.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

svn commit: r1049648 - in /cassandra/branches/cassandra-0.6: CHANGES.txt src/java/org/apache/cassandra/service/StorageProxy.java

Author: jbellis
Date: Wed Dec 15 17:42:38 2010
New Revision: 1049648

URL: http://svn.apache.org/viewvc?rev=1049648view=rev
Log:
backport CASSANDRA-1862 from 0.7

Modified:
cassandra/branches/cassandra-0.6/CHANGES.txt

cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/StorageProxy.java

Modified: cassandra/branches/cassandra-0.6/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.6/CHANGES.txt?rev=1049648r1=1049647r2=1049648view=diff
==
--- cassandra/branches/cassandra-0.6/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.6/CHANGES.txt Wed Dec 15 17:42:38 2010
@@ -19,6 +19,8 @@
  * correct ordering of drain operations so CL.recover is no longer necessary
(CASSANDRA-1408)
  * ReadResponseResolver check digests against each other (CASSANDRA-1830)
+ * change exception for read requests during bootstrap from 
+   InvalidRequest to Unavailable (CASSANDRA-1862)
 
 
 0.6.8

Modified: 
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/StorageProxy.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/StorageProxy.java?rev=1049648r1=1049647r2=1049648view=diff
==
--- 
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/StorageProxy.java
 (original)
+++ 
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/service/StorageProxy.java
 Wed Dec 15 17:42:38 2010
@@ -339,7 +339,7 @@ public class StorageProxy implements Sto
 throws IOException, UnavailableException, TimeoutException, 
InvalidRequestException
 {
 if (StorageService.instance.isBootstrapMode())
-throw new InvalidRequestException(This node cannot accept reads 
until it has bootstrapped);
+throw new UnavailableException();
 long startTime = System.nanoTime();
 
 ListRow rows;

svn commit: r1049649 - in /cassandra/branches/cassandra-0.7: ./ interface/thrift/gen-java/org/apache/cassandra/thrift/

Author: jbellis
Date: Wed Dec 15 17:43:00 2010
New Revision: 1049649

URL: http://svn.apache.org/viewvc?rev=1049649view=rev
Log:
merge from 0.6

Modified:
cassandra/branches/cassandra-0.7/   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)

Propchange: cassandra/branches/cassandra-0.7/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 17:43:00 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6:922689-1049628
+/cassandra/branches/cassandra-0.6:922689-1049628,1049648
 /cassandra/branches/cassandra-0.7:1035666
 /cassandra/trunk:1026516-1026734,1028929
 /incubator/cassandra/branches/cassandra-0.3:774578-796573

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 17:43:00 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1049628
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1049628,1049648
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1035666
 
/cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1026734,1028929
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 17:43:00 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1049628
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1049628,1049648
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1035666
 
/cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1026734,1028929
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/column_t.java:774578-792198

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 17:43:00 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1049628
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1049628,1049648
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1035666
 
/cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516-1026734,1028929
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/InvalidRequestException.java:774578-796573

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Dec 15 17:43:00 2010
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:922689-1049628
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:922689-1049628,1049648
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1035666
 
/cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1026516-1026734,1028929
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/NotFoundException.java:774578-796573

Propchange:

[jira] Commented: (CASSANDRA-1862) StorageProxy throws an InvalidRequestException in readProtocol during bootstrap


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971754#action_12971754
 ] 

Jonathan Ellis commented on CASSANDRA-1862:
---

bq. the exception declaration of the method did not change

i'll buy that.  committed

 StorageProxy throws an InvalidRequestException in readProtocol during 
 bootstrap
 ---

 Key: CASSANDRA-1862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1862
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0 rc 2
Reporter: Nate McCall
Assignee: Nate McCall
Priority: Minor
 Fix For: 0.7.0

 Attachments: 1862.txt, 1862_0.6.txt


 Though the error message provides details, IRE is supposed to signify poorly 
 formed API requests. In the context of a client request, an 
 UnavailableException is more appropriate. This would allow the client to take 
 action like removing the node from its host list. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (CASSANDRA-1866) Internal error from malformed remove with supercolumns

Internal error from malformed remove with supercolumns
--

 Key: CASSANDRA-1866
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1866
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Jonathan Ellis
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 0.7.0


From the ML: 

I just call remove method where ColumnPath structure has column_family and 
column members set (super_column not set).
{code}
ERROR 17:57:46,924 Error in ThreadPoolExecutor
java.lang.RuntimeException: java.lang.ClassCastException: 
org.apache.cassandra.db.DeletedColumn cannot be cast to 
org.apache.cassandra.db.SuperColumn
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.ClassCastException: org.apache.cassandra.db.DeletedColumn 
cannot be cast to org.apache.cassandra.db.SuperColumn
at 
org.apache.cassandra.db.SuperColumnSerializer.serialize(SuperColumn.java:318)
at 
org.apache.cassandra.db.SuperColumnSerializer.serialize(SuperColumn.java:298)
at 
org.apache.cassandra.db.ColumnFamilySerializer.serializeForSSTable(ColumnFamilySerializer.java:82)
at 
org.apache.cassandra.db.ColumnFamilySerializer.serialize(ColumnFamilySerializer.java:68)
at 
org.apache.cassandra.db.RowMutationSerializer.freezeTheMaps(RowMutation.java:344)
at 
org.apache.cassandra.db.RowMutationSerializer.serialize(RowMutation.java:355)
at 
org.apache.cassandra.db.RowMutationSerializer.serialize(RowMutation.java:333)
at 
org.apache.cassandra.db.RowMutation.getSerializedBuffer(RowMutation.java:269)
at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:194)
at 
org.apache.cassandra.service.StorageProxy$1.runMayThrow(StorageProxy.java:197)
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
ERROR 17:57:46,928 Fatal exception in thread Thread[MUTATION_STAGE:2,5,main]
java.lang.RuntimeException: java.lang.ClassCastException: 
org.apache.cassandra.db.DeletedColumn cannot be cast to 
org.apache.cassandra.db.SuperColumn
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.ClassCastException: org.apache.cassandra.db.DeletedColumn 
cannot be cast to org.apache.cassandra.db.SuperColumn
at 
org.apache.cassandra.db.SuperColumnSerializer.serialize(SuperColumn.java:318)
at 
org.apache.cassandra.db.SuperColumnSerializer.serialize(SuperColumn.java:298)
at 
org.apache.cassandra.db.ColumnFamilySerializer.serializeForSSTable(ColumnFamilySerializer.java:82)
at 
org.apache.cassandra.db.ColumnFamilySerializer.serialize(ColumnFamilySerializer.java:68)
at 
org.apache.cassandra.db.RowMutationSerializer.freezeTheMaps(RowMutation.java:344)
at 
org.apache.cassandra.db.RowMutationSerializer.serialize(RowMutation.java:355)
at 
org.apache.cassandra.db.RowMutationSerializer.serialize(RowMutation.java:333)
at 
org.apache.cassandra.db.RowMutation.getSerializedBuffer(RowMutation.java:269)
at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:194)
at 
org.apache.cassandra.service.StorageProxy$1.runMayThrow(StorageProxy.java:197)
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
{code}

Fix: add system test to catch this  raise invalidrequestexception from 
ThriftValidation

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1862) StorageProxy throws an InvalidRequestException in readProtocol during bootstrap


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971764#action_12971764
 ] 

Hudson commented on CASSANDRA-1862:
---

Integrated in Cassandra-0.6 #24 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.6/24/])
backport CASSANDRA-1862 from 0.7


 StorageProxy throws an InvalidRequestException in readProtocol during 
 bootstrap
 ---

 Key: CASSANDRA-1862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1862
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0 rc 2
Reporter: Nate McCall
Assignee: Nate McCall
Priority: Minor
 Fix For: 0.7.0

 Attachments: 1862.txt, 1862_0.6.txt


 Though the error message provides details, IRE is supposed to signify poorly 
 formed API requests. In the context of a client request, an 
 UnavailableException is more appropriate. This would allow the client to take 
 action like removing the node from its host list. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1072) Increment counters


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971794#action_12971794
 ] 

Kelvin Kakugawa commented on CASSANDRA-1072:


I was looking through the code and thought about it more.

We can't remove value ser/des, because we need to serialize it for the commit 
log (and hinted hand-off).  So, value needs to be serialized, at some point.  
So, the disk format won't change.

However, we can optimize the code to not recalculate the total on every update 
w/o read.  Looking into this.

 Increment counters
 --

 Key: CASSANDRA-1072
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1072
 Project: Cassandra
  Issue Type: Sub-task
  Components: Core
Reporter: Johan Oskarsson
Assignee: Kelvin Kakugawa
 Attachments: CASSANDRA-1072.121310.patch, CASSANDRA-1072.patch, 
 increment_test.py, Partitionedcountersdesigndoc.pdf


 Break out the increment counters out of CASSANDRA-580. Classes are shared 
 between the two features but without the plain version vector code the 
 changeset becomes smaller and more manageable.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (CASSANDRA-1072) Increment counters


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kelvin Kakugawa updated CASSANDRA-1072:
---

Comment: was deleted

(was: I was looking through the code and thought about it more.

We can't remove value ser/des, because we need to serialize it for the commit 
log (and hinted hand-off).  So, value needs to be serialized, at some point.  
So, the disk format won't change.

However, we can optimize the code to not recalculate the total on every update 
w/o read.  Looking into this.)

 Increment counters
 --

 Key: CASSANDRA-1072
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1072
 Project: Cassandra
  Issue Type: Sub-task
  Components: Core
Reporter: Johan Oskarsson
Assignee: Kelvin Kakugawa
 Attachments: CASSANDRA-1072.121310.patch, CASSANDRA-1072.patch, 
 increment_test.py, Partitionedcountersdesigndoc.pdf


 Break out the increment counters out of CASSANDRA-580. Classes are shared 
 between the two features but without the plain version vector code the 
 changeset becomes smaller and more manageable.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1072) Increment counters


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971801#action_12971801
 ] 

Kelvin Kakugawa commented on CASSANDRA-1072:


Yes, I'm onboard w/ this.  (I'm thinking about what would need to be refactored 
to support the value / partitioned counter refactor.)

 Increment counters
 --

 Key: CASSANDRA-1072
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1072
 Project: Cassandra
  Issue Type: Sub-task
  Components: Core
Reporter: Johan Oskarsson
Assignee: Kelvin Kakugawa
 Attachments: CASSANDRA-1072.121310.patch, CASSANDRA-1072.patch, 
 increment_test.py, Partitionedcountersdesigndoc.pdf


 Break out the increment counters out of CASSANDRA-580. Classes are shared 
 between the two features but without the plain version vector code the 
 changeset becomes smaller and more manageable.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1857) nodetool has invalidaterowcache but no invalidatekeycache

2010-12-15 Thread Jon Hermes (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971847#action_12971847
 ] 

Jon Hermes commented on CASSANDRA-1857:
---

Just to be clear, nodetool doesn't have this, but there is a JMX call on 
ColumnFamilyStores to invalidateRowCache() (due to CASSANDRA-761).

 nodetool has invalidaterowcache but no invalidatekeycache
 -

 Key: CASSANDRA-1857
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1857
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Robert Coli
Assignee: Jon Hermes
Priority: Trivial

 In many cases where you would want to use invalidaterowcache, you would 
 probably also want to invalidatekeycache. Currently, you can 
 invalidaterowcache, but not invalidatekeycache. It seems that users 
 should, generally, be able to do both or neither, but not one or the other. A 
 brief look at the NodeCmd/ColumnFamilyStore code suggests that the 
 stubs/hooks for this feature do not currently exist.
 =Rob

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (CASSANDRA-1072) Increment counters

2010-12-15 Thread Johan Oskarsson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Johan Oskarsson updated CASSANDRA-1072:
---

Attachment: (was: CASSANDRA-1072.patch)

 Increment counters
 --

 Key: CASSANDRA-1072
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1072
 Project: Cassandra
  Issue Type: Sub-task
  Components: Core
Reporter: Johan Oskarsson
Assignee: Kelvin Kakugawa
 Attachments: CASSANDRA-1072.121310.patch, increment_test.py, 
 Partitionedcountersdesigndoc.pdf


 Break out the increment counters out of CASSANDRA-580. Classes are shared 
 between the two features but without the plain version vector code the 
 changeset becomes smaller and more manageable.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (CASSANDRA-1072) Increment counters


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kelvin Kakugawa updated CASSANDRA-1072:
---

Attachment: CASSANDRA-1072.121510.patch

re-cut patch against tr...@e834012cd8f59d522c460e32f139cf785d1f97ee
note: apache's git repo is more up-to-date than github.com/apache

1830 code refactored w/ 1072's version of the fix.  i.e. digest check broken 
out into a separate method.

 Increment counters
 --

 Key: CASSANDRA-1072
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1072
 Project: Cassandra
  Issue Type: Sub-task
  Components: Core
Reporter: Johan Oskarsson
Assignee: Kelvin Kakugawa
 Attachments: CASSANDRA-1072.121510.patch, increment_test.py, 
 Partitionedcountersdesigndoc.pdf


 Break out the increment counters out of CASSANDRA-580. Classes are shared 
 between the two features but without the plain version vector code the 
 changeset becomes smaller and more manageable.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (CASSANDRA-1072) Increment counters


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kelvin Kakugawa updated CASSANDRA-1072:
---

Attachment: (was: CASSANDRA-1072.121310.patch)

 Increment counters
 --

 Key: CASSANDRA-1072
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1072
 Project: Cassandra
  Issue Type: Sub-task
  Components: Core
Reporter: Johan Oskarsson
Assignee: Kelvin Kakugawa
 Attachments: CASSANDRA-1072.121510.patch, increment_test.py, 
 Partitionedcountersdesigndoc.pdf


 Break out the increment counters out of CASSANDRA-580. Classes are shared 
 between the two features but without the plain version vector code the 
 changeset becomes smaller and more manageable.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1625) make the row cache continuously durable

2010-12-15 Thread Peter Schuller (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971877#action_12971877
 ] 

Peter Schuller commented on CASSANDRA-1625:
---

Synergy!

I just realized that if we combine Stu's suggestion with:

(1) CASSANDRA-1657 (in-memory cf)
(2) Just have the CF *be* the cache (I agree with Stu's problem with calling 
it a cache) rather than being used only on start-up to populate the row cache
(3) Ongoing improvements to compaction (whether it's fadvise/mincore or others) 
that eliminates the need to keep both the old and the new sstables in memory

We now have, at least when it comes to eliminating disk I/O, a in-memory cache 
that is:

(1) Memory efficient due to being backed by tightly packed sstables as usual 
(subject to compaction thresholds).
(2) Not made up of objects in the heap, so no GC pressure trade-off (except for 
tracking recenticity information).
(3) Under much more explicit control in terms of size; while it would not come 
for free, being able to say have the cache be N MB large would not be 
unrealistic.
(4) Extremely efficiently instantly available all hot and cozy on start-up 
regardless of total data set size; no seek bound operations in sight. 
Theoretically as efficiently as an mlock() with MAP_POPULATE, depending on 
CASSANDRA-1657

I say at least when it comes to eliminating disk I/O because presumably 
reading from such a CF is more expensive than using the currently existing row 
cache.

The two approaches are not mutually exclusive though and could even be used 
together for the same CF if a situation should so demand.

The lack of (3) means the memory efficiency is lessened, but in other respects 
the positive effects should remain.

But wait, there is more, as they say: If we ignore CASSANDRA-1657 for a moment, 
and instead replace it by some logic to keep a given CF on a different 
path/mountpoint, Cassandra would now have the capability of keeping the hot set 
persistent on faster storage. In other words, this is direct support for 
utilizing a fast SSD in a system which otherwise has lots of bulk storage - 
without relying on something OS specific like ZFS L2ARC, and with, it seems, 
minimal (for the gains) implementation effort.

This has me all excited, so I hope no one points out some obvious flaw in this 
plan :)


 make the row cache continuously durable
 ---

 Key: CASSANDRA-1625
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1625
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Peter Schuller
Priority: Minor

 I was looking into how the row cache worked today and realized only row keys 
 were saved and later pre-populated on start-up.
 On the premise that row caches are typically used for small rows of which 
 there may be many, this is highly likely to be seek bound on large data sets 
 during pre-population.
 The pre-population could be made faster by increasing I/O queue depth (by 
 concurrency or by libaio as in 1576), but especially on large data sets the 
 performance would be nowhere near what could be achieved if a reasonably 
 sized file containing the actual rows were to be read in a sequential fashion 
 on start.
 On the one hand, Cassandra's design means that this should be possible to do 
 efficiently much easier than in some other cases, but on the other hand it is 
 still not entirely trivial.
 The key problem with maintaining a continuously durable cache is that one 
 must never read stale data on start-up. Stale could mean either data that was 
 later deleted, or an old version of data that was updated.
 In the case of Cassandra, this means that any cache restored on start-up must 
 be up-to-date with whatever position in the commit log that commit log 
 recovery will start at. (Because the row cache is for an entire row, we can't 
 couple updating of an on-disk row cache with memtable flushes.)
 I can see two main approaches:
 (a) Periodically dump the entire row cache, deferring commit log eviction in 
 synchronization with said dumping.
 (b) Keep a change log of sorts, similar to the commit log but filtered to 
 only contain data written to the commit log that affects keys that were in 
 the row cache at the time. Eviction of commit logs or updating positional 
 markers that affect the point of commit log recovery start, would imply 
 fsync():ing this change log. An incremental traversal, or alternatively a 
 periodic full dump, would have to be used to ensure that old row change log 
 segments can be evicted without loss of cache warmness.
 I like (b), but it is also the introduction of significant complexity (and 
 potential write path overhead) for the purpose of the row cache. In the worst 
 case where hotly read data is also hotly written, the overhead could be

[jira] Issue Comment Edited: (CASSANDRA-1625) make the row cache continuously durable

2010-12-15 Thread Peter Schuller (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971877#action_12971877
 ] 

Peter Schuller edited comment on CASSANDRA-1625 at 12/15/10 5:55 PM:
-

Synergy!

I just realized that if we combine Stu's suggestion with:

(1) CASSANDRA-1657 (in-memory cf)
(2) Just have the CF *be* the cache (I agree with Stu's problem with calling 
it a cache) rather than being used only on start-up to populate the row cache
(3) Ongoing improvements to compaction (whether it's fadvise/mincore or others) 
that eliminates the need to keep both the old and the new sstables in memory

We now have, at least when it comes to eliminating disk I/O, a in-memory cache 
that is:

(1) Memory efficient due to being backed by tightly packed sstables as usual 
(subject to compaction thresholds).
(2) Not made up of objects in the heap, so no GC pressure trade-off (except for 
tracking recenticity information).
(3) Under much more explicit control in terms of size; while it would not come 
for free, being able to say have the cache be N MB large would not be 
unrealistic.
(4) Extremely efficiently instantly available all hot and cozy on start-up 
regardless of total data set size; no seek bound operations in sight. 
Theoretically as efficiently as an mmap() (edited: accidentally put mlock()) 
with MAP_POPULATE, depending on CASSANDRA-1657

I say at least when it comes to eliminating disk I/O because presumably 
reading from such a CF is more expensive than using the currently existing row 
cache.

The two approaches are not mutually exclusive though and could even be used 
together for the same CF if a situation should so demand.

The lack of (3) means the memory efficiency is lessened, but in other respects 
the positive effects should remain.

But wait, there is more, as they say: If we ignore CASSANDRA-1657 for a moment, 
and instead replace it by some logic to keep a given CF on a different 
path/mountpoint, Cassandra would now have the capability of keeping the hot set 
persistent on faster storage. In other words, this is direct support for 
utilizing a fast SSD in a system which otherwise has lots of bulk storage - 
without relying on something OS specific like ZFS L2ARC, and with, it seems, 
minimal (for the gains) implementation effort.

This has me all excited, so I hope no one points out some obvious flaw in this 
plan :)


  was (Author: scode):
Synergy!

I just realized that if we combine Stu's suggestion with:

(1) CASSANDRA-1657 (in-memory cf)
(2) Just have the CF *be* the cache (I agree with Stu's problem with calling 
it a cache) rather than being used only on start-up to populate the row cache
(3) Ongoing improvements to compaction (whether it's fadvise/mincore or others) 
that eliminates the need to keep both the old and the new sstables in memory

We now have, at least when it comes to eliminating disk I/O, a in-memory cache 
that is:

(1) Memory efficient due to being backed by tightly packed sstables as usual 
(subject to compaction thresholds).
(2) Not made up of objects in the heap, so no GC pressure trade-off (except for 
tracking recenticity information).
(3) Under much more explicit control in terms of size; while it would not come 
for free, being able to say have the cache be N MB large would not be 
unrealistic.
(4) Extremely efficiently instantly available all hot and cozy on start-up 
regardless of total data set size; no seek bound operations in sight. 
Theoretically as efficiently as an mlock() with MAP_POPULATE, depending on 
CASSANDRA-1657

I say at least when it comes to eliminating disk I/O because presumably 
reading from such a CF is more expensive than using the currently existing row 
cache.

The two approaches are not mutually exclusive though and could even be used 
together for the same CF if a situation should so demand.

The lack of (3) means the memory efficiency is lessened, but in other respects 
the positive effects should remain.

But wait, there is more, as they say: If we ignore CASSANDRA-1657 for a moment, 
and instead replace it by some logic to keep a given CF on a different 
path/mountpoint, Cassandra would now have the capability of keeping the hot set 
persistent on faster storage. In other words, this is direct support for 
utilizing a fast SSD in a system which otherwise has lots of bulk storage - 
without relying on something OS specific like ZFS L2ARC, and with, it seems, 
minimal (for the gains) implementation effort.

This has me all excited, so I hope no one points out some obvious flaw in this 
plan :)

  
 make the row cache continuously durable
 ---

 Key: CASSANDRA-1625
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1625
 Project: Cassandra
  Issue Type: Improvement
  Components: Core

[jira] Created: (CASSANDRA-1867) sstable2json runs out of memory when trying to export huge rows

sstable2json runs out of memory when trying to export huge rows
---

 Key: CASSANDRA-1867
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1867
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Affects Versions: 0.6.8
Reporter: Ilya Maykov
Priority: Minor


Currently, sstable2json can run out of memory if it encounters a huge row. The 
problem is that it creates an in-memory String for each row. Proposed solution 
is to pass the output PrintStream to the serializeRow() and serializeColumns() 
methods and write to the stream incrementally.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (CASSANDRA-1867) sstable2json runs out of memory when trying to export huge rows


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Maykov updated CASSANDRA-1867:
---

Attachment: cassandra-1867.txt

Patch against 0.6 branch attached.

 sstable2json runs out of memory when trying to export huge rows
 ---

 Key: CASSANDRA-1867
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1867
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Affects Versions: 0.6.8
Reporter: Ilya Maykov
Priority: Minor
 Attachments: cassandra-1867.txt

   Original Estimate: 1h
  Remaining Estimate: 1h

 Currently, sstable2json can run out of memory if it encounters a huge row. 
 The problem is that it creates an in-memory String for each row. Proposed 
 solution is to pass the output PrintStream to the serializeRow() and 
 serializeColumns() methods and write to the stream incrementally.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (CASSANDRA-1867) sstable2json runs out of memory when trying to export huge rows


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971948#action_12971948
 ] 

Ilya Maykov commented on CASSANDRA-1867:


Oops, the first patch is inserting an extra newline. Will fix momentarily.

 sstable2json runs out of memory when trying to export huge rows
 ---

 Key: CASSANDRA-1867
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1867
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Affects Versions: 0.6.8
Reporter: Ilya Maykov
Priority: Minor
 Attachments: cassandra-1867.txt

   Original Estimate: 1h
  Remaining Estimate: 1h

 Currently, sstable2json can run out of memory if it encounters a huge row. 
 The problem is that it creates an in-memory String for each row. Proposed 
 solution is to pass the output PrintStream to the serializeRow() and 
 serializeColumns() methods and write to the stream incrementally.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (CASSANDRA-1867) sstable2json runs out of memory when trying to export huge rows


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Maykov updated CASSANDRA-1867:
---

Attachment: (was: cassandra-1867.txt)

 sstable2json runs out of memory when trying to export huge rows
 ---

 Key: CASSANDRA-1867
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1867
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Affects Versions: 0.6.8
Reporter: Ilya Maykov
Priority: Minor
 Attachments: cassandra-1867-2.txt

   Original Estimate: 1h
  Remaining Estimate: 1h

 Currently, sstable2json can run out of memory if it encounters a huge row. 
 The problem is that it creates an in-memory String for each row. Proposed 
 solution is to pass the output PrintStream to the serializeRow() and 
 serializeColumns() methods and write to the stream incrementally.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (CASSANDRA-1867) sstable2json runs out of memory when trying to export huge rows


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Maykov updated CASSANDRA-1867:
---

Comment: was deleted

(was: Fixed patch without the extra newline.)

 sstable2json runs out of memory when trying to export huge rows
 ---

 Key: CASSANDRA-1867
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1867
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Affects Versions: 0.6.8
Reporter: Ilya Maykov
Priority: Minor
 Attachments: cassandra-1867-2.txt

   Original Estimate: 1h
  Remaining Estimate: 1h

 Currently, sstable2json can run out of memory if it encounters a huge row. 
 The problem is that it creates an in-memory String for each row. Proposed 
 solution is to pass the output PrintStream to the serializeRow() and 
 serializeColumns() methods and write to the stream incrementally.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (CASSANDRA-1867) sstable2json runs out of memory when trying to export huge rows