[jira] [Updated] (CASSANDRA-6957) testNewRepairedSSTable fails intermittently

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-6957:
---

Attachment: 0001-doh-clear-out-L0-as-well.patch

Actual bug, I didn't clear out L0 when rebuilding manifest after getting first 
repaired data.

Failed intermittently because we are not supposed to have any data in L0, but 
luckily, on cassci, we did sometimes (we wait for leveling, but an sstable is 
flushed after we disable compactions..)

 testNewRepairedSSTable fails intermittently
 ---

 Key: CASSANDRA-6957
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6957
 Project: Cassandra
  Issue Type: Bug
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
 Fix For: 2.1 beta2

 Attachments: 0001-doh-clear-out-L0-as-well.patch, system.log.txt






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-6973) timestamp data type does ISO 8601 formats with 'Z' as time zone.

[JIRA]

Juho Mäkinen created CASSANDRA-6973:
---

 Summary: timestamp data type does ISO 8601 formats with 'Z' as 
time zone.
 Key: CASSANDRA-6973
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6973
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Juho Mäkinen
Priority: Trivial
 Fix For: 2.0.6


The timestamp data type does not support format where time zone is specified 
with 'Z' (as in zulu aka. UTC+0 aka + time zone). Example:

create table foo(ts timestamp primary key);
insert into foo(ts) values('2014-04-01T20:17:35+'); -- this works
cqlsh:test insert into foo(ts) values('2014-04-01T20:17:35Z');
Bad Request: unable to coerce '2014-04-01T20:17:35Z' to a  formatted date (long)

The example date was copied directly from ISO 8601 Wikipedia page. The standard 
says that If the time is in UTC, add a Z directly after the time without a 
space. Z is the zone designator for the zero UTC offset.

Tested with cqlsh with 2.0.6 version.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Fix prepared statement on thrift post-6659

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 80b3e6cc3 - 79da4a6e0


Fix prepared statement on thrift post-6659


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/79da4a6e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/79da4a6e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/79da4a6e

Branch: refs/heads/cassandra-2.0
Commit: 79da4a6e0e1b33f880f6228e93905d85122238a2
Parents: 80b3e6c
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 11:32:48 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 11:33:05 2014 +0200

--
 src/java/org/apache/cassandra/cql3/QueryProcessor.java | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/79da4a6e/src/java/org/apache/cassandra/cql3/QueryProcessor.java
--
diff --git a/src/java/org/apache/cassandra/cql3/QueryProcessor.java 
b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
index fe818fd..64ea5e5 100644
--- a/src/java/org/apache/cassandra/cql3/QueryProcessor.java
+++ b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
@@ -35,6 +35,7 @@ import org.apache.cassandra.db.*;
 import org.apache.cassandra.exceptions.*;
 import org.apache.cassandra.service.ClientState;
 import org.apache.cassandra.service.QueryState;
+import org.apache.cassandra.thrift.ThriftClientState;
 import org.apache.cassandra.tracing.Tracing;
 import org.apache.cassandra.utils.FBUtilities;
 import org.apache.cassandra.utils.MD5Digest;
@@ -237,7 +238,8 @@ public class QueryProcessor implements QueryHandler
 public ResultMessage.Prepared prepare(String queryString, QueryState 
queryState)
 throws RequestValidationException
 {
-return prepare(queryString, queryState.getClientState(), false);
+ClientState cState = queryState.getClientState();
+return prepare(queryString, cState, cState instanceof 
ThriftClientState);
 }
 
 public static ResultMessage.Prepared prepare(String queryString, 
ClientState clientState, boolean forThrift)

[jira] [Commented] (CASSANDRA-6893) Unintended update with conditional statement

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957499#comment-13957499
 ] 

Sylvain Lebresne commented on CASSANDRA-6893:
-

Fair enough, the test should be handling timeouts. But outside of that, I was 
wrong when I said that this was a test to reproduce that issue since that issue 
only happen with prepared statements and the dtest wasn't using them. So 
anyway, I pushed a fix of the paxos dtest so that it does uses a prepared 
statement and properly handles timeouts. The test doesn't pass on cassandra-2.0 
(there is no {{errors}] but the value is incorrect), but does pass with the 
patch.

 Unintended update with conditional statement
 

 Key: CASSANDRA-6893
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6893
 Project: Cassandra
  Issue Type: Bug
 Environment: Ubuntu Precise 64bit / Cassandra 2.0.6
Reporter: Suguru Namura
Assignee: Sylvain Lebresne
 Fix For: 2.0.7

 Attachments: 6893.txt, ConcurrentCASUpdate.java


 After updated to 2.0.6, I have encountered the strange behavior of 
 conditional updates.
 When I executed CQL like UPDATE test SET value = ? WHERE id = ? IF value = ? 
 in concurrent, sometimes cassandra returns true even if value is not 
 satisfied the condition.
 I have attached the program which reproduce this issue. The program works 
 fine in cassandra 2.0.5. But it seems that resets values while execution in 
 2.0.6.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Fix Cassandra 2.0.x validates Thrift columns incorrectly

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 79da4a6e0 - b42feea64


Fix Cassandra 2.0.x validates Thrift columns incorrectly

patch by thobbs and slebresne; reviewed by thobbs and slebresne for 
CASSANDRA-6892


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b42feea6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b42feea6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b42feea6

Branch: refs/heads/cassandra-2.0
Commit: b42feea6432905946083e0e287b3545a5799e4a7
Parents: 79da4a6
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 11:58:00 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 11:58:00 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java | 32 ++
 .../cassandra/config/ColumnDefinition.java  |  9 +++
 .../org/apache/cassandra/config/Schema.java | 14 
 .../apache/cassandra/cql/QueryProcessor.java|  4 +-
 .../apache/cassandra/cql/SelectStatement.java   |  5 --
 .../apache/cassandra/cql/UpdateStatement.java   |  7 +-
 src/java/org/apache/cassandra/db/Column.java| 10 +--
 .../cassandra/thrift/ThriftValidation.java  |  7 +-
 .../apache/cassandra/tools/SSTableExport.java   |  2 +-
 .../apache/cassandra/tools/SSTableImport.java   |  2 +-
 .../unit/org/apache/cassandra/SchemaLoader.java | 18 +-
 .../unit/org/apache/cassandra/db/ScrubTest.java | 67 +++-
 .../cassandra/thrift/ThriftValidationTest.java  | 53 ++--
 .../cassandra/tools/SSTableExportTest.java  | 42 ++--
 15 files changed, 204 insertions(+), 69 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b42feea6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8bfc8b9..9d27ebf 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -38,6 +38,7 @@
  * Track liveRatio per-memtable, not per-CF (CASSANDRA-6945)
  * Make sure upgradesstables keeps sstable level (CASSANDRA-6958)
  * Fix LIMT with static columns (CASSANDRA-6956)
+ * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b42feea6/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 1f25cea..be18d1b 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -853,16 +853,13 @@ public final class CFMetaData
 .toHashCode();
 }
 
-public AbstractType? getValueValidator(ByteBuffer column)
-{
-return getValueValidator(getColumnDefinition(column));
-}
-
-public AbstractType? getValueValidator(ColumnDefinition columnDefinition)
+/**
+ * Like getColumnDefinitionFromColumnName, the argument must be an 
internal column/cell name.
+ */
+public AbstractType? getValueValidatorFromColumnName(ByteBuffer 
columnName)
 {
-return columnDefinition == null
-   ? defaultValidator
-   : columnDefinition.getValidator();
+ColumnDefinition def = getColumnDefinitionFromColumnName(columnName);
+return def == null ? defaultValidator : def.getValidator();
 }
 
 /** applies implicit defaults to cf definition. useful in updates */
@@ -1212,7 +1209,7 @@ public final class CFMetaData
 {
 CompositeType composite = (CompositeType)comparator;
 ByteBuffer[] components = composite.split(columnName);
-for (ColumnDefinition def : column_metadata.values())
+for (ColumnDefinition def : regularAndStaticColumns())
 {
 ByteBuffer toCompare;
 if (def.componentIndex == null)
@@ -1233,12 +1230,19 @@ public final class CFMetaData
 }
 else
 {
-return column_metadata.get(columnName);
+ColumnDefinition def = column_metadata.get(columnName);
+// It's possible that the def is a PRIMARY KEY or COMPACT_VALUE 
one in case a concrete cell
+// name conflicts with a CQL column name, which can happen in 2 
cases:
+// 1) because the user inserted a cell through Thrift that 
conflicts with a default alias used
+//by CQL for thrift tables (see #6892).
+// 2) for COMPACT STORAGE tables with a single utf8 clustering 
column, the cell name can 

[jira] [Commented] (CASSANDRA-6912) SSTableReader.isReplaced does not allow for safe resource cleanup

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957521#comment-13957521
 ] 

Benedict commented on CASSANDRA-6912:
-

I simply modified the deleting task back to schedule and dropped the deleteFile 
test, as this is really not necessary with this patch - I'll reintroduce it in 
CASSANDRA-6916, and possibly make CASSANDRA-6916 depend on CASSANDRA-6919 
(which is the reason for moving all of these tasks into a scheduled operation). 
Possibly I'll then flatten the SSTableDeletingTask and this scheduled tidying 
up into one concept.

The repository has been updated

 SSTableReader.isReplaced does not allow for safe resource cleanup
 -

 Key: CASSANDRA-6912
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6912
 Project: Cassandra
  Issue Type: Bug
Reporter: Benedict
Assignee: Benedict
 Fix For: 2.1 beta2


 There are a number of possible race conditions on resource cleanup from the 
 use of cloneWithNewSummarySamplingLevel, because the replacement sstable can 
 be itself replaced/obsoleted while the prior sstable is still referenced 
 (this is actually quite easy with compaction, but can happen in other 
 circumstances less commonly).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6957) testNewRepairedSSTable fails intermittently

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-6957:
---

Since Version: 2.1 beta1

 testNewRepairedSSTable fails intermittently
 ---

 Key: CASSANDRA-6957
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6957
 Project: Cassandra
  Issue Type: Bug
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
 Fix For: 2.1 beta2

 Attachments: 0001-doh-clear-out-L0-as-well.patch, system.log.txt






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6818) SSTable references not released if stream session fails before it starts

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957537#comment-13957537
 ] 

Marcus Eriksson commented on CASSANDRA-6818:


[~yukim] could this be the cause of CASSANDRA-6568 as well?

 SSTable references not released if stream session fails before it starts
 

 Key: CASSANDRA-6818
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6818
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Richard Low
Assignee: Yuki Morishita
 Fix For: 1.2.16, 2.0.7, 2.1 beta2

 Attachments: 6818-1.2.txt, 6818-2.0-v2.txt, 6818-2.0-v3.txt, 
 6818-2.0.txt


 I observed a large number of 'orphan' SSTables - SSTables that are in the 
 data directory but not loaded by Cassandra - on a 1.1.12 node that had a 
 large stream fail before it started. These orphan files are particularly 
 dangerous because if the node is restarted and picks up these SSTables it 
 could bring data back to life if tombstones have been GCed. To confirm the 
 SSTables are orphan, I created a snapshot and it didn't contain these files. 
 I can see in the logs that they have been compacted so should have been 
 deleted.
 The log entries for the stream are:
 {{INFO [StreamStage:1] 2014-02-21 19:41:48,742 StreamOut.java (line 115) 
 Beginning transfer to /10.0.0.1}}
 {{INFO [StreamStage:1] 2014-02-21 19:41:48,743 StreamOut.java (line 96) 
 Flushing memtables for [CFS(Keyspace='ks', ColumnFamily='cf1'), 
 CFS(Keyspace='ks', ColumnFamily='cf2')]...}}
 {{ERROR [GossipTasks:1] 2014-02-21 19:41:49,239 AbstractStreamSession.java 
 (line 113) Stream failed because /10.0.0.1 died or was restarted/removed 
 (streams may still be active in background, but further streams won't be 
 started)}}
 {{INFO [StreamStage:1] 2014-02-21 19:41:51,783 StreamOut.java (line 161) 
 Stream context metadata [...] 2267 sstables.}}
 {{INFO [StreamStage:1] 2014-02-21 19:41:51,789 StreamOutSession.java (line 
 182) Streaming to /10.0.0.1}}
 {{INFO [Streaming to /10.0.0.1:1] 2014-02-21 19:42:02,218 FileStreamTask.java 
 (line 99) Found no stream out session at end of file stream task - this is 
 expected if the receiver went down}}
 After digging in the code, here's what I think the issue is:
 1. StreamOutSession.transferRanges() creates a streaming session, which is 
 registered with the failure detector in AbstractStreamSession's constructor.
 2. Memtables are flushed, potentially taking a long time.
 3. The remote node fails, convict() is called and the StreamOutSession is 
 closed. However, at this time StreamOutSession.files is empty because it's 
 still waiting for the memtables to flush.
 4. Memtables finish flusing, references are obtained to SSTables to be 
 streamed and the PendingFiles are added to StreamOutSession.files.
 5. The first stream fails but the StreamOutSession isn't found so is never 
 closed and the references are never released.
 This code is more or less the same on 1.2 so I would expect it to reproduce 
 there. I looked at 2.0 and can't even see where SSTable references are 
 released when the stream fails.
 Some possible fixes for 1.1/1.2:
 1. Don't register with the failure detector until after the PendingFiles are 
 set up. I think this is the behaviour in 2.0 but I don't know if it was done 
 like this to avoid this issue.
 2. Detect the above case in (e.g.) StreamOutSession.begin() by noticing the 
 session has been closed with care to avoid double frees.
 3. Add some synchronization so closeInternal() doesn't race with setting up 
 the session.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[1/3] git commit: Fix prepared statement on thrift post-6659

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 92b40fd90 - 8afe1090b


Fix prepared statement on thrift post-6659


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/79da4a6e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/79da4a6e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/79da4a6e

Branch: refs/heads/cassandra-2.1
Commit: 79da4a6e0e1b33f880f6228e93905d85122238a2
Parents: 80b3e6c
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 11:32:48 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 11:33:05 2014 +0200

--
 src/java/org/apache/cassandra/cql3/QueryProcessor.java | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/79da4a6e/src/java/org/apache/cassandra/cql3/QueryProcessor.java
--
diff --git a/src/java/org/apache/cassandra/cql3/QueryProcessor.java 
b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
index fe818fd..64ea5e5 100644
--- a/src/java/org/apache/cassandra/cql3/QueryProcessor.java
+++ b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
@@ -35,6 +35,7 @@ import org.apache.cassandra.db.*;
 import org.apache.cassandra.exceptions.*;
 import org.apache.cassandra.service.ClientState;
 import org.apache.cassandra.service.QueryState;
+import org.apache.cassandra.thrift.ThriftClientState;
 import org.apache.cassandra.tracing.Tracing;
 import org.apache.cassandra.utils.FBUtilities;
 import org.apache.cassandra.utils.MD5Digest;
@@ -237,7 +238,8 @@ public class QueryProcessor implements QueryHandler
 public ResultMessage.Prepared prepare(String queryString, QueryState 
queryState)
 throws RequestValidationException
 {
-return prepare(queryString, queryState.getClientState(), false);
+ClientState cState = queryState.getClientState();
+return prepare(queryString, cState, cState instanceof 
ThriftClientState);
 }
 
 public static ResultMessage.Prepared prepare(String queryString, 
ClientState clientState, boolean forThrift)

[2/3] git commit: Fix Cassandra 2.0.x validates Thrift columns incorrectly

[slebresne]

Fix Cassandra 2.0.x validates Thrift columns incorrectly

patch by thobbs and slebresne; reviewed by thobbs and slebresne for 
CASSANDRA-6892


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b42feea6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b42feea6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b42feea6

Branch: refs/heads/cassandra-2.1
Commit: b42feea6432905946083e0e287b3545a5799e4a7
Parents: 79da4a6
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 11:58:00 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 11:58:00 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java | 32 ++
 .../cassandra/config/ColumnDefinition.java  |  9 +++
 .../org/apache/cassandra/config/Schema.java | 14 
 .../apache/cassandra/cql/QueryProcessor.java|  4 +-
 .../apache/cassandra/cql/SelectStatement.java   |  5 --
 .../apache/cassandra/cql/UpdateStatement.java   |  7 +-
 src/java/org/apache/cassandra/db/Column.java| 10 +--
 .../cassandra/thrift/ThriftValidation.java  |  7 +-
 .../apache/cassandra/tools/SSTableExport.java   |  2 +-
 .../apache/cassandra/tools/SSTableImport.java   |  2 +-
 .../unit/org/apache/cassandra/SchemaLoader.java | 18 +-
 .../unit/org/apache/cassandra/db/ScrubTest.java | 67 +++-
 .../cassandra/thrift/ThriftValidationTest.java  | 53 ++--
 .../cassandra/tools/SSTableExportTest.java  | 42 ++--
 15 files changed, 204 insertions(+), 69 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b42feea6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8bfc8b9..9d27ebf 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -38,6 +38,7 @@
  * Track liveRatio per-memtable, not per-CF (CASSANDRA-6945)
  * Make sure upgradesstables keeps sstable level (CASSANDRA-6958)
  * Fix LIMT with static columns (CASSANDRA-6956)
+ * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b42feea6/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 1f25cea..be18d1b 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -853,16 +853,13 @@ public final class CFMetaData
 .toHashCode();
 }
 
-public AbstractType? getValueValidator(ByteBuffer column)
-{
-return getValueValidator(getColumnDefinition(column));
-}
-
-public AbstractType? getValueValidator(ColumnDefinition columnDefinition)
+/**
+ * Like getColumnDefinitionFromColumnName, the argument must be an 
internal column/cell name.
+ */
+public AbstractType? getValueValidatorFromColumnName(ByteBuffer 
columnName)
 {
-return columnDefinition == null
-   ? defaultValidator
-   : columnDefinition.getValidator();
+ColumnDefinition def = getColumnDefinitionFromColumnName(columnName);
+return def == null ? defaultValidator : def.getValidator();
 }
 
 /** applies implicit defaults to cf definition. useful in updates */
@@ -1212,7 +1209,7 @@ public final class CFMetaData
 {
 CompositeType composite = (CompositeType)comparator;
 ByteBuffer[] components = composite.split(columnName);
-for (ColumnDefinition def : column_metadata.values())
+for (ColumnDefinition def : regularAndStaticColumns())
 {
 ByteBuffer toCompare;
 if (def.componentIndex == null)
@@ -1233,12 +1230,19 @@ public final class CFMetaData
 }
 else
 {
-return column_metadata.get(columnName);
+ColumnDefinition def = column_metadata.get(columnName);
+// It's possible that the def is a PRIMARY KEY or COMPACT_VALUE 
one in case a concrete cell
+// name conflicts with a CQL column name, which can happen in 2 
cases:
+// 1) because the user inserted a cell through Thrift that 
conflicts with a default alias used
+//by CQL for thrift tables (see #6892).
+// 2) for COMPACT STORAGE tables with a single utf8 clustering 
column, the cell name can be anything,
+//including a CQL column name (without this being a 

[3/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[slebresne]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/config/CFMetaData.java
src/java/org/apache/cassandra/config/Schema.java
src/java/org/apache/cassandra/cql/QueryProcessor.java
src/java/org/apache/cassandra/cql/SelectStatement.java
src/java/org/apache/cassandra/cql/UpdateStatement.java
src/java/org/apache/cassandra/db/Column.java
src/java/org/apache/cassandra/thrift/ThriftValidation.java
src/java/org/apache/cassandra/tools/SSTableExport.java
src/java/org/apache/cassandra/tools/SSTableImport.java
test/unit/org/apache/cassandra/SchemaLoader.java
test/unit/org/apache/cassandra/db/ScrubTest.java
test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java
test/unit/org/apache/cassandra/tools/SSTableExportTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8afe1090
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8afe1090
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8afe1090

Branch: refs/heads/cassandra-2.1
Commit: 8afe1090bc4d04346197be6aa696b3af174f3674
Parents: 92b40fd b42feea
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 14:01:43 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 14:01:43 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java | 39 +---
 .../cassandra/config/ColumnDefinition.java  |  9 +++
 .../apache/cassandra/cql/QueryProcessor.java|  2 +-
 .../apache/cassandra/cql3/QueryProcessor.java   |  4 +-
 .../cassandra/thrift/ThriftValidation.java  |  5 +-
 .../apache/cassandra/tools/SSTableImport.java   |  2 +-
 .../unit/org/apache/cassandra/SchemaLoader.java |  7 ++-
 .../unit/org/apache/cassandra/db/ScrubTest.java | 65 
 .../cassandra/thrift/ThriftValidationTest.java  | 53 ++--
 .../cassandra/tools/SSTableExportTest.java  | 35 +++
 11 files changed, 188 insertions(+), 34 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8afe1090/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8afe1090/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --cc src/java/org/apache/cassandra/config/CFMetaData.java
index b46fafd,be18d1b..e930de4
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@@ -919,25 -853,13 +919,10 @@@ public final class CFMetaDat
  .toHashCode();
  }
  
- public AbstractType? getValueValidator(CellName name)
 -/**
 - * Like getColumnDefinitionFromColumnName, the argument must be an 
internal column/cell name.
 - */
 -public AbstractType? getValueValidatorFromColumnName(ByteBuffer 
columnName)
++public AbstractType? getValueValidator(CellName cellName)
  {
- return getValueValidator(getColumnDefinition(name));
- }
- 
- public AbstractType? getValueValidatorForFullCellName(ByteBuffer name)
- {
- // If this is a CQL table, we need to pull out the CQL column name to 
look up the correct column type.
- if (!isCQL3Table())
- return getValueValidator(getColumnDefinition(name));
- 
- return 
getValueValidator(getColumnDefinition(comparator.cellFromByteBuffer(name)));
- }
- 
- public AbstractType? getValueValidator(ColumnDefinition 
columnDefinition)
- {
- return columnDefinition == null
-? defaultValidator
-: columnDefinition.type;
 -ColumnDefinition def = getColumnDefinitionFromColumnName(columnName);
 -return def == null ? defaultValidator : def.getValidator();
++ColumnDefinition def = getColumnDefinition(cellName);
++return def == null ? defaultValidator : def.type;
  }
  
  /** applies implicit defaults to cf definition. useful in updates */
@@@ -1312,14 -1201,46 +1297,24 @@@
  }
  
  /**
 - * Returns a ColumnDefinition given a full (internal) column name.
 + * Returns a ColumnDefinition given a cell name.
   */
 -public ColumnDefinition getColumnDefinitionFromColumnName(ByteBuffer 
columnName)
 +public ColumnDefinition getColumnDefinition(CellName cellName)
  {
 -if (!isSuper()  (comparator instanceof CompositeType))
 -{
 -CompositeType composite = (CompositeType)comparator;
 -ByteBuffer[] components = composite.split(columnName);
 -for (ColumnDefinition def : regularAndStaticColumns())
 -{
 - 

[1/4] git commit: Fix prepared statement on thrift post-6659

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 2eecd4039 - 04f74a915


Fix prepared statement on thrift post-6659


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/79da4a6e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/79da4a6e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/79da4a6e

Branch: refs/heads/trunk
Commit: 79da4a6e0e1b33f880f6228e93905d85122238a2
Parents: 80b3e6c
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 11:32:48 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 11:33:05 2014 +0200

--
 src/java/org/apache/cassandra/cql3/QueryProcessor.java | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/79da4a6e/src/java/org/apache/cassandra/cql3/QueryProcessor.java
--
diff --git a/src/java/org/apache/cassandra/cql3/QueryProcessor.java 
b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
index fe818fd..64ea5e5 100644
--- a/src/java/org/apache/cassandra/cql3/QueryProcessor.java
+++ b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
@@ -35,6 +35,7 @@ import org.apache.cassandra.db.*;
 import org.apache.cassandra.exceptions.*;
 import org.apache.cassandra.service.ClientState;
 import org.apache.cassandra.service.QueryState;
+import org.apache.cassandra.thrift.ThriftClientState;
 import org.apache.cassandra.tracing.Tracing;
 import org.apache.cassandra.utils.FBUtilities;
 import org.apache.cassandra.utils.MD5Digest;
@@ -237,7 +238,8 @@ public class QueryProcessor implements QueryHandler
 public ResultMessage.Prepared prepare(String queryString, QueryState 
queryState)
 throws RequestValidationException
 {
-return prepare(queryString, queryState.getClientState(), false);
+ClientState cState = queryState.getClientState();
+return prepare(queryString, cState, cState instanceof 
ThriftClientState);
 }
 
 public static ResultMessage.Prepared prepare(String queryString, 
ClientState clientState, boolean forThrift)

[4/4] git commit: Merge branch 'cassandra-2.1' into trunk

[slebresne]

Merge branch 'cassandra-2.1' into trunk

Conflicts:
src/java/org/apache/cassandra/cql/QueryProcessor.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/04f74a91
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/04f74a91
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/04f74a91

Branch: refs/heads/trunk
Commit: 04f74a915c41bb7b7263c0ec2304e2d0f7f105c0
Parents: 2eecd40 8afe109
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 14:02:31 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 14:02:31 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java | 39 +---
 .../cassandra/config/ColumnDefinition.java  |  9 +++
 .../apache/cassandra/cql3/QueryProcessor.java   |  4 +-
 .../cassandra/thrift/ThriftValidation.java  |  5 +-
 .../apache/cassandra/tools/SSTableImport.java   |  2 +-
 .../unit/org/apache/cassandra/SchemaLoader.java |  7 ++-
 .../unit/org/apache/cassandra/db/ScrubTest.java | 65 
 .../cassandra/thrift/ThriftValidationTest.java  | 53 ++--
 .../cassandra/tools/SSTableExportTest.java  | 35 +++
 10 files changed, 187 insertions(+), 33 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/04f74a91/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/04f74a91/src/java/org/apache/cassandra/config/CFMetaData.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/04f74a91/src/java/org/apache/cassandra/config/ColumnDefinition.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/04f74a91/src/java/org/apache/cassandra/cql3/QueryProcessor.java
--

[3/4] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[slebresne]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/config/CFMetaData.java
src/java/org/apache/cassandra/config/Schema.java
src/java/org/apache/cassandra/cql/QueryProcessor.java
src/java/org/apache/cassandra/cql/SelectStatement.java
src/java/org/apache/cassandra/cql/UpdateStatement.java
src/java/org/apache/cassandra/db/Column.java
src/java/org/apache/cassandra/thrift/ThriftValidation.java
src/java/org/apache/cassandra/tools/SSTableExport.java
src/java/org/apache/cassandra/tools/SSTableImport.java
test/unit/org/apache/cassandra/SchemaLoader.java
test/unit/org/apache/cassandra/db/ScrubTest.java
test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java
test/unit/org/apache/cassandra/tools/SSTableExportTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8afe1090
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8afe1090
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8afe1090

Branch: refs/heads/trunk
Commit: 8afe1090bc4d04346197be6aa696b3af174f3674
Parents: 92b40fd b42feea
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 14:01:43 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 14:01:43 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java | 39 +---
 .../cassandra/config/ColumnDefinition.java  |  9 +++
 .../apache/cassandra/cql/QueryProcessor.java|  2 +-
 .../apache/cassandra/cql3/QueryProcessor.java   |  4 +-
 .../cassandra/thrift/ThriftValidation.java  |  5 +-
 .../apache/cassandra/tools/SSTableImport.java   |  2 +-
 .../unit/org/apache/cassandra/SchemaLoader.java |  7 ++-
 .../unit/org/apache/cassandra/db/ScrubTest.java | 65 
 .../cassandra/thrift/ThriftValidationTest.java  | 53 ++--
 .../cassandra/tools/SSTableExportTest.java  | 35 +++
 11 files changed, 188 insertions(+), 34 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8afe1090/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8afe1090/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --cc src/java/org/apache/cassandra/config/CFMetaData.java
index b46fafd,be18d1b..e930de4
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@@ -919,25 -853,13 +919,10 @@@ public final class CFMetaDat
  .toHashCode();
  }
  
- public AbstractType? getValueValidator(CellName name)
 -/**
 - * Like getColumnDefinitionFromColumnName, the argument must be an 
internal column/cell name.
 - */
 -public AbstractType? getValueValidatorFromColumnName(ByteBuffer 
columnName)
++public AbstractType? getValueValidator(CellName cellName)
  {
- return getValueValidator(getColumnDefinition(name));
- }
- 
- public AbstractType? getValueValidatorForFullCellName(ByteBuffer name)
- {
- // If this is a CQL table, we need to pull out the CQL column name to 
look up the correct column type.
- if (!isCQL3Table())
- return getValueValidator(getColumnDefinition(name));
- 
- return 
getValueValidator(getColumnDefinition(comparator.cellFromByteBuffer(name)));
- }
- 
- public AbstractType? getValueValidator(ColumnDefinition 
columnDefinition)
- {
- return columnDefinition == null
-? defaultValidator
-: columnDefinition.type;
 -ColumnDefinition def = getColumnDefinitionFromColumnName(columnName);
 -return def == null ? defaultValidator : def.getValidator();
++ColumnDefinition def = getColumnDefinition(cellName);
++return def == null ? defaultValidator : def.type;
  }
  
  /** applies implicit defaults to cf definition. useful in updates */
@@@ -1312,14 -1201,46 +1297,24 @@@
  }
  
  /**
 - * Returns a ColumnDefinition given a full (internal) column name.
 + * Returns a ColumnDefinition given a cell name.
   */
 -public ColumnDefinition getColumnDefinitionFromColumnName(ByteBuffer 
columnName)
 +public ColumnDefinition getColumnDefinition(CellName cellName)
  {
 -if (!isSuper()  (comparator instanceof CompositeType))
 -{
 -CompositeType composite = (CompositeType)comparator;
 -ByteBuffer[] components = composite.split(columnName);
 -for (ColumnDefinition def : regularAndStaticColumns())
 -{
 - 

[2/4] git commit: Fix Cassandra 2.0.x validates Thrift columns incorrectly

[slebresne]

Fix Cassandra 2.0.x validates Thrift columns incorrectly

patch by thobbs and slebresne; reviewed by thobbs and slebresne for 
CASSANDRA-6892


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b42feea6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b42feea6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b42feea6

Branch: refs/heads/trunk
Commit: b42feea6432905946083e0e287b3545a5799e4a7
Parents: 79da4a6
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 11:58:00 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 11:58:00 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java | 32 ++
 .../cassandra/config/ColumnDefinition.java  |  9 +++
 .../org/apache/cassandra/config/Schema.java | 14 
 .../apache/cassandra/cql/QueryProcessor.java|  4 +-
 .../apache/cassandra/cql/SelectStatement.java   |  5 --
 .../apache/cassandra/cql/UpdateStatement.java   |  7 +-
 src/java/org/apache/cassandra/db/Column.java| 10 +--
 .../cassandra/thrift/ThriftValidation.java  |  7 +-
 .../apache/cassandra/tools/SSTableExport.java   |  2 +-
 .../apache/cassandra/tools/SSTableImport.java   |  2 +-
 .../unit/org/apache/cassandra/SchemaLoader.java | 18 +-
 .../unit/org/apache/cassandra/db/ScrubTest.java | 67 +++-
 .../cassandra/thrift/ThriftValidationTest.java  | 53 ++--
 .../cassandra/tools/SSTableExportTest.java  | 42 ++--
 15 files changed, 204 insertions(+), 69 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b42feea6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8bfc8b9..9d27ebf 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -38,6 +38,7 @@
  * Track liveRatio per-memtable, not per-CF (CASSANDRA-6945)
  * Make sure upgradesstables keeps sstable level (CASSANDRA-6958)
  * Fix LIMT with static columns (CASSANDRA-6956)
+ * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b42feea6/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 1f25cea..be18d1b 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -853,16 +853,13 @@ public final class CFMetaData
 .toHashCode();
 }
 
-public AbstractType? getValueValidator(ByteBuffer column)
-{
-return getValueValidator(getColumnDefinition(column));
-}
-
-public AbstractType? getValueValidator(ColumnDefinition columnDefinition)
+/**
+ * Like getColumnDefinitionFromColumnName, the argument must be an 
internal column/cell name.
+ */
+public AbstractType? getValueValidatorFromColumnName(ByteBuffer 
columnName)
 {
-return columnDefinition == null
-   ? defaultValidator
-   : columnDefinition.getValidator();
+ColumnDefinition def = getColumnDefinitionFromColumnName(columnName);
+return def == null ? defaultValidator : def.getValidator();
 }
 
 /** applies implicit defaults to cf definition. useful in updates */
@@ -1212,7 +1209,7 @@ public final class CFMetaData
 {
 CompositeType composite = (CompositeType)comparator;
 ByteBuffer[] components = composite.split(columnName);
-for (ColumnDefinition def : column_metadata.values())
+for (ColumnDefinition def : regularAndStaticColumns())
 {
 ByteBuffer toCompare;
 if (def.componentIndex == null)
@@ -1233,12 +1230,19 @@ public final class CFMetaData
 }
 else
 {
-return column_metadata.get(columnName);
+ColumnDefinition def = column_metadata.get(columnName);
+// It's possible that the def is a PRIMARY KEY or COMPACT_VALUE 
one in case a concrete cell
+// name conflicts with a CQL column name, which can happen in 2 
cases:
+// 1) because the user inserted a cell through Thrift that 
conflicts with a default alias used
+//by CQL for thrift tables (see #6892).
+// 2) for COMPACT STORAGE tables with a single utf8 clustering 
column, the cell name can be anything,
+//including a CQL column name (without this being a problem).
+  

git commit: Fix error with super columns in mixed 1.2-2.0 cluster

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 b42feea64 - e1a90936c


Fix error with super columns in mixed 1.2-2.0 cluster

patch by slebresne; reviewed by iamaleksey for CASSANDRA-6966


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e1a90936
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e1a90936
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e1a90936

Branch: refs/heads/cassandra-2.0
Commit: e1a90936c709d0da1a601ef418d9b97a14ff5bb2
Parents: b42feea
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 14:26:14 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 14:26:14 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/SuperColumns.java   | 24 
 2 files changed, 11 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e1a90936/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9d27ebf..483ee0b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -39,6 +39,7 @@
  * Make sure upgradesstables keeps sstable level (CASSANDRA-6958)
  * Fix LIMT with static columns (CASSANDRA-6956)
  * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
+ * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e1a90936/src/java/org/apache/cassandra/db/SuperColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/SuperColumns.java 
b/src/java/org/apache/cassandra/db/SuperColumns.java
index 513db38..0f74587 100644
--- a/src/java/org/apache/cassandra/db/SuperColumns.java
+++ b/src/java/org/apache/cassandra/db/SuperColumns.java
@@ -260,21 +260,13 @@ public class SuperColumns
 
 public static SCFilter namesFilterToSC(CompositeType type, 
NamesQueryFilter filter)
 {
-ByteBuffer scName = null;
-SortedSetByteBuffer newColumns = new 
TreeSetByteBuffer(filter.columns.comparator());
+SortedSetByteBuffer newColumns = new TreeSet(type.types.get(1));
+ByteBuffer scName = scName(filter.columns.first());
 for (ByteBuffer name : filter.columns)
 {
-ByteBuffer newScName = scName(name);
-
-if (scName == null)
-{
-scName = newScName;
-}
-else if (type.types.get(0).compare(scName, newScName) != 0)
-{
-// If we're selecting column across multiple SC, it's not 
something we can translate for an old node
+// If we're selecting column across multiple SC, it's not 
something we can translate for an old node
+if (type.types.get(0).compare(scName, scName(name)) != 0)
 throw new RuntimeException(Cannot convert filter to old super 
column format. Update all nodes to Cassandra 2.0 first.);
-}
 
 newColumns.add(subName(name));
 }
@@ -331,10 +323,14 @@ public class SuperColumns
 return new SCFilter(null, new 
SliceQueryFilter(scName(start), scName(finish), reversed, filter.count));
 }
 }
-else if (filter.compositesToGroup == 0  
type.types.get(0).compare(scName(start), scName(finish)) == 0)
+else if (filter.compositesToGroup == -1  
type.types.get(0).compare(scName(start), scName(finish)) == 0)
 {
 // A slice of subcolumns
-return new SCFilter(scName(start), 
filter.withUpdatedSlice(subName(start), subName(finish)));
+ByteBuffer newStart = subName(start);
+ByteBuffer newFinish = subName(finish);
+return new SCFilter(scName(start),
+filter.withUpdatedSlice(newStart  == null 
? ByteBufferUtil.EMPTY_BYTE_BUFFER : newStart,
+newFinish == null 
? ByteBufferUtil.EMPTY_BYTE_BUFFER : newFinish));
 }
 }
 else if (!reversed)

[1/2] git commit: Fix error with super columns in mixed 1.2-2.0 cluster

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 8afe1090b - ef08e6258


Fix error with super columns in mixed 1.2-2.0 cluster

patch by slebresne; reviewed by iamaleksey for CASSANDRA-6966


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e1a90936
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e1a90936
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e1a90936

Branch: refs/heads/cassandra-2.1
Commit: e1a90936c709d0da1a601ef418d9b97a14ff5bb2
Parents: b42feea
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 14:26:14 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 14:26:14 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/SuperColumns.java   | 24 
 2 files changed, 11 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e1a90936/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9d27ebf..483ee0b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -39,6 +39,7 @@
  * Make sure upgradesstables keeps sstable level (CASSANDRA-6958)
  * Fix LIMT with static columns (CASSANDRA-6956)
  * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
+ * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e1a90936/src/java/org/apache/cassandra/db/SuperColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/SuperColumns.java 
b/src/java/org/apache/cassandra/db/SuperColumns.java
index 513db38..0f74587 100644
--- a/src/java/org/apache/cassandra/db/SuperColumns.java
+++ b/src/java/org/apache/cassandra/db/SuperColumns.java
@@ -260,21 +260,13 @@ public class SuperColumns
 
 public static SCFilter namesFilterToSC(CompositeType type, 
NamesQueryFilter filter)
 {
-ByteBuffer scName = null;
-SortedSetByteBuffer newColumns = new 
TreeSetByteBuffer(filter.columns.comparator());
+SortedSetByteBuffer newColumns = new TreeSet(type.types.get(1));
+ByteBuffer scName = scName(filter.columns.first());
 for (ByteBuffer name : filter.columns)
 {
-ByteBuffer newScName = scName(name);
-
-if (scName == null)
-{
-scName = newScName;
-}
-else if (type.types.get(0).compare(scName, newScName) != 0)
-{
-// If we're selecting column across multiple SC, it's not 
something we can translate for an old node
+// If we're selecting column across multiple SC, it's not 
something we can translate for an old node
+if (type.types.get(0).compare(scName, scName(name)) != 0)
 throw new RuntimeException(Cannot convert filter to old super 
column format. Update all nodes to Cassandra 2.0 first.);
-}
 
 newColumns.add(subName(name));
 }
@@ -331,10 +323,14 @@ public class SuperColumns
 return new SCFilter(null, new 
SliceQueryFilter(scName(start), scName(finish), reversed, filter.count));
 }
 }
-else if (filter.compositesToGroup == 0  
type.types.get(0).compare(scName(start), scName(finish)) == 0)
+else if (filter.compositesToGroup == -1  
type.types.get(0).compare(scName(start), scName(finish)) == 0)
 {
 // A slice of subcolumns
-return new SCFilter(scName(start), 
filter.withUpdatedSlice(subName(start), subName(finish)));
+ByteBuffer newStart = subName(start);
+ByteBuffer newFinish = subName(finish);
+return new SCFilter(scName(start),
+filter.withUpdatedSlice(newStart  == null 
? ByteBufferUtil.EMPTY_BYTE_BUFFER : newStart,
+newFinish == null 
? ByteBufferUtil.EMPTY_BYTE_BUFFER : newFinish));
 }
 }
 else if (!reversed)

[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[slebresne]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/db/SuperColumns.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ef08e625
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ef08e625
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ef08e625

Branch: refs/heads/cassandra-2.1
Commit: ef08e62589f9a27c76f610058373780ce36a7d89
Parents: 8afe109 e1a9093
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 14:28:54 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 14:28:54 2014 +0200

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/db/SuperColumns.java | 1 -
 2 files changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ef08e625/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ef08e625/src/java/org/apache/cassandra/db/SuperColumns.java
--
diff --cc src/java/org/apache/cassandra/db/SuperColumns.java
index 46c5577,0f74587..bab46eb
--- a/src/java/org/apache/cassandra/db/SuperColumns.java
+++ b/src/java/org/apache/cassandra/db/SuperColumns.java
@@@ -124,52 -205,155 +124,51 @@@ public class SuperColumn
  }
  }
  
 -public static AbstractType? getComparatorFor(CFMetaData metadata, 
ByteBuffer superColumn)
 -{
 -return getComparatorFor(metadata, superColumn != null);
 -}
 -
 -public static AbstractType? getComparatorFor(CFMetaData metadata, 
boolean subColumn)
 +private static CellNameType subType(CellNameType type)
  {
 -return metadata.isSuper()
 - ? ((CompositeType)metadata.comparator).types.get(subColumn ? 1 : 
0)
 - : metadata.comparator;
 +return new SimpleDenseCellNameType(type.subtype(1));
  }
  
 -// Extract the first component of a columnName, i.e. the super column name
 -public static ByteBuffer scName(ByteBuffer columnName)
 +public static CellNameType scNameType(CellNameType type)
  {
 -return CompositeType.extractComponent(columnName, 0);
 +return new SimpleDenseCellNameType(type.subtype(0));
  }
  
 -// Extract the 2nd component of a columnName, i.e. the sub-column name
 -public static ByteBuffer subName(ByteBuffer columnName)
 +public static AbstractType? getComparatorFor(CFMetaData metadata, 
ByteBuffer superColumn)
  {
 -return CompositeType.extractComponent(columnName, 1);
 +return getComparatorFor(metadata, superColumn != null);
  }
  
 -// We don't use CompositeType.Builder mostly because we want to avoid 
having to provide the comparator.
 -public static ByteBuffer startOf(ByteBuffer scName)
 +public static AbstractType? getComparatorFor(CFMetaData metadata, 
boolean subColumn)
  {
 -int length = scName.remaining();
 -ByteBuffer bb = ByteBuffer.allocate(2 + length + 1);
 -
 -bb.put((byte) ((length  8)  0xFF));
 -bb.put((byte) (length  0xFF));
 -bb.put(scName.duplicate());
 -bb.put((byte) 0);
 -bb.flip();
 -return bb;
 +return metadata.isSuper()
 + ? metadata.comparator.subtype(subColumn ? 1 : 0)
 + : metadata.comparator.asAbstractType();
  }
  
 -public static ByteBuffer endOf(ByteBuffer scName)
 +// Extract the first component of a columnName, i.e. the super column name
 +public static ByteBuffer scName(Composite columnName)
  {
 -ByteBuffer bb = startOf(scName);
 -bb.put(bb.remaining() - 1, (byte)1);
 -return bb;
 +return columnName.get(0);
  }
  
 -public static SCFilter filterToSC(CompositeType type, IDiskAtomFilter 
filter)
 +// Extract the 2nd component of a columnName, i.e. the sub-column name
 +public static ByteBuffer subName(Composite columnName)
  {
 -if (filter instanceof NamesQueryFilter)
 -return namesFilterToSC(type, (NamesQueryFilter)filter);
 -else
 -return sliceFilterToSC(type, (SliceQueryFilter)filter);
 +return columnName.get(1);
  }
  
 -public static SCFilter namesFilterToSC(CompositeType type, 
NamesQueryFilter filter)
 +public static Composite startOf(ByteBuffer scName)
  {
 -SortedSetByteBuffer newColumns = new TreeSet(type.types.get(1));
 -ByteBuffer scName = scName(filter.columns.first());
 -for (ByteBuffer name : filter.columns)
 -{
 -// If we're selecting column across multiple SC, it's not 
something we can translate for an old node
 -if 

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

[slebresne]

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/386ed8fa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/386ed8fa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/386ed8fa

Branch: refs/heads/trunk
Commit: 386ed8fa6c981f144433edc2ab13105734f16fae
Parents: 04f74a9 ef08e62
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 14:29:10 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 14:29:10 2014 +0200

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/db/SuperColumns.java | 1 -
 2 files changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/386ed8fa/CHANGES.txt
--

[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[slebresne]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/db/SuperColumns.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ef08e625
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ef08e625
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ef08e625

Branch: refs/heads/trunk
Commit: ef08e62589f9a27c76f610058373780ce36a7d89
Parents: 8afe109 e1a9093
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 14:28:54 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 14:28:54 2014 +0200

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/db/SuperColumns.java | 1 -
 2 files changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ef08e625/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ef08e625/src/java/org/apache/cassandra/db/SuperColumns.java
--
diff --cc src/java/org/apache/cassandra/db/SuperColumns.java
index 46c5577,0f74587..bab46eb
--- a/src/java/org/apache/cassandra/db/SuperColumns.java
+++ b/src/java/org/apache/cassandra/db/SuperColumns.java
@@@ -124,52 -205,155 +124,51 @@@ public class SuperColumn
  }
  }
  
 -public static AbstractType? getComparatorFor(CFMetaData metadata, 
ByteBuffer superColumn)
 -{
 -return getComparatorFor(metadata, superColumn != null);
 -}
 -
 -public static AbstractType? getComparatorFor(CFMetaData metadata, 
boolean subColumn)
 +private static CellNameType subType(CellNameType type)
  {
 -return metadata.isSuper()
 - ? ((CompositeType)metadata.comparator).types.get(subColumn ? 1 : 
0)
 - : metadata.comparator;
 +return new SimpleDenseCellNameType(type.subtype(1));
  }
  
 -// Extract the first component of a columnName, i.e. the super column name
 -public static ByteBuffer scName(ByteBuffer columnName)
 +public static CellNameType scNameType(CellNameType type)
  {
 -return CompositeType.extractComponent(columnName, 0);
 +return new SimpleDenseCellNameType(type.subtype(0));
  }
  
 -// Extract the 2nd component of a columnName, i.e. the sub-column name
 -public static ByteBuffer subName(ByteBuffer columnName)
 +public static AbstractType? getComparatorFor(CFMetaData metadata, 
ByteBuffer superColumn)
  {
 -return CompositeType.extractComponent(columnName, 1);
 +return getComparatorFor(metadata, superColumn != null);
  }
  
 -// We don't use CompositeType.Builder mostly because we want to avoid 
having to provide the comparator.
 -public static ByteBuffer startOf(ByteBuffer scName)
 +public static AbstractType? getComparatorFor(CFMetaData metadata, 
boolean subColumn)
  {
 -int length = scName.remaining();
 -ByteBuffer bb = ByteBuffer.allocate(2 + length + 1);
 -
 -bb.put((byte) ((length  8)  0xFF));
 -bb.put((byte) (length  0xFF));
 -bb.put(scName.duplicate());
 -bb.put((byte) 0);
 -bb.flip();
 -return bb;
 +return metadata.isSuper()
 + ? metadata.comparator.subtype(subColumn ? 1 : 0)
 + : metadata.comparator.asAbstractType();
  }
  
 -public static ByteBuffer endOf(ByteBuffer scName)
 +// Extract the first component of a columnName, i.e. the super column name
 +public static ByteBuffer scName(Composite columnName)
  {
 -ByteBuffer bb = startOf(scName);
 -bb.put(bb.remaining() - 1, (byte)1);
 -return bb;
 +return columnName.get(0);
  }
  
 -public static SCFilter filterToSC(CompositeType type, IDiskAtomFilter 
filter)
 +// Extract the 2nd component of a columnName, i.e. the sub-column name
 +public static ByteBuffer subName(Composite columnName)
  {
 -if (filter instanceof NamesQueryFilter)
 -return namesFilterToSC(type, (NamesQueryFilter)filter);
 -else
 -return sliceFilterToSC(type, (SliceQueryFilter)filter);
 +return columnName.get(1);
  }
  
 -public static SCFilter namesFilterToSC(CompositeType type, 
NamesQueryFilter filter)
 +public static Composite startOf(ByteBuffer scName)
  {
 -SortedSetByteBuffer newColumns = new TreeSet(type.types.get(1));
 -ByteBuffer scName = scName(filter.columns.first());
 -for (ByteBuffer name : filter.columns)
 -{
 -// If we're selecting column across multiple SC, it's not 
something we can translate for an old node
 -if 

[1/3] git commit: Fix error with super columns in mixed 1.2-2.0 cluster

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 04f74a915 - 386ed8fa6


Fix error with super columns in mixed 1.2-2.0 cluster

patch by slebresne; reviewed by iamaleksey for CASSANDRA-6966


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e1a90936
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e1a90936
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e1a90936

Branch: refs/heads/trunk
Commit: e1a90936c709d0da1a601ef418d9b97a14ff5bb2
Parents: b42feea
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 14:26:14 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 14:26:14 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/SuperColumns.java   | 24 
 2 files changed, 11 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e1a90936/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9d27ebf..483ee0b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -39,6 +39,7 @@
  * Make sure upgradesstables keeps sstable level (CASSANDRA-6958)
  * Fix LIMT with static columns (CASSANDRA-6956)
  * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
+ * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e1a90936/src/java/org/apache/cassandra/db/SuperColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/SuperColumns.java 
b/src/java/org/apache/cassandra/db/SuperColumns.java
index 513db38..0f74587 100644
--- a/src/java/org/apache/cassandra/db/SuperColumns.java
+++ b/src/java/org/apache/cassandra/db/SuperColumns.java
@@ -260,21 +260,13 @@ public class SuperColumns
 
 public static SCFilter namesFilterToSC(CompositeType type, 
NamesQueryFilter filter)
 {
-ByteBuffer scName = null;
-SortedSetByteBuffer newColumns = new 
TreeSetByteBuffer(filter.columns.comparator());
+SortedSetByteBuffer newColumns = new TreeSet(type.types.get(1));
+ByteBuffer scName = scName(filter.columns.first());
 for (ByteBuffer name : filter.columns)
 {
-ByteBuffer newScName = scName(name);
-
-if (scName == null)
-{
-scName = newScName;
-}
-else if (type.types.get(0).compare(scName, newScName) != 0)
-{
-// If we're selecting column across multiple SC, it's not 
something we can translate for an old node
+// If we're selecting column across multiple SC, it's not 
something we can translate for an old node
+if (type.types.get(0).compare(scName, scName(name)) != 0)
 throw new RuntimeException(Cannot convert filter to old super 
column format. Update all nodes to Cassandra 2.0 first.);
-}
 
 newColumns.add(subName(name));
 }
@@ -331,10 +323,14 @@ public class SuperColumns
 return new SCFilter(null, new 
SliceQueryFilter(scName(start), scName(finish), reversed, filter.count));
 }
 }
-else if (filter.compositesToGroup == 0  
type.types.get(0).compare(scName(start), scName(finish)) == 0)
+else if (filter.compositesToGroup == -1  
type.types.get(0).compare(scName(start), scName(finish)) == 0)
 {
 // A slice of subcolumns
-return new SCFilter(scName(start), 
filter.withUpdatedSlice(subName(start), subName(finish)));
+ByteBuffer newStart = subName(start);
+ByteBuffer newFinish = subName(finish);
+return new SCFilter(scName(start),
+filter.withUpdatedSlice(newStart  == null 
? ByteBufferUtil.EMPTY_BYTE_BUFFER : newStart,
+newFinish == null 
? ByteBufferUtil.EMPTY_BYTE_BUFFER : newFinish));
 }
 }
 else if (!reversed)

[jira] [Resolved] (CASSANDRA-6973) timestamp data type does ISO 8601 formats with 'Z' as time zone.

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-6973.
---

   Resolution: Duplicate
Fix Version/s: (was: 2.0.6)

 timestamp data type does ISO 8601 formats with 'Z' as time zone.
 

 Key: CASSANDRA-6973
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6973
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Juho Mäkinen
Priority: Trivial

 The timestamp data type does not support format where time zone is specified 
 with 'Z' (as in zulu aka. UTC+0 aka + time zone). Example:
 create table foo(ts timestamp primary key);
 insert into foo(ts) values('2014-04-01T20:17:35+'); -- this works
 cqlsh:test insert into foo(ts) values('2014-04-01T20:17:35Z');
 Bad Request: unable to coerce '2014-04-01T20:17:35Z' to a  formatted date 
 (long)
 The example date was copied directly from ISO 8601 Wikipedia page. The 
 standard says that If the time is in UTC, add a Z directly after the time 
 without a space. Z is the zone designator for the zero UTC offset.
 Tested with cqlsh with 2.0.6 version.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Reopened] (CASSANDRA-6973) timestamp data type does ISO 8601 formats with 'Z' as time zone.

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne reopened CASSANDRA-6973:
-


 timestamp data type does ISO 8601 formats with 'Z' as time zone.
 

 Key: CASSANDRA-6973
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6973
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Juho Mäkinen
Priority: Trivial

 The timestamp data type does not support format where time zone is specified 
 with 'Z' (as in zulu aka. UTC+0 aka + time zone). Example:
 create table foo(ts timestamp primary key);
 insert into foo(ts) values('2014-04-01T20:17:35+'); -- this works
 cqlsh:test insert into foo(ts) values('2014-04-01T20:17:35Z');
 Bad Request: unable to coerce '2014-04-01T20:17:35Z' to a  formatted date 
 (long)
 The example date was copied directly from ISO 8601 Wikipedia page. The 
 standard says that If the time is in UTC, add a Z directly after the time 
 without a space. Z is the zone designator for the zero UTC offset.
 Tested with cqlsh with 2.0.6 version.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6973) timestamp data type does ISO 8601 formats with 'Z' as time zone.

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957634#comment-13957634
 ] 

Sylvain Lebresne commented on CASSANDRA-6973:
-

Hum, I'm not sure at all that's a duplicate of CASSANDRA-6350. And truth is, 
this should work since it's supposed to be one of the recognized pattern in 
TimestampSerializer.iso8601Patterns. Not sure what's the problem, but my 
understanding is that this should work (we certainly pretend to support IS08601 
in the doc) and is not really related to CASSANDRA-6350.

 timestamp data type does ISO 8601 formats with 'Z' as time zone.
 

 Key: CASSANDRA-6973
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6973
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Juho Mäkinen
Priority: Trivial

 The timestamp data type does not support format where time zone is specified 
 with 'Z' (as in zulu aka. UTC+0 aka + time zone). Example:
 create table foo(ts timestamp primary key);
 insert into foo(ts) values('2014-04-01T20:17:35+'); -- this works
 cqlsh:test insert into foo(ts) values('2014-04-01T20:17:35Z');
 Bad Request: unable to coerce '2014-04-01T20:17:35Z' to a  formatted date 
 (long)
 The example date was copied directly from ISO 8601 Wikipedia page. The 
 standard says that If the time is in UTC, add a Z directly after the time 
 without a space. Z is the zone designator for the zero UTC offset.
 Tested with cqlsh with 2.0.6 version.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6350) Timestamp-with-timezone type

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957637#comment-13957637
 ] 

Sylvain Lebresne commented on CASSANDRA-6350:
-

Actually, I'm confused here, isn't that just a cqlsh display problem?

 Timestamp-with-timezone type
 

 Key: CASSANDRA-6350
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6350
 Project: Cassandra
  Issue Type: Wish
  Components: Core
Reporter: Ramkumar S
Priority: Minor
  Labels: lhf

 Create a table with a timestamp column.
 Insert a value from US time Zone.
 Try querying the value from a different time zone like India.
 The timestamp column  value shown in the select query result is converted to 
 Indian Local time instead of showing the actual value.
 This becomes a problem when we want to narrow down the query using where 
 condition.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6825) Slice Queries Can Skip Intersecting SSTables

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957648#comment-13957648
 ] 

Sylvain Lebresne commented on CASSANDRA-6825:
-

I'm not sure the example above is really the one you meant (because I believe 
that the current code does return that there is intersection for that example), 
I think you meant something along the lines of a slice of say {{((1, 3), )}} 
with min={{(0, 1)}} and max={{(3, 2)}}, but I agree there's a problem.

The patch looks mostly good, but regarding the optimization inner loop in 
CompositeType.intersects, I believe it could be simplified to:
{noformat}
for (int i = 0; i  Math.min(Math.min(start.length, finish.length), 
minColumnNames.size()); i++)
{
AbstractType? t = types.get(i);
if (t.compare(start[i], finish[i]) != 0)
break;

// we already know the first component falls within its min/max range 
(otherwise we wouldn't get here)
if (i  0  !t.intersects(minColumnNames.get(i), maxColumnNames.get(i), 
start[i], finish[i]))
continue outer;
}
{noformat}
Note in particular that we shouldn't assume that minColumnNames/maxColumnNames 
length is always bigger or equal that the slice bounds. But +1 if we agree on 
that change.


 Slice Queries Can Skip Intersecting SSTables
 

 Key: CASSANDRA-6825
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6825
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: quad core Windows7 x64, single node cluster
 Cassandra 2.0.5
Reporter: Bill Mitchell
Assignee: Tyler Hobbs
 Fix For: 2.0.7, 2.1 beta2

 Attachments: 6825-2.0.txt, cassandra.log, selectpartitions.zip, 
 selectrowcounts.txt, testdb_1395372407904.zip, testdb_1395372407904.zip


 Investigating another problem, I needed to do COUNT(*) on the several 
 partitions of a table immediately after a test case ran, and I discovered 
 that count(*) on the full table and on each of the partitions returned 
 different counts.  
 In particular case, SELECT COUNT(*) FROM sr LIMIT 100; returned the 
 expected count from the test 9 rows.  The composite primary key splits 
 the logical row into six distinct partitions, and when I issue a query asking 
 for the total across all six partitions, the returned result is only 83999.  
 Drilling down, I find that SELECT * from sr WHERE s = 5 AND l = 11 AND 
 partition = 0; returns 30,000 rows, but a SELECT COUNT(*) with the identical 
 WHERE predicate reports only 14,000. 
 This is failing immediately after running a single small test, such that 
 there are only two SSTables, sr-jb-1 and sr-jb-2.  Compaction never needed to 
 run.  
 In selectrowcounts.txt is a copy of the cqlsh output showing the incorrect 
 count(*) results.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6818) SSTable references not released if stream session fails before it starts

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957667#comment-13957667
 ] 

Yuki Morishita commented on CASSANDRA-6818:
---

[~krummas] This can make SSTables that are compacted but not deleted. But for 
the SSTable in system/hints that cburroughs saw is different story since there 
should be no stream involved.

 SSTable references not released if stream session fails before it starts
 

 Key: CASSANDRA-6818
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6818
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Richard Low
Assignee: Yuki Morishita
 Fix For: 1.2.16, 2.0.7, 2.1 beta2

 Attachments: 6818-1.2.txt, 6818-2.0-v2.txt, 6818-2.0-v3.txt, 
 6818-2.0.txt


 I observed a large number of 'orphan' SSTables - SSTables that are in the 
 data directory but not loaded by Cassandra - on a 1.1.12 node that had a 
 large stream fail before it started. These orphan files are particularly 
 dangerous because if the node is restarted and picks up these SSTables it 
 could bring data back to life if tombstones have been GCed. To confirm the 
 SSTables are orphan, I created a snapshot and it didn't contain these files. 
 I can see in the logs that they have been compacted so should have been 
 deleted.
 The log entries for the stream are:
 {{INFO [StreamStage:1] 2014-02-21 19:41:48,742 StreamOut.java (line 115) 
 Beginning transfer to /10.0.0.1}}
 {{INFO [StreamStage:1] 2014-02-21 19:41:48,743 StreamOut.java (line 96) 
 Flushing memtables for [CFS(Keyspace='ks', ColumnFamily='cf1'), 
 CFS(Keyspace='ks', ColumnFamily='cf2')]...}}
 {{ERROR [GossipTasks:1] 2014-02-21 19:41:49,239 AbstractStreamSession.java 
 (line 113) Stream failed because /10.0.0.1 died or was restarted/removed 
 (streams may still be active in background, but further streams won't be 
 started)}}
 {{INFO [StreamStage:1] 2014-02-21 19:41:51,783 StreamOut.java (line 161) 
 Stream context metadata [...] 2267 sstables.}}
 {{INFO [StreamStage:1] 2014-02-21 19:41:51,789 StreamOutSession.java (line 
 182) Streaming to /10.0.0.1}}
 {{INFO [Streaming to /10.0.0.1:1] 2014-02-21 19:42:02,218 FileStreamTask.java 
 (line 99) Found no stream out session at end of file stream task - this is 
 expected if the receiver went down}}
 After digging in the code, here's what I think the issue is:
 1. StreamOutSession.transferRanges() creates a streaming session, which is 
 registered with the failure detector in AbstractStreamSession's constructor.
 2. Memtables are flushed, potentially taking a long time.
 3. The remote node fails, convict() is called and the StreamOutSession is 
 closed. However, at this time StreamOutSession.files is empty because it's 
 still waiting for the memtables to flush.
 4. Memtables finish flusing, references are obtained to SSTables to be 
 streamed and the PendingFiles are added to StreamOutSession.files.
 5. The first stream fails but the StreamOutSession isn't found so is never 
 closed and the references are never released.
 This code is more or less the same on 1.2 so I would expect it to reproduce 
 there. I looked at 2.0 and can't even see where SSTable references are 
 released when the stream fails.
 Some possible fixes for 1.1/1.2:
 1. Don't register with the failure detector until after the PendingFiles are 
 set up. I think this is the behaviour in 2.0 but I don't know if it was done 
 like this to avoid this issue.
 2. Detect the above case in (e.g.) StreamOutSession.begin() by noticing the 
 session has been closed with care to avoid double frees.
 3. Add some synchronization so closeInternal() doesn't race with setting up 
 the session.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6855) Native protocol V3

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957675#comment-13957675
 ] 

Sylvain Lebresne commented on CASSANDRA-6855:
-

While I'm not necessarily against in principle, I'm not entirely sold for 
practical reasons. First, we already had to handle a few minor deprecation and 
so far we've logged a warning in the log. And while it's not perfect, it 
doesn't feel too bad either to me especially since we shouldn't deprecate stuff 
all the time. This to say that while sending the deprecation warning to the 
client would be nice, it doesn't seem absolutely necessary to me. On the cons 
side of things, I don't see a very clean way to add this to all possible type 
of RESULT messages without adding some bytes even when no warning is sent 
(which will be by far the majority of case): it's trivial to add a new flag to 
Result.Rows  messages, because we have the flags and we can dedicate one bit 
for an optional warning message easily, but for Result.Void, we can't make it 
entirely optional (unless maybe we create a new Result.Void_with_warning 
type, but that feels rather ugly). Sure just adding a warning field to 
Result.Void, just having it be the empty string when there is no warning, is 
only 2 bytes of overhead, but it's 2 bytes that is sent all the time for 
something that is very rarely needed, and is really just a convenience when it 
is. There is also the fact that currently we wouldn't have much use of this.

So, well, that doesn't seem too useful as of now, and I'm not too sure how to 
add it cleanly/without adding overhead in general. That said, if you have a 
concrete proposition, I'm fine looking at it.

But *please*, lets not that slow down the review of what's already here. As 
long as we decide what we want to do for this warnings idea before the final 
release of 2.1, we can deal with it asynchronously from the bulk of this issue 
(at best this should be minor modification anyway). And I really think the 
sooner we have the main parts of this in, the sooner drivers can test that new 
version and hopefully shake the bugs out before the final.

 Native protocol V3
 --

 Key: CASSANDRA-6855
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6855
 Project: Cassandra
  Issue Type: New Feature
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 2.1 beta2


 I think we need a V3 of the protocol for 2.1. The things that this 
 could/should includes are:
 # Adding an optional Serial CL for protocol batches (like we have for QUERY 
 and EXECUTE). It was an oversight of V2 of not adding it, and now that we can 
 batch conditional updates, it's definitively missing.
 # Proper type codes for UDT. This is not *strictly* needed to be able to 
 support UDT since currently a UDT will be sent as a custom type with his 
 fully class name + arguments. But parsing that is no fun nor convenient for 
 clients. It's also not particular space efficient (though that's probably not 
 a huge concern since with prepared statement you can avoid sending the 
 ResultSet metadata every time).
 # Serialization format for collections. Currently the serialization format 
 only allow for 65K elements, each of 65K bytes size at most. While 
 collections are not meant to store large amount of data, having the 
 limitation in the protocol serialization format is the wrong way to deal with 
 that. Concretely, the current workaround for CASSANDRA-5428 is ugly. I'll 
 note that the current serialization format is also an obstacle to supporting 
 null inside collections (whether or not we want to support null there is a 
 good question, but here again I'm not sure being limited by the serialization 
 format is a good idea).
 # CASSANDRA-6178: I continue to believe that in many case it makes somewhat 
 more sense to have the default timestamp provided by the client (this is a 
 necessary condition for true idempotent retries in particular). I'm 
 absolutely fine making that optional and leaving server-side generated 
 timestamps by default, but since client can already provide timestamp in 
 query string anyway, I don't see a big deal in making it easier for client 
 driver to control that without messing with the query string.
 # Optional names for values in QUERY messages: it has been brought to my 
 attention that while V2 allows to send a query string with values for a 
 one-roundtrip bind-and-execute, a driver can't really support named bind 
 marker with that feature properly without parsing the query. The proposition 
 is thus to make it (optionally) possible to ship the name of the marker each 
 value is supposed to be bound to.
 I think that 1) and 2) are enough reason to make a V3 (even if there is 
 disagreement on the rest that is).
 3) is a little bit more 

[jira] [Updated] (CASSANDRA-6948) After Bootstrap or Replace node startup, EXPIRING_MAP_REAPER is shutdown and cannot be restarted, causing callbacks to collect indefinitely

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-6948:


Reviewer: Brandon Williams

 After Bootstrap or Replace node startup, EXPIRING_MAP_REAPER is shutdown and 
 cannot be restarted, causing callbacks to collect indefinitely
 ---

 Key: CASSANDRA-6948
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6948
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Keith Wright
Assignee: Benedict
 Fix For: 2.0.7

 Attachments: 6948.debug.txt, 6948.txt, Screen Shot 2014-03-28 at 
 11.27.56 AM.png, Screen Shot 2014-03-28 at 11.29.24 AM.png, 
 cassandra.log.min, cassandra.yaml, logs.old.tar.gz, logs.tar.gz, 
 system.log.1.gz, system.log.gz


 Since ExpiringMap.shutdown() shuts down the static executor service, it 
 cannot be restarted (and in fact reset() makes no attempt to do so). As such 
 callbacks that receive no response are never removed from the map, and 
 eventually either than server will run out of memory or will loop around the 
 integer space and start reusing messageids that have not been expired, 
 causing assertions to be thrown and messages to fail to be sent. It appears 
 that this situation only arises on bootstrap or node replacement, as 
 MessagingService is shutdown before being attached to the listen address.
 This can cause the following errors to begin occurring in the log:
 ERROR [Native-Transport-Requests:7636] 2014-03-28 13:32:10,638 
 ErrorMessage.java (line 222) Unexpected exception during request
 java.lang.AssertionError: Callback already exists for id -1665979622! 
 (CallbackInfo(target=/10.106.160.84, 
 callback=org.apache.cassandra.service.WriteResponseHandler@5d36d8ea, 
 serializer=org.apache.cassandra.db.WriteResponse$WriteResponseSerializer@6ed37f0b))
   at 
 org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:549)
   at 
 org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:601)
   at 
 org.apache.cassandra.service.StorageProxy.mutateCounter(StorageProxy.java:984)
   at 
 org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:449)
   at 
 org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:524)
   at 
 org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:521)
   at 
 org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:505)
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:188)
   at 
 org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:358)
   at 
 org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:131)
   at 
 org.apache.cassandra.transport.Message$Dispatcher.messageReceived(Message.java:304)
   at 
 org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43)
   at 
 org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 ERROR [ReplicateOnWriteStage:102766] 2014-03-28 13:32:10,638 
 CassandraDaemon.java (line 196) Exception in thread 
 Thread[ReplicateOnWriteStage:102766,5,main]
 java.lang.AssertionError: Callback already exists for id -1665979620! 
 (CallbackInfo(target=/10.106.160.84, 
 callback=org.apache.cassandra.service.WriteResponseHandler@3bdb1a75, 
 serializer=org.apache.cassandra.db.WriteResponse$WriteResponseSerializer@6ed37f0b))
   at 
 org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:549)
   at 
 org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:601)
   at 
 org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:806)
   at 
 org.apache.cassandra.service.StorageProxy$8$1.runMayThrow(StorageProxy.java:1074)
   at 
 org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1896)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6971) nodes not catching up to creation of new keyspace

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957713#comment-13957713
 ] 

Sylvain Lebresne commented on CASSANDRA-6971:
-

I think we should consider moving the dtests to use the new python driver, 
which I believe should give use the check for schema agreement for free. But if 
nodes can't get to agree even after 2 minutes, then there is indeed probably 
something fishy going on.

 nodes not catching up to creation of new keyspace
 -

 Key: CASSANDRA-6971
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6971
 Project: Cassandra
  Issue Type: Bug
Reporter: Russ Hatch

 The dtest suite is running a test which creates a 3 node cluster, then adds a 
 keyspace and column family. For some reason the 3 nodes are not agreeing on 
 the schema version. The problem is intermittent -- either the nodes all agree 
 on schema quickly, or they seem to stay stuck in limbo.
 The simplest way to reproduce is to run the dtest (simple_increment_test):
 https://github.com/riptano/cassandra-dtest/blob/master/counter_tests.py
 using nosetests:
 {noformat}
 nosetests -vs counter_tests.py:TestCounters.simple_increment_test
 {noformat}
 If the problem is reproduced nose will return this:
 ProgrammingError: Bad Request: Keyspace 'ks' does not exist
 I am not yet sure if the bug is reproducible outside of the dtest suite.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Fix tests post-merge

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 ef08e6258 - 486cc4cc0


Fix tests post-merge


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/486cc4cc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/486cc4cc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/486cc4cc

Branch: refs/heads/cassandra-2.1
Commit: 486cc4cc0a1aae49ef8d09fd884d7e4095c05f1b
Parents: ef08e62
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 17:02:49 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 17:02:49 2014 +0200

--
 src/java/org/apache/cassandra/tools/SSTableImport.java  | 2 +-
 test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/486cc4cc/src/java/org/apache/cassandra/tools/SSTableImport.java
--
diff --git a/src/java/org/apache/cassandra/tools/SSTableImport.java 
b/src/java/org/apache/cassandra/tools/SSTableImport.java
index c0d8e00..6bd2119 100644
--- a/src/java/org/apache/cassandra/tools/SSTableImport.java
+++ b/src/java/org/apache/cassandra/tools/SSTableImport.java
@@ -162,7 +162,7 @@ public class SSTableImport
 }
 else
 {
-value = stringAsType((String) fields.get(1), 
meta.getValueValidator(meta.comparator.cellFromByteBuffer(name)));
+value = stringAsType((String) fields.get(1), 
meta.getValueValidator(comparator.cellFromByteBuffer(name)));
 }
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/486cc4cc/test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java
--
diff --git a/test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java 
b/test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java
index 1d47c18..df0f98c 100644
--- a/test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java
+++ b/test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java
@@ -101,7 +101,7 @@ public class ThriftValidationTest extends SchemaLoader
 CFMetaData metaData = Schema.instance.getCFMetaData(Keyspace1, 
UUIDKeys);
 ColumnDefinition definition = 
metaData.getColumnDefinition(ByteBufferUtil.bytes(CFMetaData.DEFAULT_KEY_ALIAS));
 assertNotNull(definition);
-assertEquals(ColumnDefinition.Kind.PARTITION_KEY, definition.type);
+assertEquals(ColumnDefinition.Kind.PARTITION_KEY, definition.kind);
 
 // make sure the key alias does not affect validation of columns with 
the same name (CASSANDRA-6892)
 Column column = new 
Column(ByteBufferUtil.bytes(CFMetaData.DEFAULT_KEY_ALIAS));

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

[slebresne]

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3e0dac5a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3e0dac5a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3e0dac5a

Branch: refs/heads/trunk
Commit: 3e0dac5a443c6e8c089a832ddc05617241cdfb48
Parents: 386ed8f 486cc4c
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 17:03:15 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 17:03:15 2014 +0200

--
 src/java/org/apache/cassandra/tools/SSTableImport.java  | 2 +-
 test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
--

[1/2] git commit: Fix tests post-merge

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 386ed8fa6 - 3e0dac5a4


Fix tests post-merge


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/486cc4cc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/486cc4cc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/486cc4cc

Branch: refs/heads/trunk
Commit: 486cc4cc0a1aae49ef8d09fd884d7e4095c05f1b
Parents: ef08e62
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 17:02:49 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 17:02:49 2014 +0200

--
 src/java/org/apache/cassandra/tools/SSTableImport.java  | 2 +-
 test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/486cc4cc/src/java/org/apache/cassandra/tools/SSTableImport.java
--
diff --git a/src/java/org/apache/cassandra/tools/SSTableImport.java 
b/src/java/org/apache/cassandra/tools/SSTableImport.java
index c0d8e00..6bd2119 100644
--- a/src/java/org/apache/cassandra/tools/SSTableImport.java
+++ b/src/java/org/apache/cassandra/tools/SSTableImport.java
@@ -162,7 +162,7 @@ public class SSTableImport
 }
 else
 {
-value = stringAsType((String) fields.get(1), 
meta.getValueValidator(meta.comparator.cellFromByteBuffer(name)));
+value = stringAsType((String) fields.get(1), 
meta.getValueValidator(comparator.cellFromByteBuffer(name)));
 }
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/486cc4cc/test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java
--
diff --git a/test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java 
b/test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java
index 1d47c18..df0f98c 100644
--- a/test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java
+++ b/test/unit/org/apache/cassandra/thrift/ThriftValidationTest.java
@@ -101,7 +101,7 @@ public class ThriftValidationTest extends SchemaLoader
 CFMetaData metaData = Schema.instance.getCFMetaData(Keyspace1, 
UUIDKeys);
 ColumnDefinition definition = 
metaData.getColumnDefinition(ByteBufferUtil.bytes(CFMetaData.DEFAULT_KEY_ALIAS));
 assertNotNull(definition);
-assertEquals(ColumnDefinition.Kind.PARTITION_KEY, definition.type);
+assertEquals(ColumnDefinition.Kind.PARTITION_KEY, definition.kind);
 
 // make sure the key alias does not affect validation of columns with 
the same name (CASSANDRA-6892)
 Column column = new 
Column(ByteBufferUtil.bytes(CFMetaData.DEFAULT_KEY_ALIAS));

[jira] [Commented] (CASSANDRA-6971) nodes not catching up to creation of new keyspace

[Brandon Williams (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957787#comment-13957787
 ] 

Brandon Williams commented on CASSANDRA-6971:
-

We probably need logs from the 2 minute case.

 nodes not catching up to creation of new keyspace
 -

 Key: CASSANDRA-6971
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6971
 Project: Cassandra
  Issue Type: Bug
Reporter: Russ Hatch

 The dtest suite is running a test which creates a 3 node cluster, then adds a 
 keyspace and column family. For some reason the 3 nodes are not agreeing on 
 the schema version. The problem is intermittent -- either the nodes all agree 
 on schema quickly, or they seem to stay stuck in limbo.
 The simplest way to reproduce is to run the dtest (simple_increment_test):
 https://github.com/riptano/cassandra-dtest/blob/master/counter_tests.py
 using nosetests:
 {noformat}
 nosetests -vs counter_tests.py:TestCounters.simple_increment_test
 {noformat}
 If the problem is reproduced nose will return this:
 ProgrammingError: Bad Request: Keyspace 'ks' does not exist
 I am not yet sure if the bug is reproducible outside of the dtest suite.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6965) sstableloader fails to stream data

[Ryan McGuire (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957804#comment-13957804
 ] 

Ryan McGuire commented on CASSANDRA-6965:
-

The patch makes my test pass. Thanks Vijay.

 sstableloader fails to stream data
 --

 Key: CASSANDRA-6965
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6965
 Project: Cassandra
  Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Vijay
 Attachments: 0001-CASSANDRA-6965.patch


 When loading a snapshot via sstableloader I get this NullPointerException:
 {code}
 Established connection to initial hosts
 Opening sstables and calculating sections to stream
 Streaming relevant part of /tmp/tmpmusajE/ks/cf/ks-cf-ka-1-Data.db to 
 [/127.0.0.1]
 Streaming session ID: 276a9d20-b944-11e3-9f60-4191ec2d6f66
 Exception in thread STREAM-OUT-/127.0.0.1 java.lang.NullPointerException
   at 
 org.apache.cassandra.streaming.ConnectionHandler$MessageHandler.signalCloseDone(ConnectionHandler.java:249)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:375)
   at java.lang.Thread.run(Thread.java:744)
 {code}
 Which at first I thought meant it was unable to connect, but I do see this in 
 system.log, indicating the connection was successful:
 {code}
 INFO  [STREAM-INIT-/127.0.0.1:55820] 2014-03-31 22:20:12,265 
 StreamResultFuture.java:121 - [Stream #276a9d20-b944-11e3-9f60-4191ec2d6f66] 
 Received streaming plan for Bulk Load
 INFO  [STREAM-IN-/10.13.37.130] 2014-03-31 22:20:12,272 
 StreamResultFuture.java:173 - [Stream #276a9d20-b944-11e3-9f60-4191ec2d6f66] 
 Prepare completed. Receiving 1 files(1726 bytes), sending 0 files(0 bytes)
 INFO  [STREAM-IN-/10.13.37.130] 2014-03-31 22:20:12,280 
 CompressedStreamReader.java:65 - reading file from /10.13.37.130, repairedAt 
 = 0
 {code}
 To reproduce this behaviour, run the 
 [snapshot_test.py:TestSnapshot.test_basic_snapshot 
 dtest|https://github.com/riptano/cassandra-dtest/blob/85692be39cb6b1b5ebc96905834ff7bb174de0bc/snapshot_test.py#L40]
  on 2.1 HEAD.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6350) Timestamp-with-timezone type

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957806#comment-13957806
 ] 

Jonathan Ellis commented on CASSANDRA-6350:
---

No -- current TS type does not preserve TZ internally.

I.e. if I insert 2014:05:24 11:11:11 EDT and select it back out, I should get 
EDT and not whatever the server's TZ is or UTC.

 Timestamp-with-timezone type
 

 Key: CASSANDRA-6350
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6350
 Project: Cassandra
  Issue Type: Wish
  Components: Core
Reporter: Ramkumar S
Priority: Minor
  Labels: lhf

 Create a table with a timestamp column.
 Insert a value from US time Zone.
 Try querying the value from a different time zone like India.
 The timestamp column  value shown in the select query result is converted to 
 Indian Local time instead of showing the actual value.
 This becomes a problem when we want to narrow down the query using where 
 condition.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6965) sstableloader fails to stream data

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957809#comment-13957809
 ] 

Yuki Morishita commented on CASSANDRA-6965:
---

+1 from me too.

 sstableloader fails to stream data
 --

 Key: CASSANDRA-6965
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6965
 Project: Cassandra
  Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Vijay
 Attachments: 0001-CASSANDRA-6965.patch


 When loading a snapshot via sstableloader I get this NullPointerException:
 {code}
 Established connection to initial hosts
 Opening sstables and calculating sections to stream
 Streaming relevant part of /tmp/tmpmusajE/ks/cf/ks-cf-ka-1-Data.db to 
 [/127.0.0.1]
 Streaming session ID: 276a9d20-b944-11e3-9f60-4191ec2d6f66
 Exception in thread STREAM-OUT-/127.0.0.1 java.lang.NullPointerException
   at 
 org.apache.cassandra.streaming.ConnectionHandler$MessageHandler.signalCloseDone(ConnectionHandler.java:249)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:375)
   at java.lang.Thread.run(Thread.java:744)
 {code}
 Which at first I thought meant it was unable to connect, but I do see this in 
 system.log, indicating the connection was successful:
 {code}
 INFO  [STREAM-INIT-/127.0.0.1:55820] 2014-03-31 22:20:12,265 
 StreamResultFuture.java:121 - [Stream #276a9d20-b944-11e3-9f60-4191ec2d6f66] 
 Received streaming plan for Bulk Load
 INFO  [STREAM-IN-/10.13.37.130] 2014-03-31 22:20:12,272 
 StreamResultFuture.java:173 - [Stream #276a9d20-b944-11e3-9f60-4191ec2d6f66] 
 Prepare completed. Receiving 1 files(1726 bytes), sending 0 files(0 bytes)
 INFO  [STREAM-IN-/10.13.37.130] 2014-03-31 22:20:12,280 
 CompressedStreamReader.java:65 - reading file from /10.13.37.130, repairedAt 
 = 0
 {code}
 To reproduce this behaviour, run the 
 [snapshot_test.py:TestSnapshot.test_basic_snapshot 
 dtest|https://github.com/riptano/cassandra-dtest/blob/85692be39cb6b1b5ebc96905834ff7bb174de0bc/snapshot_test.py#L40]
  on 2.1 HEAD.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6350) Timestamp-with-timezone type

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957813#comment-13957813
 ] 

Jonathan Ellis commented on CASSANDRA-6350:
---

Actually I am wrong and that is not how PG's TSTZ support works: 
http://it.toolbox.com/blogs/database-soup/zone-of-misunderstanding-48608

 Timestamp-with-timezone type
 

 Key: CASSANDRA-6350
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6350
 Project: Cassandra
  Issue Type: Wish
  Components: Core
Reporter: Ramkumar S
Priority: Minor
  Labels: lhf

 Create a table with a timestamp column.
 Insert a value from US time Zone.
 Try querying the value from a different time zone like India.
 The timestamp column  value shown in the select query result is converted to 
 Indian Local time instead of showing the actual value.
 This becomes a problem when we want to narrow down the query using where 
 condition.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-6974) Replaying archived commitlogs isn't working

[Ryan McGuire (JIRA)]

Ryan McGuire created CASSANDRA-6974:
---

 Summary: Replaying archived commitlogs isn't working
 Key: CASSANDRA-6974
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6974
 Project: Cassandra
  Issue Type: Bug
Reporter: Ryan McGuire


I have a test for restoring archived commitlogs, which is not working in 2.1 
HEAD.  My commitlogs consist of 30,000 inserts, but system.log indicates there 
were only 2 mutations replayed:

{code}
INFO  [main] 2014-04-02 11:49:54,173 CommitLog.java:115 - Log replay complete, 
2 replayed mutations
{code}

There are several warnings in the logs about bad headers and invalid CRCs: 

{code}
WARN  [main] 2014-04-02 11:49:54,156 CommitLogReplayer.java:138 - Encountered 
bad header at position 0 of commit log /tmp/dtest
-mZIlPE/test/node1/commitlogs/CommitLog-4-1396453793570.log, with invalid CRC. 
The end of segment marker should be zero.
{code}

compare that to the same test run on 2.0, where it replayed many more mutations:

{code}
 INFO [main] 2014-04-02 11:49:04,673 CommitLog.java (line 132) Log replay 
complete, 35960 replayed mutations
{code}

I'll attach the system logs for reference.

[Here is the dtest to reproduce 
this|https://github.com/riptano/cassandra-dtest/blob/master/snapshot_test.py#L75]
 - (This currently relies on the fix for snapshots available in CASSANDRA-6965.)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6350) Timestamp-with-timezone type

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957824#comment-13957824
 ] 

Benedict commented on CASSANDRA-6350:
-

You're more talking about: 
[datetimeoffset|http://msdn.microsoft.com/en-us/library/bb630289.aspx] in SQL 
Server, which lets you store the timezone along with the value.

 Timestamp-with-timezone type
 

 Key: CASSANDRA-6350
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6350
 Project: Cassandra
  Issue Type: Wish
  Components: Core
Reporter: Ramkumar S
Priority: Minor
  Labels: lhf

 Create a table with a timestamp column.
 Insert a value from US time Zone.
 Try querying the value from a different time zone like India.
 The timestamp column  value shown in the select query result is converted to 
 Indian Local time instead of showing the actual value.
 This becomes a problem when we want to narrow down the query using where 
 condition.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6974) Replaying archived commitlogs isn't working

[Ryan McGuire (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire updated CASSANDRA-6974:


Attachment: 2.1.system.log
2.0.system.log

 Replaying archived commitlogs isn't working
 ---

 Key: CASSANDRA-6974
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6974
 Project: Cassandra
  Issue Type: Bug
Reporter: Ryan McGuire
 Attachments: 2.0.system.log, 2.1.system.log


 I have a test for restoring archived commitlogs, which is not working in 2.1 
 HEAD.  My commitlogs consist of 30,000 inserts, but system.log indicates 
 there were only 2 mutations replayed:
 {code}
 INFO  [main] 2014-04-02 11:49:54,173 CommitLog.java:115 - Log replay 
 complete, 2 replayed mutations
 {code}
 There are several warnings in the logs about bad headers and invalid CRCs: 
 {code}
 WARN  [main] 2014-04-02 11:49:54,156 CommitLogReplayer.java:138 - Encountered 
 bad header at position 0 of commit log /tmp/dtest
 -mZIlPE/test/node1/commitlogs/CommitLog-4-1396453793570.log, with invalid 
 CRC. The end of segment marker should be zero.
 {code}
 compare that to the same test run on 2.0, where it replayed many more 
 mutations:
 {code}
  INFO [main] 2014-04-02 11:49:04,673 CommitLog.java (line 132) Log replay 
 complete, 35960 replayed mutations
 {code}
 I'll attach the system logs for reference.
 [Here is the dtest to reproduce 
 this|https://github.com/riptano/cassandra-dtest/blob/master/snapshot_test.py#L75]
  - (This currently relies on the fix for snapshots available in 
 CASSANDRA-6965.)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6825) Slice Queries Can Skip Intersecting SSTables

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-6825:
---

Reviewer: Sylvain Lebresne

 Slice Queries Can Skip Intersecting SSTables
 

 Key: CASSANDRA-6825
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6825
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: quad core Windows7 x64, single node cluster
 Cassandra 2.0.5
Reporter: Bill Mitchell
Assignee: Tyler Hobbs
 Fix For: 2.0.7, 2.1 beta2

 Attachments: 6825-2.0-v2.txt, 6825-2.0.txt, cassandra.log, 
 selectpartitions.zip, selectrowcounts.txt, testdb_1395372407904.zip, 
 testdb_1395372407904.zip


 Investigating another problem, I needed to do COUNT(*) on the several 
 partitions of a table immediately after a test case ran, and I discovered 
 that count(*) on the full table and on each of the partitions returned 
 different counts.  
 In particular case, SELECT COUNT(*) FROM sr LIMIT 100; returned the 
 expected count from the test 9 rows.  The composite primary key splits 
 the logical row into six distinct partitions, and when I issue a query asking 
 for the total across all six partitions, the returned result is only 83999.  
 Drilling down, I find that SELECT * from sr WHERE s = 5 AND l = 11 AND 
 partition = 0; returns 30,000 rows, but a SELECT COUNT(*) with the identical 
 WHERE predicate reports only 14,000. 
 This is failing immediately after running a single small test, such that 
 there are only two SSTables, sr-jb-1 and sr-jb-2.  Compaction never needed to 
 run.  
 In selectrowcounts.txt is a copy of the cqlsh output showing the incorrect 
 count(*) results.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6825) Slice Queries Can Skip Intersecting SSTables

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-6825:
---

Attachment: 6825-2.0-v2.txt

bq. I think you meant something along the lines of a slice of say {{((1, 3), 
)}} with min={{(0, 1)}} and max={{(3, 2)}}

Oops, you're correct there.

Your simplification is good, except that the equality check for the start and 
finish needs to happen after the min/max intersects check.  Non-equal 
start/finish can still fall outside the min/max range.

The v2 patch uses the simplification with that correction.  (The branch is also 
updated.)

 Slice Queries Can Skip Intersecting SSTables
 

 Key: CASSANDRA-6825
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6825
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: quad core Windows7 x64, single node cluster
 Cassandra 2.0.5
Reporter: Bill Mitchell
Assignee: Tyler Hobbs
 Fix For: 2.0.7, 2.1 beta2

 Attachments: 6825-2.0-v2.txt, 6825-2.0.txt, cassandra.log, 
 selectpartitions.zip, selectrowcounts.txt, testdb_1395372407904.zip, 
 testdb_1395372407904.zip


 Investigating another problem, I needed to do COUNT(*) on the several 
 partitions of a table immediately after a test case ran, and I discovered 
 that count(*) on the full table and on each of the partitions returned 
 different counts.  
 In particular case, SELECT COUNT(*) FROM sr LIMIT 100; returned the 
 expected count from the test 9 rows.  The composite primary key splits 
 the logical row into six distinct partitions, and when I issue a query asking 
 for the total across all six partitions, the returned result is only 83999.  
 Drilling down, I find that SELECT * from sr WHERE s = 5 AND l = 11 AND 
 partition = 0; returns 30,000 rows, but a SELECT COUNT(*) with the identical 
 WHERE predicate reports only 14,000. 
 This is failing immediately after running a single small test, such that 
 there are only two SSTables, sr-jb-1 and sr-jb-2.  Compaction never needed to 
 run.  
 In selectrowcounts.txt is a copy of the cqlsh output showing the incorrect 
 count(*) results.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6825) Slice Queries Can Skip Intersecting SSTables

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957845#comment-13957845
 ] 

Sylvain Lebresne commented on CASSANDRA-6825:
-

bq. except that the equality check for the start and finish needs to happen 
after the min/max intersects check

Hum, you're right, it just felt less natural for me to check the equals 
afterwards for some reason, but fair enough, +1 on v2

 Slice Queries Can Skip Intersecting SSTables
 

 Key: CASSANDRA-6825
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6825
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: quad core Windows7 x64, single node cluster
 Cassandra 2.0.5
Reporter: Bill Mitchell
Assignee: Tyler Hobbs
 Fix For: 2.0.7, 2.1 beta2

 Attachments: 6825-2.0-v2.txt, 6825-2.0.txt, cassandra.log, 
 selectpartitions.zip, selectrowcounts.txt, testdb_1395372407904.zip, 
 testdb_1395372407904.zip


 Investigating another problem, I needed to do COUNT(*) on the several 
 partitions of a table immediately after a test case ran, and I discovered 
 that count(*) on the full table and on each of the partitions returned 
 different counts.  
 In particular case, SELECT COUNT(*) FROM sr LIMIT 100; returned the 
 expected count from the test 9 rows.  The composite primary key splits 
 the logical row into six distinct partitions, and when I issue a query asking 
 for the total across all six partitions, the returned result is only 83999.  
 Drilling down, I find that SELECT * from sr WHERE s = 5 AND l = 11 AND 
 partition = 0; returns 30,000 rows, but a SELECT COUNT(*) with the identical 
 WHERE predicate reports only 14,000. 
 This is failing immediately after running a single small test, such that 
 there are only two SSTables, sr-jb-1 and sr-jb-2.  Compaction never needed to 
 run.  
 In selectrowcounts.txt is a copy of the cqlsh output showing the incorrect 
 count(*) results.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6971) nodes not catching up to creation of new keyspace

[Russ Hatch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Russ Hatch updated CASSANDRA-6971:
--

Attachment: node3.log
node2.log
node1.log

attaching logs

 nodes not catching up to creation of new keyspace
 -

 Key: CASSANDRA-6971
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6971
 Project: Cassandra
  Issue Type: Bug
Reporter: Russ Hatch
 Attachments: node1.log, node2.log, node3.log


 The dtest suite is running a test which creates a 3 node cluster, then adds a 
 keyspace and column family. For some reason the 3 nodes are not agreeing on 
 the schema version. The problem is intermittent -- either the nodes all agree 
 on schema quickly, or they seem to stay stuck in limbo.
 The simplest way to reproduce is to run the dtest (simple_increment_test):
 https://github.com/riptano/cassandra-dtest/blob/master/counter_tests.py
 using nosetests:
 {noformat}
 nosetests -vs counter_tests.py:TestCounters.simple_increment_test
 {noformat}
 If the problem is reproduced nose will return this:
 ProgrammingError: Bad Request: Keyspace 'ks' does not exist
 I am not yet sure if the bug is reproducible outside of the dtest suite.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6914) Map element is not allowed in CAS condition with DELETE/UPDATE query

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957880#comment-13957880
 ] 

Tyler Hobbs commented on CASSANDRA-6914:


Overall it looks good.

In ColumnCondition.equalsTo(), this check is wrong:
{code}
else if (other.collectionElement == null || 
!other.collectionElement.bindAndGet(variables).equals(other.collectionElement.bindAndGet(variables)))
{code}

it should be:
{code}
else if (other.collectionElement == null || 
!collectionElement.bindAndGet(variables).equals(other.collectionElement.bindAndGet(other.variables)))
{code}

I'd like to see the error cases tested as well:
* Negative list index
* List index out of range
* Element access on sets
* Element access on non-collection types
* Null element access for maps and sets
* Two non-equal conditions

 Map element is not allowed in CAS condition with DELETE/UPDATE query
 

 Key: CASSANDRA-6914
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6914
 Project: Cassandra
  Issue Type: Bug
Reporter: Dmitriy Ukhlov
Assignee: Sylvain Lebresne
 Fix For: 2.0.7

 Attachments: 6914.txt


 {code}
 CREATE TABLE test (id int, data maptext,text, PRIMARY KEY(id));
 INSERT INTO test (id, data) VALUES (1,{'a':'1'});
 DELETE FROM test WHERE id=1 IF data['a']=null;
 Bad Request: line 1:40 missing EOF at '='
 UPDATE test SET data['b']='2' WHERE id=1 IF data['a']='1';
 Bad Request: line 1:53 missing EOF at '='
 {code}
 These queries was successfuly executed with cassandra 2.0.5, but don't work 
 in 2.0.6 release



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6893) Unintended update with conditional statement

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957883#comment-13957883
 ] 

Aleksey Yeschenko commented on CASSANDRA-6893:
--

All right, after switching castorture to use prepared statements was able to 
repro immediately with default params (w/ 30% of acknowledged writes lost). The 
attached patch fixes the issue. Reviewing now.

 Unintended update with conditional statement
 

 Key: CASSANDRA-6893
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6893
 Project: Cassandra
  Issue Type: Bug
 Environment: Ubuntu Precise 64bit / Cassandra 2.0.6
Reporter: Suguru Namura
Assignee: Sylvain Lebresne
 Fix For: 2.0.7

 Attachments: 6893.txt, ConcurrentCASUpdate.java


 After updated to 2.0.6, I have encountered the strange behavior of 
 conditional updates.
 When I executed CQL like UPDATE test SET value = ? WHERE id = ? IF value = ? 
 in concurrent, sometimes cassandra returns true even if value is not 
 satisfied the condition.
 I have attached the program which reproduce this issue. The program works 
 fine in cassandra 2.0.5. But it seems that resets values while execution in 
 2.0.6.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Fix bad skip of sstables on slice query with composite start/finish

[tylerhobbs]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 e1a90936c - b218536db


Fix bad skip of sstables on slice query with composite start/finish

Patch by Tyler Hobbs; reviewed by Sylvain Lebresne for CASSANDRA-6825


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b218536d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b218536d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b218536d

Branch: refs/heads/cassandra-2.0
Commit: b218536dbc55232a73a81f6288d82ef2b0fa0e6f
Parents: e1a9093
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Apr 2 12:02:47 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Apr 2 12:02:47 2014 -0500

--
 CHANGES.txt |   1 +
 .../cql3/statements/SelectStatement.java|   7 +-
 .../cassandra/db/marshal/CompositeType.java |  40 ++-
 .../service/pager/AbstractQueryPager.java   |  11 +
 .../service/pager/SliceQueryPager.java  |   5 +
 .../cassandra/db/marshal/CompositeTypeTest.java | 294 +++
 6 files changed, 350 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b218536d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 483ee0b..3cc9937 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -40,6 +40,7 @@
  * Fix LIMT with static columns (CASSANDRA-6956)
  * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
  * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
+ * Fix bad skip of sstables on slice query with composite start/finish 
(CASSANDRA-6825)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b218536d/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 53b2c05..56e87e8 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -22,9 +22,7 @@ import java.util.*;
 
 import com.google.common.base.Objects;
 import com.google.common.base.Predicate;
-import com.google.common.collect.AbstractIterator;
 import com.google.common.collect.Iterables;
-import com.google.common.collect.Lists;
 
 import org.github.jamm.MemoryMeter;
 
@@ -53,6 +51,8 @@ import org.apache.cassandra.serializers.MarshalException;
 import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.FBUtilities;
 import org.apache.cassandra.utils.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 /**
  * Encapsulates a completely parsed SELECT query, including the target
@@ -61,6 +61,8 @@ import org.apache.cassandra.utils.Pair;
  */
 public class SelectStatement implements CQLStatement, 
MeasurableForPreparedCache
 {
+private static final Logger logger = 
LoggerFactory.getLogger(SelectStatement.class);
+
 private static final int DEFAULT_COUNT_PAGE_SIZE = 1;
 
 private final int boundTerms;
@@ -257,6 +259,7 @@ public class SelectStatement implements CQLStatement, 
MeasurableForPreparedCache
 while (!pager.isExhausted())
 {
 int maxLimit = pager.maxRemaining();
+logger.debug(New maxLimit for paged count query is {}, maxLimit);
 ResultSet rset = process(pager.fetchPage(pageSize), variables, 
maxLimit, now);
 count += rset.rows.size();
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b218536d/src/java/org/apache/cassandra/db/marshal/CompositeType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/CompositeType.java 
b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
index 83e3b97..7f08219 100644
--- a/src/java/org/apache/cassandra/db/marshal/CompositeType.java
+++ b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
@@ -264,26 +264,54 @@ public class CompositeType extends AbstractCompositeType
 public boolean intersects(ListByteBuffer minColumnNames, 
ListByteBuffer maxColumnNames, SliceQueryFilter filter)
 {
 assert minColumnNames.size() == maxColumnNames.size();
+
+// If any of the slices in the filter intersect, return true
 outer:
 for (ColumnSlice slice : filter.slices)
 {
-// This slices intersects if all component intersect. And we don't 
intersect
-// only 

[jira] [Commented] (CASSANDRA-6818) SSTable references not released if stream session fails before it starts

[Richard Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957886#comment-13957886
 ] 

Richard Low commented on CASSANDRA-6818:


[~yukim] looking better, a couple more issues though:

1. The timeoutExecutor shouldn't be static any more now it's per task
2. When the timeout occurs, inside StreamTransferTask.complete, 
timeoutTask.cancel is called while running the task. It's unclear from the 
java.util.concurrent code how safe this is. Did you test it? If it doesn't work 
a simple solution is to remove the task from timeoutTasks from within the 
runnable before calling complete.

 SSTable references not released if stream session fails before it starts
 

 Key: CASSANDRA-6818
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6818
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Richard Low
Assignee: Yuki Morishita
 Fix For: 1.2.16, 2.0.7, 2.1 beta2

 Attachments: 6818-1.2.txt, 6818-2.0-v2.txt, 6818-2.0-v3.txt, 
 6818-2.0.txt


 I observed a large number of 'orphan' SSTables - SSTables that are in the 
 data directory but not loaded by Cassandra - on a 1.1.12 node that had a 
 large stream fail before it started. These orphan files are particularly 
 dangerous because if the node is restarted and picks up these SSTables it 
 could bring data back to life if tombstones have been GCed. To confirm the 
 SSTables are orphan, I created a snapshot and it didn't contain these files. 
 I can see in the logs that they have been compacted so should have been 
 deleted.
 The log entries for the stream are:
 {{INFO [StreamStage:1] 2014-02-21 19:41:48,742 StreamOut.java (line 115) 
 Beginning transfer to /10.0.0.1}}
 {{INFO [StreamStage:1] 2014-02-21 19:41:48,743 StreamOut.java (line 96) 
 Flushing memtables for [CFS(Keyspace='ks', ColumnFamily='cf1'), 
 CFS(Keyspace='ks', ColumnFamily='cf2')]...}}
 {{ERROR [GossipTasks:1] 2014-02-21 19:41:49,239 AbstractStreamSession.java 
 (line 113) Stream failed because /10.0.0.1 died or was restarted/removed 
 (streams may still be active in background, but further streams won't be 
 started)}}
 {{INFO [StreamStage:1] 2014-02-21 19:41:51,783 StreamOut.java (line 161) 
 Stream context metadata [...] 2267 sstables.}}
 {{INFO [StreamStage:1] 2014-02-21 19:41:51,789 StreamOutSession.java (line 
 182) Streaming to /10.0.0.1}}
 {{INFO [Streaming to /10.0.0.1:1] 2014-02-21 19:42:02,218 FileStreamTask.java 
 (line 99) Found no stream out session at end of file stream task - this is 
 expected if the receiver went down}}
 After digging in the code, here's what I think the issue is:
 1. StreamOutSession.transferRanges() creates a streaming session, which is 
 registered with the failure detector in AbstractStreamSession's constructor.
 2. Memtables are flushed, potentially taking a long time.
 3. The remote node fails, convict() is called and the StreamOutSession is 
 closed. However, at this time StreamOutSession.files is empty because it's 
 still waiting for the memtables to flush.
 4. Memtables finish flusing, references are obtained to SSTables to be 
 streamed and the PendingFiles are added to StreamOutSession.files.
 5. The first stream fails but the StreamOutSession isn't found so is never 
 closed and the references are never released.
 This code is more or less the same on 1.2 so I would expect it to reproduce 
 there. I looked at 2.0 and can't even see where SSTable references are 
 released when the stream fails.
 Some possible fixes for 1.1/1.2:
 1. Don't register with the failure detector until after the PendingFiles are 
 set up. I think this is the behaviour in 2.0 but I don't know if it was done 
 like this to avoid this issue.
 2. Detect the above case in (e.g.) StreamOutSession.begin() by noticing the 
 session has been closed with care to avoid double frees.
 3. Add some synchronization so closeInternal() doesn't race with setting up 
 the session.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6893) Unintended update with conditional statement

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957923#comment-13957923
 ] 

Aleksey Yeschenko commented on CASSANDRA-6893:
--

{code}
private boolean listAppliesTo(CFMetaData cfm, IteratorColumn iter, 
ListByteBuffer elements)
{
for (ByteBuffer e : elements)
if (!iter.hasNext() || iter.next().value().equals(e))
return false;
{code}

should be !iter.next().value().equals(e), I think. Other than that LGTM.

 Unintended update with conditional statement
 

 Key: CASSANDRA-6893
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6893
 Project: Cassandra
  Issue Type: Bug
 Environment: Ubuntu Precise 64bit / Cassandra 2.0.6
Reporter: Suguru Namura
Assignee: Sylvain Lebresne
 Fix For: 2.0.7

 Attachments: 6893.txt, ConcurrentCASUpdate.java


 After updated to 2.0.6, I have encountered the strange behavior of 
 conditional updates.
 When I executed CQL like UPDATE test SET value = ? WHERE id = ? IF value = ? 
 in concurrent, sometimes cassandra returns true even if value is not 
 satisfied the condition.
 I have attached the program which reproduce this issue. The program works 
 fine in cassandra 2.0.5. But it seems that resets values while execution in 
 2.0.6.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Fix unintended update with conditional statement

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 b218536db - 167380fb0


Fix unintended update with conditional statement

patch by slebresne; reviewed by iamaleksey for CASSANDRA-6893


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/167380fb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/167380fb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/167380fb

Branch: refs/heads/cassandra-2.0
Commit: 167380fb0d7fa3fc6dc9879c839419babaea2a16
Parents: b218536
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 19:49:16 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 19:49:16 2014 +0200

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  | 198 ++-
 .../cql3/statements/CQL3CasConditions.java  |  11 +-
 3 files changed, 114 insertions(+), 96 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/167380fb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3cc9937..66196d0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -41,6 +41,7 @@
  * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
  * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
  * Fix bad skip of sstables on slice query with composite start/finish 
(CASSANDRA-6825)
+ * Fix unintended update with conditional statement (CASSANDRA-6893)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/167380fb/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index ce44c3b..0ebd4af 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -39,8 +39,6 @@ public class ColumnCondition
 public final CFDefinition.Name column;
 private final Term value;
 
-private ListByteBuffer variables;
-
 private ColumnCondition(CFDefinition.Name column, Term value)
 {
 this.column = column;
@@ -53,13 +51,6 @@ public class ColumnCondition
 return new ColumnCondition(column, value);
 }
 
-// See CQL3CasConditions for why it's convenient to have this
-public ColumnCondition attach(ListByteBuffer variables)
-{
-this.variables = variables;
-return this;
-}
-
 /**
  * Collects the column specification for the bind variables of this 
operation.
  *
@@ -71,109 +62,134 @@ public class ColumnCondition
 value.collectMarkerSpecification(boundNames);
 }
 
-// Not overriding equals() because we need the variables to have been 
attached when this is
-// called and so having a non standard method name might help avoid 
mistakes
-public boolean equalsTo(ColumnCondition other) throws 
InvalidRequestException
+public ColumnCondition.WithVariables with(ListByteBuffer variables)
 {
-return column.equals(other.column)
- 
value.bindAndGet(variables).equals(other.value.bindAndGet(other.variables));
+return new WithVariables(variables);
 }
 
-private ColumnNameBuilder copyOrUpdatePrefix(CFMetaData cfm, 
ColumnNameBuilder rowPrefix)
+public class WithVariables
 {
-return column.kind == CFDefinition.Name.Kind.STATIC ? 
cfm.getStaticColumnNameBuilder() : rowPrefix.copy();
-}
+private final ListByteBuffer variables;
 
-/**
- * Validates whether this condition applies to {@code current}.
- */
-public boolean appliesTo(ColumnNameBuilder rowPrefix, ColumnFamily 
current, long now) throws InvalidRequestException
-{
-if (column.type instanceof CollectionType)
-return collectionAppliesTo((CollectionType)column.type, rowPrefix, 
current, now);
-
-ColumnNameBuilder prefix = copyOrUpdatePrefix(current.metadata(), 
rowPrefix);
-ByteBuffer columnName = column.kind == 
CFDefinition.Name.Kind.VALUE_ALIAS
-  ? prefix.build()
-  : prefix.add(column.name.key).build();
-
-Column c = current.getColumn(columnName);
-ByteBuffer v = value.bindAndGet(variables);
-return v == null
- ? c == null || !c.isLive(now)
- : c != null  c.isLive(now)  c.value().equals(v);
-}
+private WithVariables(ListByteBuffer variables)
+{
+this.variables = variables;
+  

[1/2] git commit: Fix bad skip of sstables on slice query with composite start/finish

[tylerhobbs]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 486cc4cc0 - ab1a02cfa


Fix bad skip of sstables on slice query with composite start/finish

Patch by Tyler Hobbs; reviewed by Sylvain Lebresne for CASSANDRA-6825


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b218536d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b218536d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b218536d

Branch: refs/heads/cassandra-2.1
Commit: b218536dbc55232a73a81f6288d82ef2b0fa0e6f
Parents: e1a9093
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Apr 2 12:02:47 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Apr 2 12:02:47 2014 -0500

--
 CHANGES.txt |   1 +
 .../cql3/statements/SelectStatement.java|   7 +-
 .../cassandra/db/marshal/CompositeType.java |  40 ++-
 .../service/pager/AbstractQueryPager.java   |  11 +
 .../service/pager/SliceQueryPager.java  |   5 +
 .../cassandra/db/marshal/CompositeTypeTest.java | 294 +++
 6 files changed, 350 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b218536d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 483ee0b..3cc9937 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -40,6 +40,7 @@
  * Fix LIMT with static columns (CASSANDRA-6956)
  * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
  * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
+ * Fix bad skip of sstables on slice query with composite start/finish 
(CASSANDRA-6825)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b218536d/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 53b2c05..56e87e8 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -22,9 +22,7 @@ import java.util.*;
 
 import com.google.common.base.Objects;
 import com.google.common.base.Predicate;
-import com.google.common.collect.AbstractIterator;
 import com.google.common.collect.Iterables;
-import com.google.common.collect.Lists;
 
 import org.github.jamm.MemoryMeter;
 
@@ -53,6 +51,8 @@ import org.apache.cassandra.serializers.MarshalException;
 import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.FBUtilities;
 import org.apache.cassandra.utils.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 /**
  * Encapsulates a completely parsed SELECT query, including the target
@@ -61,6 +61,8 @@ import org.apache.cassandra.utils.Pair;
  */
 public class SelectStatement implements CQLStatement, 
MeasurableForPreparedCache
 {
+private static final Logger logger = 
LoggerFactory.getLogger(SelectStatement.class);
+
 private static final int DEFAULT_COUNT_PAGE_SIZE = 1;
 
 private final int boundTerms;
@@ -257,6 +259,7 @@ public class SelectStatement implements CQLStatement, 
MeasurableForPreparedCache
 while (!pager.isExhausted())
 {
 int maxLimit = pager.maxRemaining();
+logger.debug(New maxLimit for paged count query is {}, maxLimit);
 ResultSet rset = process(pager.fetchPage(pageSize), variables, 
maxLimit, now);
 count += rset.rows.size();
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b218536d/src/java/org/apache/cassandra/db/marshal/CompositeType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/CompositeType.java 
b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
index 83e3b97..7f08219 100644
--- a/src/java/org/apache/cassandra/db/marshal/CompositeType.java
+++ b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
@@ -264,26 +264,54 @@ public class CompositeType extends AbstractCompositeType
 public boolean intersects(ListByteBuffer minColumnNames, 
ListByteBuffer maxColumnNames, SliceQueryFilter filter)
 {
 assert minColumnNames.size() == maxColumnNames.size();
+
+// If any of the slices in the filter intersect, return true
 outer:
 for (ColumnSlice slice : filter.slices)
 {
-// This slices intersects if all component intersect. And we don't 
intersect
-// only 

[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[tylerhobbs]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
src/java/org/apache/cassandra/db/marshal/CompositeType.java
test/unit/org/apache/cassandra/db/marshal/CompositeTypeTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ab1a02cf
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ab1a02cf
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ab1a02cf

Branch: refs/heads/cassandra-2.1
Commit: ab1a02cfa69e0c57579b312202b477e43bb90b45
Parents: 486cc4c b218536
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Apr 2 13:17:18 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Apr 2 13:17:18 2014 -0500

--
 CHANGES.txt |   1 +
 .../cql3/statements/SelectStatement.java|   7 +-
 .../apache/cassandra/db/filter/ColumnSlice.java |  32 +-
 .../service/pager/AbstractQueryPager.java   |  11 +
 .../service/pager/SliceQueryPager.java  |   5 +
 .../cassandra/db/filter/ColumnSliceTest.java| 290 +++
 6 files changed, 342 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab1a02cf/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab1a02cf/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --cc src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index edc7f5c,56e87e8..7ca7d93
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@@ -22,9 -22,8 +22,10 @@@ import java.util.*
  
  import com.google.common.base.Objects;
  import com.google.common.base.Predicate;
 +import com.google.common.collect.AbstractIterator;
  import com.google.common.collect.Iterables;
 +import com.google.common.collect.Iterators;
+ 
  import org.github.jamm.MemoryMeter;
  
  import org.apache.cassandra.auth.Permission;
@@@ -38,7 -37,7 +39,6 @@@ import org.apache.cassandra.db.filter.*
  import org.apache.cassandra.db.marshal.*;
  import org.apache.cassandra.dht.*;
  import org.apache.cassandra.exceptions.*;
--import org.apache.cassandra.net.MessagingService;
  import org.apache.cassandra.service.ClientState;
  import org.apache.cassandra.service.QueryState;
  import org.apache.cassandra.service.StorageProxy;
@@@ -49,6 -50,9 +49,8 @@@ import org.apache.cassandra.thrift.Thri
  import org.apache.cassandra.serializers.MarshalException;
  import org.apache.cassandra.utils.ByteBufferUtil;
  import org.apache.cassandra.utils.FBUtilities;
 -import org.apache.cassandra.utils.Pair;
+ import org.slf4j.Logger;
+ import org.slf4j.LoggerFactory;
  
  /**
   * Encapsulates a completely parsed SELECT query, including the target

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab1a02cf/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
--
diff --cc src/java/org/apache/cassandra/db/filter/ColumnSlice.java
index 262ebae,9eff12a..3838ee5
--- a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
@@@ -52,39 -47,22 +52,65 @@@ public class ColumnSlic
  this.finish = finish;
  }
  
 -public boolean isAlwaysEmpty(AbstractType? comparator, boolean reversed)
 +public boolean isAlwaysEmpty(CellNameType comparator, boolean reversed)
  {
 -ComparatorByteBuffer orderedComparator = reversed ? 
comparator.reverseComparator : comparator;
 -return (start.remaining()  0  finish.remaining()  0  
orderedComparator.compare(start, finish)  0);
 +ComparatorComposite orderedComparator = reversed ? 
comparator.reverseComparator() : comparator;
 +return !start.isEmpty()  !finish.isEmpty()  
orderedComparator.compare(start, finish)  0;
  }
  
 -public boolean includes(ComparatorByteBuffer cmp, ByteBuffer name)
 +public boolean includes(ComparatorComposite cmp, Composite name)
  {
 -return cmp.compare(start, name) = 0  
(finish.equals(ByteBufferUtil.EMPTY_BYTE_BUFFER) || cmp.compare(finish, name) 
= 0);
 +return cmp.compare(start, name) = 0  (finish.isEmpty() || 
cmp.compare(finish, name) = 0);
 +}
 +
 +public boolean isBefore(ComparatorComposite cmp, Composite name)
 +{
 +return !finish.isEmpty()  cmp.compare(finish, name)  0;
 +}
 +
 +public boolean intersects(ListByteBuffer minCellNames, ListByteBuffer 
maxCellNames, CellNameType comparator, boolean reversed)
 +{
 +assert 

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

[tylerhobbs]

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/500503d6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/500503d6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/500503d6

Branch: refs/heads/trunk
Commit: 500503d6d573eac02dba8332d13ac502648b6ac7
Parents: 3e0dac5 ab1a02c
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Apr 2 13:17:54 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Apr 2 13:17:54 2014 -0500

--
 CHANGES.txt |   1 +
 .../cql3/statements/SelectStatement.java|   7 +-
 .../apache/cassandra/db/filter/ColumnSlice.java |  32 +-
 .../service/pager/AbstractQueryPager.java   |  11 +
 .../service/pager/SliceQueryPager.java  |   5 +
 .../cassandra/db/filter/ColumnSliceTest.java| 290 +++
 6 files changed, 342 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/500503d6/CHANGES.txt
--

[1/3] git commit: Fix bad skip of sstables on slice query with composite start/finish

[tylerhobbs]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 3e0dac5a4 - 500503d6d


Fix bad skip of sstables on slice query with composite start/finish

Patch by Tyler Hobbs; reviewed by Sylvain Lebresne for CASSANDRA-6825


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b218536d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b218536d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b218536d

Branch: refs/heads/trunk
Commit: b218536dbc55232a73a81f6288d82ef2b0fa0e6f
Parents: e1a9093
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Apr 2 12:02:47 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Apr 2 12:02:47 2014 -0500

--
 CHANGES.txt |   1 +
 .../cql3/statements/SelectStatement.java|   7 +-
 .../cassandra/db/marshal/CompositeType.java |  40 ++-
 .../service/pager/AbstractQueryPager.java   |  11 +
 .../service/pager/SliceQueryPager.java  |   5 +
 .../cassandra/db/marshal/CompositeTypeTest.java | 294 +++
 6 files changed, 350 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b218536d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 483ee0b..3cc9937 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -40,6 +40,7 @@
  * Fix LIMT with static columns (CASSANDRA-6956)
  * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
  * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
+ * Fix bad skip of sstables on slice query with composite start/finish 
(CASSANDRA-6825)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b218536d/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 53b2c05..56e87e8 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -22,9 +22,7 @@ import java.util.*;
 
 import com.google.common.base.Objects;
 import com.google.common.base.Predicate;
-import com.google.common.collect.AbstractIterator;
 import com.google.common.collect.Iterables;
-import com.google.common.collect.Lists;
 
 import org.github.jamm.MemoryMeter;
 
@@ -53,6 +51,8 @@ import org.apache.cassandra.serializers.MarshalException;
 import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.FBUtilities;
 import org.apache.cassandra.utils.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 /**
  * Encapsulates a completely parsed SELECT query, including the target
@@ -61,6 +61,8 @@ import org.apache.cassandra.utils.Pair;
  */
 public class SelectStatement implements CQLStatement, 
MeasurableForPreparedCache
 {
+private static final Logger logger = 
LoggerFactory.getLogger(SelectStatement.class);
+
 private static final int DEFAULT_COUNT_PAGE_SIZE = 1;
 
 private final int boundTerms;
@@ -257,6 +259,7 @@ public class SelectStatement implements CQLStatement, 
MeasurableForPreparedCache
 while (!pager.isExhausted())
 {
 int maxLimit = pager.maxRemaining();
+logger.debug(New maxLimit for paged count query is {}, maxLimit);
 ResultSet rset = process(pager.fetchPage(pageSize), variables, 
maxLimit, now);
 count += rset.rows.size();
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b218536d/src/java/org/apache/cassandra/db/marshal/CompositeType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/CompositeType.java 
b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
index 83e3b97..7f08219 100644
--- a/src/java/org/apache/cassandra/db/marshal/CompositeType.java
+++ b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
@@ -264,26 +264,54 @@ public class CompositeType extends AbstractCompositeType
 public boolean intersects(ListByteBuffer minColumnNames, 
ListByteBuffer maxColumnNames, SliceQueryFilter filter)
 {
 assert minColumnNames.size() == maxColumnNames.size();
+
+// If any of the slices in the filter intersect, return true
 outer:
 for (ColumnSlice slice : filter.slices)
 {
-// This slices intersects if all component intersect. And we don't 
intersect
-// only if no slice 

[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[tylerhobbs]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
src/java/org/apache/cassandra/db/marshal/CompositeType.java
test/unit/org/apache/cassandra/db/marshal/CompositeTypeTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ab1a02cf
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ab1a02cf
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ab1a02cf

Branch: refs/heads/trunk
Commit: ab1a02cfa69e0c57579b312202b477e43bb90b45
Parents: 486cc4c b218536
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Apr 2 13:17:18 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Apr 2 13:17:18 2014 -0500

--
 CHANGES.txt |   1 +
 .../cql3/statements/SelectStatement.java|   7 +-
 .../apache/cassandra/db/filter/ColumnSlice.java |  32 +-
 .../service/pager/AbstractQueryPager.java   |  11 +
 .../service/pager/SliceQueryPager.java  |   5 +
 .../cassandra/db/filter/ColumnSliceTest.java| 290 +++
 6 files changed, 342 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab1a02cf/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab1a02cf/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --cc src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index edc7f5c,56e87e8..7ca7d93
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@@ -22,9 -22,8 +22,10 @@@ import java.util.*
  
  import com.google.common.base.Objects;
  import com.google.common.base.Predicate;
 +import com.google.common.collect.AbstractIterator;
  import com.google.common.collect.Iterables;
 +import com.google.common.collect.Iterators;
+ 
  import org.github.jamm.MemoryMeter;
  
  import org.apache.cassandra.auth.Permission;
@@@ -38,7 -37,7 +39,6 @@@ import org.apache.cassandra.db.filter.*
  import org.apache.cassandra.db.marshal.*;
  import org.apache.cassandra.dht.*;
  import org.apache.cassandra.exceptions.*;
--import org.apache.cassandra.net.MessagingService;
  import org.apache.cassandra.service.ClientState;
  import org.apache.cassandra.service.QueryState;
  import org.apache.cassandra.service.StorageProxy;
@@@ -49,6 -50,9 +49,8 @@@ import org.apache.cassandra.thrift.Thri
  import org.apache.cassandra.serializers.MarshalException;
  import org.apache.cassandra.utils.ByteBufferUtil;
  import org.apache.cassandra.utils.FBUtilities;
 -import org.apache.cassandra.utils.Pair;
+ import org.slf4j.Logger;
+ import org.slf4j.LoggerFactory;
  
  /**
   * Encapsulates a completely parsed SELECT query, including the target

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab1a02cf/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
--
diff --cc src/java/org/apache/cassandra/db/filter/ColumnSlice.java
index 262ebae,9eff12a..3838ee5
--- a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
@@@ -52,39 -47,22 +52,65 @@@ public class ColumnSlic
  this.finish = finish;
  }
  
 -public boolean isAlwaysEmpty(AbstractType? comparator, boolean reversed)
 +public boolean isAlwaysEmpty(CellNameType comparator, boolean reversed)
  {
 -ComparatorByteBuffer orderedComparator = reversed ? 
comparator.reverseComparator : comparator;
 -return (start.remaining()  0  finish.remaining()  0  
orderedComparator.compare(start, finish)  0);
 +ComparatorComposite orderedComparator = reversed ? 
comparator.reverseComparator() : comparator;
 +return !start.isEmpty()  !finish.isEmpty()  
orderedComparator.compare(start, finish)  0;
  }
  
 -public boolean includes(ComparatorByteBuffer cmp, ByteBuffer name)
 +public boolean includes(ComparatorComposite cmp, Composite name)
  {
 -return cmp.compare(start, name) = 0  
(finish.equals(ByteBufferUtil.EMPTY_BYTE_BUFFER) || cmp.compare(finish, name) 
= 0);
 +return cmp.compare(start, name) = 0  (finish.isEmpty() || 
cmp.compare(finish, name) = 0);
 +}
 +
 +public boolean isBefore(ComparatorComposite cmp, Composite name)
 +{
 +return !finish.isEmpty()  cmp.compare(finish, name)  0;
 +}
 +
 +public boolean intersects(ListByteBuffer minCellNames, ListByteBuffer 
maxCellNames, CellNameType comparator, boolean reversed)
 +{
 +assert minCellNames.size() == 

git commit: Replace equals by type compare

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 167380fb0 - 17278b3a3


Replace equals by type compare


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/17278b3a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/17278b3a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/17278b3a

Branch: refs/heads/cassandra-2.0
Commit: 17278b3a343575f4c32414cfffb52cdc2e7c7d00
Parents: 167380f
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 20:14:42 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 20:14:42 2014 +0200

--
 .../apache/cassandra/cql3/ColumnCondition.java  | 27 ++--
 1 file changed, 14 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/17278b3a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index 0ebd4af..e6cdf43 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -26,9 +26,7 @@ import com.google.common.collect.Iterators;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.db.*;
 import org.apache.cassandra.db.filter.ColumnSlice;
-import org.apache.cassandra.db.marshal.CollectionType;
-import org.apache.cassandra.db.marshal.CounterColumnType;
-import org.apache.cassandra.db.marshal.CompositeType;
+import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.exceptions.InvalidRequestException;
 
 /**
@@ -140,9 +138,9 @@ public class ColumnCondition
 
 switch (type.kind)
 {
-case LIST: return listAppliesTo(current.metadata(), iter, 
((Lists.Value)v).elements);
-case SET: return setAppliesTo(current.metadata(), iter, 
((Sets.Value)v).elements);
-case MAP: return mapAppliesTo(current.metadata(), iter, 
((Maps.Value)v).map);
+case LIST: return listAppliesTo((ListType)type, 
current.metadata(), iter, ((Lists.Value)v).elements);
+case SET: return setAppliesTo((SetType)type, 
current.metadata(), iter, ((Sets.Value)v).elements);
+case MAP: return mapAppliesTo((MapType)type, 
current.metadata(), iter, ((Maps.Value)v).map);
 }
 throw new AssertionError();
 }
@@ -153,18 +151,19 @@ public class ColumnCondition
 return bbs[bbs.length - 1];
 }
 
-private boolean listAppliesTo(CFMetaData cfm, IteratorColumn iter, 
ListByteBuffer elements)
+private boolean listAppliesTo(ListType type, CFMetaData cfm, 
IteratorColumn iter, ListByteBuffer elements)
 {
 for (ByteBuffer e : elements)
-if (!iter.hasNext() || !iter.next().value().equals(e))
+if (!iter.hasNext() || 
type.elements.compare(iter.next().value(), e) != 0)
 return false;
 // We must not have more elements than expected
 return !iter.hasNext();
 }
 
-private boolean setAppliesTo(CFMetaData cfm, IteratorColumn iter, 
SetByteBuffer elements)
+private boolean setAppliesTo(SetType type, CFMetaData cfm, 
IteratorColumn iter, SetByteBuffer elements)
 {
-SetByteBuffer remaining = new HashSet(elements);
+SetByteBuffer remaining = new TreeSet(type.elements);
+remaining.addAll(elements);
 while (iter.hasNext())
 {
 if (remaining.isEmpty())
@@ -176,16 +175,18 @@ public class ColumnCondition
 return remaining.isEmpty();
 }
 
-private boolean mapAppliesTo(CFMetaData cfm, IteratorColumn iter, 
MapByteBuffer, ByteBuffer elements)
+private boolean mapAppliesTo(MapType type, CFMetaData cfm, 
IteratorColumn iter, MapByteBuffer, ByteBuffer elements)
 {
-MapByteBuffer, ByteBuffer remaining = new HashMap(elements);
+MapByteBuffer, ByteBuffer remaining = new TreeMap(type.keys);
+remaining.putAll(elements);
 while (iter.hasNext())
 {
 if (remaining.isEmpty())
 return false;
 
 Column c = iter.next();
-if (!remaining.remove(collectionKey(cfm, c)).equals(c.value()))
+ByteBuffer previous = remaining.remove(collectionKey(cfm, c));
+if (previous == null || type.values.compare(previous, 
c.value()) != 0)
 return false;
 }
 return remaining.isEmpty();

[2/3] git commit: Replace equals by type compare

[slebresne]

Replace equals by type compare


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/17278b3a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/17278b3a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/17278b3a

Branch: refs/heads/cassandra-2.1
Commit: 17278b3a343575f4c32414cfffb52cdc2e7c7d00
Parents: 167380f
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 20:14:42 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 20:14:42 2014 +0200

--
 .../apache/cassandra/cql3/ColumnCondition.java  | 27 ++--
 1 file changed, 14 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/17278b3a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index 0ebd4af..e6cdf43 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -26,9 +26,7 @@ import com.google.common.collect.Iterators;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.db.*;
 import org.apache.cassandra.db.filter.ColumnSlice;
-import org.apache.cassandra.db.marshal.CollectionType;
-import org.apache.cassandra.db.marshal.CounterColumnType;
-import org.apache.cassandra.db.marshal.CompositeType;
+import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.exceptions.InvalidRequestException;
 
 /**
@@ -140,9 +138,9 @@ public class ColumnCondition
 
 switch (type.kind)
 {
-case LIST: return listAppliesTo(current.metadata(), iter, 
((Lists.Value)v).elements);
-case SET: return setAppliesTo(current.metadata(), iter, 
((Sets.Value)v).elements);
-case MAP: return mapAppliesTo(current.metadata(), iter, 
((Maps.Value)v).map);
+case LIST: return listAppliesTo((ListType)type, 
current.metadata(), iter, ((Lists.Value)v).elements);
+case SET: return setAppliesTo((SetType)type, 
current.metadata(), iter, ((Sets.Value)v).elements);
+case MAP: return mapAppliesTo((MapType)type, 
current.metadata(), iter, ((Maps.Value)v).map);
 }
 throw new AssertionError();
 }
@@ -153,18 +151,19 @@ public class ColumnCondition
 return bbs[bbs.length - 1];
 }
 
-private boolean listAppliesTo(CFMetaData cfm, IteratorColumn iter, 
ListByteBuffer elements)
+private boolean listAppliesTo(ListType type, CFMetaData cfm, 
IteratorColumn iter, ListByteBuffer elements)
 {
 for (ByteBuffer e : elements)
-if (!iter.hasNext() || !iter.next().value().equals(e))
+if (!iter.hasNext() || 
type.elements.compare(iter.next().value(), e) != 0)
 return false;
 // We must not have more elements than expected
 return !iter.hasNext();
 }
 
-private boolean setAppliesTo(CFMetaData cfm, IteratorColumn iter, 
SetByteBuffer elements)
+private boolean setAppliesTo(SetType type, CFMetaData cfm, 
IteratorColumn iter, SetByteBuffer elements)
 {
-SetByteBuffer remaining = new HashSet(elements);
+SetByteBuffer remaining = new TreeSet(type.elements);
+remaining.addAll(elements);
 while (iter.hasNext())
 {
 if (remaining.isEmpty())
@@ -176,16 +175,18 @@ public class ColumnCondition
 return remaining.isEmpty();
 }
 
-private boolean mapAppliesTo(CFMetaData cfm, IteratorColumn iter, 
MapByteBuffer, ByteBuffer elements)
+private boolean mapAppliesTo(MapType type, CFMetaData cfm, 
IteratorColumn iter, MapByteBuffer, ByteBuffer elements)
 {
-MapByteBuffer, ByteBuffer remaining = new HashMap(elements);
+MapByteBuffer, ByteBuffer remaining = new TreeMap(type.keys);
+remaining.putAll(elements);
 while (iter.hasNext())
 {
 if (remaining.isEmpty())
 return false;
 
 Column c = iter.next();
-if (!remaining.remove(collectionKey(cfm, c)).equals(c.value()))
+ByteBuffer previous = remaining.remove(collectionKey(cfm, c));
+if (previous == null || type.values.compare(previous, 
c.value()) != 0)
 return false;
 }
 return remaining.isEmpty();

[4/4] git commit: Merge branch 'cassandra-2.1' into trunk

[slebresne]

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5b85be04
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5b85be04
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5b85be04

Branch: refs/heads/trunk
Commit: 5b85be04cbf49e306cd43563ee85652484179812
Parents: 500503d f7e690d
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 20:32:49 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 20:32:49 2014 +0200

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  | 176 ++-
 .../cql3/statements/CQL3CasConditions.java  |  11 +-
 3 files changed, 104 insertions(+), 84 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5b85be04/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5b85be04/src/java/org/apache/cassandra/cql3/statements/CQL3CasConditions.java
--

[1/3] git commit: Fix unintended update with conditional statement

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 ab1a02cfa - f7e690d7f


Fix unintended update with conditional statement

patch by slebresne; reviewed by iamaleksey for CASSANDRA-6893


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/167380fb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/167380fb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/167380fb

Branch: refs/heads/cassandra-2.1
Commit: 167380fb0d7fa3fc6dc9879c839419babaea2a16
Parents: b218536
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 19:49:16 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 19:49:16 2014 +0200

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  | 198 ++-
 .../cql3/statements/CQL3CasConditions.java  |  11 +-
 3 files changed, 114 insertions(+), 96 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/167380fb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3cc9937..66196d0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -41,6 +41,7 @@
  * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
  * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
  * Fix bad skip of sstables on slice query with composite start/finish 
(CASSANDRA-6825)
+ * Fix unintended update with conditional statement (CASSANDRA-6893)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/167380fb/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index ce44c3b..0ebd4af 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -39,8 +39,6 @@ public class ColumnCondition
 public final CFDefinition.Name column;
 private final Term value;
 
-private ListByteBuffer variables;
-
 private ColumnCondition(CFDefinition.Name column, Term value)
 {
 this.column = column;
@@ -53,13 +51,6 @@ public class ColumnCondition
 return new ColumnCondition(column, value);
 }
 
-// See CQL3CasConditions for why it's convenient to have this
-public ColumnCondition attach(ListByteBuffer variables)
-{
-this.variables = variables;
-return this;
-}
-
 /**
  * Collects the column specification for the bind variables of this 
operation.
  *
@@ -71,109 +62,134 @@ public class ColumnCondition
 value.collectMarkerSpecification(boundNames);
 }
 
-// Not overriding equals() because we need the variables to have been 
attached when this is
-// called and so having a non standard method name might help avoid 
mistakes
-public boolean equalsTo(ColumnCondition other) throws 
InvalidRequestException
+public ColumnCondition.WithVariables with(ListByteBuffer variables)
 {
-return column.equals(other.column)
- 
value.bindAndGet(variables).equals(other.value.bindAndGet(other.variables));
+return new WithVariables(variables);
 }
 
-private ColumnNameBuilder copyOrUpdatePrefix(CFMetaData cfm, 
ColumnNameBuilder rowPrefix)
+public class WithVariables
 {
-return column.kind == CFDefinition.Name.Kind.STATIC ? 
cfm.getStaticColumnNameBuilder() : rowPrefix.copy();
-}
+private final ListByteBuffer variables;
 
-/**
- * Validates whether this condition applies to {@code current}.
- */
-public boolean appliesTo(ColumnNameBuilder rowPrefix, ColumnFamily 
current, long now) throws InvalidRequestException
-{
-if (column.type instanceof CollectionType)
-return collectionAppliesTo((CollectionType)column.type, rowPrefix, 
current, now);
-
-ColumnNameBuilder prefix = copyOrUpdatePrefix(current.metadata(), 
rowPrefix);
-ByteBuffer columnName = column.kind == 
CFDefinition.Name.Kind.VALUE_ALIAS
-  ? prefix.build()
-  : prefix.add(column.name.key).build();
-
-Column c = current.getColumn(columnName);
-ByteBuffer v = value.bindAndGet(variables);
-return v == null
- ? c == null || !c.isLive(now)
- : c != null  c.isLive(now)  c.value().equals(v);
-}
+private WithVariables(ListByteBuffer variables)
+{
+this.variables = variables;
+  

[2/4] git commit: Replace equals by type compare

[slebresne]

Replace equals by type compare


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/17278b3a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/17278b3a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/17278b3a

Branch: refs/heads/trunk
Commit: 17278b3a343575f4c32414cfffb52cdc2e7c7d00
Parents: 167380f
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 20:14:42 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 20:14:42 2014 +0200

--
 .../apache/cassandra/cql3/ColumnCondition.java  | 27 ++--
 1 file changed, 14 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/17278b3a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index 0ebd4af..e6cdf43 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -26,9 +26,7 @@ import com.google.common.collect.Iterators;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.db.*;
 import org.apache.cassandra.db.filter.ColumnSlice;
-import org.apache.cassandra.db.marshal.CollectionType;
-import org.apache.cassandra.db.marshal.CounterColumnType;
-import org.apache.cassandra.db.marshal.CompositeType;
+import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.exceptions.InvalidRequestException;
 
 /**
@@ -140,9 +138,9 @@ public class ColumnCondition
 
 switch (type.kind)
 {
-case LIST: return listAppliesTo(current.metadata(), iter, 
((Lists.Value)v).elements);
-case SET: return setAppliesTo(current.metadata(), iter, 
((Sets.Value)v).elements);
-case MAP: return mapAppliesTo(current.metadata(), iter, 
((Maps.Value)v).map);
+case LIST: return listAppliesTo((ListType)type, 
current.metadata(), iter, ((Lists.Value)v).elements);
+case SET: return setAppliesTo((SetType)type, 
current.metadata(), iter, ((Sets.Value)v).elements);
+case MAP: return mapAppliesTo((MapType)type, 
current.metadata(), iter, ((Maps.Value)v).map);
 }
 throw new AssertionError();
 }
@@ -153,18 +151,19 @@ public class ColumnCondition
 return bbs[bbs.length - 1];
 }
 
-private boolean listAppliesTo(CFMetaData cfm, IteratorColumn iter, 
ListByteBuffer elements)
+private boolean listAppliesTo(ListType type, CFMetaData cfm, 
IteratorColumn iter, ListByteBuffer elements)
 {
 for (ByteBuffer e : elements)
-if (!iter.hasNext() || !iter.next().value().equals(e))
+if (!iter.hasNext() || 
type.elements.compare(iter.next().value(), e) != 0)
 return false;
 // We must not have more elements than expected
 return !iter.hasNext();
 }
 
-private boolean setAppliesTo(CFMetaData cfm, IteratorColumn iter, 
SetByteBuffer elements)
+private boolean setAppliesTo(SetType type, CFMetaData cfm, 
IteratorColumn iter, SetByteBuffer elements)
 {
-SetByteBuffer remaining = new HashSet(elements);
+SetByteBuffer remaining = new TreeSet(type.elements);
+remaining.addAll(elements);
 while (iter.hasNext())
 {
 if (remaining.isEmpty())
@@ -176,16 +175,18 @@ public class ColumnCondition
 return remaining.isEmpty();
 }
 
-private boolean mapAppliesTo(CFMetaData cfm, IteratorColumn iter, 
MapByteBuffer, ByteBuffer elements)
+private boolean mapAppliesTo(MapType type, CFMetaData cfm, 
IteratorColumn iter, MapByteBuffer, ByteBuffer elements)
 {
-MapByteBuffer, ByteBuffer remaining = new HashMap(elements);
+MapByteBuffer, ByteBuffer remaining = new TreeMap(type.keys);
+remaining.putAll(elements);
 while (iter.hasNext())
 {
 if (remaining.isEmpty())
 return false;
 
 Column c = iter.next();
-if (!remaining.remove(collectionKey(cfm, c)).equals(c.value()))
+ByteBuffer previous = remaining.remove(collectionKey(cfm, c));
+if (previous == null || type.values.compare(previous, 
c.value()) != 0)
 return false;
 }
 return remaining.isEmpty();

[1/4] git commit: Fix unintended update with conditional statement

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 500503d6d - 5b85be04c


Fix unintended update with conditional statement

patch by slebresne; reviewed by iamaleksey for CASSANDRA-6893


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/167380fb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/167380fb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/167380fb

Branch: refs/heads/trunk
Commit: 167380fb0d7fa3fc6dc9879c839419babaea2a16
Parents: b218536
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 19:49:16 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 19:49:16 2014 +0200

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  | 198 ++-
 .../cql3/statements/CQL3CasConditions.java  |  11 +-
 3 files changed, 114 insertions(+), 96 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/167380fb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3cc9937..66196d0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -41,6 +41,7 @@
  * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
  * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
  * Fix bad skip of sstables on slice query with composite start/finish 
(CASSANDRA-6825)
+ * Fix unintended update with conditional statement (CASSANDRA-6893)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/167380fb/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index ce44c3b..0ebd4af 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -39,8 +39,6 @@ public class ColumnCondition
 public final CFDefinition.Name column;
 private final Term value;
 
-private ListByteBuffer variables;
-
 private ColumnCondition(CFDefinition.Name column, Term value)
 {
 this.column = column;
@@ -53,13 +51,6 @@ public class ColumnCondition
 return new ColumnCondition(column, value);
 }
 
-// See CQL3CasConditions for why it's convenient to have this
-public ColumnCondition attach(ListByteBuffer variables)
-{
-this.variables = variables;
-return this;
-}
-
 /**
  * Collects the column specification for the bind variables of this 
operation.
  *
@@ -71,109 +62,134 @@ public class ColumnCondition
 value.collectMarkerSpecification(boundNames);
 }
 
-// Not overriding equals() because we need the variables to have been 
attached when this is
-// called and so having a non standard method name might help avoid 
mistakes
-public boolean equalsTo(ColumnCondition other) throws 
InvalidRequestException
+public ColumnCondition.WithVariables with(ListByteBuffer variables)
 {
-return column.equals(other.column)
- 
value.bindAndGet(variables).equals(other.value.bindAndGet(other.variables));
+return new WithVariables(variables);
 }
 
-private ColumnNameBuilder copyOrUpdatePrefix(CFMetaData cfm, 
ColumnNameBuilder rowPrefix)
+public class WithVariables
 {
-return column.kind == CFDefinition.Name.Kind.STATIC ? 
cfm.getStaticColumnNameBuilder() : rowPrefix.copy();
-}
+private final ListByteBuffer variables;
 
-/**
- * Validates whether this condition applies to {@code current}.
- */
-public boolean appliesTo(ColumnNameBuilder rowPrefix, ColumnFamily 
current, long now) throws InvalidRequestException
-{
-if (column.type instanceof CollectionType)
-return collectionAppliesTo((CollectionType)column.type, rowPrefix, 
current, now);
-
-ColumnNameBuilder prefix = copyOrUpdatePrefix(current.metadata(), 
rowPrefix);
-ByteBuffer columnName = column.kind == 
CFDefinition.Name.Kind.VALUE_ALIAS
-  ? prefix.build()
-  : prefix.add(column.name.key).build();
-
-Column c = current.getColumn(columnName);
-ByteBuffer v = value.bindAndGet(variables);
-return v == null
- ? c == null || !c.isLive(now)
- : c != null  c.isLive(now)  c.value().equals(v);
-}
+private WithVariables(ListByteBuffer variables)
+{
+this.variables = variables;
+}
 
-

[3/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[slebresne]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/cql3/ColumnCondition.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f7e690d7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f7e690d7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f7e690d7

Branch: refs/heads/cassandra-2.1
Commit: f7e690d7f0298c7fb47036f2ec6051df733cd145
Parents: ab1a02c 17278b3
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 20:32:26 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 20:32:26 2014 +0200

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  | 176 ++-
 .../cql3/statements/CQL3CasConditions.java  |  11 +-
 3 files changed, 104 insertions(+), 84 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7e690d7/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7e690d7/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --cc src/java/org/apache/cassandra/cql3/ColumnCondition.java
index 520858f,e6cdf43..64c2032
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@@ -23,13 -23,10 +23,12 @@@ import java.util.*
  import com.google.common.base.Predicate;
  import com.google.common.collect.Iterators;
  
 -import org.apache.cassandra.config.CFMetaData;
 +import org.apache.cassandra.config.ColumnDefinition;
  import org.apache.cassandra.db.*;
 +import org.apache.cassandra.db.composites.CellName;
 +import org.apache.cassandra.db.composites.Composite;
  import org.apache.cassandra.db.filter.ColumnSlice;
- import org.apache.cassandra.db.marshal.CollectionType;
- import org.apache.cassandra.db.marshal.CounterColumnType;
+ import org.apache.cassandra.db.marshal.*;
  import org.apache.cassandra.exceptions.InvalidRequestException;
  
  /**
@@@ -37,12 -34,10 +36,10 @@@
   */
  public class ColumnCondition
  {
 -public final CFDefinition.Name column;
 +public final ColumnDefinition column;
  private final Term value;
  
- private ListByteBuffer variables;
- 
 -private ColumnCondition(CFDefinition.Name column, Term value)
 +private ColumnCondition(ColumnDefinition column, Term value)
  {
  this.column = column;
  this.value = value;
@@@ -72,93 -60,137 +62,121 @@@
  value.collectMarkerSpecification(boundNames);
  }
  
- // Not overriding equals() because we need the variables to have been 
attached when this is
- // called and so having a non standard method name might help avoid 
mistakes
- public boolean equalsTo(ColumnCondition other) throws 
InvalidRequestException
+ public ColumnCondition.WithVariables with(ListByteBuffer variables)
  {
- return column.equals(other.column)
-  
value.bindAndGet(variables).equals(other.value.bindAndGet(other.variables));
+ return new WithVariables(variables);
  }
  
- /**
-  * Validates whether this condition applies to {@code current}.
-  */
- public boolean appliesTo(Composite rowPrefix, ColumnFamily current, long 
now) throws InvalidRequestException
+ public class WithVariables
  {
- if (column.type instanceof CollectionType)
- return collectionAppliesTo((CollectionType)column.type, 
rowPrefix, current, now);
- 
- Cell c = 
current.getColumn(current.metadata().comparator.create(rowPrefix, column));
- ByteBuffer v = value.bindAndGet(variables);
- return v == null
-  ? c == null || !c.isLive(now)
-  : c != null  c.isLive(now)  c.value().equals(v);
- }
+ private final ListByteBuffer variables;
  
- private boolean collectionAppliesTo(CollectionType type, Composite 
rowPrefix, ColumnFamily current, final long now) throws InvalidRequestException
- {
- CellName name = current.metadata().comparator.create(rowPrefix, 
column);
- // We are testing for collection equality, so we need to have the 
expected values *and* only those.
- ColumnSlice[] collectionSlice = new ColumnSlice[]{ name.slice() };
- // Filter live columns, this makes things simpler afterwards
- IteratorCell iter = 
Iterators.filter(current.iterator(collectionSlice), new PredicateCell()
+ private WithVariables(ListByteBuffer variables)
  {
- public boolean apply(Cell c)
- {
- // we only care about live columns
- return c.isLive(now);
- }
- 

[3/4] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[slebresne]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/cql3/ColumnCondition.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f7e690d7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f7e690d7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f7e690d7

Branch: refs/heads/trunk
Commit: f7e690d7f0298c7fb47036f2ec6051df733cd145
Parents: ab1a02c 17278b3
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 20:32:26 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 20:32:26 2014 +0200

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  | 176 ++-
 .../cql3/statements/CQL3CasConditions.java  |  11 +-
 3 files changed, 104 insertions(+), 84 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7e690d7/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7e690d7/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --cc src/java/org/apache/cassandra/cql3/ColumnCondition.java
index 520858f,e6cdf43..64c2032
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@@ -23,13 -23,10 +23,12 @@@ import java.util.*
  import com.google.common.base.Predicate;
  import com.google.common.collect.Iterators;
  
 -import org.apache.cassandra.config.CFMetaData;
 +import org.apache.cassandra.config.ColumnDefinition;
  import org.apache.cassandra.db.*;
 +import org.apache.cassandra.db.composites.CellName;
 +import org.apache.cassandra.db.composites.Composite;
  import org.apache.cassandra.db.filter.ColumnSlice;
- import org.apache.cassandra.db.marshal.CollectionType;
- import org.apache.cassandra.db.marshal.CounterColumnType;
+ import org.apache.cassandra.db.marshal.*;
  import org.apache.cassandra.exceptions.InvalidRequestException;
  
  /**
@@@ -37,12 -34,10 +36,10 @@@
   */
  public class ColumnCondition
  {
 -public final CFDefinition.Name column;
 +public final ColumnDefinition column;
  private final Term value;
  
- private ListByteBuffer variables;
- 
 -private ColumnCondition(CFDefinition.Name column, Term value)
 +private ColumnCondition(ColumnDefinition column, Term value)
  {
  this.column = column;
  this.value = value;
@@@ -72,93 -60,137 +62,121 @@@
  value.collectMarkerSpecification(boundNames);
  }
  
- // Not overriding equals() because we need the variables to have been 
attached when this is
- // called and so having a non standard method name might help avoid 
mistakes
- public boolean equalsTo(ColumnCondition other) throws 
InvalidRequestException
+ public ColumnCondition.WithVariables with(ListByteBuffer variables)
  {
- return column.equals(other.column)
-  
value.bindAndGet(variables).equals(other.value.bindAndGet(other.variables));
+ return new WithVariables(variables);
  }
  
- /**
-  * Validates whether this condition applies to {@code current}.
-  */
- public boolean appliesTo(Composite rowPrefix, ColumnFamily current, long 
now) throws InvalidRequestException
+ public class WithVariables
  {
- if (column.type instanceof CollectionType)
- return collectionAppliesTo((CollectionType)column.type, 
rowPrefix, current, now);
- 
- Cell c = 
current.getColumn(current.metadata().comparator.create(rowPrefix, column));
- ByteBuffer v = value.bindAndGet(variables);
- return v == null
-  ? c == null || !c.isLive(now)
-  : c != null  c.isLive(now)  c.value().equals(v);
- }
+ private final ListByteBuffer variables;
  
- private boolean collectionAppliesTo(CollectionType type, Composite 
rowPrefix, ColumnFamily current, final long now) throws InvalidRequestException
- {
- CellName name = current.metadata().comparator.create(rowPrefix, 
column);
- // We are testing for collection equality, so we need to have the 
expected values *and* only those.
- ColumnSlice[] collectionSlice = new ColumnSlice[]{ name.slice() };
- // Filter live columns, this makes things simpler afterwards
- IteratorCell iter = 
Iterators.filter(current.iterator(collectionSlice), new PredicateCell()
+ private WithVariables(ListByteBuffer variables)
  {
- public boolean apply(Cell c)
- {
- // we only care about live columns
- return c.isLive(now);
- }
- });
+ 

[jira] [Assigned] (CASSANDRA-6974) Replaying archived commitlogs isn't working

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-6974:
-

Assignee: Benedict

 Replaying archived commitlogs isn't working
 ---

 Key: CASSANDRA-6974
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6974
 Project: Cassandra
  Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Benedict
 Attachments: 2.0.system.log, 2.1.system.log


 I have a test for restoring archived commitlogs, which is not working in 2.1 
 HEAD.  My commitlogs consist of 30,000 inserts, but system.log indicates 
 there were only 2 mutations replayed:
 {code}
 INFO  [main] 2014-04-02 11:49:54,173 CommitLog.java:115 - Log replay 
 complete, 2 replayed mutations
 {code}
 There are several warnings in the logs about bad headers and invalid CRCs: 
 {code}
 WARN  [main] 2014-04-02 11:49:54,156 CommitLogReplayer.java:138 - Encountered 
 bad header at position 0 of commit log /tmp/dtest
 -mZIlPE/test/node1/commitlogs/CommitLog-4-1396453793570.log, with invalid 
 CRC. The end of segment marker should be zero.
 {code}
 compare that to the same test run on 2.0, where it replayed many more 
 mutations:
 {code}
  INFO [main] 2014-04-02 11:49:04,673 CommitLog.java (line 132) Log replay 
 complete, 35960 replayed mutations
 {code}
 I'll attach the system logs for reference.
 [Here is the dtest to reproduce 
 this|https://github.com/riptano/cassandra-dtest/blob/master/snapshot_test.py#L75]
  - (This currently relies on the fix for snapshots available in 
 CASSANDRA-6965.)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Reopened] (CASSANDRA-6825) Slice Queries Can Skip Intersecting SSTables

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs reopened CASSANDRA-6825:



 Slice Queries Can Skip Intersecting SSTables
 

 Key: CASSANDRA-6825
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6825
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: quad core Windows7 x64, single node cluster
 Cassandra 2.0.5
Reporter: Bill Mitchell
Assignee: Tyler Hobbs
 Fix For: 2.0.7, 2.1 beta2

 Attachments: 6825-2.0-v2.txt, 6825-2.0.txt, cassandra.log, 
 selectpartitions.zip, selectrowcounts.txt, testdb_1395372407904.zip, 
 testdb_1395372407904.zip


 Investigating another problem, I needed to do COUNT(*) on the several 
 partitions of a table immediately after a test case ran, and I discovered 
 that count(*) on the full table and on each of the partitions returned 
 different counts.  
 In particular case, SELECT COUNT(*) FROM sr LIMIT 100; returned the 
 expected count from the test 9 rows.  The composite primary key splits 
 the logical row into six distinct partitions, and when I issue a query asking 
 for the total across all six partitions, the returned result is only 83999.  
 Drilling down, I find that SELECT * from sr WHERE s = 5 AND l = 11 AND 
 partition = 0; returns 30,000 rows, but a SELECT COUNT(*) with the identical 
 WHERE predicate reports only 14,000. 
 This is failing immediately after running a single small test, such that 
 there are only two SSTables, sr-jb-1 and sr-jb-2.  Compaction never needed to 
 run.  
 In selectrowcounts.txt is a copy of the cqlsh output showing the incorrect 
 count(*) results.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-5483) Repair tracing

[Lyuben Todorov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13958055#comment-13958055
 ] 

Lyuben Todorov commented on CASSANDRA-5483:
---

v9 has indeed fixed the high cpu usage. I pushed v8 and v9 to 
[d9abbfc62319935ef42d4685037f80b292e03e0f|https://github.com/lyubent/cassandra/commit/d9abbfc62319935ef42d4685037f80b292e03e0f]
 with only the minor change of giving a name to the thread that [carries out 
polling|https://github.com/lyubent/cassandra/commit/d9abbfc62319935ef42d4685037f80b292e03e0f#diff-b76a607445d53f18a98c9df14323c7ddR2649]
 for nodetool output. The thread name is currently {{RepairTracePolling}} but 
I'm open to suggestions. 

[~usrbincc] I think we should rebase to the latest trunk but the patch is 
looking good. 

 Repair tracing
 --

 Key: CASSANDRA-5483
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5483
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Yuki Morishita
Assignee: Ben Chan
Priority: Minor
  Labels: repair
 Attachments: 5483-full-trunk.txt, 
 5483-v06-04-Allow-tracing-ttl-to-be-configured.patch, 
 5483-v06-05-Add-a-command-column-to-system_traces.events.patch, 
 5483-v06-06-Fix-interruption-in-tracestate-propagation.patch, 
 5483-v07-07-Better-constructor-parameters-for-DebuggableThreadPoolExecutor.patch,
  5483-v07-08-Fix-brace-style.patch, 
 5483-v07-09-Add-trace-option-to-a-more-complete-set-of-repair-functions.patch,
  5483-v07-10-Correct-name-of-boolean-repairedAt-to-fullRepair.patch, 
 5483-v08-11-Shorten-trace-messages.-Use-Tracing-begin.patch, 
 5483-v08-12-Trace-streaming-in-Differencer-StreamingRepairTask.patch, 
 5483-v08-13-sendNotification-of-local-traces-back-to-nodetool.patch, 
 5483-v08-14-Poll-system_traces.events.patch, 
 5483-v08-15-Limit-trace-notifications.-Add-exponential-backoff.patch, 
 5483-v09-16-Fix-hang-caused-by-incorrect-exit-code.patch, ccm-repair-test, 
 cqlsh-left-justify-text-columns.patch, prerepair-vs-postbuggedrepair.diff, 
 test-5483-system_traces-events.txt, 
 trunk@4620823-5483-v02-0001-Trace-filtering-and-tracestate-propagation.patch, 
 trunk@4620823-5483-v02-0002-Put-a-few-traces-parallel-to-the-repair-logging.patch,
  tr...@8ebeee1-5483-v01-001-trace-filtering-and-tracestate-propagation.txt, 
 tr...@8ebeee1-5483-v01-002-simple-repair-tracing.txt, 
 v02p02-5483-v03-0003-Make-repair-tracing-controllable-via-nodetool.patch, 
 v02p02-5483-v04-0003-This-time-use-an-EnumSet-to-pass-boolean-repair-options.patch,
  v02p02-5483-v05-0003-Use-long-instead-of-EnumSet-to-work-with-JMX.patch


 I think it would be nice to log repair stats and results like query tracing 
 stores traces to system keyspace. With it, you don't have to lookup each log 
 file to see what was the status and how it performed the repair you invoked. 
 Instead, you can query the repair log with session ID to see the state and 
 stats of all nodes involved in that repair session.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6825) Slice Queries Can Skip Intersecting SSTables

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-6825:
---

Attachment: 6825-2.0-part2.txt

Looks like I missed one case, which ended up breaking 
KeyspaceTest.testLimitSSTableComposites().  In particular, the sstable-skipping 
optimization can't stop at {{min(min(start.length, finish.length), 
minColumnNames.size())}}, because it needs to handle starts and finishes of 
different lengths.

6825-2.0-part2.txt corrects that and adds another case to CompositeTypeTest to 
explicitly cover this.

 Slice Queries Can Skip Intersecting SSTables
 

 Key: CASSANDRA-6825
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6825
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: quad core Windows7 x64, single node cluster
 Cassandra 2.0.5
Reporter: Bill Mitchell
Assignee: Tyler Hobbs
 Fix For: 2.0.7, 2.1 beta2

 Attachments: 6825-2.0-part2.txt, 6825-2.0-v2.txt, 6825-2.0.txt, 
 cassandra.log, selectpartitions.zip, selectrowcounts.txt, 
 testdb_1395372407904.zip, testdb_1395372407904.zip


 Investigating another problem, I needed to do COUNT(*) on the several 
 partitions of a table immediately after a test case ran, and I discovered 
 that count(*) on the full table and on each of the partitions returned 
 different counts.  
 In particular case, SELECT COUNT(*) FROM sr LIMIT 100; returned the 
 expected count from the test 9 rows.  The composite primary key splits 
 the logical row into six distinct partitions, and when I issue a query asking 
 for the total across all six partitions, the returned result is only 83999.  
 Drilling down, I find that SELECT * from sr WHERE s = 5 AND l = 11 AND 
 partition = 0; returns 30,000 rows, but a SELECT COUNT(*) with the identical 
 WHERE predicate reports only 14,000. 
 This is failing immediately after running a single small test, such that 
 there are only two SSTables, sr-jb-1 and sr-jb-2.  Compaction never needed to 
 run.  
 In selectrowcounts.txt is a copy of the cqlsh output showing the incorrect 
 count(*) results.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[5/6] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[brandonwilliams]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8234bc15
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8234bc15
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8234bc15

Branch: refs/heads/cassandra-2.1
Commit: 8234bc15fdc63095d0457c9bf6757a750fbbf4f2
Parents: f7e690d d4ec31f
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Apr 2 14:15:41 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Apr 2 14:15:41 2014 -0500

--
 CHANGES.txt |  1 +
 .../apache/cassandra/net/MessagingService.java  | 13 +-
 .../org/apache/cassandra/utils/ExpiringMap.java | 47 +---
 .../apache/cassandra/service/RemoveTest.java|  1 -
 4 files changed, 15 insertions(+), 47 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8234bc15/CHANGES.txt
--
diff --cc CHANGES.txt
index dc493dc,9bbcf07..db734b2
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,44 -1,5 +1,45 @@@
 -2.0.7
 +2.1.0-beta2
 + * Fail write instead of logging a warning when unable to append to CL
 +   (CASSANDRA-6764)
 + * Eliminate possibility of CL segment appearing twice in active list 
 +   (CASSANDRA-6557)
 + * Apply DONTNEED fadvise to commitlog segments (CASSANDRA-6759)
 + * Switch CRC component to Adler and include it for compressed sstables 
 +   (CASSANDRA-4165)
 + * Allow cassandra-stress to set compaction strategy options (CASSANDRA-6451)
 + * Add broadcast_rpc_address option to cassandra.yaml (CASSANDRA-5899)
 + * Auto reload GossipingPropertyFileSnitch config (CASSANDRA-5897)
 + * Fix overflow of memtable_total_space_in_mb (CASSANDRA-6573)
 + * Fix ABTC NPE and apply update function correctly (CASSANDRA-6692)
 + * Allow nodetool to use a file or prompt for password (CASSANDRA-6660)
 + * Fix AIOOBE when concurrently accessing ABSC (CASSANDRA-6742)
 + * Fix assertion error in ALTER TYPE RENAME (CASSANDRA-6705)
 + * Scrub should not always clear out repaired status (CASSANDRA-5351)
 + * Improve handling of range tombstone for wide partitions (CASSANDRA-6446)
 + * Fix ClassCastException for compact table with composites (CASSANDRA-6738)
 + * Fix potentially repairing with wrong nodes (CASSANDRA-6808)
 + * Change caching option syntax (CASSANDRA-6745)
 + * Fix stress to do proper counter reads (CASSANDRA-6835)
 + * Fix help message for stress counter_write (CASSANDRA-6824)
 + * Fix stress smart Thrift client to pick servers correctly (CASSANDRA-6848)
 + * Add logging levels (minimal, normal or verbose) to stress tool 
(CASSANDRA-6849)
 + * Fix race condition in Batch CLE (CASSANDRA-6860)
 + * Improve cleanup/scrub/upgradesstables failure handling (CASSANDRA-6774)
 + * ByteBuffer write() methods for serializing sstables (CASSANDRA-6781)
 + * Proper compare function for CollectionType (CASSANDRA-6783)
 + * Update native server to Netty 4 (CASSANDRA-6236)
 + * Fix off-by-one error in stress (CASSANDRA-6883)
 + * Make OpOrder AutoCloseable (CASSANDRA-6901)
 + * Remove sync repair JMX interface (CASSANDRA-6900)
 + * Add multiple memory allocation options for memtables (CASSANDRA-6689)
 + * Remove adjusted op rate from stress output (CASSANDRA-6921)
 + * Add optimized CF.hasColumns() implementations (CASSANDRA-6941)
 + * Serialize batchlog mutations with the version of the target node
 +   (CASSANDRA-6931)
 + * Optimize CounterColumn#reconcile() (CASSANDRA-6953)
 + * Properly remove 1.2 sstable support in 2.1 (CASSANDRA-6869)
 +Merged from 2.0:
+  * Don't shut ExpiringMap down (CASSANDRA-6948)
   * Restrict Windows to parallel repairs (CASSANDRA-6907)
   * (Hadoop) Allow manually specifying start/end tokens in CFIF 
(CASSANDRA-6436)
   * Fix NPE in MeteredFlusher (CASSANDRA-6820)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8234bc15/src/java/org/apache/cassandra/net/MessagingService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8234bc15/test/unit/org/apache/cassandra/service/RemoveTest.java
--

[6/6] git commit: Merge branch 'cassandra-2.1' into trunk

[brandonwilliams]

Merge branch 'cassandra-2.1' into trunk

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/90387f63
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/90387f63
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/90387f63

Branch: refs/heads/trunk
Commit: 90387f63bc908b5ed91b47ee33454e85a98886de
Parents: 5b85be0 8234bc1
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Apr 2 14:16:07 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Apr 2 14:16:07 2014 -0500

--
 CHANGES.txt |  2 +
 .../apache/cassandra/net/MessagingService.java  | 13 +-
 .../org/apache/cassandra/utils/ExpiringMap.java | 47 +---
 .../apache/cassandra/service/RemoveTest.java|  1 -
 4 files changed, 16 insertions(+), 47 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/90387f63/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/90387f63/src/java/org/apache/cassandra/net/MessagingService.java
--

[1/6] git commit: Don't shut ExpiringMap down.

[brandonwilliams]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 17278b3a3 - d4ec31f21
  refs/heads/cassandra-2.1 f7e690d7f - 8234bc15f
  refs/heads/trunk 5b85be04c - 90387f63b


Don't shut ExpiringMap down.

Patch by Benedict, reviewed by brandonwilliams for CASSANDRA-6948


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d4ec31f2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d4ec31f2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d4ec31f2

Branch: refs/heads/cassandra-2.0
Commit: d4ec31f21eb83502bd523c057bce2d3b249b3d80
Parents: 17278b3
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Apr 2 14:05:14 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Apr 2 14:14:30 2014 -0500

--
 CHANGES.txt |  1 +
 .../apache/cassandra/net/MessagingService.java  | 13 +-
 .../org/apache/cassandra/utils/ExpiringMap.java | 47 +---
 .../apache/cassandra/service/RemoveTest.java|  1 -
 4 files changed, 15 insertions(+), 47 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4ec31f2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 66196d0..9bbcf07 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.7
+ * Don't shut ExpiringMap down (CASSANDRA-6948)
  * Restrict Windows to parallel repairs (CASSANDRA-6907)
  * (Hadoop) Allow manually specifying start/end tokens in CFIF (CASSANDRA-6436)
  * Fix NPE in MeteredFlusher (CASSANDRA-6820)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4ec31f2/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java 
b/src/java/org/apache/cassandra/net/MessagingService.java
index ad86bbd..094e861 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -392,7 +392,6 @@ public final class MessagingService implements 
MessagingServiceMBean
  */
 public void listen(InetAddress localEp) throws ConfigurationException
 {
-callbacks.reset(); // hack to allow tests to stop/restart MS
 for (ServerSocket ss : getServerSockets(localEp))
 {
 SocketThread th = new SocketThread(ss, ACCEPT- + localEp);
@@ -536,7 +535,7 @@ public final class MessagingService implements 
MessagingServiceMBean
 {
 assert message.verb != Verb.MUTATION; // mutations need to call the 
overload with a ConsistencyLevel
 int messageId = nextId();
-CallbackInfo previous = callbacks.put(messageId, new CallbackInfo(to, 
cb, callbackDeserializers.get(message.verb)), timeout);
+ExpiringMap.CacheableObjectCallbackInfo previous = 
callbacks.put(messageId, new CallbackInfo(to, cb, 
callbackDeserializers.get(message.verb)), timeout);
 assert previous == null : String.format(Callback already exists for 
id %d! (%s), messageId, previous);
 return messageId;
 }
@@ -545,7 +544,7 @@ public final class MessagingService implements 
MessagingServiceMBean
 {
 assert message.verb == Verb.MUTATION || message.verb == 
Verb.COUNTER_MUTATION;
 int messageId = nextId();
-CallbackInfo previous = callbacks.put(messageId, new 
WriteCallbackInfo(to, cb, message, callbackDeserializers.get(message.verb), 
consistencyLevel), timeout);
+ExpiringMap.CacheableObjectCallbackInfo previous = 
callbacks.put(messageId, new WriteCallbackInfo(to, cb, message, 
callbackDeserializers.get(message.verb), consistencyLevel), timeout);
 assert previous == null : String.format(Callback already exists for 
id %d! (%s), messageId, previous);
 return messageId;
 }
@@ -654,11 +653,6 @@ public final class MessagingService implements 
MessagingServiceMBean
 subscribers.add(subcriber);
 }
 
-public void clearCallbacksUnsafe()
-{
-callbacks.reset();
-}
-
 /**
  * Wait for callbacks and don't allow any more to be created (since they 
could require writing hints)
  */
@@ -668,9 +662,6 @@ public final class MessagingService implements 
MessagingServiceMBean
 // We may need to schedule hints on the mutation stage, so it's 
erroneous to shut down the mutation stage first
 assert !StageManager.getStage(Stage.MUTATION).isShutdown();
 
-// the important part
-callbacks.shutdownBlocking();
-
 // attempt to humor tests that try to stop and restart MS
 try
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4ec31f2/src/java/org/apache/cassandra/utils/ExpiringMap.java

[3/6] git commit: Don't shut ExpiringMap down.

[brandonwilliams]

Don't shut ExpiringMap down.

Patch by Benedict, reviewed by brandonwilliams for CASSANDRA-6948


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d4ec31f2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d4ec31f2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d4ec31f2

Branch: refs/heads/trunk
Commit: d4ec31f21eb83502bd523c057bce2d3b249b3d80
Parents: 17278b3
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Apr 2 14:05:14 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Apr 2 14:14:30 2014 -0500

--
 CHANGES.txt |  1 +
 .../apache/cassandra/net/MessagingService.java  | 13 +-
 .../org/apache/cassandra/utils/ExpiringMap.java | 47 +---
 .../apache/cassandra/service/RemoveTest.java|  1 -
 4 files changed, 15 insertions(+), 47 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4ec31f2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 66196d0..9bbcf07 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.7
+ * Don't shut ExpiringMap down (CASSANDRA-6948)
  * Restrict Windows to parallel repairs (CASSANDRA-6907)
  * (Hadoop) Allow manually specifying start/end tokens in CFIF (CASSANDRA-6436)
  * Fix NPE in MeteredFlusher (CASSANDRA-6820)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4ec31f2/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java 
b/src/java/org/apache/cassandra/net/MessagingService.java
index ad86bbd..094e861 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -392,7 +392,6 @@ public final class MessagingService implements 
MessagingServiceMBean
  */
 public void listen(InetAddress localEp) throws ConfigurationException
 {
-callbacks.reset(); // hack to allow tests to stop/restart MS
 for (ServerSocket ss : getServerSockets(localEp))
 {
 SocketThread th = new SocketThread(ss, ACCEPT- + localEp);
@@ -536,7 +535,7 @@ public final class MessagingService implements 
MessagingServiceMBean
 {
 assert message.verb != Verb.MUTATION; // mutations need to call the 
overload with a ConsistencyLevel
 int messageId = nextId();
-CallbackInfo previous = callbacks.put(messageId, new CallbackInfo(to, 
cb, callbackDeserializers.get(message.verb)), timeout);
+ExpiringMap.CacheableObjectCallbackInfo previous = 
callbacks.put(messageId, new CallbackInfo(to, cb, 
callbackDeserializers.get(message.verb)), timeout);
 assert previous == null : String.format(Callback already exists for 
id %d! (%s), messageId, previous);
 return messageId;
 }
@@ -545,7 +544,7 @@ public final class MessagingService implements 
MessagingServiceMBean
 {
 assert message.verb == Verb.MUTATION || message.verb == 
Verb.COUNTER_MUTATION;
 int messageId = nextId();
-CallbackInfo previous = callbacks.put(messageId, new 
WriteCallbackInfo(to, cb, message, callbackDeserializers.get(message.verb), 
consistencyLevel), timeout);
+ExpiringMap.CacheableObjectCallbackInfo previous = 
callbacks.put(messageId, new WriteCallbackInfo(to, cb, message, 
callbackDeserializers.get(message.verb), consistencyLevel), timeout);
 assert previous == null : String.format(Callback already exists for 
id %d! (%s), messageId, previous);
 return messageId;
 }
@@ -654,11 +653,6 @@ public final class MessagingService implements 
MessagingServiceMBean
 subscribers.add(subcriber);
 }
 
-public void clearCallbacksUnsafe()
-{
-callbacks.reset();
-}
-
 /**
  * Wait for callbacks and don't allow any more to be created (since they 
could require writing hints)
  */
@@ -668,9 +662,6 @@ public final class MessagingService implements 
MessagingServiceMBean
 // We may need to schedule hints on the mutation stage, so it's 
erroneous to shut down the mutation stage first
 assert !StageManager.getStage(Stage.MUTATION).isShutdown();
 
-// the important part
-callbacks.shutdownBlocking();
-
 // attempt to humor tests that try to stop and restart MS
 try
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4ec31f2/src/java/org/apache/cassandra/utils/ExpiringMap.java
--
diff --git a/src/java/org/apache/cassandra/utils/ExpiringMap.java 
b/src/java/org/apache/cassandra/utils/ExpiringMap.java

[4/6] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[brandonwilliams]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8234bc15
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8234bc15
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8234bc15

Branch: refs/heads/trunk
Commit: 8234bc15fdc63095d0457c9bf6757a750fbbf4f2
Parents: f7e690d d4ec31f
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Apr 2 14:15:41 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Apr 2 14:15:41 2014 -0500

--
 CHANGES.txt |  1 +
 .../apache/cassandra/net/MessagingService.java  | 13 +-
 .../org/apache/cassandra/utils/ExpiringMap.java | 47 +---
 .../apache/cassandra/service/RemoveTest.java|  1 -
 4 files changed, 15 insertions(+), 47 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8234bc15/CHANGES.txt
--
diff --cc CHANGES.txt
index dc493dc,9bbcf07..db734b2
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,44 -1,5 +1,45 @@@
 -2.0.7
 +2.1.0-beta2
 + * Fail write instead of logging a warning when unable to append to CL
 +   (CASSANDRA-6764)
 + * Eliminate possibility of CL segment appearing twice in active list 
 +   (CASSANDRA-6557)
 + * Apply DONTNEED fadvise to commitlog segments (CASSANDRA-6759)
 + * Switch CRC component to Adler and include it for compressed sstables 
 +   (CASSANDRA-4165)
 + * Allow cassandra-stress to set compaction strategy options (CASSANDRA-6451)
 + * Add broadcast_rpc_address option to cassandra.yaml (CASSANDRA-5899)
 + * Auto reload GossipingPropertyFileSnitch config (CASSANDRA-5897)
 + * Fix overflow of memtable_total_space_in_mb (CASSANDRA-6573)
 + * Fix ABTC NPE and apply update function correctly (CASSANDRA-6692)
 + * Allow nodetool to use a file or prompt for password (CASSANDRA-6660)
 + * Fix AIOOBE when concurrently accessing ABSC (CASSANDRA-6742)
 + * Fix assertion error in ALTER TYPE RENAME (CASSANDRA-6705)
 + * Scrub should not always clear out repaired status (CASSANDRA-5351)
 + * Improve handling of range tombstone for wide partitions (CASSANDRA-6446)
 + * Fix ClassCastException for compact table with composites (CASSANDRA-6738)
 + * Fix potentially repairing with wrong nodes (CASSANDRA-6808)
 + * Change caching option syntax (CASSANDRA-6745)
 + * Fix stress to do proper counter reads (CASSANDRA-6835)
 + * Fix help message for stress counter_write (CASSANDRA-6824)
 + * Fix stress smart Thrift client to pick servers correctly (CASSANDRA-6848)
 + * Add logging levels (minimal, normal or verbose) to stress tool 
(CASSANDRA-6849)
 + * Fix race condition in Batch CLE (CASSANDRA-6860)
 + * Improve cleanup/scrub/upgradesstables failure handling (CASSANDRA-6774)
 + * ByteBuffer write() methods for serializing sstables (CASSANDRA-6781)
 + * Proper compare function for CollectionType (CASSANDRA-6783)
 + * Update native server to Netty 4 (CASSANDRA-6236)
 + * Fix off-by-one error in stress (CASSANDRA-6883)
 + * Make OpOrder AutoCloseable (CASSANDRA-6901)
 + * Remove sync repair JMX interface (CASSANDRA-6900)
 + * Add multiple memory allocation options for memtables (CASSANDRA-6689)
 + * Remove adjusted op rate from stress output (CASSANDRA-6921)
 + * Add optimized CF.hasColumns() implementations (CASSANDRA-6941)
 + * Serialize batchlog mutations with the version of the target node
 +   (CASSANDRA-6931)
 + * Optimize CounterColumn#reconcile() (CASSANDRA-6953)
 + * Properly remove 1.2 sstable support in 2.1 (CASSANDRA-6869)
 +Merged from 2.0:
+  * Don't shut ExpiringMap down (CASSANDRA-6948)
   * Restrict Windows to parallel repairs (CASSANDRA-6907)
   * (Hadoop) Allow manually specifying start/end tokens in CFIF 
(CASSANDRA-6436)
   * Fix NPE in MeteredFlusher (CASSANDRA-6820)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8234bc15/src/java/org/apache/cassandra/net/MessagingService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8234bc15/test/unit/org/apache/cassandra/service/RemoveTest.java
--

[2/6] git commit: Don't shut ExpiringMap down.

[brandonwilliams]

Don't shut ExpiringMap down.

Patch by Benedict, reviewed by brandonwilliams for CASSANDRA-6948


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d4ec31f2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d4ec31f2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d4ec31f2

Branch: refs/heads/cassandra-2.1
Commit: d4ec31f21eb83502bd523c057bce2d3b249b3d80
Parents: 17278b3
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Apr 2 14:05:14 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Apr 2 14:14:30 2014 -0500

--
 CHANGES.txt |  1 +
 .../apache/cassandra/net/MessagingService.java  | 13 +-
 .../org/apache/cassandra/utils/ExpiringMap.java | 47 +---
 .../apache/cassandra/service/RemoveTest.java|  1 -
 4 files changed, 15 insertions(+), 47 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4ec31f2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 66196d0..9bbcf07 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.7
+ * Don't shut ExpiringMap down (CASSANDRA-6948)
  * Restrict Windows to parallel repairs (CASSANDRA-6907)
  * (Hadoop) Allow manually specifying start/end tokens in CFIF (CASSANDRA-6436)
  * Fix NPE in MeteredFlusher (CASSANDRA-6820)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4ec31f2/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java 
b/src/java/org/apache/cassandra/net/MessagingService.java
index ad86bbd..094e861 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -392,7 +392,6 @@ public final class MessagingService implements 
MessagingServiceMBean
  */
 public void listen(InetAddress localEp) throws ConfigurationException
 {
-callbacks.reset(); // hack to allow tests to stop/restart MS
 for (ServerSocket ss : getServerSockets(localEp))
 {
 SocketThread th = new SocketThread(ss, ACCEPT- + localEp);
@@ -536,7 +535,7 @@ public final class MessagingService implements 
MessagingServiceMBean
 {
 assert message.verb != Verb.MUTATION; // mutations need to call the 
overload with a ConsistencyLevel
 int messageId = nextId();
-CallbackInfo previous = callbacks.put(messageId, new CallbackInfo(to, 
cb, callbackDeserializers.get(message.verb)), timeout);
+ExpiringMap.CacheableObjectCallbackInfo previous = 
callbacks.put(messageId, new CallbackInfo(to, cb, 
callbackDeserializers.get(message.verb)), timeout);
 assert previous == null : String.format(Callback already exists for 
id %d! (%s), messageId, previous);
 return messageId;
 }
@@ -545,7 +544,7 @@ public final class MessagingService implements 
MessagingServiceMBean
 {
 assert message.verb == Verb.MUTATION || message.verb == 
Verb.COUNTER_MUTATION;
 int messageId = nextId();
-CallbackInfo previous = callbacks.put(messageId, new 
WriteCallbackInfo(to, cb, message, callbackDeserializers.get(message.verb), 
consistencyLevel), timeout);
+ExpiringMap.CacheableObjectCallbackInfo previous = 
callbacks.put(messageId, new WriteCallbackInfo(to, cb, message, 
callbackDeserializers.get(message.verb), consistencyLevel), timeout);
 assert previous == null : String.format(Callback already exists for 
id %d! (%s), messageId, previous);
 return messageId;
 }
@@ -654,11 +653,6 @@ public final class MessagingService implements 
MessagingServiceMBean
 subscribers.add(subcriber);
 }
 
-public void clearCallbacksUnsafe()
-{
-callbacks.reset();
-}
-
 /**
  * Wait for callbacks and don't allow any more to be created (since they 
could require writing hints)
  */
@@ -668,9 +662,6 @@ public final class MessagingService implements 
MessagingServiceMBean
 // We may need to schedule hints on the mutation stage, so it's 
erroneous to shut down the mutation stage first
 assert !StageManager.getStage(Stage.MUTATION).isShutdown();
 
-// the important part
-callbacks.shutdownBlocking();
-
 // attempt to humor tests that try to stop and restart MS
 try
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4ec31f2/src/java/org/apache/cassandra/utils/ExpiringMap.java
--
diff --git a/src/java/org/apache/cassandra/utils/ExpiringMap.java 

[jira] [Updated] (CASSANDRA-6906) Skip Replica Calculation for Range Slice on LocalStrategy Keyspace

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-6906:
---

Attachment: 6906.txt

The attached patch (and 
[branch|https://github.com/thobbs/cassandra/tree/CASSANDRA-6906]) avoid 
splitting the queried range when working with LocalStrategy keyspaces.

 Skip Replica Calculation for Range Slice on LocalStrategy Keyspace
 --

 Key: CASSANDRA-6906
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6906
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
Priority: Minor
 Attachments: 6906.txt


 For vnode-enabled clusters, the Determining replicas to query portion of 
 range slice commands can be expensive.  When querying LocalStrategy 
 keyspaces, we can safely skip this step.  On a 15 node cluster with vnodes, 
 skipping this saves about 100ms.  This makes a big difference for the 
 drivers, which frequently execute queries like select * from system.peers 
 and select * from system.local.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6948) After Bootstrap or Replace node startup, EXPIRING_MAP_REAPER is shutdown and cannot be restarted, causing callbacks to collect indefinitely

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6948:


Fix Version/s: 2.1 beta2

 After Bootstrap or Replace node startup, EXPIRING_MAP_REAPER is shutdown and 
 cannot be restarted, causing callbacks to collect indefinitely
 ---

 Key: CASSANDRA-6948
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6948
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Keith Wright
Assignee: Benedict
 Fix For: 2.0.7, 2.1 beta2

 Attachments: 6948.debug.txt, 6948.txt, Screen Shot 2014-03-28 at 
 11.27.56 AM.png, Screen Shot 2014-03-28 at 11.29.24 AM.png, 
 cassandra.log.min, cassandra.yaml, logs.old.tar.gz, logs.tar.gz, 
 system.log.1.gz, system.log.gz


 Since ExpiringMap.shutdown() shuts down the static executor service, it 
 cannot be restarted (and in fact reset() makes no attempt to do so). As such 
 callbacks that receive no response are never removed from the map, and 
 eventually either than server will run out of memory or will loop around the 
 integer space and start reusing messageids that have not been expired, 
 causing assertions to be thrown and messages to fail to be sent. It appears 
 that this situation only arises on bootstrap or node replacement, as 
 MessagingService is shutdown before being attached to the listen address.
 This can cause the following errors to begin occurring in the log:
 ERROR [Native-Transport-Requests:7636] 2014-03-28 13:32:10,638 
 ErrorMessage.java (line 222) Unexpected exception during request
 java.lang.AssertionError: Callback already exists for id -1665979622! 
 (CallbackInfo(target=/10.106.160.84, 
 callback=org.apache.cassandra.service.WriteResponseHandler@5d36d8ea, 
 serializer=org.apache.cassandra.db.WriteResponse$WriteResponseSerializer@6ed37f0b))
   at 
 org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:549)
   at 
 org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:601)
   at 
 org.apache.cassandra.service.StorageProxy.mutateCounter(StorageProxy.java:984)
   at 
 org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:449)
   at 
 org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:524)
   at 
 org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:521)
   at 
 org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:505)
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:188)
   at 
 org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:358)
   at 
 org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:131)
   at 
 org.apache.cassandra.transport.Message$Dispatcher.messageReceived(Message.java:304)
   at 
 org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43)
   at 
 org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 ERROR [ReplicateOnWriteStage:102766] 2014-03-28 13:32:10,638 
 CassandraDaemon.java (line 196) Exception in thread 
 Thread[ReplicateOnWriteStage:102766,5,main]
 java.lang.AssertionError: Callback already exists for id -1665979620! 
 (CallbackInfo(target=/10.106.160.84, 
 callback=org.apache.cassandra.service.WriteResponseHandler@3bdb1a75, 
 serializer=org.apache.cassandra.db.WriteResponse$WriteResponseSerializer@6ed37f0b))
   at 
 org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:549)
   at 
 org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:601)
   at 
 org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:806)
   at 
 org.apache.cassandra.service.StorageProxy$8$1.runMayThrow(StorageProxy.java:1074)
   at 
 org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1896)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6825) Slice Queries Can Skip Intersecting SSTables

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13958067#comment-13958067
 ] 

Sylvain Lebresne commented on CASSANDRA-6825:
-

Good catch, +1

 Slice Queries Can Skip Intersecting SSTables
 

 Key: CASSANDRA-6825
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6825
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: quad core Windows7 x64, single node cluster
 Cassandra 2.0.5
Reporter: Bill Mitchell
Assignee: Tyler Hobbs
 Fix For: 2.0.7, 2.1 beta2

 Attachments: 6825-2.0-part2.txt, 6825-2.0-v2.txt, 6825-2.0.txt, 
 cassandra.log, selectpartitions.zip, selectrowcounts.txt, 
 testdb_1395372407904.zip, testdb_1395372407904.zip


 Investigating another problem, I needed to do COUNT(*) on the several 
 partitions of a table immediately after a test case ran, and I discovered 
 that count(*) on the full table and on each of the partitions returned 
 different counts.  
 In particular case, SELECT COUNT(*) FROM sr LIMIT 100; returned the 
 expected count from the test 9 rows.  The composite primary key splits 
 the logical row into six distinct partitions, and when I issue a query asking 
 for the total across all six partitions, the returned result is only 83999.  
 Drilling down, I find that SELECT * from sr WHERE s = 5 AND l = 11 AND 
 partition = 0; returns 30,000 rows, but a SELECT COUNT(*) with the identical 
 WHERE predicate reports only 14,000. 
 This is failing immediately after running a single small test, such that 
 there are only two SSTables, sr-jb-1 and sr-jb-2.  Compaction never needed to 
 run.  
 In selectrowcounts.txt is a copy of the cqlsh output showing the incorrect 
 count(*) results.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6930) Dynamic Snitch isWorthMergingForRangeQuery Doesn't Handle Some Cases Optimally

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-6930:
---

Priority: Minor  (was: Major)
  Labels: lhf  (was: )

 Dynamic Snitch isWorthMergingForRangeQuery Doesn't Handle Some Cases Optimally
 --

 Key: CASSANDRA-6930
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6930
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Priority: Minor
  Labels: lhf
 Fix For: 2.0.8


 I was investigating slow responses for queries like {{select * from 
 system.peers}} and noticed that the dynamic endpoint snitch was reporting 
 that the query was _not_ worth merging.  In this case, the local host had a 
 score of 0, so {{return maxMerged  maxL1 + maxL2}} was returning false.  I 
 believe using a {{=}} condition is the proper fix there.
 Additionally, because scores are looked up three separate times, this method 
 is a prone to race conditions.  I don't think it's worth fixing the race 
 condition for a multi-node scenario, but at least in the single-node case, we 
 can immediately return true and avoid any race conditions that would cause it 
 to erroneously return false.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Fix collection element access in IF

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 d4ec31f21 - 5aafa9876


Fix collection element access in IF

patch by slebresne; reviewed by thobbs for CASSANDRA-6914


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5aafa987
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5aafa987
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5aafa987

Branch: refs/heads/cassandra-2.0
Commit: 5aafa98768d0e309c2b19f3a20bdf46e114b0c94
Parents: d4ec31f
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Tue Mar 25 08:48:38 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 21:28:03 2014 +0200

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  | 126 +--
 src/java/org/apache/cassandra/cql3/Cql.g|   4 +-
 .../cql3/statements/CQL3CasConditions.java  |   5 +-
 4 files changed, 123 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5aafa987/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9bbcf07..3a08c33 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -43,6 +43,7 @@
  * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
  * Fix bad skip of sstables on slice query with composite start/finish 
(CASSANDRA-6825)
  * Fix unintended update with conditional statement (CASSANDRA-6893)
+ * Fix map element access in IF (CASSANDRA-6914)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5aafa987/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index e6cdf43..9fb3390 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -28,6 +28,7 @@ import org.apache.cassandra.db.*;
 import org.apache.cassandra.db.filter.ColumnSlice;
 import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.exceptions.InvalidRequestException;
+import org.apache.cassandra.utils.ByteBufferUtil;
 
 /**
  * A CQL3 condition.
@@ -35,18 +36,27 @@ import 
org.apache.cassandra.exceptions.InvalidRequestException;
 public class ColumnCondition
 {
 public final CFDefinition.Name column;
+
+// For collection, when testing the equality of a specific element, null 
otherwise.
+private final Term collectionElement;
+
 private final Term value;
 
-private ColumnCondition(CFDefinition.Name column, Term value)
+private ColumnCondition(CFDefinition.Name column, Term collectionElement, 
Term value)
 {
 this.column = column;
+this.collectionElement = collectionElement;
 this.value = value;
 }
 
-// The only ones we support so far
 public static ColumnCondition equal(CFDefinition.Name column, Term value)
 {
-return new ColumnCondition(column, value);
+return new ColumnCondition(column, null, value);
+}
+
+public static ColumnCondition equal(CFDefinition.Name column, Term 
collectionElement, Term value)
+{
+return new ColumnCondition(column, collectionElement, value);
 }
 
 /**
@@ -57,6 +67,8 @@ public class ColumnCondition
  */
 public void collectMarkerSpecification(VariableSpecifications boundNames)
 {
+if (collectionElement != null)
+collectionElement.collectMarkerSpecification(boundNames);
 value.collectMarkerSpecification(boundNames);
 }
 
@@ -74,12 +86,26 @@ public class ColumnCondition
 this.variables = variables;
 }
 
-// Not overriding equals() because we need the variables to have been 
attached when this is
-// called and so having a non standard method name might help avoid 
mistakes
 public boolean equalsTo(WithVariables other) throws 
InvalidRequestException
 {
-return column.equals(other.column())
- 
value.bindAndGet(variables).equals(other.value().bindAndGet(other.variables));
+if (!column().equals(other.column()))
+return false;
+
+if ((collectionElement() == null) != (other.collectionElement() == 
null))
+return false;
+
+if (collectionElement() != null)
+{
+assert column.type instanceof ListType || column.type 
instanceof MapType;
+AbstractType? comparator = column.type instanceof ListType

[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[slebresne]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/cql3/ColumnCondition.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/13510d41
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/13510d41
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/13510d41

Branch: refs/heads/cassandra-2.1
Commit: 13510d414fbb0da489ecd062054aa49d059ba79d
Parents: 8234bc1 5aafa98
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 21:38:03 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 21:38:03 2014 +0200

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  | 126 +--
 src/java/org/apache/cassandra/cql3/Cql.g|   4 +-
 .../cql3/statements/CQL3CasConditions.java  |   5 +-
 4 files changed, 123 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/13510d41/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/13510d41/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --cc src/java/org/apache/cassandra/cql3/ColumnCondition.java
index 64c2032,9fb3390..67e7174
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@@ -36,19 -35,28 +37,28 @@@ import org.apache.cassandra.utils.ByteB
   */
  public class ColumnCondition
  {
 -public final CFDefinition.Name column;
 +public final ColumnDefinition column;
+ 
+ // For collection, when testing the equality of a specific element, null 
otherwise.
+ private final Term collectionElement;
+ 
  private final Term value;
  
- private ColumnCondition(ColumnDefinition column, Term value)
 -private ColumnCondition(CFDefinition.Name column, Term collectionElement, 
Term value)
++private ColumnCondition(ColumnDefinition column, Term collectionElement, 
Term value)
  {
  this.column = column;
+ this.collectionElement = collectionElement;
  this.value = value;
  }
  
- // The only ones we support so far
 -public static ColumnCondition equal(CFDefinition.Name column, Term value)
 +public static ColumnCondition equal(ColumnDefinition column, Term value)
  {
- return new ColumnCondition(column, value);
+ return new ColumnCondition(column, null, value);
+ }
+ 
 -public static ColumnCondition equal(CFDefinition.Name column, Term 
collectionElement, Term value)
++public static ColumnCondition equal(ColumnDefinition column, Term 
collectionElement, Term value)
+ {
+ return new ColumnCondition(column, collectionElement, value);
  }
  
  /**
@@@ -76,15 -86,29 +88,29 @@@
  this.variables = variables;
  }
  
- // Not overriding equals() because we need the variables to have been 
attached when this is
- // called and so having a non standard method name might help avoid 
mistakes
  public boolean equalsTo(WithVariables other) throws 
InvalidRequestException
  {
- return column.equals(other.column())
-  
value.bindAndGet(variables).equals(other.value().bindAndGet(other.variables));
+ if (!column().equals(other.column()))
+ return false;
+ 
+ if ((collectionElement() == null) != (other.collectionElement() 
== null))
+ return false;
+ 
+ if (collectionElement() != null)
+ {
+ assert column.type instanceof ListType || column.type 
instanceof MapType;
+ AbstractType? comparator = column.type instanceof ListType
+? Int32Type.instance
+: ((MapType)column.type).keys;
+ 
+ if 
(comparator.compare(collectionElement().bindAndGet(variables), 
other.collectionElement().bindAndGet(variables)) != 0)
+ return false;
+ }
+ 
+ return 
value().bindAndGet(variables).equals(other.value().bindAndGet(other.variables));
  }
  
 -private CFDefinition.Name column()
 +private ColumnDefinition column()
  {
  return column;
  }
@@@ -94,6 -123,16 +125,11 @@@
  return value;
  }
  
+ public ByteBuffer getCollectionElementValue() throws 
InvalidRequestException
+ {
+ return collectionElement == null ? null : 
collectionElement.bindAndGet(variables);
+ }
+ 
 -private ColumnNameBuilder 

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

[slebresne]

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/70517107
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/70517107
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/70517107

Branch: refs/heads/trunk
Commit: 705171074f0270a7bd5bb406d5a6b5192d9bd44d
Parents: 90387f6 13510d4
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 21:38:29 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 21:38:29 2014 +0200

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  | 126 +--
 src/java/org/apache/cassandra/cql3/Cql.g|   4 +-
 .../cql3/statements/CQL3CasConditions.java  |   5 +-
 4 files changed, 123 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/70517107/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/70517107/src/java/org/apache/cassandra/cql3/statements/CQL3CasConditions.java
--

[1/2] git commit: Fix collection element access in IF

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 8234bc15f - 13510d414


Fix collection element access in IF

patch by slebresne; reviewed by thobbs for CASSANDRA-6914


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5aafa987
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5aafa987
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5aafa987

Branch: refs/heads/cassandra-2.1
Commit: 5aafa98768d0e309c2b19f3a20bdf46e114b0c94
Parents: d4ec31f
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Tue Mar 25 08:48:38 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 21:28:03 2014 +0200

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  | 126 +--
 src/java/org/apache/cassandra/cql3/Cql.g|   4 +-
 .../cql3/statements/CQL3CasConditions.java  |   5 +-
 4 files changed, 123 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5aafa987/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9bbcf07..3a08c33 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -43,6 +43,7 @@
  * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
  * Fix bad skip of sstables on slice query with composite start/finish 
(CASSANDRA-6825)
  * Fix unintended update with conditional statement (CASSANDRA-6893)
+ * Fix map element access in IF (CASSANDRA-6914)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5aafa987/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index e6cdf43..9fb3390 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -28,6 +28,7 @@ import org.apache.cassandra.db.*;
 import org.apache.cassandra.db.filter.ColumnSlice;
 import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.exceptions.InvalidRequestException;
+import org.apache.cassandra.utils.ByteBufferUtil;
 
 /**
  * A CQL3 condition.
@@ -35,18 +36,27 @@ import 
org.apache.cassandra.exceptions.InvalidRequestException;
 public class ColumnCondition
 {
 public final CFDefinition.Name column;
+
+// For collection, when testing the equality of a specific element, null 
otherwise.
+private final Term collectionElement;
+
 private final Term value;
 
-private ColumnCondition(CFDefinition.Name column, Term value)
+private ColumnCondition(CFDefinition.Name column, Term collectionElement, 
Term value)
 {
 this.column = column;
+this.collectionElement = collectionElement;
 this.value = value;
 }
 
-// The only ones we support so far
 public static ColumnCondition equal(CFDefinition.Name column, Term value)
 {
-return new ColumnCondition(column, value);
+return new ColumnCondition(column, null, value);
+}
+
+public static ColumnCondition equal(CFDefinition.Name column, Term 
collectionElement, Term value)
+{
+return new ColumnCondition(column, collectionElement, value);
 }
 
 /**
@@ -57,6 +67,8 @@ public class ColumnCondition
  */
 public void collectMarkerSpecification(VariableSpecifications boundNames)
 {
+if (collectionElement != null)
+collectionElement.collectMarkerSpecification(boundNames);
 value.collectMarkerSpecification(boundNames);
 }
 
@@ -74,12 +86,26 @@ public class ColumnCondition
 this.variables = variables;
 }
 
-// Not overriding equals() because we need the variables to have been 
attached when this is
-// called and so having a non standard method name might help avoid 
mistakes
 public boolean equalsTo(WithVariables other) throws 
InvalidRequestException
 {
-return column.equals(other.column())
- 
value.bindAndGet(variables).equals(other.value().bindAndGet(other.variables));
+if (!column().equals(other.column()))
+return false;
+
+if ((collectionElement() == null) != (other.collectionElement() == 
null))
+return false;
+
+if (collectionElement() != null)
+{
+assert column.type instanceof ListType || column.type 
instanceof MapType;
+AbstractType? comparator = column.type instanceof ListType

[1/3] git commit: Fix collection element access in IF

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 90387f63b - 705171074


Fix collection element access in IF

patch by slebresne; reviewed by thobbs for CASSANDRA-6914


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5aafa987
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5aafa987
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5aafa987

Branch: refs/heads/trunk
Commit: 5aafa98768d0e309c2b19f3a20bdf46e114b0c94
Parents: d4ec31f
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Tue Mar 25 08:48:38 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 21:28:03 2014 +0200

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  | 126 +--
 src/java/org/apache/cassandra/cql3/Cql.g|   4 +-
 .../cql3/statements/CQL3CasConditions.java  |   5 +-
 4 files changed, 123 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5aafa987/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9bbcf07..3a08c33 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -43,6 +43,7 @@
  * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
  * Fix bad skip of sstables on slice query with composite start/finish 
(CASSANDRA-6825)
  * Fix unintended update with conditional statement (CASSANDRA-6893)
+ * Fix map element access in IF (CASSANDRA-6914)
 Merged from 1.2:
  * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
  * add extra SSL cipher suites (CASSANDRA-6613)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5aafa987/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index e6cdf43..9fb3390 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -28,6 +28,7 @@ import org.apache.cassandra.db.*;
 import org.apache.cassandra.db.filter.ColumnSlice;
 import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.exceptions.InvalidRequestException;
+import org.apache.cassandra.utils.ByteBufferUtil;
 
 /**
  * A CQL3 condition.
@@ -35,18 +36,27 @@ import 
org.apache.cassandra.exceptions.InvalidRequestException;
 public class ColumnCondition
 {
 public final CFDefinition.Name column;
+
+// For collection, when testing the equality of a specific element, null 
otherwise.
+private final Term collectionElement;
+
 private final Term value;
 
-private ColumnCondition(CFDefinition.Name column, Term value)
+private ColumnCondition(CFDefinition.Name column, Term collectionElement, 
Term value)
 {
 this.column = column;
+this.collectionElement = collectionElement;
 this.value = value;
 }
 
-// The only ones we support so far
 public static ColumnCondition equal(CFDefinition.Name column, Term value)
 {
-return new ColumnCondition(column, value);
+return new ColumnCondition(column, null, value);
+}
+
+public static ColumnCondition equal(CFDefinition.Name column, Term 
collectionElement, Term value)
+{
+return new ColumnCondition(column, collectionElement, value);
 }
 
 /**
@@ -57,6 +67,8 @@ public class ColumnCondition
  */
 public void collectMarkerSpecification(VariableSpecifications boundNames)
 {
+if (collectionElement != null)
+collectionElement.collectMarkerSpecification(boundNames);
 value.collectMarkerSpecification(boundNames);
 }
 
@@ -74,12 +86,26 @@ public class ColumnCondition
 this.variables = variables;
 }
 
-// Not overriding equals() because we need the variables to have been 
attached when this is
-// called and so having a non standard method name might help avoid 
mistakes
 public boolean equalsTo(WithVariables other) throws 
InvalidRequestException
 {
-return column.equals(other.column())
- 
value.bindAndGet(variables).equals(other.value().bindAndGet(other.variables));
+if (!column().equals(other.column()))
+return false;
+
+if ((collectionElement() == null) != (other.collectionElement() == 
null))
+return false;
+
+if (collectionElement() != null)
+{
+assert column.type instanceof ListType || column.type 
instanceof MapType;
+AbstractType? comparator = column.type instanceof ListType
+   

[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[slebresne]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/cql3/ColumnCondition.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/13510d41
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/13510d41
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/13510d41

Branch: refs/heads/trunk
Commit: 13510d414fbb0da489ecd062054aa49d059ba79d
Parents: 8234bc1 5aafa98
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 2 21:38:03 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 2 21:38:03 2014 +0200

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  | 126 +--
 src/java/org/apache/cassandra/cql3/Cql.g|   4 +-
 .../cql3/statements/CQL3CasConditions.java  |   5 +-
 4 files changed, 123 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/13510d41/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/13510d41/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --cc src/java/org/apache/cassandra/cql3/ColumnCondition.java
index 64c2032,9fb3390..67e7174
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@@ -36,19 -35,28 +37,28 @@@ import org.apache.cassandra.utils.ByteB
   */
  public class ColumnCondition
  {
 -public final CFDefinition.Name column;
 +public final ColumnDefinition column;
+ 
+ // For collection, when testing the equality of a specific element, null 
otherwise.
+ private final Term collectionElement;
+ 
  private final Term value;
  
- private ColumnCondition(ColumnDefinition column, Term value)
 -private ColumnCondition(CFDefinition.Name column, Term collectionElement, 
Term value)
++private ColumnCondition(ColumnDefinition column, Term collectionElement, 
Term value)
  {
  this.column = column;
+ this.collectionElement = collectionElement;
  this.value = value;
  }
  
- // The only ones we support so far
 -public static ColumnCondition equal(CFDefinition.Name column, Term value)
 +public static ColumnCondition equal(ColumnDefinition column, Term value)
  {
- return new ColumnCondition(column, value);
+ return new ColumnCondition(column, null, value);
+ }
+ 
 -public static ColumnCondition equal(CFDefinition.Name column, Term 
collectionElement, Term value)
++public static ColumnCondition equal(ColumnDefinition column, Term 
collectionElement, Term value)
+ {
+ return new ColumnCondition(column, collectionElement, value);
  }
  
  /**
@@@ -76,15 -86,29 +88,29 @@@
  this.variables = variables;
  }
  
- // Not overriding equals() because we need the variables to have been 
attached when this is
- // called and so having a non standard method name might help avoid 
mistakes
  public boolean equalsTo(WithVariables other) throws 
InvalidRequestException
  {
- return column.equals(other.column())
-  
value.bindAndGet(variables).equals(other.value().bindAndGet(other.variables));
+ if (!column().equals(other.column()))
+ return false;
+ 
+ if ((collectionElement() == null) != (other.collectionElement() 
== null))
+ return false;
+ 
+ if (collectionElement() != null)
+ {
+ assert column.type instanceof ListType || column.type 
instanceof MapType;
+ AbstractType? comparator = column.type instanceof ListType
+? Int32Type.instance
+: ((MapType)column.type).keys;
+ 
+ if 
(comparator.compare(collectionElement().bindAndGet(variables), 
other.collectionElement().bindAndGet(variables)) != 0)
+ return false;
+ }
+ 
+ return 
value().bindAndGet(variables).equals(other.value().bindAndGet(other.variables));
  }
  
 -private CFDefinition.Name column()
 +private ColumnDefinition column()
  {
  return column;
  }
@@@ -94,6 -123,16 +125,11 @@@
  return value;
  }
  
+ public ByteBuffer getCollectionElementValue() throws 
InvalidRequestException
+ {
+ return collectionElement == null ? null : 
collectionElement.bindAndGet(variables);
+ }
+ 
 -private ColumnNameBuilder 

[1/8] git commit: Allow compaction of system tables during startup patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-6913

[jbellis]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 5aafa9876 - 56d84a7c0
  refs/heads/cassandra-2.1 13510d414 - 5ab1a341a
  refs/heads/trunk 705171074 - c7324473f


Allow compaction of system tables during startup
patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-6913


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/56d84a7c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/56d84a7c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/56d84a7c

Branch: refs/heads/cassandra-2.0
Commit: 56d84a7c028c0498158efb1a3cadea149ab7c1cd
Parents: 5aafa98
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Apr 2 14:35:09 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Apr 2 14:41:29 2014 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 8 +---
 2 files changed, 6 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/56d84a7c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3a08c33..d999e88 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.7
+ * Allow compaction of system tables during startup (CASSANDRA-6913)
  * Don't shut ExpiringMap down (CASSANDRA-6948)
  * Restrict Windows to parallel repairs (CASSANDRA-6907)
  * (Hadoop) Allow manually specifying start/end tokens in CFIF (CASSANDRA-6436)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/56d84a7c/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 6b70987..53bbb77 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -34,8 +34,6 @@ import javax.management.Notification;
 import javax.management.NotificationBroadcasterSupport;
 import javax.management.ObjectName;
 
-import static java.nio.charset.StandardCharsets.ISO_8859_1;
-
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Predicate;
 import com.google.common.collect.*;
@@ -88,6 +86,8 @@ import org.apache.cassandra.thrift.cassandraConstants;
 import org.apache.cassandra.tracing.Tracing;
 import org.apache.cassandra.utils.*;
 
+import static java.nio.charset.StandardCharsets.ISO_8859_1;
+
 /**
  * This abstraction contains the token/identifier of this node
  * on the identifier space. This token gets gossiped around.
@@ -2048,7 +2048,9 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 double bytes = 0;
 for (String keyspaceName : Schema.instance.getKeyspaces())
 {
-Keyspace keyspace = Keyspace.open(keyspaceName);
+Keyspace keyspace = 
Schema.instance.getKeyspaceInstance(keyspaceName);
+if (keyspace == null)
+continue;
 for (ColumnFamilyStore cfs : keyspace.getColumnFamilyStores())
 bytes += cfs.getLiveDiskSpaceUsed();
 }

[2/8] git commit: Allow compaction of system tables during startup patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-6913

[jbellis]

Allow compaction of system tables during startup
patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-6913


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/56d84a7c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/56d84a7c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/56d84a7c

Branch: refs/heads/cassandra-2.1
Commit: 56d84a7c028c0498158efb1a3cadea149ab7c1cd
Parents: 5aafa98
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Apr 2 14:35:09 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Apr 2 14:41:29 2014 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 8 +---
 2 files changed, 6 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/56d84a7c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3a08c33..d999e88 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.7
+ * Allow compaction of system tables during startup (CASSANDRA-6913)
  * Don't shut ExpiringMap down (CASSANDRA-6948)
  * Restrict Windows to parallel repairs (CASSANDRA-6907)
  * (Hadoop) Allow manually specifying start/end tokens in CFIF (CASSANDRA-6436)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/56d84a7c/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 6b70987..53bbb77 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -34,8 +34,6 @@ import javax.management.Notification;
 import javax.management.NotificationBroadcasterSupport;
 import javax.management.ObjectName;
 
-import static java.nio.charset.StandardCharsets.ISO_8859_1;
-
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Predicate;
 import com.google.common.collect.*;
@@ -88,6 +86,8 @@ import org.apache.cassandra.thrift.cassandraConstants;
 import org.apache.cassandra.tracing.Tracing;
 import org.apache.cassandra.utils.*;
 
+import static java.nio.charset.StandardCharsets.ISO_8859_1;
+
 /**
  * This abstraction contains the token/identifier of this node
  * on the identifier space. This token gets gossiped around.
@@ -2048,7 +2048,9 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 double bytes = 0;
 for (String keyspaceName : Schema.instance.getKeyspaces())
 {
-Keyspace keyspace = Keyspace.open(keyspaceName);
+Keyspace keyspace = 
Schema.instance.getKeyspaceInstance(keyspaceName);
+if (keyspace == null)
+continue;
 for (ColumnFamilyStore cfs : keyspace.getColumnFamilyStores())
 bytes += cfs.getLiveDiskSpaceUsed();
 }

[3/8] git commit: Allow compaction of system tables during startup patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-6913

[jbellis]

Allow compaction of system tables during startup
patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-6913


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/56d84a7c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/56d84a7c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/56d84a7c

Branch: refs/heads/trunk
Commit: 56d84a7c028c0498158efb1a3cadea149ab7c1cd
Parents: 5aafa98
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Apr 2 14:35:09 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Apr 2 14:41:29 2014 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 8 +---
 2 files changed, 6 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/56d84a7c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3a08c33..d999e88 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.7
+ * Allow compaction of system tables during startup (CASSANDRA-6913)
  * Don't shut ExpiringMap down (CASSANDRA-6948)
  * Restrict Windows to parallel repairs (CASSANDRA-6907)
  * (Hadoop) Allow manually specifying start/end tokens in CFIF (CASSANDRA-6436)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/56d84a7c/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 6b70987..53bbb77 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -34,8 +34,6 @@ import javax.management.Notification;
 import javax.management.NotificationBroadcasterSupport;
 import javax.management.ObjectName;
 
-import static java.nio.charset.StandardCharsets.ISO_8859_1;
-
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Predicate;
 import com.google.common.collect.*;
@@ -88,6 +86,8 @@ import org.apache.cassandra.thrift.cassandraConstants;
 import org.apache.cassandra.tracing.Tracing;
 import org.apache.cassandra.utils.*;
 
+import static java.nio.charset.StandardCharsets.ISO_8859_1;
+
 /**
  * This abstraction contains the token/identifier of this node
  * on the identifier space. This token gets gossiped around.
@@ -2048,7 +2048,9 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 double bytes = 0;
 for (String keyspaceName : Schema.instance.getKeyspaces())
 {
-Keyspace keyspace = Keyspace.open(keyspaceName);
+Keyspace keyspace = 
Schema.instance.getKeyspaceInstance(keyspaceName);
+if (keyspace == null)
+continue;
 for (ColumnFamilyStore cfs : keyspace.getColumnFamilyStores())
 bytes += cfs.getLiveDiskSpaceUsed();
 }

[4/8] git commit: merge from 2.0

[jbellis]

merge from 2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8e69acec
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8e69acec
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8e69acec

Branch: refs/heads/cassandra-2.1
Commit: 8e69acec7b3943e07e1031c5b3e7c495409d2942
Parents: 13510d4 56d84a7
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Apr 2 14:42:35 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Apr 2 14:42:35 2014 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 8 +---
 2 files changed, 6 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8e69acec/CHANGES.txt
--
diff --cc CHANGES.txt
index b30d432,d999e88..d5c8372
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,44 -1,5 +1,45 @@@
 -2.0.7
 +2.1.0-beta2
 + * Fail write instead of logging a warning when unable to append to CL
 +   (CASSANDRA-6764)
 + * Eliminate possibility of CL segment appearing twice in active list 
 +   (CASSANDRA-6557)
 + * Apply DONTNEED fadvise to commitlog segments (CASSANDRA-6759)
 + * Switch CRC component to Adler and include it for compressed sstables 
 +   (CASSANDRA-4165)
 + * Allow cassandra-stress to set compaction strategy options (CASSANDRA-6451)
 + * Add broadcast_rpc_address option to cassandra.yaml (CASSANDRA-5899)
 + * Auto reload GossipingPropertyFileSnitch config (CASSANDRA-5897)
 + * Fix overflow of memtable_total_space_in_mb (CASSANDRA-6573)
 + * Fix ABTC NPE and apply update function correctly (CASSANDRA-6692)
 + * Allow nodetool to use a file or prompt for password (CASSANDRA-6660)
 + * Fix AIOOBE when concurrently accessing ABSC (CASSANDRA-6742)
 + * Fix assertion error in ALTER TYPE RENAME (CASSANDRA-6705)
 + * Scrub should not always clear out repaired status (CASSANDRA-5351)
 + * Improve handling of range tombstone for wide partitions (CASSANDRA-6446)
 + * Fix ClassCastException for compact table with composites (CASSANDRA-6738)
 + * Fix potentially repairing with wrong nodes (CASSANDRA-6808)
 + * Change caching option syntax (CASSANDRA-6745)
 + * Fix stress to do proper counter reads (CASSANDRA-6835)
 + * Fix help message for stress counter_write (CASSANDRA-6824)
 + * Fix stress smart Thrift client to pick servers correctly (CASSANDRA-6848)
 + * Add logging levels (minimal, normal or verbose) to stress tool 
(CASSANDRA-6849)
 + * Fix race condition in Batch CLE (CASSANDRA-6860)
 + * Improve cleanup/scrub/upgradesstables failure handling (CASSANDRA-6774)
 + * ByteBuffer write() methods for serializing sstables (CASSANDRA-6781)
 + * Proper compare function for CollectionType (CASSANDRA-6783)
 + * Update native server to Netty 4 (CASSANDRA-6236)
 + * Fix off-by-one error in stress (CASSANDRA-6883)
 + * Make OpOrder AutoCloseable (CASSANDRA-6901)
 + * Remove sync repair JMX interface (CASSANDRA-6900)
 + * Add multiple memory allocation options for memtables (CASSANDRA-6689)
 + * Remove adjusted op rate from stress output (CASSANDRA-6921)
 + * Add optimized CF.hasColumns() implementations (CASSANDRA-6941)
 + * Serialize batchlog mutations with the version of the target node
 +   (CASSANDRA-6931)
 + * Optimize CounterColumn#reconcile() (CASSANDRA-6953)
 + * Properly remove 1.2 sstable support in 2.1 (CASSANDRA-6869)
 +Merged from 2.0:
+  * Allow compaction of system tables during startup (CASSANDRA-6913)
   * Don't shut ExpiringMap down (CASSANDRA-6948)
   * Restrict Windows to parallel repairs (CASSANDRA-6907)
   * (Hadoop) Allow manually specifying start/end tokens in CFIF 
(CASSANDRA-6436)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8e69acec/src/java/org/apache/cassandra/service/StorageService.java
--
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index 042e2bc,53bbb77..53f7bb0
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -34,11 -33,7 +34,9 @@@ import javax.management.MBeanServer
  import javax.management.Notification;
  import javax.management.NotificationBroadcasterSupport;
  import javax.management.ObjectName;
 +import javax.management.openmbean.TabularData;
 +import javax.management.openmbean.TabularDataSupport;
  
- import static java.nio.charset.StandardCharsets.ISO_8859_1;
- 
  import com.google.common.annotations.VisibleForTesting;
  import com.google.common.base.Predicate;
  import com.google.common.collect.*;

[7/8] git commit: add asserts to make sure we don't call KS.open during startup patch by Benedict Elliott Smith; reviewed by jbellis

[jbellis]

add asserts to make sure we don't call KS.open during startup
patch by Benedict Elliott Smith; reviewed by jbellis


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5ab1a341
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5ab1a341
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5ab1a341

Branch: refs/heads/cassandra-2.1
Commit: 5ab1a341a47d43f74398443f1ec1c17edb95e550
Parents: 8e69ace
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Apr 2 14:35:54 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Apr 2 14:43:15 2014 -0500

--
 src/java/org/apache/cassandra/db/Keyspace.java   |  8 
 .../cassandra/service/CassandraDaemon.java   |  6 +++---
 test/unit/org/apache/cassandra/SchemaLoader.java |  7 ---
 .../cassandra/config/DatabaseDescriptorTest.java | 13 +
 .../io/sstable/CQLSSTableWriterTest.java | 10 +++---
 .../cassandra/io/sstable/LegacySSTableTest.java  | 19 ++-
 .../cassandra/io/sstable/SSTableLoaderTest.java  |  1 +
 .../apache/cassandra/locator/EC2SnitchTest.java  | 12 +++-
 .../service/StorageServiceServerTest.java|  1 +
 9 files changed, 54 insertions(+), 23 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5ab1a341/src/java/org/apache/cassandra/db/Keyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/Keyspace.java 
b/src/java/org/apache/cassandra/db/Keyspace.java
index d6746da..c0a8690 100644
--- a/src/java/org/apache/cassandra/db/Keyspace.java
+++ b/src/java/org/apache/cassandra/db/Keyspace.java
@@ -83,13 +83,21 @@ public class Keyspace
 }
 };
 
+private static volatile boolean initialized = false;
+public static void setInitialized()
+{
+initialized = true;
+}
+
 public static Keyspace open(String keyspaceName)
 {
+assert initialized || keyspaceName.equals(SYSTEM_KS);
 return open(keyspaceName, Schema.instance, true);
 }
 
 public static Keyspace openWithoutSSTables(String keyspaceName)
 {
+assert initialized || keyspaceName.equals(SYSTEM_KS);
 return open(keyspaceName, Schema.instance, false);
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5ab1a341/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java 
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index 7259a17..7171b68 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -31,14 +31,12 @@ import javax.management.MBeanServer;
 import javax.management.ObjectName;
 import javax.management.StandardMBean;
 
-import com.addthis.metrics.reporter.config.ReporterConfig;
 import com.google.common.collect.Iterables;
 import com.google.common.util.concurrent.Uninterruptibles;
-
-import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import com.addthis.metrics.reporter.config.ReporterConfig;
 import org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor;
 import org.apache.cassandra.concurrent.Stage;
 import org.apache.cassandra.concurrent.StageManager;
@@ -53,6 +51,7 @@ import org.apache.cassandra.db.commitlog.CommitLog;
 import org.apache.cassandra.db.compaction.CompactionManager;
 import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.io.FSError;
+import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.apache.cassandra.io.util.FileUtils;
 import org.apache.cassandra.metrics.StorageMetrics;
 import org.apache.cassandra.thrift.ThriftServer;
@@ -258,6 +257,7 @@ public class CassandraDaemon
 ColumnFamilyStore.scrubDataDirectories(cfm);
 }
 
+Keyspace.setInitialized();
 // initialize keyspaces
 for (String keyspaceName : Schema.instance.getKeyspaces())
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5ab1a341/test/unit/org/apache/cassandra/SchemaLoader.java
--
diff --git a/test/unit/org/apache/cassandra/SchemaLoader.java 
b/test/unit/org/apache/cassandra/SchemaLoader.java
index 43a6dc3..943cf96 100644
--- a/test/unit/org/apache/cassandra/SchemaLoader.java
+++ b/test/unit/org/apache/cassandra/SchemaLoader.java
@@ -21,19 +21,19 @@ import java.io.File;
 import java.nio.ByteBuffer;
 import java.util.*;
 
-import org.apache.cassandra.cache.CachingOptions;
-import org.apache.cassandra.db.index.PerRowSecondaryIndexTest;
-import 

[5/8] git commit: merge from 2.0

[jbellis]

merge from 2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8e69acec
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8e69acec
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8e69acec

Branch: refs/heads/trunk
Commit: 8e69acec7b3943e07e1031c5b3e7c495409d2942
Parents: 13510d4 56d84a7
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Apr 2 14:42:35 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Apr 2 14:42:35 2014 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 8 +---
 2 files changed, 6 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8e69acec/CHANGES.txt
--
diff --cc CHANGES.txt
index b30d432,d999e88..d5c8372
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,44 -1,5 +1,45 @@@
 -2.0.7
 +2.1.0-beta2
 + * Fail write instead of logging a warning when unable to append to CL
 +   (CASSANDRA-6764)
 + * Eliminate possibility of CL segment appearing twice in active list 
 +   (CASSANDRA-6557)
 + * Apply DONTNEED fadvise to commitlog segments (CASSANDRA-6759)
 + * Switch CRC component to Adler and include it for compressed sstables 
 +   (CASSANDRA-4165)
 + * Allow cassandra-stress to set compaction strategy options (CASSANDRA-6451)
 + * Add broadcast_rpc_address option to cassandra.yaml (CASSANDRA-5899)
 + * Auto reload GossipingPropertyFileSnitch config (CASSANDRA-5897)
 + * Fix overflow of memtable_total_space_in_mb (CASSANDRA-6573)
 + * Fix ABTC NPE and apply update function correctly (CASSANDRA-6692)
 + * Allow nodetool to use a file or prompt for password (CASSANDRA-6660)
 + * Fix AIOOBE when concurrently accessing ABSC (CASSANDRA-6742)
 + * Fix assertion error in ALTER TYPE RENAME (CASSANDRA-6705)
 + * Scrub should not always clear out repaired status (CASSANDRA-5351)
 + * Improve handling of range tombstone for wide partitions (CASSANDRA-6446)
 + * Fix ClassCastException for compact table with composites (CASSANDRA-6738)
 + * Fix potentially repairing with wrong nodes (CASSANDRA-6808)
 + * Change caching option syntax (CASSANDRA-6745)
 + * Fix stress to do proper counter reads (CASSANDRA-6835)
 + * Fix help message for stress counter_write (CASSANDRA-6824)
 + * Fix stress smart Thrift client to pick servers correctly (CASSANDRA-6848)
 + * Add logging levels (minimal, normal or verbose) to stress tool 
(CASSANDRA-6849)
 + * Fix race condition in Batch CLE (CASSANDRA-6860)
 + * Improve cleanup/scrub/upgradesstables failure handling (CASSANDRA-6774)
 + * ByteBuffer write() methods for serializing sstables (CASSANDRA-6781)
 + * Proper compare function for CollectionType (CASSANDRA-6783)
 + * Update native server to Netty 4 (CASSANDRA-6236)
 + * Fix off-by-one error in stress (CASSANDRA-6883)
 + * Make OpOrder AutoCloseable (CASSANDRA-6901)
 + * Remove sync repair JMX interface (CASSANDRA-6900)
 + * Add multiple memory allocation options for memtables (CASSANDRA-6689)
 + * Remove adjusted op rate from stress output (CASSANDRA-6921)
 + * Add optimized CF.hasColumns() implementations (CASSANDRA-6941)
 + * Serialize batchlog mutations with the version of the target node
 +   (CASSANDRA-6931)
 + * Optimize CounterColumn#reconcile() (CASSANDRA-6953)
 + * Properly remove 1.2 sstable support in 2.1 (CASSANDRA-6869)
 +Merged from 2.0:
+  * Allow compaction of system tables during startup (CASSANDRA-6913)
   * Don't shut ExpiringMap down (CASSANDRA-6948)
   * Restrict Windows to parallel repairs (CASSANDRA-6907)
   * (Hadoop) Allow manually specifying start/end tokens in CFIF 
(CASSANDRA-6436)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8e69acec/src/java/org/apache/cassandra/service/StorageService.java
--
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index 042e2bc,53bbb77..53f7bb0
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -34,11 -33,7 +34,9 @@@ import javax.management.MBeanServer
  import javax.management.Notification;
  import javax.management.NotificationBroadcasterSupport;
  import javax.management.ObjectName;
 +import javax.management.openmbean.TabularData;
 +import javax.management.openmbean.TabularDataSupport;
  
- import static java.nio.charset.StandardCharsets.ISO_8859_1;
- 
  import com.google.common.annotations.VisibleForTesting;
  import com.google.common.base.Predicate;
  import com.google.common.collect.*;

[6/8] git commit: add asserts to make sure we don't call KS.open during startup patch by Benedict Elliott Smith; reviewed by jbellis

[jbellis]

add asserts to make sure we don't call KS.open during startup
patch by Benedict Elliott Smith; reviewed by jbellis


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5ab1a341
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5ab1a341
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5ab1a341

Branch: refs/heads/trunk
Commit: 5ab1a341a47d43f74398443f1ec1c17edb95e550
Parents: 8e69ace
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Apr 2 14:35:54 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Apr 2 14:43:15 2014 -0500

--
 src/java/org/apache/cassandra/db/Keyspace.java   |  8 
 .../cassandra/service/CassandraDaemon.java   |  6 +++---
 test/unit/org/apache/cassandra/SchemaLoader.java |  7 ---
 .../cassandra/config/DatabaseDescriptorTest.java | 13 +
 .../io/sstable/CQLSSTableWriterTest.java | 10 +++---
 .../cassandra/io/sstable/LegacySSTableTest.java  | 19 ++-
 .../cassandra/io/sstable/SSTableLoaderTest.java  |  1 +
 .../apache/cassandra/locator/EC2SnitchTest.java  | 12 +++-
 .../service/StorageServiceServerTest.java|  1 +
 9 files changed, 54 insertions(+), 23 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5ab1a341/src/java/org/apache/cassandra/db/Keyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/Keyspace.java 
b/src/java/org/apache/cassandra/db/Keyspace.java
index d6746da..c0a8690 100644
--- a/src/java/org/apache/cassandra/db/Keyspace.java
+++ b/src/java/org/apache/cassandra/db/Keyspace.java
@@ -83,13 +83,21 @@ public class Keyspace
 }
 };
 
+private static volatile boolean initialized = false;
+public static void setInitialized()
+{
+initialized = true;
+}
+
 public static Keyspace open(String keyspaceName)
 {
+assert initialized || keyspaceName.equals(SYSTEM_KS);
 return open(keyspaceName, Schema.instance, true);
 }
 
 public static Keyspace openWithoutSSTables(String keyspaceName)
 {
+assert initialized || keyspaceName.equals(SYSTEM_KS);
 return open(keyspaceName, Schema.instance, false);
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5ab1a341/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java 
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index 7259a17..7171b68 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -31,14 +31,12 @@ import javax.management.MBeanServer;
 import javax.management.ObjectName;
 import javax.management.StandardMBean;
 
-import com.addthis.metrics.reporter.config.ReporterConfig;
 import com.google.common.collect.Iterables;
 import com.google.common.util.concurrent.Uninterruptibles;
-
-import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import com.addthis.metrics.reporter.config.ReporterConfig;
 import org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor;
 import org.apache.cassandra.concurrent.Stage;
 import org.apache.cassandra.concurrent.StageManager;
@@ -53,6 +51,7 @@ import org.apache.cassandra.db.commitlog.CommitLog;
 import org.apache.cassandra.db.compaction.CompactionManager;
 import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.io.FSError;
+import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.apache.cassandra.io.util.FileUtils;
 import org.apache.cassandra.metrics.StorageMetrics;
 import org.apache.cassandra.thrift.ThriftServer;
@@ -258,6 +257,7 @@ public class CassandraDaemon
 ColumnFamilyStore.scrubDataDirectories(cfm);
 }
 
+Keyspace.setInitialized();
 // initialize keyspaces
 for (String keyspaceName : Schema.instance.getKeyspaces())
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5ab1a341/test/unit/org/apache/cassandra/SchemaLoader.java
--
diff --git a/test/unit/org/apache/cassandra/SchemaLoader.java 
b/test/unit/org/apache/cassandra/SchemaLoader.java
index 43a6dc3..943cf96 100644
--- a/test/unit/org/apache/cassandra/SchemaLoader.java
+++ b/test/unit/org/apache/cassandra/SchemaLoader.java
@@ -21,19 +21,19 @@ import java.io.File;
 import java.nio.ByteBuffer;
 import java.util.*;
 
-import org.apache.cassandra.cache.CachingOptions;
-import org.apache.cassandra.db.index.PerRowSecondaryIndexTest;
-import 

[8/8] git commit: Merge branch 'cassandra-2.1' into trunk

[jbellis]

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c7324473
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c7324473
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c7324473

Branch: refs/heads/trunk
Commit: c7324473fe07a92e21d3e8d3762fd0560b36ea5a
Parents: 7051710 5ab1a34
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Apr 2 14:43:25 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Apr 2 14:43:25 2014 -0500

--
 CHANGES.txt  |  1 +
 src/java/org/apache/cassandra/db/Keyspace.java   |  8 
 .../cassandra/service/CassandraDaemon.java   |  6 +++---
 .../apache/cassandra/service/StorageService.java |  8 +---
 test/unit/org/apache/cassandra/SchemaLoader.java |  7 ---
 .../cassandra/config/DatabaseDescriptorTest.java | 13 +
 .../io/sstable/CQLSSTableWriterTest.java | 10 +++---
 .../cassandra/io/sstable/LegacySSTableTest.java  | 19 ++-
 .../cassandra/io/sstable/SSTableLoaderTest.java  |  1 +
 .../apache/cassandra/locator/EC2SnitchTest.java  | 12 +++-
 .../service/StorageServiceServerTest.java|  1 +
 11 files changed, 60 insertions(+), 26 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c7324473/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c7324473/src/java/org/apache/cassandra/service/StorageService.java
--

[jira] [Reopened] (CASSANDRA-6948) After Bootstrap or Replace node startup, EXPIRING_MAP_REAPER is shutdown and cannot be restarted, causing callbacks to collect indefinitely

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reopened CASSANDRA-6948:
---


Hang on, I think I remember why we shut down EM: we don't want to be processing 
callbacks that are going to kick off hint storage or read repair once we're 
supposed to be Done.

(This should be done on Drain, maybe on normal shutdown, not during Bootstrap.  
Point is that just ripping out the shutdown code is probably not the right fix.)

 After Bootstrap or Replace node startup, EXPIRING_MAP_REAPER is shutdown and 
 cannot be restarted, causing callbacks to collect indefinitely
 ---

 Key: CASSANDRA-6948
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6948
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Keith Wright
Assignee: Benedict
 Fix For: 2.0.7, 2.1 beta2

 Attachments: 6948.debug.txt, 6948.txt, Screen Shot 2014-03-28 at 
 11.27.56 AM.png, Screen Shot 2014-03-28 at 11.29.24 AM.png, 
 cassandra.log.min, cassandra.yaml, logs.old.tar.gz, logs.tar.gz, 
 system.log.1.gz, system.log.gz


 Since ExpiringMap.shutdown() shuts down the static executor service, it 
 cannot be restarted (and in fact reset() makes no attempt to do so). As such 
 callbacks that receive no response are never removed from the map, and 
 eventually either than server will run out of memory or will loop around the 
 integer space and start reusing messageids that have not been expired, 
 causing assertions to be thrown and messages to fail to be sent. It appears 
 that this situation only arises on bootstrap or node replacement, as 
 MessagingService is shutdown before being attached to the listen address.
 This can cause the following errors to begin occurring in the log:
 ERROR [Native-Transport-Requests:7636] 2014-03-28 13:32:10,638 
 ErrorMessage.java (line 222) Unexpected exception during request
 java.lang.AssertionError: Callback already exists for id -1665979622! 
 (CallbackInfo(target=/10.106.160.84, 
 callback=org.apache.cassandra.service.WriteResponseHandler@5d36d8ea, 
 serializer=org.apache.cassandra.db.WriteResponse$WriteResponseSerializer@6ed37f0b))
   at 
 org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:549)
   at 
 org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:601)
   at 
 org.apache.cassandra.service.StorageProxy.mutateCounter(StorageProxy.java:984)
   at 
 org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:449)
   at 
 org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:524)
   at 
 org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:521)
   at 
 org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:505)
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:188)
   at 
 org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:358)
   at 
 org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:131)
   at 
 org.apache.cassandra.transport.Message$Dispatcher.messageReceived(Message.java:304)
   at 
 org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43)
   at 
 org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 ERROR [ReplicateOnWriteStage:102766] 2014-03-28 13:32:10,638 
 CassandraDaemon.java (line 196) Exception in thread 
 Thread[ReplicateOnWriteStage:102766,5,main]
 java.lang.AssertionError: Callback already exists for id -1665979620! 
 (CallbackInfo(target=/10.106.160.84, 
 callback=org.apache.cassandra.service.WriteResponseHandler@3bdb1a75, 
 serializer=org.apache.cassandra.db.WriteResponse$WriteResponseSerializer@6ed37f0b))
   at 
 org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:549)
   at 
 org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:601)
   at 
 org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:806)
   at 
 org.apache.cassandra.service.StorageProxy$8$1.runMayThrow(StorageProxy.java:1074)
   at 
 org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1896)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 

[jira] [Commented] (CASSANDRA-6906) Skip Replica Calculation for Range Slice on LocalStrategy Keyspace

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13958094#comment-13958094
 ] 

Jonathan Ellis commented on CASSANDRA-6906:
---

+1

 Skip Replica Calculation for Range Slice on LocalStrategy Keyspace
 --

 Key: CASSANDRA-6906
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6906
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.0.7, 2.1 beta2

 Attachments: 6906.txt


 For vnode-enabled clusters, the Determining replicas to query portion of 
 range slice commands can be expensive.  When querying LocalStrategy 
 keyspaces, we can safely skip this step.  On a 15 node cluster with vnodes, 
 skipping this saves about 100ms.  This makes a big difference for the 
 drivers, which frequently execute queries like select * from system.peers 
 and select * from system.local.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6948) After Bootstrap or Replace node startup, EXPIRING_MAP_REAPER is shutdown and cannot be restarted, causing callbacks to collect indefinitely

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13958103#comment-13958103
 ] 

Benedict commented on CASSANDRA-6948:
-

Note that this wasn't being achieved before - which was my main justification 
for saying it was safe. The EM shutdown was only turning off the reaper thread, 
not preventing items being inserted nor destroying items already present.

What we want to do is clear the EM on MS shutdown, and until registration 
perhaps throw an error if we try to send a message. The wait forever )that 
was never run but was ostensibly the tack taken before) looks dangerous to me, 
since MS can restart and leave threads hanging forever. Either way, I don't 
think the EM needs to know anything about it, nor have a shutdown method.

 After Bootstrap or Replace node startup, EXPIRING_MAP_REAPER is shutdown and 
 cannot be restarted, causing callbacks to collect indefinitely
 ---

 Key: CASSANDRA-6948
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6948
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Keith Wright
Assignee: Benedict
 Fix For: 2.0.7, 2.1 beta2

 Attachments: 6948.debug.txt, 6948.txt, Screen Shot 2014-03-28 at 
 11.27.56 AM.png, Screen Shot 2014-03-28 at 11.29.24 AM.png, 
 cassandra.log.min, cassandra.yaml, logs.old.tar.gz, logs.tar.gz, 
 system.log.1.gz, system.log.gz


 Since ExpiringMap.shutdown() shuts down the static executor service, it 
 cannot be restarted (and in fact reset() makes no attempt to do so). As such 
 callbacks that receive no response are never removed from the map, and 
 eventually either than server will run out of memory or will loop around the 
 integer space and start reusing messageids that have not been expired, 
 causing assertions to be thrown and messages to fail to be sent. It appears 
 that this situation only arises on bootstrap or node replacement, as 
 MessagingService is shutdown before being attached to the listen address.
 This can cause the following errors to begin occurring in the log:
 ERROR [Native-Transport-Requests:7636] 2014-03-28 13:32:10,638 
 ErrorMessage.java (line 222) Unexpected exception during request
 java.lang.AssertionError: Callback already exists for id -1665979622! 
 (CallbackInfo(target=/10.106.160.84, 
 callback=org.apache.cassandra.service.WriteResponseHandler@5d36d8ea, 
 serializer=org.apache.cassandra.db.WriteResponse$WriteResponseSerializer@6ed37f0b))
   at 
 org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:549)
   at 
 org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:601)
   at 
 org.apache.cassandra.service.StorageProxy.mutateCounter(StorageProxy.java:984)
   at 
 org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:449)
   at 
 org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:524)
   at 
 org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:521)
   at 
 org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:505)
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:188)
   at 
 org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:358)
   at 
 org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:131)
   at 
 org.apache.cassandra.transport.Message$Dispatcher.messageReceived(Message.java:304)
   at 
 org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43)
   at 
 org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 ERROR [ReplicateOnWriteStage:102766] 2014-03-28 13:32:10,638 
 CassandraDaemon.java (line 196) Exception in thread 
 Thread[ReplicateOnWriteStage:102766,5,main]
 java.lang.AssertionError: Callback already exists for id -1665979620! 
 (CallbackInfo(target=/10.106.160.84, 
 callback=org.apache.cassandra.service.WriteResponseHandler@3bdb1a75, 
 serializer=org.apache.cassandra.db.WriteResponse$WriteResponseSerializer@6ed37f0b))
   at 
 org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:549)
   at 
 org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:601)
   at 
 

[jira] [Commented] (CASSANDRA-6913) Compaction of system keyspaces during startup can cause early loading of non-system keyspaces

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13958102#comment-13958102
 ] 

Jonathan Ellis commented on CASSANDRA-6913:
---

committed with a minor fix to the initialized logic

 Compaction of system keyspaces during startup can cause early loading of 
 non-system keyspaces
 -

 Key: CASSANDRA-6913
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6913
 Project: Cassandra
  Issue Type: Bug
Reporter: Benedict
Assignee: Benedict
Priority: Minor
 Fix For: 2.0.7, 2.1 beta2

 Attachments: 6913.txt


 This then can result in an inconsistent CFS state, as cleanup of e.g. 
 compaction leftovers does not get reflected in DataTracker. It happens 
 because StorageService.getLoad() iterates over and opens all CFS, and this is 
 called by Compaction.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[tylerhobbs]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/db/marshal/CompositeType.java
test/unit/org/apache/cassandra/db/marshal/CompositeTypeTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e34d1af9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e34d1af9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e34d1af9

Branch: refs/heads/cassandra-2.1
Commit: e34d1af963b85c46afc973221a296498eaba8264
Parents: 5ab1a34 f4f7417
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Apr 2 14:55:48 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Apr 2 14:55:48 2014 -0500

--
 src/java/org/apache/cassandra/db/filter/ColumnSlice.java| 9 ++---
 .../org/apache/cassandra/db/filter/ColumnSliceTest.java | 4 
 2 files changed, 10 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e34d1af9/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
--
diff --cc src/java/org/apache/cassandra/db/filter/ColumnSlice.java
index 3838ee5,9eff12a..227297e
--- a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
@@@ -52,63 -47,20 +52,66 @@@ public class ColumnSlic
  this.finish = finish;
  }
  
 -public boolean isAlwaysEmpty(AbstractType? comparator, boolean reversed)
 +public boolean isAlwaysEmpty(CellNameType comparator, boolean reversed)
  {
 -ComparatorByteBuffer orderedComparator = reversed ? 
comparator.reverseComparator : comparator;
 -return (start.remaining()  0  finish.remaining()  0  
orderedComparator.compare(start, finish)  0);
 +ComparatorComposite orderedComparator = reversed ? 
comparator.reverseComparator() : comparator;
 +return !start.isEmpty()  !finish.isEmpty()  
orderedComparator.compare(start, finish)  0;
  }
  
 -public boolean includes(ComparatorByteBuffer cmp, ByteBuffer name)
 +public boolean includes(ComparatorComposite cmp, Composite name)
  {
 -return cmp.compare(start, name) = 0  
(finish.equals(ByteBufferUtil.EMPTY_BYTE_BUFFER) || cmp.compare(finish, name) 
= 0);
 +return cmp.compare(start, name) = 0  (finish.isEmpty() || 
cmp.compare(finish, name) = 0);
  }
  
 -public boolean isBefore(ComparatorByteBuffer cmp, ByteBuffer name)
 +public boolean isBefore(ComparatorComposite cmp, Composite name)
  {
 -return !finish.equals(ByteBufferUtil.EMPTY_BYTE_BUFFER)  
cmp.compare(finish, name)  0;
 +return !finish.isEmpty()  cmp.compare(finish, name)  0;
 +}
 +
 +public boolean intersects(ListByteBuffer minCellNames, ListByteBuffer 
maxCellNames, CellNameType comparator, boolean reversed)
 +{
 +assert minCellNames.size() == maxCellNames.size();
 +
 +Composite sStart = reversed ? finish : start;
 +Composite sEnd = reversed ? start : finish;
 +
 +if (compare(sStart, maxCellNames, comparator, true)  0 || 
compare(sEnd, minCellNames, comparator, false)  0)
 +return false;
 +
 +// We could safely return true here, but there's a minor 
optimization: if the first component is restricted
 +// to a single value, we can check that the second component falls 
within the min/max for that component
 +// (and repeat for all components).
- for (int i = 0; i  Math.min(Math.min(sStart.size(), sEnd.size()), 
minCellNames.size()); i++)
++for (int i = 0; i  minCellNames.size(); i++)
 +{
 +AbstractType? t = comparator.subtype(i);
++ByteBuffer s = i  sStart.size() ? sStart.get(i) : 
ByteBufferUtil.EMPTY_BYTE_BUFFER;
++ByteBuffer f = i  sEnd.size() ? sEnd.get(i) : 
ByteBufferUtil.EMPTY_BYTE_BUFFER;
++
 +// we already know the first component falls within its min/max 
range (otherwise we wouldn't get here)
- if (i  0  (t.compare(sEnd.get(i), minCellNames.get(i))  0 || 
t.compare(sStart.get(i), maxCellNames.get(i))  0))
++if (i  0  (t.compare(f, minCellNames.get(i))  0 || 
t.compare(s, maxCellNames.get(i))  0))
 +return false;
 +
 +// if this component isn't equal in the start and finish, we 
don't need to check any more
- if (t.compare(sStart.get(i), sEnd.get(i)) != 0)
++if (i = sStart.size() || i = sEnd.size() || t.compare(s, f) != 
0)
 +break;
 +}
 +
 +return true;
 +}
 +
 +/** Helper method for intersects() */
 +private int compare(Composite sliceBounds, ListByteBuffer 
sstableBounds, CellNameType comparator, boolean isSliceStart)
 +{
 +for 

[1/2] git commit: Handle short slice start/finishes of different lengths

[tylerhobbs]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 5ab1a341a - e34d1af96


Handle short slice start/finishes of different lengths

Patch by Tyler Hobbs; reviewed by Sylvain Lebresne as a follow-up for
CASSANDRA-6825


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f4f74177
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f4f74177
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f4f74177

Branch: refs/heads/cassandra-2.1
Commit: f4f7417735e179c73334ecfb3df6aeb1467b6842
Parents: 5aafa98
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Apr 2 14:27:35 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Apr 2 14:29:44 2014 -0500

--
 src/java/org/apache/cassandra/db/marshal/CompositeType.java  | 8 +---
 .../org/apache/cassandra/db/marshal/CompositeTypeTest.java   | 4 
 2 files changed, 9 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4f74177/src/java/org/apache/cassandra/db/marshal/CompositeType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/CompositeType.java 
b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
index 7f08219..32fc432 100644
--- a/src/java/org/apache/cassandra/db/marshal/CompositeType.java
+++ b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
@@ -278,16 +278,18 @@ public class CompositeType extends AbstractCompositeType
 // We could safely return true here, but there's a minor 
optimization: if the first component is restricted
 // to a single value, we can check that the second component falls 
within the min/max for that component
 // (and repeat for all components).
-for (int i = 0; i  Math.min(Math.min(start.length, 
finish.length), minColumnNames.size()); i++)
+for (int i = 0; i  minColumnNames.size(); i++)
 {
 AbstractType? t = types.get(i);
+ByteBuffer s = i  start.length ? start[i] : 
ByteBufferUtil.EMPTY_BYTE_BUFFER;
+ByteBuffer f = i  finish.length ? finish[i] : 
ByteBufferUtil.EMPTY_BYTE_BUFFER;
 
 // we already know the first component falls within its 
min/max range (otherwise we wouldn't get here)
-if (i  0  !t.intersects(minColumnNames.get(i), 
maxColumnNames.get(i), start[i], finish[i]))
+if (i  0  !t.intersects(minColumnNames.get(i), 
maxColumnNames.get(i), s, f))
 continue outer;
 
 // if this component isn't equal in the start and finish, we 
don't need to check any more
-if (t.compare(start[i], finish[i]) != 0)
+if (i = start.length || i = finish.length || t.compare(s, f) 
!= 0)
 break;
 }
 return true;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4f74177/test/unit/org/apache/cassandra/db/marshal/CompositeTypeTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/marshal/CompositeTypeTest.java 
b/test/unit/org/apache/cassandra/db/marshal/CompositeTypeTest.java
index 20cb5ef..df6f5e1 100644
--- a/test/unit/org/apache/cassandra/db/marshal/CompositeTypeTest.java
+++ b/test/unit/org/apache/cassandra/db/marshal/CompositeTypeTest.java
@@ -458,6 +458,10 @@ public class CompositeTypeTest extends SchemaLoader
 filter = new SliceQueryFilter(composite(1, 2), composite(1, 3), false, 
1);
 assertFalse(comparator.intersects(columnNames(1, 0, 0), columnNames(2, 
1, 0), filter));
 
+// same case, but with missing start and end components and different 
lengths for start and end
+filter = new SliceQueryFilter(composite(1, 2), composite(1), false, 1);
+assertFalse(comparator.intersects(columnNames(1, 0, 0), columnNames(2, 
1, 0), filter));
+
 
 // same as the previous set of tests, but the second component is 
equal in the slice start and end
 filter = new SliceQueryFilter(composite(1, 2, 0), composite(1, 2, 0), 
false, 1);

[jira] [Commented] (CASSANDRA-6906) Skip Replica Calculation for Range Slice on LocalStrategy Keyspace

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13958105#comment-13958105
 ] 

Benedict commented on CASSANDRA-6906:
-

bq. On a 15 node cluster with vnodes, skipping this saves about 100ms

milliseconds? Something is wrong if so.

 Skip Replica Calculation for Range Slice on LocalStrategy Keyspace
 --

 Key: CASSANDRA-6906
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6906
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.0.7, 2.1 beta2

 Attachments: 6906.txt


 For vnode-enabled clusters, the Determining replicas to query portion of 
 range slice commands can be expensive.  When querying LocalStrategy 
 keyspaces, we can safely skip this step.  On a 15 node cluster with vnodes, 
 skipping this saves about 100ms.  This makes a big difference for the 
 drivers, which frequently execute queries like select * from system.peers 
 and select * from system.local.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[2/8] git commit: Revert Don't shut ExpiringMap down.

[brandonwilliams]

Revert Don't shut ExpiringMap down.

This reverts commit d4ec31f21eb83502bd523c057bce2d3b249b3d80.

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4f47a44d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4f47a44d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4f47a44d

Branch: refs/heads/cassandra-2.1
Commit: 4f47a44d05a1c9b8e284c2d8c084edb7c0376d1d
Parents: 56d84a7
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Apr 2 14:53:25 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Apr 2 14:53:25 2014 -0500

--
 CHANGES.txt |  1 -
 .../apache/cassandra/net/MessagingService.java  | 13 +-
 .../org/apache/cassandra/utils/ExpiringMap.java | 47 +++-
 .../apache/cassandra/service/RemoveTest.java|  1 +
 4 files changed, 47 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4f47a44d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d999e88..9003309 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,5 @@
 2.0.7
  * Allow compaction of system tables during startup (CASSANDRA-6913)
- * Don't shut ExpiringMap down (CASSANDRA-6948)
  * Restrict Windows to parallel repairs (CASSANDRA-6907)
  * (Hadoop) Allow manually specifying start/end tokens in CFIF (CASSANDRA-6436)
  * Fix NPE in MeteredFlusher (CASSANDRA-6820)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4f47a44d/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java 
b/src/java/org/apache/cassandra/net/MessagingService.java
index 094e861..ad86bbd 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -392,6 +392,7 @@ public final class MessagingService implements 
MessagingServiceMBean
  */
 public void listen(InetAddress localEp) throws ConfigurationException
 {
+callbacks.reset(); // hack to allow tests to stop/restart MS
 for (ServerSocket ss : getServerSockets(localEp))
 {
 SocketThread th = new SocketThread(ss, ACCEPT- + localEp);
@@ -535,7 +536,7 @@ public final class MessagingService implements 
MessagingServiceMBean
 {
 assert message.verb != Verb.MUTATION; // mutations need to call the 
overload with a ConsistencyLevel
 int messageId = nextId();
-ExpiringMap.CacheableObjectCallbackInfo previous = 
callbacks.put(messageId, new CallbackInfo(to, cb, 
callbackDeserializers.get(message.verb)), timeout);
+CallbackInfo previous = callbacks.put(messageId, new CallbackInfo(to, 
cb, callbackDeserializers.get(message.verb)), timeout);
 assert previous == null : String.format(Callback already exists for 
id %d! (%s), messageId, previous);
 return messageId;
 }
@@ -544,7 +545,7 @@ public final class MessagingService implements 
MessagingServiceMBean
 {
 assert message.verb == Verb.MUTATION || message.verb == 
Verb.COUNTER_MUTATION;
 int messageId = nextId();
-ExpiringMap.CacheableObjectCallbackInfo previous = 
callbacks.put(messageId, new WriteCallbackInfo(to, cb, message, 
callbackDeserializers.get(message.verb), consistencyLevel), timeout);
+CallbackInfo previous = callbacks.put(messageId, new 
WriteCallbackInfo(to, cb, message, callbackDeserializers.get(message.verb), 
consistencyLevel), timeout);
 assert previous == null : String.format(Callback already exists for 
id %d! (%s), messageId, previous);
 return messageId;
 }
@@ -653,6 +654,11 @@ public final class MessagingService implements 
MessagingServiceMBean
 subscribers.add(subcriber);
 }
 
+public void clearCallbacksUnsafe()
+{
+callbacks.reset();
+}
+
 /**
  * Wait for callbacks and don't allow any more to be created (since they 
could require writing hints)
  */
@@ -662,6 +668,9 @@ public final class MessagingService implements 
MessagingServiceMBean
 // We may need to schedule hints on the mutation stage, so it's 
erroneous to shut down the mutation stage first
 assert !StageManager.getStage(Stage.MUTATION).isShutdown();
 
+// the important part
+callbacks.shutdownBlocking();
+
 // attempt to humor tests that try to stop and restart MS
 try
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4f47a44d/src/java/org/apache/cassandra/utils/ExpiringMap.java
--
diff