date:20140430


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985278#comment-13985278
 ] 

Benedict commented on CASSANDRA-6861:
-

See CASSANDRA-7045 for one possible explanation. For comparison see performance 
of async and hsha thrift, which in my experiments perform, either as poorly, or 
half as poorly. Further evidence for this hypothesis is that the gap narrows 
for larger messages.

 Optimise our Netty 4 integration
 

 Key: CASSANDRA-6861
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6861
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: T Jake Luciani
Priority: Minor
  Labels: performance
 Fix For: 2.1 beta2


 Now we've upgraded to Netty 4, we're generating a lot of garbage that could 
 be avoided, so we should probably stop that. Should be reasonably easy to 
 hook into Netty's pooled buffers, returning them to the pool once a given 
 message is completed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Preserves CQL metadata when updating table from thrift

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-1.2 7f019804c - 10527498a


Preserves CQL metadata when updating table from thrift

patch by mishail; reviewed by iamaleksey  slebresne for CASSANDRA-6831


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/10527498
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/10527498
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/10527498

Branch: refs/heads/cassandra-1.2
Commit: 10527498a340feb7333b3c2b0252029fe6a840c7
Parents: 7f01980
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 30 11:19:57 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 30 11:19:57 2014 +0200

--
 CHANGES.txt  |  1 +
 src/java/org/apache/cassandra/config/CFMetaData.java | 11 ---
 .../org/apache/cassandra/thrift/CassandraServer.java | 10 ++
 3 files changed, 11 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/10527498/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e8d6a8d..fa9a156 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -14,6 +14,7 @@
  * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
  * Always clean up references in SerializingCache (CASSANDRA-6994)
  * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
+ * Preserves CQL metadata when updating table from thrift (CASSANDRA-6831)
 
 
 1.2.16

http://git-wip-us.apache.org/repos/asf/cassandra/blob/10527498/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 85c3dcb..9e3ceb7 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -802,17 +802,6 @@ public final class CFMetaData
 minCompactionThreshold = cfm.minCompactionThreshold;
 maxCompactionThreshold = cfm.maxCompactionThreshold;
 
-/*
- * Because thrift updates don't know about aliases, we should ignore
- * the case where the new aliases are empty.
- */
-if (!cfm.keyAliases.isEmpty())
-keyAliases = cfm.keyAliases;
-if (!cfm.columnAliases.isEmpty())
-columnAliases = cfm.columnAliases;
-if (cfm.valueAlias != null)
-valueAlias = cfm.valueAlias;
-
 bloomFilterFpChance = cfm.bloomFilterFpChance;
 caching = cfm.caching;
 populateIoCacheOnFlush = cfm.populateIoCacheOnFlush;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/10527498/src/java/org/apache/cassandra/thrift/CassandraServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/CassandraServer.java 
b/src/java/org/apache/cassandra/thrift/CassandraServer.java
index ec7a37d..588f732 100644
--- a/src/java/org/apache/cassandra/thrift/CassandraServer.java
+++ b/src/java/org/apache/cassandra/thrift/CassandraServer.java
@@ -1427,6 +1427,16 @@ public class CassandraServer implements Cassandra.Iface
 
 CFMetaData.applyImplicitDefaults(cf_def);
 CFMetaData cfm = CFMetaData.fromThrift(cf_def);
+
+/*
+ * CASSANDRA-6831: Because thrift updates don't know about aliases,
+ * we should copy them from the original CFM
+ */
+if (!cf_def.isSetKey_alias())
+cfm.keyAliases(oldCfm.getKeyAliases());
+cfm.columnAliases(oldCfm.getColumnAliases());
+cfm.valueAlias(oldCfm.getValueAlias());
+
 CFMetaData.validateCompactionOptions(cfm.compactionStrategyClass, 
cfm.compactionStrategyOptions, false);
 cfm.addDefaultIndexNames();
 MigrationManager.announceColumnFamilyUpdate(cfm);

[2/2] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

Merge branch 'cassandra-1.2' into cassandra-2.0

Conflicts:
src/java/org/apache/cassandra/thrift/CassandraServer.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b4337228
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b4337228
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b4337228

Branch: refs/heads/cassandra-2.0
Commit: b4337228df7178183a643a8f201e9389cab36ab3
Parents: 90d086a 1052749
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 30 11:24:21 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 30 11:24:21 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java | 47 +---
 .../cassandra/thrift/CassandraServer.java   |  3 +-
 .../unit/org/apache/cassandra/SchemaLoader.java | 16 ++-
 .../apache/cassandra/config/CFMetaDataTest.java | 32 -
 5 files changed, 65 insertions(+), 34 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4337228/CHANGES.txt
--
diff --cc CHANGES.txt
index 428f8fc,fa9a156..e25e71f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -23,64 -11,13 +23,65 @@@ Merged from 1.2
   * Fix CQLSH parsing of functions and BLOB literals (CASSANDRA-7018)
   * Require nodetool rebuild_index to specify index names (CASSANDRA-7038)
   * Ensure that batchlog and hint timeouts do not produce hints 
(CASSANDRA-7058)
 - * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
   * Always clean up references in SerializingCache (CASSANDRA-6994)
 + * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
   * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
+  * Preserves CQL metadata when updating table from thrift (CASSANDRA-6831)
  
  
 -1.2.16
 +2.0.7
 + * Put nodes in hibernate when join_ring is false (CASSANDRA-6961)
 + * Avoid early loading of non-system keyspaces before compaction-leftovers 
 +   cleanup at startup (CASSANDRA-6913)
 + * Restrict Windows to parallel repairs (CASSANDRA-6907)
 + * (Hadoop) Allow manually specifying start/end tokens in CFIF 
(CASSANDRA-6436)
 + * Fix NPE in MeteredFlusher (CASSANDRA-6820)
 + * Fix race processing range scan responses (CASSANDRA-6820)
 + * Allow deleting snapshots from dropped keyspaces (CASSANDRA-6821)
 + * Add uuid() function (CASSANDRA-6473)
 + * Omit tombstones from schema digests (CASSANDRA-6862)
 + * Include correct consistencyLevel in LWT timeout (CASSANDRA-6884)
 + * Lower chances for losing new SSTables during nodetool refresh and
 +   ColumnFamilyStore.loadNewSSTables (CASSANDRA-6514)
 + * Add support for DELETE ... IF EXISTS to CQL3 (CASSANDRA-5708)
 + * Update hadoop_cql3_word_count example (CASSANDRA-6793)
 + * Fix handling of RejectedExecution in sync Thrift server (CASSANDRA-6788)
 + * Log more information when exceeding tombstone_warn_threshold 
(CASSANDRA-6865)
 + * Fix truncate to not abort due to unreachable fat clients (CASSANDRA-6864)
 + * Fix schema concurrency exceptions (CASSANDRA-6841)
 + * Fix leaking validator FH in StreamWriter (CASSANDRA-6832)
 + * Fix saving triggers to schema (CASSANDRA-6789)
 + * Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
 + * Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
 + * Fix static counter columns (CASSANDRA-6827)
 + * Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
 + * Fix CompactionManager.needsCleanup (CASSANDRA-6845)
 + * Correctly compare BooleanType values other than 0 and 1 (CASSANDRA-6779)
 + * Read message id as string from earlier versions (CASSANDRA-6840)
 + * Properly use the Paxos consistency for (non-protocol) batch 
(CASSANDRA-6837)
 + * Add paranoid disk failure option (CASSANDRA-6646)
 + * Improve PerRowSecondaryIndex performance (CASSANDRA-6876)
 + * Extend triggers to support CAS updates (CASSANDRA-6882)
 + * Static columns with IF NOT EXISTS don't always work as expected 
(CASSANDRA-6873)
 + * Fix paging with SELECT DISTINCT (CASSANDRA-6857)
 + * Fix UnsupportedOperationException on CAS timeout (CASSANDRA-6923)
 + * Improve MeteredFlusher handling of MF-unaffected column families
 +   (CASSANDRA-6867)
 + * Add CqlRecordReader using native pagination (CASSANDRA-6311)
 + * Add QueryHandler interface (CASSANDRA-6659)
 + * Track liveRatio per-memtable, not per-CF (CASSANDRA-6945)
 + * Make sure upgradesstables keeps sstable level (CASSANDRA-6958)
 + * Fix LIMIT with static columns (CASSANDRA-6956)
 + * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
 + * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
 + * Fix bad skip of sstables on

[1/2] git commit: Preserves CQL metadata when updating table from thrift

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 90d086a15 - b4337228d


Preserves CQL metadata when updating table from thrift

patch by mishail; reviewed by iamaleksey  slebresne for CASSANDRA-6831


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/10527498
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/10527498
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/10527498

Branch: refs/heads/cassandra-2.0
Commit: 10527498a340feb7333b3c2b0252029fe6a840c7
Parents: 7f01980
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 30 11:19:57 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 30 11:19:57 2014 +0200

--
 CHANGES.txt  |  1 +
 src/java/org/apache/cassandra/config/CFMetaData.java | 11 ---
 .../org/apache/cassandra/thrift/CassandraServer.java | 10 ++
 3 files changed, 11 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/10527498/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e8d6a8d..fa9a156 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -14,6 +14,7 @@
  * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
  * Always clean up references in SerializingCache (CASSANDRA-6994)
  * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
+ * Preserves CQL metadata when updating table from thrift (CASSANDRA-6831)
 
 
 1.2.16

http://git-wip-us.apache.org/repos/asf/cassandra/blob/10527498/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 85c3dcb..9e3ceb7 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -802,17 +802,6 @@ public final class CFMetaData
 minCompactionThreshold = cfm.minCompactionThreshold;
 maxCompactionThreshold = cfm.maxCompactionThreshold;
 
-/*
- * Because thrift updates don't know about aliases, we should ignore
- * the case where the new aliases are empty.
- */
-if (!cfm.keyAliases.isEmpty())
-keyAliases = cfm.keyAliases;
-if (!cfm.columnAliases.isEmpty())
-columnAliases = cfm.columnAliases;
-if (cfm.valueAlias != null)
-valueAlias = cfm.valueAlias;
-
 bloomFilterFpChance = cfm.bloomFilterFpChance;
 caching = cfm.caching;
 populateIoCacheOnFlush = cfm.populateIoCacheOnFlush;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/10527498/src/java/org/apache/cassandra/thrift/CassandraServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/CassandraServer.java 
b/src/java/org/apache/cassandra/thrift/CassandraServer.java
index ec7a37d..588f732 100644
--- a/src/java/org/apache/cassandra/thrift/CassandraServer.java
+++ b/src/java/org/apache/cassandra/thrift/CassandraServer.java
@@ -1427,6 +1427,16 @@ public class CassandraServer implements Cassandra.Iface
 
 CFMetaData.applyImplicitDefaults(cf_def);
 CFMetaData cfm = CFMetaData.fromThrift(cf_def);
+
+/*
+ * CASSANDRA-6831: Because thrift updates don't know about aliases,
+ * we should copy them from the original CFM
+ */
+if (!cf_def.isSetKey_alias())
+cfm.keyAliases(oldCfm.getKeyAliases());
+cfm.columnAliases(oldCfm.getColumnAliases());
+cfm.valueAlias(oldCfm.getValueAlias());
+
 CFMetaData.validateCompactionOptions(cfm.compactionStrategyClass, 
cfm.compactionStrategyOptions, false);
 cfm.addDefaultIndexNames();
 MigrationManager.announceColumnFamilyUpdate(cfm);

[2/3] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

Merge branch 'cassandra-1.2' into cassandra-2.0

Conflicts:
src/java/org/apache/cassandra/thrift/CassandraServer.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b4337228
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b4337228
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b4337228

Branch: refs/heads/cassandra-2.1
Commit: b4337228df7178183a643a8f201e9389cab36ab3
Parents: 90d086a 1052749
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 30 11:24:21 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 30 11:24:21 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java | 47 +---
 .../cassandra/thrift/CassandraServer.java   |  3 +-
 .../unit/org/apache/cassandra/SchemaLoader.java | 16 ++-
 .../apache/cassandra/config/CFMetaDataTest.java | 32 -
 5 files changed, 65 insertions(+), 34 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4337228/CHANGES.txt
--
diff --cc CHANGES.txt
index 428f8fc,fa9a156..e25e71f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -23,64 -11,13 +23,65 @@@ Merged from 1.2
   * Fix CQLSH parsing of functions and BLOB literals (CASSANDRA-7018)
   * Require nodetool rebuild_index to specify index names (CASSANDRA-7038)
   * Ensure that batchlog and hint timeouts do not produce hints 
(CASSANDRA-7058)
 - * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
   * Always clean up references in SerializingCache (CASSANDRA-6994)
 + * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
   * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
+  * Preserves CQL metadata when updating table from thrift (CASSANDRA-6831)
  
  
 -1.2.16
 +2.0.7
 + * Put nodes in hibernate when join_ring is false (CASSANDRA-6961)
 + * Avoid early loading of non-system keyspaces before compaction-leftovers 
 +   cleanup at startup (CASSANDRA-6913)
 + * Restrict Windows to parallel repairs (CASSANDRA-6907)
 + * (Hadoop) Allow manually specifying start/end tokens in CFIF 
(CASSANDRA-6436)
 + * Fix NPE in MeteredFlusher (CASSANDRA-6820)
 + * Fix race processing range scan responses (CASSANDRA-6820)
 + * Allow deleting snapshots from dropped keyspaces (CASSANDRA-6821)
 + * Add uuid() function (CASSANDRA-6473)
 + * Omit tombstones from schema digests (CASSANDRA-6862)
 + * Include correct consistencyLevel in LWT timeout (CASSANDRA-6884)
 + * Lower chances for losing new SSTables during nodetool refresh and
 +   ColumnFamilyStore.loadNewSSTables (CASSANDRA-6514)
 + * Add support for DELETE ... IF EXISTS to CQL3 (CASSANDRA-5708)
 + * Update hadoop_cql3_word_count example (CASSANDRA-6793)
 + * Fix handling of RejectedExecution in sync Thrift server (CASSANDRA-6788)
 + * Log more information when exceeding tombstone_warn_threshold 
(CASSANDRA-6865)
 + * Fix truncate to not abort due to unreachable fat clients (CASSANDRA-6864)
 + * Fix schema concurrency exceptions (CASSANDRA-6841)
 + * Fix leaking validator FH in StreamWriter (CASSANDRA-6832)
 + * Fix saving triggers to schema (CASSANDRA-6789)
 + * Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
 + * Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
 + * Fix static counter columns (CASSANDRA-6827)
 + * Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
 + * Fix CompactionManager.needsCleanup (CASSANDRA-6845)
 + * Correctly compare BooleanType values other than 0 and 1 (CASSANDRA-6779)
 + * Read message id as string from earlier versions (CASSANDRA-6840)
 + * Properly use the Paxos consistency for (non-protocol) batch 
(CASSANDRA-6837)
 + * Add paranoid disk failure option (CASSANDRA-6646)
 + * Improve PerRowSecondaryIndex performance (CASSANDRA-6876)
 + * Extend triggers to support CAS updates (CASSANDRA-6882)
 + * Static columns with IF NOT EXISTS don't always work as expected 
(CASSANDRA-6873)
 + * Fix paging with SELECT DISTINCT (CASSANDRA-6857)
 + * Fix UnsupportedOperationException on CAS timeout (CASSANDRA-6923)
 + * Improve MeteredFlusher handling of MF-unaffected column families
 +   (CASSANDRA-6867)
 + * Add CqlRecordReader using native pagination (CASSANDRA-6311)
 + * Add QueryHandler interface (CASSANDRA-6659)
 + * Track liveRatio per-memtable, not per-CF (CASSANDRA-6945)
 + * Make sure upgradesstables keeps sstable level (CASSANDRA-6958)
 + * Fix LIMIT with static columns (CASSANDRA-6956)
 + * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
 + * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
 + * Fix bad skip of sstables on

[1/3] git commit: Preserves CQL metadata when updating table from thrift

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 6b5b7f519 - 2269adba6


Preserves CQL metadata when updating table from thrift

patch by mishail; reviewed by iamaleksey  slebresne for CASSANDRA-6831


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/10527498
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/10527498
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/10527498

Branch: refs/heads/cassandra-2.1
Commit: 10527498a340feb7333b3c2b0252029fe6a840c7
Parents: 7f01980
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 30 11:19:57 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 30 11:19:57 2014 +0200

--
 CHANGES.txt  |  1 +
 src/java/org/apache/cassandra/config/CFMetaData.java | 11 ---
 .../org/apache/cassandra/thrift/CassandraServer.java | 10 ++
 3 files changed, 11 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/10527498/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e8d6a8d..fa9a156 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -14,6 +14,7 @@
  * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
  * Always clean up references in SerializingCache (CASSANDRA-6994)
  * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
+ * Preserves CQL metadata when updating table from thrift (CASSANDRA-6831)
 
 
 1.2.16

http://git-wip-us.apache.org/repos/asf/cassandra/blob/10527498/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 85c3dcb..9e3ceb7 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -802,17 +802,6 @@ public final class CFMetaData
 minCompactionThreshold = cfm.minCompactionThreshold;
 maxCompactionThreshold = cfm.maxCompactionThreshold;
 
-/*
- * Because thrift updates don't know about aliases, we should ignore
- * the case where the new aliases are empty.
- */
-if (!cfm.keyAliases.isEmpty())
-keyAliases = cfm.keyAliases;
-if (!cfm.columnAliases.isEmpty())
-columnAliases = cfm.columnAliases;
-if (cfm.valueAlias != null)
-valueAlias = cfm.valueAlias;
-
 bloomFilterFpChance = cfm.bloomFilterFpChance;
 caching = cfm.caching;
 populateIoCacheOnFlush = cfm.populateIoCacheOnFlush;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/10527498/src/java/org/apache/cassandra/thrift/CassandraServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/CassandraServer.java 
b/src/java/org/apache/cassandra/thrift/CassandraServer.java
index ec7a37d..588f732 100644
--- a/src/java/org/apache/cassandra/thrift/CassandraServer.java
+++ b/src/java/org/apache/cassandra/thrift/CassandraServer.java
@@ -1427,6 +1427,16 @@ public class CassandraServer implements Cassandra.Iface
 
 CFMetaData.applyImplicitDefaults(cf_def);
 CFMetaData cfm = CFMetaData.fromThrift(cf_def);
+
+/*
+ * CASSANDRA-6831: Because thrift updates don't know about aliases,
+ * we should copy them from the original CFM
+ */
+if (!cf_def.isSetKey_alias())
+cfm.keyAliases(oldCfm.getKeyAliases());
+cfm.columnAliases(oldCfm.getColumnAliases());
+cfm.valueAlias(oldCfm.getValueAlias());
+
 CFMetaData.validateCompactionOptions(cfm.compactionStrategyClass, 
cfm.compactionStrategyOptions, false);
 cfm.addDefaultIndexNames();
 MigrationManager.announceColumnFamilyUpdate(cfm);

[3/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/config/CFMetaData.java
test/unit/org/apache/cassandra/config/CFMetaDataTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2269adba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2269adba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2269adba

Branch: refs/heads/cassandra-2.1
Commit: 2269adba6c28e24f60883700f657d36fff7b9d3f
Parents: 6b5b7f5 b433722
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 30 11:35:15 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 30 11:35:15 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java | 45 +---
 .../cassandra/thrift/CassandraServer.java   |  3 +-
 .../unit/org/apache/cassandra/SchemaLoader.java | 16 ++-
 .../apache/cassandra/config/CFMetaDataTest.java |  5 ++-
 5 files changed, 60 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2269adba/CHANGES.txt
--
diff --cc CHANGES.txt
index b60feb8,e25e71f..64e5afb
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -111,15 -81,7 +111,16 @@@ Merged from 2.0
 (CASSANDRA-6906)
   * Fix SSTable not released if stream session fails (CASSANDRA-6818)
   * Avoid build failure due to ANTLR timeout (CASSANDRA-6991)
 + * Queries on compact tables can return more rows that requested 
(CASSANDRA-7052)
 + * USING TIMESTAMP for batches does not work (CASSANDRA-7053)
 + * Fix performance regression from CASSANDRA-5614 (CASSANDRA-6949)
 + * Ensure that batchlog and hint timeouts do not produce hints 
(CASSANDRA-7058)
 + * Merge groupable mutations in TriggerExecutor#execute() (CASSANDRA-7047)
 + * Plug holes in resource release when wiring up StreamSession 
(CASSANDRA-7073)
 + * Re-add parameter columns to tracing session (CASSANDRA-6942)
++ * Preserves CQL metadata when updating table from thrift (CASSANDRA-6831)
  Merged from 1.2:
 + * Fix nodetool display with vnodes (CASSANDRA-7082)
   * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
   * add extra SSL cipher suites (CASSANDRA-6613)
   * fix nodetool getsstables for blob PK (CASSANDRA-6803)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2269adba/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --cc src/java/org/apache/cassandra/config/CFMetaData.java
index b4b3fbe,9ca41ac..2e531e8
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@@ -1001,6 -895,44 +1001,44 @@@ public final class CFMetaDat
  
  public static CFMetaData fromThrift(org.apache.cassandra.thrift.CfDef 
cf_def) throws InvalidRequestException, ConfigurationException
  {
+ CFMetaData cfm = internalFromThrift(cf_def);
+ 
+ if (cf_def.isSetKey_alias()  !(cfm.keyValidator instanceof 
CompositeType))
 -cfm.column_metadata.put(cf_def.key_alias, 
ColumnDefinition.partitionKeyDef(cf_def.key_alias, cfm.keyValidator, null));
++
cfm.addOrReplaceColumnDefinition(ColumnDefinition.partitionKeyDef(cfm, 
cf_def.key_alias, cfm.keyValidator, null));
+ 
+ return cfm.rebuild();
+ }
+ 
+ public static CFMetaData 
fromThriftForUpdate(org.apache.cassandra.thrift.CfDef cf_def, CFMetaData 
toUpdate) throws InvalidRequestException, ConfigurationException
+ {
+ CFMetaData cfm = internalFromThrift(cf_def);
+ 
+ // Thrift update can't have CQL metadata, and so we'll copy the ones 
of the updated metadata (to make
+ // sure we don't override anything existing -- see #6831). One 
exception (for historical reasons) is
+ // the partition key column name however, which can be provided 
through thrift. If it is, make sure
+ // we use the one of the update.
+ boolean hasKeyAlias = cf_def.isSetKey_alias()  !(cfm.keyValidator 
instanceof CompositeType);
+ if (hasKeyAlias)
 -cfm.column_metadata.put(cf_def.key_alias, 
ColumnDefinition.partitionKeyDef(cf_def.key_alias, cfm.keyValidator, null));
++
cfm.addOrReplaceColumnDefinition(ColumnDefinition.partitionKeyDef(cfm, 
cf_def.key_alias, cfm.keyValidator, null));
+ 
+ for (ColumnDefinition def : toUpdate.allColumns())
+ {
+ // isPartOfCellName basically means 'is not just a CQL metadata'
+ if (def.isPartOfCellName())
+ continue;
+ 
 -if (def.type == ColumnDefinition.Type.PARTITION_KEY  
hasKeyAlias)
++if (def.kind ==

[1/4] git commit: Preserves CQL metadata when updating table from thrift

Repository: cassandra
Updated Branches:
  refs/heads/trunk c06ba25a5 - d9f06a3be


Preserves CQL metadata when updating table from thrift

patch by mishail; reviewed by iamaleksey  slebresne for CASSANDRA-6831


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/10527498
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/10527498
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/10527498

Branch: refs/heads/trunk
Commit: 10527498a340feb7333b3c2b0252029fe6a840c7
Parents: 7f01980
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 30 11:19:57 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 30 11:19:57 2014 +0200

--
 CHANGES.txt  |  1 +
 src/java/org/apache/cassandra/config/CFMetaData.java | 11 ---
 .../org/apache/cassandra/thrift/CassandraServer.java | 10 ++
 3 files changed, 11 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/10527498/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e8d6a8d..fa9a156 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -14,6 +14,7 @@
  * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
  * Always clean up references in SerializingCache (CASSANDRA-6994)
  * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
+ * Preserves CQL metadata when updating table from thrift (CASSANDRA-6831)
 
 
 1.2.16

http://git-wip-us.apache.org/repos/asf/cassandra/blob/10527498/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 85c3dcb..9e3ceb7 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -802,17 +802,6 @@ public final class CFMetaData
 minCompactionThreshold = cfm.minCompactionThreshold;
 maxCompactionThreshold = cfm.maxCompactionThreshold;
 
-/*
- * Because thrift updates don't know about aliases, we should ignore
- * the case where the new aliases are empty.
- */
-if (!cfm.keyAliases.isEmpty())
-keyAliases = cfm.keyAliases;
-if (!cfm.columnAliases.isEmpty())
-columnAliases = cfm.columnAliases;
-if (cfm.valueAlias != null)
-valueAlias = cfm.valueAlias;
-
 bloomFilterFpChance = cfm.bloomFilterFpChance;
 caching = cfm.caching;
 populateIoCacheOnFlush = cfm.populateIoCacheOnFlush;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/10527498/src/java/org/apache/cassandra/thrift/CassandraServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/CassandraServer.java 
b/src/java/org/apache/cassandra/thrift/CassandraServer.java
index ec7a37d..588f732 100644
--- a/src/java/org/apache/cassandra/thrift/CassandraServer.java
+++ b/src/java/org/apache/cassandra/thrift/CassandraServer.java
@@ -1427,6 +1427,16 @@ public class CassandraServer implements Cassandra.Iface
 
 CFMetaData.applyImplicitDefaults(cf_def);
 CFMetaData cfm = CFMetaData.fromThrift(cf_def);
+
+/*
+ * CASSANDRA-6831: Because thrift updates don't know about aliases,
+ * we should copy them from the original CFM
+ */
+if (!cf_def.isSetKey_alias())
+cfm.keyAliases(oldCfm.getKeyAliases());
+cfm.columnAliases(oldCfm.getColumnAliases());
+cfm.valueAlias(oldCfm.getValueAlias());
+
 CFMetaData.validateCompactionOptions(cfm.compactionStrategyClass, 
cfm.compactionStrategyOptions, false);
 cfm.addDefaultIndexNames();
 MigrationManager.announceColumnFamilyUpdate(cfm);

[4/4] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d9f06a3b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d9f06a3b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d9f06a3b

Branch: refs/heads/trunk
Commit: d9f06a3be095a6576974533d921a14819e43a91e
Parents: c06ba25 2269adb
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 30 11:36:04 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 30 11:36:04 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java | 45 +---
 .../cassandra/thrift/CassandraServer.java   |  3 +-
 .../unit/org/apache/cassandra/SchemaLoader.java | 16 ++-
 .../apache/cassandra/config/CFMetaDataTest.java |  5 ++-
 5 files changed, 60 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d9f06a3b/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d9f06a3b/src/java/org/apache/cassandra/config/CFMetaData.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d9f06a3b/src/java/org/apache/cassandra/thrift/CassandraServer.java
--

[2/4] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

Merge branch 'cassandra-1.2' into cassandra-2.0

Conflicts:
src/java/org/apache/cassandra/thrift/CassandraServer.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b4337228
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b4337228
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b4337228

Branch: refs/heads/trunk
Commit: b4337228df7178183a643a8f201e9389cab36ab3
Parents: 90d086a 1052749
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 30 11:24:21 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 30 11:24:21 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java | 47 +---
 .../cassandra/thrift/CassandraServer.java   |  3 +-
 .../unit/org/apache/cassandra/SchemaLoader.java | 16 ++-
 .../apache/cassandra/config/CFMetaDataTest.java | 32 -
 5 files changed, 65 insertions(+), 34 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4337228/CHANGES.txt
--
diff --cc CHANGES.txt
index 428f8fc,fa9a156..e25e71f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -23,64 -11,13 +23,65 @@@ Merged from 1.2
   * Fix CQLSH parsing of functions and BLOB literals (CASSANDRA-7018)
   * Require nodetool rebuild_index to specify index names (CASSANDRA-7038)
   * Ensure that batchlog and hint timeouts do not produce hints 
(CASSANDRA-7058)
 - * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
   * Always clean up references in SerializingCache (CASSANDRA-6994)
 + * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
   * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
+  * Preserves CQL metadata when updating table from thrift (CASSANDRA-6831)
  
  
 -1.2.16
 +2.0.7
 + * Put nodes in hibernate when join_ring is false (CASSANDRA-6961)
 + * Avoid early loading of non-system keyspaces before compaction-leftovers 
 +   cleanup at startup (CASSANDRA-6913)
 + * Restrict Windows to parallel repairs (CASSANDRA-6907)
 + * (Hadoop) Allow manually specifying start/end tokens in CFIF 
(CASSANDRA-6436)
 + * Fix NPE in MeteredFlusher (CASSANDRA-6820)
 + * Fix race processing range scan responses (CASSANDRA-6820)
 + * Allow deleting snapshots from dropped keyspaces (CASSANDRA-6821)
 + * Add uuid() function (CASSANDRA-6473)
 + * Omit tombstones from schema digests (CASSANDRA-6862)
 + * Include correct consistencyLevel in LWT timeout (CASSANDRA-6884)
 + * Lower chances for losing new SSTables during nodetool refresh and
 +   ColumnFamilyStore.loadNewSSTables (CASSANDRA-6514)
 + * Add support for DELETE ... IF EXISTS to CQL3 (CASSANDRA-5708)
 + * Update hadoop_cql3_word_count example (CASSANDRA-6793)
 + * Fix handling of RejectedExecution in sync Thrift server (CASSANDRA-6788)
 + * Log more information when exceeding tombstone_warn_threshold 
(CASSANDRA-6865)
 + * Fix truncate to not abort due to unreachable fat clients (CASSANDRA-6864)
 + * Fix schema concurrency exceptions (CASSANDRA-6841)
 + * Fix leaking validator FH in StreamWriter (CASSANDRA-6832)
 + * Fix saving triggers to schema (CASSANDRA-6789)
 + * Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
 + * Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
 + * Fix static counter columns (CASSANDRA-6827)
 + * Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
 + * Fix CompactionManager.needsCleanup (CASSANDRA-6845)
 + * Correctly compare BooleanType values other than 0 and 1 (CASSANDRA-6779)
 + * Read message id as string from earlier versions (CASSANDRA-6840)
 + * Properly use the Paxos consistency for (non-protocol) batch 
(CASSANDRA-6837)
 + * Add paranoid disk failure option (CASSANDRA-6646)
 + * Improve PerRowSecondaryIndex performance (CASSANDRA-6876)
 + * Extend triggers to support CAS updates (CASSANDRA-6882)
 + * Static columns with IF NOT EXISTS don't always work as expected 
(CASSANDRA-6873)
 + * Fix paging with SELECT DISTINCT (CASSANDRA-6857)
 + * Fix UnsupportedOperationException on CAS timeout (CASSANDRA-6923)
 + * Improve MeteredFlusher handling of MF-unaffected column families
 +   (CASSANDRA-6867)
 + * Add CqlRecordReader using native pagination (CASSANDRA-6311)
 + * Add QueryHandler interface (CASSANDRA-6659)
 + * Track liveRatio per-memtable, not per-CF (CASSANDRA-6945)
 + * Make sure upgradesstables keeps sstable level (CASSANDRA-6958)
 + * Fix LIMIT with static columns (CASSANDRA-6956)
 + * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
 + * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
 + * Fix bad skip of sstables on slice

[3/4] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/config/CFMetaData.java
test/unit/org/apache/cassandra/config/CFMetaDataTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2269adba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2269adba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2269adba

Branch: refs/heads/trunk
Commit: 2269adba6c28e24f60883700f657d36fff7b9d3f
Parents: 6b5b7f5 b433722
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Apr 30 11:35:15 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Apr 30 11:35:15 2014 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java | 45 +---
 .../cassandra/thrift/CassandraServer.java   |  3 +-
 .../unit/org/apache/cassandra/SchemaLoader.java | 16 ++-
 .../apache/cassandra/config/CFMetaDataTest.java |  5 ++-
 5 files changed, 60 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2269adba/CHANGES.txt
--
diff --cc CHANGES.txt
index b60feb8,e25e71f..64e5afb
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -111,15 -81,7 +111,16 @@@ Merged from 2.0
 (CASSANDRA-6906)
   * Fix SSTable not released if stream session fails (CASSANDRA-6818)
   * Avoid build failure due to ANTLR timeout (CASSANDRA-6991)
 + * Queries on compact tables can return more rows that requested 
(CASSANDRA-7052)
 + * USING TIMESTAMP for batches does not work (CASSANDRA-7053)
 + * Fix performance regression from CASSANDRA-5614 (CASSANDRA-6949)
 + * Ensure that batchlog and hint timeouts do not produce hints 
(CASSANDRA-7058)
 + * Merge groupable mutations in TriggerExecutor#execute() (CASSANDRA-7047)
 + * Plug holes in resource release when wiring up StreamSession 
(CASSANDRA-7073)
 + * Re-add parameter columns to tracing session (CASSANDRA-6942)
++ * Preserves CQL metadata when updating table from thrift (CASSANDRA-6831)
  Merged from 1.2:
 + * Fix nodetool display with vnodes (CASSANDRA-7082)
   * Add UNLOGGED, COUNTER options to BATCH documentation (CASSANDRA-6816)
   * add extra SSL cipher suites (CASSANDRA-6613)
   * fix nodetool getsstables for blob PK (CASSANDRA-6803)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2269adba/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --cc src/java/org/apache/cassandra/config/CFMetaData.java
index b4b3fbe,9ca41ac..2e531e8
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@@ -1001,6 -895,44 +1001,44 @@@ public final class CFMetaDat
  
  public static CFMetaData fromThrift(org.apache.cassandra.thrift.CfDef 
cf_def) throws InvalidRequestException, ConfigurationException
  {
+ CFMetaData cfm = internalFromThrift(cf_def);
+ 
+ if (cf_def.isSetKey_alias()  !(cfm.keyValidator instanceof 
CompositeType))
 -cfm.column_metadata.put(cf_def.key_alias, 
ColumnDefinition.partitionKeyDef(cf_def.key_alias, cfm.keyValidator, null));
++
cfm.addOrReplaceColumnDefinition(ColumnDefinition.partitionKeyDef(cfm, 
cf_def.key_alias, cfm.keyValidator, null));
+ 
+ return cfm.rebuild();
+ }
+ 
+ public static CFMetaData 
fromThriftForUpdate(org.apache.cassandra.thrift.CfDef cf_def, CFMetaData 
toUpdate) throws InvalidRequestException, ConfigurationException
+ {
+ CFMetaData cfm = internalFromThrift(cf_def);
+ 
+ // Thrift update can't have CQL metadata, and so we'll copy the ones 
of the updated metadata (to make
+ // sure we don't override anything existing -- see #6831). One 
exception (for historical reasons) is
+ // the partition key column name however, which can be provided 
through thrift. If it is, make sure
+ // we use the one of the update.
+ boolean hasKeyAlias = cf_def.isSetKey_alias()  !(cfm.keyValidator 
instanceof CompositeType);
+ if (hasKeyAlias)
 -cfm.column_metadata.put(cf_def.key_alias, 
ColumnDefinition.partitionKeyDef(cf_def.key_alias, cfm.keyValidator, null));
++
cfm.addOrReplaceColumnDefinition(ColumnDefinition.partitionKeyDef(cfm, 
cf_def.key_alias, cfm.keyValidator, null));
+ 
+ for (ColumnDefinition def : toUpdate.allColumns())
+ {
+ // isPartOfCellName basically means 'is not just a CQL metadata'
+ if (def.isPartOfCellName())
+ continue;
+ 
 -if (def.type == ColumnDefinition.Type.PARTITION_KEY  
hasKeyAlias)
++if (def.kind ==

[jira] [Resolved] (CASSANDRA-6874) CQL3 docs doesn't document conditional DELETE

2014-04-30 Thread Sylvain Lebresne (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-6874.
-

Resolution: Duplicate

I actually fixed that part of the doc as part of CASSANDRA-7055. I'll probably 
only push the update public when 2.0.8 is out though because due to 
CASSANDRA-7055 the doc references a CQL version that will only be in 2.0.8.

 CQL3 docs doesn't document conditional DELETE
 -

 Key: CASSANDRA-6874
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6874
 Project: Cassandra
  Issue Type: Bug
  Components: Documentation  website
Reporter: Mohica Jasha
Assignee: Tyler Hobbs
  Labels: documentation

 http://cassandra.apache.org/doc/cql3/CQL.html#deleteStmt doesn't document 
 conditional {{DELETE}}. But support for if clause for {{DELETE}} was there 
 from C* 2.0



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6572) Workload recording / playback


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985381#comment-13985381
 ] 

Benedict commented on CASSANDRA-6572:
-

I wouldn't worry about the negligible amount of memory wasted storing the data 
- after all, if there are no new messages to log it means the server isn't 
serving any requests. A shutdown hook is probably easiest and sufficient, as a 
periodic flush would also miss any messages logged between last flush and 
shutdown.

 Workload recording / playback
 -

 Key: CASSANDRA-6572
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572
 Project: Cassandra
  Issue Type: New Feature
  Components: Core, Tools
Reporter: Jonathan Ellis
Assignee: Lyuben Todorov
 Fix For: 2.1.1

 Attachments: 6572-trunk.diff


 Write sample mode gets us part way to testing new versions against a real 
 world workload, but we need an easy way to test the query side as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (CASSANDRA-7099) Concurrent instances of same Prepared Statement seeing intermingled result sets

2014-04-30 Thread Bill Mitchell (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bill Mitchell resolved CASSANDRA-7099.
--

Resolution: Not a Problem

This problem would seem to be my fault.

In the normal, non parallel case, one can cheat. One can bind a
PreparedStatement, execute it, process its result set, then bind a different
parameter value and execute the same BoundStatement again.

This does not work when the resultSet size exceeds the fetch size. The initial
segments are all fetched fine, but the Java Driver apparently uses the
BoundStatement to distinguish the queries. If one executes the same
BoundStatement object, with different values, to generate multiple result sets,
the Java driver or Cassandra get quite confused as to which results to return
to which query.

Building distinct BoundStatement objects and executing each just once avoids
the confusion.

Concurrent instances of same Prepared Statement seeing intermingled result
sets
---

Key: CASSANDRA-7099
URL: https://issues.apache.org/jira/browse/CASSANDRA-7099
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassandra 2.0.7 with single node cluster
Windows dual-core laptop
DataStax Java driver 2.0.1
Reporter: Bill Mitchell

I have a schema in which a wide row is partitioned into smaller rows. (See
CASSANDRA-6826, CASSANDRA-6825 for more detail on this schema.) In this
case, I randomly assigned the rows across the partitions based on the first
four hex digits of a hash value modulo the number of partitions.
Occasionally I need to retrieve the rows in order of insertion irrespective
of the partitioning. Cassandra, of course, does not support this when paging
by fetch size is enabled, so I am issuing a query against each of the
partitions to obtain their rows in order, and merging the results:
SELECT l, partition, cd, rd, ec, ea FROM sr WHERE s = ?, l = ?, partition = ?
ORDER BY cd ASC, ec ASC ALLOW FILTERING;
These parallel queries are all instances of a single PreparedStatement.
What I saw was identical values from multiple queries, which by construction
should never happen, and after further investigation, discovered that rows
from partition 5 are being returned in the result set for the query against
another partition, e.g., 1. This was so unbelievable that I added diagnostic
code in my test case to detect this:
After reading 167 rows, returned partition 5 does not match query partition 4
The merge logic works fine and delivers correct results when I use LIMIT to
avoid fetch size paging. Even if there were a bug there, it is hard to see
how any client error explains ResultSet.one() returning a row whose values
don't match the constraints in that ResultSet's query.
I'm not sure of the exact significance of 167, as I have configured the
queryFetchSize for the cluster to 1000, and in this merge logic I divide that
by the number of partitions, 7, so the fetchSize for each of these parallel
queries was set to 142. I suspect this is being treated as a minimum
fetchSize, and the driver or server is rounding this up to fill a
transmission block. When I prime the pump, issuing the query against each of
the partitions, the initial contents of the result sets are correct. The
failure appears after we advance two of these queries to the next page.
Although I had been experimenting with fetchMoreResults() for prefetching, I
disabled that to isolate this problem, so that is not a factor.
I have not yet tried preparing separate instances of the query, as I already
have common logic to cache and reuse already prepared statements.
I have not proven that it is a server bug and not a Java driver bug, but on
first glance it was not obvious how the Java driver might associate the
responses with the wrong requests. Were that happening, one would expect to
see the right overall collection of rows, just to the wrong queries, and not
duplicates, which is what I saw.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6861) Optimise our Netty 4 integration

2014-04-30 Thread T Jake Luciani (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985516#comment-13985516
 ] 

T Jake Luciani commented on CASSANDRA-6861:
---

Yeah, looks like thrift hsha is about as poor.

 Optimise our Netty 4 integration
 

 Key: CASSANDRA-6861
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6861
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: T Jake Luciani
Priority: Minor
  Labels: performance
 Fix For: 2.1 beta2


 Now we've upgraded to Netty 4, we're generating a lot of garbage that could 
 be avoided, so we should probably stop that. Should be reasonably easy to 
 hook into Netty's pooled buffers, returning them to the pool once a given 
 message is completed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7099) Concurrent instances of same Prepared Statement seeing intermingled result sets

2014-04-30 Thread Jack Krupansky (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985529#comment-13985529
 ] 

Jack Krupansky commented on CASSANDRA-7099:
---

It may have been your mistake, but could C* or the driver have detected the 
difficulty and reported an error?

 Concurrent instances of same Prepared Statement seeing intermingled result 
 sets
 ---

 Key: CASSANDRA-7099
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7099
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.0.7 with single node cluster
 Windows dual-core laptop
 DataStax Java driver 2.0.1
Reporter: Bill Mitchell

 I have a schema in which a wide row is partitioned into smaller rows.  (See 
 CASSANDRA-6826, CASSANDRA-6825 for more detail on this schema.)  In this 
 case, I randomly assigned the rows across the partitions based on the first 
 four hex digits of a hash value modulo the number of partitions.  
 Occasionally I need to retrieve the rows in order of insertion irrespective 
 of the partitioning.  Cassandra, of course, does not support this when paging 
 by fetch size is enabled, so I am issuing a query against each of the 
 partitions to obtain their rows in order, and merging the results:
 SELECT l, partition, cd, rd, ec, ea FROM sr WHERE s = ?, l = ?, partition = ? 
 ORDER BY cd ASC, ec ASC ALLOW FILTERING;
 These parallel queries are all instances of a single PreparedStatement.  
 What I saw was identical values from multiple queries, which by construction 
 should never happen, and after further investigation, discovered that rows 
 from partition 5 are being returned in the result set for the query against 
 another partition, e.g., 1.  This was so unbelievable that I added diagnostic 
 code in my test case to detect this:
 After reading 167 rows, returned partition 5 does not match query partition 4
 The merge logic works fine and delivers correct results when I use LIMIT to 
 avoid fetch size paging.  Even if there were a bug there, it is hard to see 
 how any client error explains ResultSet.one() returning a row whose values 
 don't match the constraints in that ResultSet's query.
 I'm not sure of the exact significance of 167, as I have configured the 
 queryFetchSize for the cluster to 1000, and in this merge logic I divide that 
 by the number of partitions, 7, so the fetchSize for each of these parallel 
 queries was set to 142.  I suspect this is being treated as a minimum 
 fetchSize, and the driver or server is rounding this up to fill a 
 transmission block.  When I prime the pump, issuing the query against each of 
 the partitions, the initial contents of the result sets are correct.  The 
 failure appears after we advance two of these queries to the next page.
 Although I had been experimenting with fetchMoreResults() for prefetching, I 
 disabled that to isolate this problem, so that is not a factor.   
 I have not yet tried preparing separate instances of the query, as I already 
 have common logic to cache and reuse already prepared statements.
 I have not proven that it is a server bug and not a Java driver bug, but on 
 first glance it was not obvious how the Java driver might associate the 
 responses with the wrong requests.  Were that happening, one would expect to 
 see the right overall collection of rows, just to the wrong queries, and not 
 duplicates, which is what I saw.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6875) CQL3: select multiple CQL rows in a single partition using IN

2014-04-30 Thread Sylvain Lebresne (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985587#comment-13985587
 ] 

Sylvain Lebresne commented on CASSANDRA-6875:
-

The overall principle looks good, but I feel this could use some more comments 
and/or made a little more clear. Mainly, the {{isMultiColumn}} path in 
{{SelectStatement.buildBound}} looks weird at face value: we're inside a loop 
that populates a {{ColumnNameBuilder}}, but the {{isMultiColumn}} path 
completely ignores both the iterated object and the builder. This work because 
it relies on the fact that when there is a multi-column restriction then it's 
the only one restriction (which is duplicated in the 
{{SelectStatement.columnRestrictions}} array), and that the value from such 
restriction is not a single column value but rather a fully built composite 
serialized value (which is not self evident from the method naming in 
particular).  But it's hard to piece it all that together when you look at 
{{buildBound}} currently.  Some comments would help, but I'd prefer going even 
further and move the {{isMultiColumn}} path outside of the loop (by looking up 
first if the first restriction in {{columnRestrictions}} is a multi-column one 
or not) since it has no reason to be in the loop. In fact, I'd go a tad further 
by making SelectStatement abstract and have 2 subClass, one for single-column 
restrictions with a {{SingleColumnRelation[] columnRestrictions}} array field 
as we have now, and one for multi-column that has just one non-array 
{{MultiColumnRestriction columnsRestriction}} field. After all, both cases 
exclude one another in the current implementation.

Somewhat related, I'm slightly afraid that the parts about multi-column 
restrictions returning fully serialized composites (through Tuples.Value.get()) 
will not play nice with the 2.1 code, where we don't manipulate composites as 
opaque ByteBuffers anymore (concretely, Tuples will serialize the composite, 
but SelectStatement will have to deserialize it back right away to get a 
Composite, which will be both ugly and inefficient).  So to avoid having to 
change everything on merge, I think it would be cleaner to make Tuples.Value 
return a list of (individual column) values instead of just one, and let 
SelectStatement build back the full composite name using a ColumnNameBuilder.  
Especially if you make the per-column and multi-column paths a tad more 
separated as suggested above, I suspect it might clarify things a bit.  

Other than that, a bunch of more minor comments and nits:
* The {{SelectStatement.RawStatement.prepare()}} re-org patch breaks proper 
indentation at places (for instance, the indentation of parameters to 'new 
ColumnSpecification' in the first branch of udpateSingleColumnRestriction, 
though there is a few other places). Would be nice to fix those.
* Can't we use {{QueryProcessor.instance.getPrepared}} instead of creating a 
only-for-test {{QueryProcessor.staticGetPrepared}}? Or at the very least leave 
such shortcuts in the tests where it belongs.
* In Tuples.Literal.prepare, I'd prefer good ol'fashion indexed loop to iterate 
over 2 lists (feels clearer, and saves the allocator creation as a bonus).
* In Tuples.Raw.makeInReceiver should probably be called makeReceiver (it's not 
related to IN). I'd also drop the spaces in the string generated (if only for 
consistency with the string generated in INRaw). As a side node, 
Raw.makeReceiver uses indexed iteration while INRaw.makeInReceiver don't, can't 
we make both consistent style wise for OCD sakes?
* Why make methods of CQLStatement abstract (it's an interface)?  Also, I'd 
rather add the QueryOptions parameter to the existing executeInternal and 
default to QueryOptions.DEFAULT when calling it, rather than having 2 methods. 
Though tbh, my preference would be to move the tests to dtest and leave those 
somewhat unrelated changes to another ticket, see below.
* SingleColumnRelation.previousInTuple is now unused but not removed.
* We could save one list allocation (instead of both toCreate and toUpdate) in 
SelectStatement.updateRestrictionsForRelation (for EQ and IN, we know it can 
only be a create, and for slices we can lookup with getExisting).
* In Restriction, the {{values}} method is already at top-level, no reason to 
re-declare it for EQ.
* It bothers me to start adding unit tests for CQL queries when all of our CQL 
tests are currently in dtest. I'd *much* rather keep it all in the dtests to 
avoid the confusion on where is what tested.


 CQL3: select multiple CQL rows in a single partition using IN
 -

 Key: CASSANDRA-6875
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6875
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Nicolas

[jira] [Commented] (CASSANDRA-7099) Concurrent instances of same Prepared Statement seeing intermingled result sets

2014-04-30 Thread Bill Mitchell (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985608#comment-13985608
 ] 

Bill Mitchell commented on CASSANDRA-7099:
--

My thought was that, if the Java driver were more clever, it might be possible 
to use the ResultSet to determine the correlation id when paging in more 
results, instead of the Statement.  But there may be reasons why it wants to 
assume the Statement parameters have not changed, e.g., to avoid having to copy 
the bound parameters if it needs these to generate the later paged requests.  

 Concurrent instances of same Prepared Statement seeing intermingled result 
 sets
 ---

 Key: CASSANDRA-7099
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7099
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.0.7 with single node cluster
 Windows dual-core laptop
 DataStax Java driver 2.0.1
Reporter: Bill Mitchell

 I have a schema in which a wide row is partitioned into smaller rows.  (See 
 CASSANDRA-6826, CASSANDRA-6825 for more detail on this schema.)  In this 
 case, I randomly assigned the rows across the partitions based on the first 
 four hex digits of a hash value modulo the number of partitions.  
 Occasionally I need to retrieve the rows in order of insertion irrespective 
 of the partitioning.  Cassandra, of course, does not support this when paging 
 by fetch size is enabled, so I am issuing a query against each of the 
 partitions to obtain their rows in order, and merging the results:
 SELECT l, partition, cd, rd, ec, ea FROM sr WHERE s = ?, l = ?, partition = ? 
 ORDER BY cd ASC, ec ASC ALLOW FILTERING;
 These parallel queries are all instances of a single PreparedStatement.  
 What I saw was identical values from multiple queries, which by construction 
 should never happen, and after further investigation, discovered that rows 
 from partition 5 are being returned in the result set for the query against 
 another partition, e.g., 1.  This was so unbelievable that I added diagnostic 
 code in my test case to detect this:
 After reading 167 rows, returned partition 5 does not match query partition 4
 The merge logic works fine and delivers correct results when I use LIMIT to 
 avoid fetch size paging.  Even if there were a bug there, it is hard to see 
 how any client error explains ResultSet.one() returning a row whose values 
 don't match the constraints in that ResultSet's query.
 I'm not sure of the exact significance of 167, as I have configured the 
 queryFetchSize for the cluster to 1000, and in this merge logic I divide that 
 by the number of partitions, 7, so the fetchSize for each of these parallel 
 queries was set to 142.  I suspect this is being treated as a minimum 
 fetchSize, and the driver or server is rounding this up to fill a 
 transmission block.  When I prime the pump, issuing the query against each of 
 the partitions, the initial contents of the result sets are correct.  The 
 failure appears after we advance two of these queries to the next page.
 Although I had been experimenting with fetchMoreResults() for prefetching, I 
 disabled that to isolate this problem, so that is not a factor.   
 I have not yet tried preparing separate instances of the query, as I already 
 have common logic to cache and reuse already prepared statements.
 I have not proven that it is a server bug and not a Java driver bug, but on 
 first glance it was not obvious how the Java driver might associate the 
 responses with the wrong requests.  Were that happening, one would expect to 
 see the right overall collection of rows, just to the wrong queries, and not 
 duplicates, which is what I saw.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7107) General minor tidying of CollationController path

2014-04-30 Thread Aleksey Yeschenko (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985626#comment-13985626
 ] 

Aleksey Yeschenko commented on CASSANDRA-7107:
--

Have only skimmed it so far. Will have a deep look once the issues behind these 
two unit tests are fixed (caused by the patch):
- org.apache.cassandra.cli.CliTest
- org.apache.cassandra.db.ColumnFamilyStoreTest

There are other tests failing, but those fail w/ and w/out the patch, both.

 General minor tidying of CollationController path
 -

 Key: CASSANDRA-7107
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7107
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
Priority: Minor
 Fix For: 2.1.0


 There is a lot of unnecessary boiler plate when grabbing an iterator from an 
 in-memory column family. This patch:
 * Removes FakeCellName
 * Avoids wrapping a non-OnDiskAtomIterator as an OnDiskAtomIterator except 
 when the wrapping is useful
 * Removes ColumnSlice.NavigableSetIterator and creates a simpler more direct 
 equivalent in ABTC
 * Does not construct a SliceIterator in either ABSC or ABTC if only one slice 
 is requested (just returns that slice as an Iterator)
 * Does not construct multiple list indirections in ABSC when constructing a 
 slice
 * Shares forward/reverse iterators in ABSC between slices and full-iteration
 * Avoids O(N) comparisons during collation of results into an ABSC, by using 
 the knowledge that all columns are provided in insertion order from a merge 
 iterator



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7116) Use ConcurrentLinkedQueue instead of NonBlockingHashSet in AbstractRowResolver


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7116:


Attachment: 7116.txt

Trivial one-line patch

 Use ConcurrentLinkedQueue instead of NonBlockingHashSet in AbstractRowResolver
 --

 Key: CASSANDRA-7116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7116
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
Priority: Trivial
 Fix For: 2.1 rc1

 Attachments: 7116.txt


 There's no reason to use a NBHM in this class, as we only add unique 
 elements. A CLQ seems a more appropriate lightweight datastructure to use



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6572) Workload recording / playback

2014-04-30 Thread Lyuben Todorov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985649#comment-13985649
 ] 

Lyuben Todorov commented on CASSANDRA-6572:
---

Since queries are logged as their string form, this doesn't accommodate 
prepared statements. One way I think we can log the ps' is to log the string 
query during the prepare phase along with the query's id, e.g. 
{{b7693b50da63a31229b8413754bc72c0 INSERT INTO ks.cf (col1) VALUES (?) }} and 
then in ExecuteMessage#execute the values and the id can be logged again later 
on in the log, then during replay we can match values to the queryString by 
using the id. A better way to do it would be to get access to the statementId 
in QP#executePrepared but I'm not sure whether it's worth changing the 
statement to store it's id. 

 Workload recording / playback
 -

 Key: CASSANDRA-6572
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572
 Project: Cassandra
  Issue Type: New Feature
  Components: Core, Tools
Reporter: Jonathan Ellis
Assignee: Lyuben Todorov
 Fix For: 2.1.1

 Attachments: 6572-trunk.diff


 Write sample mode gets us part way to testing new versions against a real 
 world workload, but we need an easy way to test the query side as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7099) Concurrent instances of same Prepared Statement seeing intermingled result sets

2014-04-30 Thread Sylvain Lebresne (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985654#comment-13985654
]

Sylvain Lebresne commented on CASSANDRA-7099:
-

bq. it might be possible to use the ResultSet to determine the correlation id
when paging in more results

Fyi, the driver does need to send the full query (including bound parameters)
for every page, not just an ID. This is not specific to the java driver, this
is how the paging work in the protocol, and this is done so so that pages can
be fetched from another coordinator than the one of the first page. That said,
it's probably possible to make it easier driver side to reuse a BoundStatement
more safely, or at least to clarify in the document when it's safe or not to do
so. But that's a driver concern, so let's keep further discussion, if further
discussion there is, on the driver mailing list/jira.

Concurrent instances of same Prepared Statement seeing intermingled result
sets
---

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (CASSANDRA-6572) Workload recording / playback

2014-04-30 Thread Lyuben Todorov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985649#comment-13985649
 ] 

Lyuben Todorov edited comment on CASSANDRA-6572 at 4/30/14 3:55 PM:


Since queries are logged as their string form, this doesn't accommodate 
prepared statements. One way I think we can log the ps' is to log the string 
query during the prepare phase along with the query's id, e.g. 
{{b7693b50da63a31229b8413754bc72c0 INSERT INTO ks.cf (col1) VALUES ( \? )}} and 
then in {{ExecuteMessage#execute}} the values and the id can be logged again 
later on in the log, then during replay we can match values to the queryString 
by using the id. A better way to do it would be to get access to the 
statementId in QP#executePrepared but I'm not sure whether it's worth changing 
the statement to store it's id. 


was (Author: lyubent):
Since queries are logged as their string form, this doesn't accommodate 
prepared statements. One way I think we can log the ps' is to log the string 
query during the prepare phase along with the query's id, e.g. 
{{b7693b50da63a31229b8413754bc72c0 INSERT INTO ks.cf (col1) VALUES (?) }} and 
then in ExecuteMessage#execute the values and the id can be logged again later 
on in the log, then during replay we can match values to the queryString by 
using the id. A better way to do it would be to get access to the statementId 
in QP#executePrepared but I'm not sure whether it's worth changing the 
statement to store it's id. 

 Workload recording / playback
 -

 Key: CASSANDRA-6572
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572
 Project: Cassandra
  Issue Type: New Feature
  Components: Core, Tools
Reporter: Jonathan Ellis
Assignee: Lyuben Todorov
 Fix For: 2.1.1

 Attachments: 6572-trunk.diff


 Write sample mode gets us part way to testing new versions against a real 
 world workload, but we need an easy way to test the query side as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-7116) Use ConcurrentLinkedQueue instead of NonBlockingHashSet in AbstractRowResolver

Benedict created CASSANDRA-7116:
---

 Summary: Use ConcurrentLinkedQueue instead of NonBlockingHashSet 
in AbstractRowResolver
 Key: CASSANDRA-7116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7116
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
Priority: Trivial
 Fix For: 2.1 rc1
 Attachments: 7116.txt

There's no reason to use a NBHM in this class, as we only add unique elements. 
A CLQ seems a more appropriate lightweight datastructure to use



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985658#comment-13985658
 ] 

Benedict commented on CASSANDRA-4718:
-

I've uploaded a slight variant of the patch 
[here|https://github.com/belliottsmith/cassandra/tree/4718-lowsignal] - this 
introduces a special FJP for that processing native transport work, that avoids 
blocking on enqueue to the pool unless the configured limit has been reached. 
Instead we schedule a ForkJoinTask that sleeps for 5us, forking any work that 
has been queued in the interval (and going to sleep only if no work has been 
seen in the past 5ms). This permits the connection worker threads to return to 
servicing their connections more promptly.

It has only a modest effect on my box, but it does give a 5-10% bump in native 
transport performance.

 More-efficient ExecutorService for improved throughput
 --

 Key: CASSANDRA-4718
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4718
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jonathan Ellis
Assignee: Jason Brown
Priority: Minor
  Labels: performance
 Fix For: 2.1.0

 Attachments: 4718-v1.patch, PerThreadQueue.java, baq vs trunk.png, op 
 costs of various queues.ods, stress op rate with various queues.ods, 
 v1-stress.out


 Currently all our execution stages dequeue tasks one at a time.  This can 
 result in contention between producers and consumers (although we do our best 
 to minimize this by using LinkedBlockingQueue).
 One approach to mitigating this would be to make consumer threads do more 
 work in bulk instead of just one task per dequeue.  (Producer threads tend 
 to be single-task oriented by nature, so I don't see an equivalent 
 opportunity there.)
 BlockingQueue has a drainTo(collection, int) method that would be perfect for 
 this.  However, no ExecutorService in the jdk supports using drainTo, nor 
 could I google one.
 What I would like to do here is create just such a beast and wire it into (at 
 least) the write and read stages.  (Other possible candidates for such an 
 optimization, such as the CommitLog and OutboundTCPConnection, are not 
 ExecutorService-based and will need to be one-offs.)
 AbstractExecutorService may be useful.  The implementations of 
 ICommitLogExecutorService may also be useful. (Despite the name these are not 
 actual ExecutorServices, although they share the most important properties of 
 one.)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6861) Optimise our Netty 4 integration


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985659#comment-13985659
 ] 

Benedict commented on CASSANDRA-6861:
-

It may be worth trying netty's JNI epoll provider whilst we're at this

 Optimise our Netty 4 integration
 

 Key: CASSANDRA-6861
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6861
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: T Jake Luciani
Priority: Minor
  Labels: performance
 Fix For: 2.1 beta2


 Now we've upgraded to Netty 4, we're generating a lot of garbage that could 
 be avoided, so we should probably stop that. Should be reasonably easy to 
 hook into Netty's pooled buffers, returning them to the pool once a given 
 message is completed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6559) cqlsh should warn about ALLOW FILTERING


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985660#comment-13985660
 ] 

Jonathan Ellis commented on CASSANDRA-6559:
---

There's tension between smooth the learning curve for new users so they don't 
ragequit and go use mongo instead and make it harder for people to shoot 
themselves in the foot.  We're going to continue to optimize for the former, 
but adding cqlsh-side warning is reasonable.

 cqlsh should warn about ALLOW FILTERING
 ---

 Key: CASSANDRA-6559
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6559
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Tupshin Harper
Assignee: Aleksey Yeschenko
Priority: Minor
 Fix For: 2.0.8


 ALLOW FILTERING can be a convenience for preliminary exploration of your 
 data, and can be useful for batch jobs, but it is such an anti-pattern for 
 regular production queries, that cqlsh should provie an explicit warn 
 ingwhenever such a query is performed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7116) Use ConcurrentLinkedQueue instead of NonBlockingHashSet in AbstractRowResolver


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985663#comment-13985663
 ] 

Jonathan Ellis commented on CASSANDRA-7116:
---

Is CLQ lighter weight?  Because a queue implies that order is important, which 
it is not here.  So abstractly speaking I like the Set better.

 Use ConcurrentLinkedQueue instead of NonBlockingHashSet in AbstractRowResolver
 --

 Key: CASSANDRA-7116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7116
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
Priority: Trivial
 Fix For: 2.1 rc1

 Attachments: 7116.txt


 There's no reason to use a NBHM in this class, as we only add unique 
 elements. A CLQ seems a more appropriate lightweight datastructure to use



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7116) Use ConcurrentLinkedQueue instead of NonBlockingHashSet in AbstractRowResolver


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985669#comment-13985669
 ] 

Benedict commented on CASSANDRA-7116:
-

Well, for small collections (like we have here) it's lighter weight 
memory-wise, and it's faster - at the very least because it has no need to 
invoke the object hashCode() which involves revoking the biased lock on the 
object, but also because on a small map you can still have collisions which 
require at least rehashing, and possibly resizing (NBHM resizes when you have 
too many collisions, not based on total size, iirc).

CLQ is guaranteed O(1) behaviour, and a constant 16 bytes per entry

 Use ConcurrentLinkedQueue instead of NonBlockingHashSet in AbstractRowResolver
 --

 Key: CASSANDRA-7116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7116
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
Priority: Trivial
 Fix For: 2.1 rc1

 Attachments: 7116.txt


 There's no reason to use a NBHM in this class, as we only add unique 
 elements. A CLQ seems a more appropriate lightweight datastructure to use



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7116) Use ConcurrentLinkedQueue instead of NonBlockingHashSet in AbstractRowResolver


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985675#comment-13985675
 ] 

Benedict commented on CASSANDRA-7116:
-

To put it another way, the Set imposes a uniqueness constraint, and the CLQ 
imposes an ordering constraint, but the ordering constraint is cheaper to 
maintain.

That said, I'd be happy to use a lock-free stack here as well, which is even 
easier/cheaper to maintain; or simply an atomic index pointer and an atomic 
reference array.

 Use ConcurrentLinkedQueue instead of NonBlockingHashSet in AbstractRowResolver
 --

 Key: CASSANDRA-7116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7116
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
Priority: Trivial
 Fix For: 2.1 rc1

 Attachments: 7116.txt


 There's no reason to use a NBHM in this class, as we only add unique 
 elements. A CLQ seems a more appropriate lightweight datastructure to use



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7111) Include snippet of CQL query near error in SyntaxError messages


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-7111:
--

Component/s: (was: Core)
 Tools
 API

Can we target this for 2.1.1 or would it break assumptions of what a 
SyntaxError should contain?

 Include snippet of CQL query near error in SyntaxError messages
 ---

 Key: CASSANDRA-7111
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7111
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Tools
Reporter: Tyler Hobbs
Priority: Minor

 When a SyntaxError is returned, including a snippet of the query close to the 
 error would make a lot of error messages easier to understand.  For example, 
 if you did this with the python driver:
 {code}
 session.execute(SELECT * FROM users WHERE username='%s', ['Joe Smith'])
 {code}
 you would wind up with an extra set of single quotes (the driver 
 automatically escapes and quotes input).  If a snippet like {{...WHERE 
 username=''Joe Smith''}} were included in the error message, this would be 
 pretty easy to spot.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6572) Workload recording / playback


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985695#comment-13985695
 ] 

Benedict commented on CASSANDRA-6572:
-

It might be easiest to modify QP.processPrepared to accept the statementId as 
an extra parameter, to keep everything encapsulated in QP

 Workload recording / playback
 -

 Key: CASSANDRA-6572
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572
 Project: Cassandra
  Issue Type: New Feature
  Components: Core, Tools
Reporter: Jonathan Ellis
Assignee: Lyuben Todorov
 Fix For: 2.1.1

 Attachments: 6572-trunk.diff


 Write sample mode gets us part way to testing new versions against a real 
 world workload, but we need an easy way to test the query side as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7111) Include snippet of CQL query near error in SyntaxError messages

2014-04-30 Thread Tyler Hobbs (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985705#comment-13985705
 ] 

Tyler Hobbs commented on CASSANDRA-7111:


The snippet could just be inserted into the normal message without breaking 
compatibility.  I don't think any of the drivers try to parse the message or 
anything like that.

 Include snippet of CQL query near error in SyntaxError messages
 ---

 Key: CASSANDRA-7111
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7111
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Tools
Reporter: Tyler Hobbs
Priority: Minor

 When a SyntaxError is returned, including a snippet of the query close to the 
 error would make a lot of error messages easier to understand.  For example, 
 if you did this with the python driver:
 {code}
 session.execute(SELECT * FROM users WHERE username='%s', ['Joe Smith'])
 {code}
 you would wind up with an extra set of single quotes (the driver 
 automatically escapes and quotes input).  If a snippet like {{...WHERE 
 username=''Joe Smith''}} were included in the error message, this would be 
 pretty easy to spot.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-7117) cqlsh should return a non-zero error code if a query fails

2014-04-30 Thread J.B. Langston (JIRA)

J.B. Langston created CASSANDRA-7117:


 Summary: cqlsh should return a non-zero error code if a query fails
 Key: CASSANDRA-7117
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7117
 Project: Cassandra
  Issue Type: Improvement
Reporter: J.B. Langston
Priority: Minor


cqlsh should return a non-zero error code when the last query in a file or 
piped stdin fails.  This is so that shell scripts to determine if a cql script 
failed or succeeded.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7117) cqlsh should return a non-zero error code if a query fails

2014-04-30 Thread J.B. Langston (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.B. Langston updated CASSANDRA-7117:
-

Description: cqlsh should return a non-zero error code when a query in a 
file or piped stdin fails.  This is so that shell scripts to determine if a cql 
script failed or succeeded.  (was: cqlsh should return a non-zero error code 
when the last query in a file or piped stdin fails.  This is so that shell 
scripts to determine if a cql script failed or succeeded.)

 cqlsh should return a non-zero error code if a query fails
 --

 Key: CASSANDRA-7117
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7117
 Project: Cassandra
  Issue Type: Improvement
Reporter: J.B. Langston
Priority: Minor

 cqlsh should return a non-zero error code when a query in a file or piped 
 stdin fails.  This is so that shell scripts to determine if a cql script 
 failed or succeeded.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7116) Use ConcurrentLinkedQueue instead of NonBlockingHashSet in AbstractRowResolver


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985726#comment-13985726
 ] 

Jonathan Ellis commented on CASSANDRA-7116:
---

Hmm, in CASSANDRA-6933 you sneaked in a change to Collections.synchronizedList 
for trunk.  Which do you prefer? :)

 Use ConcurrentLinkedQueue instead of NonBlockingHashSet in AbstractRowResolver
 --

 Key: CASSANDRA-7116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7116
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
Priority: Trivial
 Fix For: 2.1 rc1

 Attachments: 7116.txt


 There's no reason to use a NBHM in this class, as we only add unique 
 elements. A CLQ seems a more appropriate lightweight datastructure to use



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7111) Include snippet of CQL query near error in SyntaxError messages


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-7111:
--

Fix Version/s: 2.1 rc1

 Include snippet of CQL query near error in SyntaxError messages
 ---

 Key: CASSANDRA-7111
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7111
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Tools
Reporter: Tyler Hobbs
Priority: Minor
 Fix For: 2.1 rc1


 When a SyntaxError is returned, including a snippet of the query close to the 
 error would make a lot of error messages easier to understand.  For example, 
 if you did this with the python driver:
 {code}
 session.execute(SELECT * FROM users WHERE username='%s', ['Joe Smith'])
 {code}
 you would wind up with an extra set of single quotes (the driver 
 automatically escapes and quotes input).  If a snippet like {{...WHERE 
 username=''Joe Smith''}} were included in the error message, this would be 
 pretty easy to spot.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7111) Include snippet of CQL query near error in SyntaxError messages


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-7111:
--

Priority: Major  (was: Minor)

 Include snippet of CQL query near error in SyntaxError messages
 ---

 Key: CASSANDRA-7111
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7111
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Tools
Reporter: Tyler Hobbs
 Fix For: 2.1 rc1


 When a SyntaxError is returned, including a snippet of the query close to the 
 error would make a lot of error messages easier to understand.  For example, 
 if you did this with the python driver:
 {code}
 session.execute(SELECT * FROM users WHERE username='%s', ['Joe Smith'])
 {code}
 you would wind up with an extra set of single quotes (the driver 
 automatically escapes and quotes input).  If a snippet like {{...WHERE 
 username=''Joe Smith''}} were included in the error message, this would be 
 pretty easy to spot.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7116) Use ConcurrentLinkedQueue instead of NonBlockingHashSet in AbstractRowResolver