date:20141014


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170863#comment-14170863
 ] 

Aleksey Yeschenko commented on CASSANDRA-8095:
--

What's concurrent about it?

 create table query not working in concurrent
 

 Key: CASSANDRA-8095
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8095
 Project: Cassandra
  Issue Type: Bug
 Environment: 2.1.0 on Ubuntu 14.04
Reporter: Katsutoshi Nagaoka

 After 2.1.0, I have encountered the strange behavior of create table.
 When I executed create table query in concurrent, 
 {code}
 package cassandra.test;
 import com.datastax.driver.core.Cluster;
 import com.datastax.driver.core.Session;
 import java.util.ArrayList;
 import java.util.List;
 import java.util.concurrent.ExecutorService;
 import java.util.concurrent.Executors;
 import java.util.concurrent.Future;
 import java.util.concurrent.TimeUnit;
 public class ConcurrentCreateTable {
 public static void main(String[] args) {
 ConcurrentCreateTable instance = new ConcurrentCreateTable();
 try {
 instance.execute();
 } catch (Exception e) {
 e.printStackTrace();
 }
 }
 private final Cluster cluster;
 public ConcurrentCreateTable() {
 this.cluster = Cluster.builder().addContactPoint(server1).build();
 }
 private void execute() throws Exception {
 Session session = cluster.newSession();
 session.execute(DROP KEYSPACE IF EXISTS ks);
 session.execute(CREATE KEYSPACE IF NOT EXISTS ks WITH REPLICATION = 
 {'class':'SimpleStrategy','replication_factor':3});
 ExecutorService executor = Executors.newFixedThreadPool(10);
 ListFuture? futures = new ArrayList();
 futures.add(executor.submit(new CreateTableRunner(text)));
 futures.add(executor.submit(new CreateTableRunner(int)));
 futures.add(executor.submit(new CreateTableRunner(boolean)));
 futures.add(executor.submit(new CreateTableRunner(bigint)));
 executor.shutdown();
 executor.awaitTermination(1L, TimeUnit.MINUTES);
   session.execute(DROP KEYSPACE ks);
   session.close();
 cluster.close();
 }
 private class CreateTableRunner implements Runnable {
 private final Session session;
 private final String type;
 public CreateTableRunner(String type) {
 this.session = cluster.newSession();
 this.type = type;
 }
 @Override
 public void run() {
 session.execute(CREATE TABLE IF NOT EXISTS ks.tb (key  + type + 
 , value  + type + , PRIMARY KEY (key)));
 session.close();
 }
 }
 }
 {code}
 sometimes each node has different table schema or no table.
 {code}
 [on server1] cqlsh DESCRIBE TABLE ks.tb;
 CREATE TABLE ks.tb (
 key int PRIMARY KEY,
 value int
 ) WITH bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.1
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 [on server2] cqlsh DESCRIBE TABLE ks.tb;
 CREATE TABLE ks.tb (
 key bigint PRIMARY KEY,
 value bigint
 ) WITH bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.1
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 [on server3] cqlsh DESCRIBE TABLE ks.tb;
 Column family 'tb' not found
 {code}
 I can reproduce this issue on three nodes cluster in 2.1.0. and the test 
 works fine in 2.0.10.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8112) nodetool compactionhistory can allocate memory unbounded

2014-10-14 Thread T Jake Luciani (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170886#comment-14170886
 ] 

T Jake Luciani commented on CASSANDRA-8112:
---

Here's an option.  Once CASSANDRA-7402 goes in we can check the size of the 
result set and by using a small page size we can stop loading after result set 
gets to say 1MB.

 nodetool compactionhistory can allocate memory unbounded
 

 Key: CASSANDRA-8112
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8112
 Project: Cassandra
  Issue Type: Bug
Reporter: T Jake Luciani
Priority: Minor
 Fix For: 2.1.1


 nodetool compactionhistory keeps data for 1 week by default and creates a 
 table from the result set in memory.
 For many systems a week can generate 10's of thousands of compactions in this 
 time (esp with LCS) we should guard against this command allocating too much 
 memory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8111) Create backup directories for commitlog archiving during startup


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-8111:
---
 Reviewer: Jason Brown
Reproduced In: 2.1.0, 2.0.11  (was: 2.0.11, 2.1.0)

 Create backup directories for commitlog archiving during startup
 

 Key: CASSANDRA-8111
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8111
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jan Karlsson
Priority: Trivial
  Labels: archive, commitlog
 Fix For: 2.0.11

 Attachments: CASSANDRA-8111.patch


 Cassandra currently crashes if the recovery directory in commitlog_archiving 
 does not exist (or cannot be listed). I would like to propose that Cassandra 
 creates this directory if it does not exist. This will mimic the behavior of 
 creating data, commitlog .. etc. directories during startup. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Create backup directories for commitlog archiving during startup

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 7497399d0 - d828643f5


Create backup directories for commitlog archiving during startup

patch by Jan Karlsson, reviewed by jasobrown for CASSANDRA-8111


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d828643f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d828643f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d828643f

Branch: refs/heads/cassandra-2.0
Commit: d828643f555aac14be59e7f8b35ac67cb06da20b
Parents: 7497399
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 06:31:20 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 06:31:20 2014 -0700

--
 CHANGES.txt |  1 +
 .../cassandra/db/commitlog/CommitLogArchiver.java   | 16 ++--
 2 files changed, 15 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d828643f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 84bbe75..2faea63 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * Create backup directories for commitlog archiving during startup 
(CASSANDRA-8111)
  * Reduce totalBlockFor() for LOCAL_* consistency levels (CASSANDRA-8058)
  * Fix merging schemas with re-dropped keyspaces (CASSANDRA-7256)
  * Fix counters in supercolumns during live upgrades from 1.2 (CASSANDRA-7188)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d828643f/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
index f020182..e30443e 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
@@ -45,6 +45,7 @@ public class CommitLogArchiver
 {
 private static final Logger logger = 
LoggerFactory.getLogger(CommitLogArchiver.class);
 public static final SimpleDateFormat format = new 
SimpleDateFormat(:MM:dd HH:mm:ss);
+private static final String DELIMITER = ,;
 static
 {
 format.setTimeZone(TimeZone.getTimeZone(GMT));
@@ -81,6 +82,17 @@ public class CommitLogArchiver
 archiveCommand = 
commitlog_commands.getProperty(archive_command);
 restoreCommand = 
commitlog_commands.getProperty(restore_command);
 restoreDirectories = 
commitlog_commands.getProperty(restore_directories);
+for (String dir : restoreDirectories.split(DELIMITER))
+{
+File directory = new File(dir);
+if (!directory.exists())
+{
+if (!directory.mkdir())
+{
+throw new RuntimeException(Unable to create 
directory  + dir);
+}
+}
+}
 String targetTime = 
commitlog_commands.getProperty(restore_point_in_time);
 precision = 
TimeUnit.valueOf(commitlog_commands.getProperty(precision, MICROSECONDS));
 try
@@ -151,12 +163,12 @@ public class CommitLogArchiver
 if (Strings.isNullOrEmpty(restoreDirectories))
 return;
 
-for (String dir : restoreDirectories.split(,))
+for (String dir : restoreDirectories.split(DELIMITER))
 {
 File[] files = new File(dir).listFiles();
 if (files == null)
 {
-throw new RuntimeException(Unable to list director  + dir);
+throw new RuntimeException(Unable to list directory  + dir);
 }
 for (File fromFile : files)
 {

[1/2] git commit: Create backup directories for commitlog archiving during startup

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 c5cbd58a3 - 4b7161da4


Create backup directories for commitlog archiving during startup

patch by Jan Karlsson, reviewed by jasobrown for CASSANDRA-8111


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d828643f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d828643f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d828643f

Branch: refs/heads/cassandra-2.1
Commit: d828643f555aac14be59e7f8b35ac67cb06da20b
Parents: 7497399
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 06:31:20 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 06:31:20 2014 -0700

--
 CHANGES.txt |  1 +
 .../cassandra/db/commitlog/CommitLogArchiver.java   | 16 ++--
 2 files changed, 15 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d828643f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 84bbe75..2faea63 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * Create backup directories for commitlog archiving during startup 
(CASSANDRA-8111)
  * Reduce totalBlockFor() for LOCAL_* consistency levels (CASSANDRA-8058)
  * Fix merging schemas with re-dropped keyspaces (CASSANDRA-7256)
  * Fix counters in supercolumns during live upgrades from 1.2 (CASSANDRA-7188)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d828643f/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
index f020182..e30443e 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
@@ -45,6 +45,7 @@ public class CommitLogArchiver
 {
 private static final Logger logger = 
LoggerFactory.getLogger(CommitLogArchiver.class);
 public static final SimpleDateFormat format = new 
SimpleDateFormat(:MM:dd HH:mm:ss);
+private static final String DELIMITER = ,;
 static
 {
 format.setTimeZone(TimeZone.getTimeZone(GMT));
@@ -81,6 +82,17 @@ public class CommitLogArchiver
 archiveCommand = 
commitlog_commands.getProperty(archive_command);
 restoreCommand = 
commitlog_commands.getProperty(restore_command);
 restoreDirectories = 
commitlog_commands.getProperty(restore_directories);
+for (String dir : restoreDirectories.split(DELIMITER))
+{
+File directory = new File(dir);
+if (!directory.exists())
+{
+if (!directory.mkdir())
+{
+throw new RuntimeException(Unable to create 
directory  + dir);
+}
+}
+}
 String targetTime = 
commitlog_commands.getProperty(restore_point_in_time);
 precision = 
TimeUnit.valueOf(commitlog_commands.getProperty(precision, MICROSECONDS));
 try
@@ -151,12 +163,12 @@ public class CommitLogArchiver
 if (Strings.isNullOrEmpty(restoreDirectories))
 return;
 
-for (String dir : restoreDirectories.split(,))
+for (String dir : restoreDirectories.split(DELIMITER))
 {
 File[] files = new File(dir).listFiles();
 if (files == null)
 {
-throw new RuntimeException(Unable to list director  + dir);
+throw new RuntimeException(Unable to list directory  + dir);
 }
 for (File fromFile : files)
 {

[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4b7161da
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4b7161da
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4b7161da

Branch: refs/heads/cassandra-2.1
Commit: 4b7161da4aacb2ca9e56ef78258df114d217ccde
Parents: c5cbd58 d828643
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 06:33:54 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 06:33:54 2014 -0700

--
 CHANGES.txt |  1 +
 .../cassandra/db/commitlog/CommitLogArchiver.java   | 16 ++--
 2 files changed, 15 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b7161da/CHANGES.txt
--
diff --cc CHANGES.txt
index b6b1953,2faea63..bbe6982
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,76 -1,5 +1,77 @@@
 -2.0.11:
 +2.1.1
 + * Fix high memory use due to tracking reads on incrementally opened sstable 
readers (CASSANDRA-8066)
 + * Fix EXECUTE request with skipMetadata=false returning no metadata
 +   (CASSANDRA-8054)
 + * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)
 + * Shutdown JVM on OOM (CASSANDRA-7507)
 + * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
 + * Don't duplicate sstables smaller than split size when using
 +   the sstablesplitter tool (CASSANDRA-7616)
 + * Avoid re-parsing already prepared statements (CASSANDRA-7923)
 + * Fix some Thrift slice deletions and updates of COMPACT STORAGE
 +   tables with some clustering columns omitted (CASSANDRA-7990)
 + * Fix filtering for CONTAINS on sets (CASSANDRA-8033)
 + * Properly track added size (CASSANDRA-7239)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Fix Assertion error on RangeTombstoneList diff (CASSANDRA-8013)
 + * Release references to overlapping sstables during compaction 
(CASSANDRA-7819)
 + * Send notification when opening compaction results early (CASSANDRA-8034)
 + * Make native server start block until properly bound (CASSANDRA-7885)
 + * (cqlsh) Fix IPv6 support (CASSANDRA-7988)
 + * Ignore fat clients when checking for endpoint collision (CASSANDRA-7939)
 + * Make sstablerepairedset take a list of files (CASSANDRA-7995)
 + * (cqlsh) Tab completeion for indexes on map keys (CASSANDRA-7972)
 + * (cqlsh) Fix UDT field selection in select clause (CASSANDRA-7891)
 + * Fix resource leak in event of corrupt sstable
 + * (cqlsh) Add command line option for cqlshrc file path (CASSANDRA-7131)
 + * Provide visibility into prepared statements churn (CASSANDRA-7921, 
CASSANDRA-7930)
 + * Invalidate prepared statements when their keyspace or table is
 +   dropped (CASSANDRA-7566)
 + * cassandra-stress: fix support for NetworkTopologyStrategy (CASSANDRA-7945)
 + * Fix saving caches when a table is dropped (CASSANDRA-7784)
 + * Add better error checking of new stress profile (CASSANDRA-7716)
 + * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom 
(CASSANDRA-7934)
 + * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
 + * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
 + * GCInspector more closely tracks GC; cassandra-stress and nodetool report 
it (CASSANDRA-7916)
 + * nodetool won't output bogus ownership info without a keyspace 
(CASSANDRA-7173)
 + * Add human readable option to nodetool commands (CASSANDRA-5433)
 + * Don't try to set repairedAt on old sstables (CASSANDRA-7913)
 + * Add metrics for tracking PreparedStatement use (CASSANDRA-7719)
 + * (cqlsh) tab-completion for triggers (CASSANDRA-7824)
 + * (cqlsh) Support for query paging (CASSANDRA-7514)
 + * (cqlsh) Show progress of COPY operations (CASSANDRA-7789)
 + * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
 + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
 + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
 + * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
 + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
 + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
 +   output (CASSANDRA-7659)
 + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
 + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
 + * Support list index operations with conditions (CASSANDRA-7499)
 + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
 + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
 + * (cqlsh) Error when tracing query (CASSANDRA-7613)
 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569)
 + *

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/25821864
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/25821864
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/25821864

Branch: refs/heads/trunk
Commit: 25821864f7b8a9926761b9c93d5afeb637850a26
Parents: cd60129 4b7161d
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 06:34:26 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 06:34:26 2014 -0700

--
 CHANGES.txt |  1 +
 .../cassandra/db/commitlog/CommitLogArchiver.java   | 16 ++--
 2 files changed, 15 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/25821864/CHANGES.txt
--

[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4b7161da
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4b7161da
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4b7161da

Branch: refs/heads/trunk
Commit: 4b7161da4aacb2ca9e56ef78258df114d217ccde
Parents: c5cbd58 d828643
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 06:33:54 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 06:33:54 2014 -0700

--
 CHANGES.txt |  1 +
 .../cassandra/db/commitlog/CommitLogArchiver.java   | 16 ++--
 2 files changed, 15 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b7161da/CHANGES.txt
--
diff --cc CHANGES.txt
index b6b1953,2faea63..bbe6982
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,76 -1,5 +1,77 @@@
 -2.0.11:
 +2.1.1
 + * Fix high memory use due to tracking reads on incrementally opened sstable 
readers (CASSANDRA-8066)
 + * Fix EXECUTE request with skipMetadata=false returning no metadata
 +   (CASSANDRA-8054)
 + * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)
 + * Shutdown JVM on OOM (CASSANDRA-7507)
 + * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
 + * Don't duplicate sstables smaller than split size when using
 +   the sstablesplitter tool (CASSANDRA-7616)
 + * Avoid re-parsing already prepared statements (CASSANDRA-7923)
 + * Fix some Thrift slice deletions and updates of COMPACT STORAGE
 +   tables with some clustering columns omitted (CASSANDRA-7990)
 + * Fix filtering for CONTAINS on sets (CASSANDRA-8033)
 + * Properly track added size (CASSANDRA-7239)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Fix Assertion error on RangeTombstoneList diff (CASSANDRA-8013)
 + * Release references to overlapping sstables during compaction 
(CASSANDRA-7819)
 + * Send notification when opening compaction results early (CASSANDRA-8034)
 + * Make native server start block until properly bound (CASSANDRA-7885)
 + * (cqlsh) Fix IPv6 support (CASSANDRA-7988)
 + * Ignore fat clients when checking for endpoint collision (CASSANDRA-7939)
 + * Make sstablerepairedset take a list of files (CASSANDRA-7995)
 + * (cqlsh) Tab completeion for indexes on map keys (CASSANDRA-7972)
 + * (cqlsh) Fix UDT field selection in select clause (CASSANDRA-7891)
 + * Fix resource leak in event of corrupt sstable
 + * (cqlsh) Add command line option for cqlshrc file path (CASSANDRA-7131)
 + * Provide visibility into prepared statements churn (CASSANDRA-7921, 
CASSANDRA-7930)
 + * Invalidate prepared statements when their keyspace or table is
 +   dropped (CASSANDRA-7566)
 + * cassandra-stress: fix support for NetworkTopologyStrategy (CASSANDRA-7945)
 + * Fix saving caches when a table is dropped (CASSANDRA-7784)
 + * Add better error checking of new stress profile (CASSANDRA-7716)
 + * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom 
(CASSANDRA-7934)
 + * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
 + * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
 + * GCInspector more closely tracks GC; cassandra-stress and nodetool report 
it (CASSANDRA-7916)
 + * nodetool won't output bogus ownership info without a keyspace 
(CASSANDRA-7173)
 + * Add human readable option to nodetool commands (CASSANDRA-5433)
 + * Don't try to set repairedAt on old sstables (CASSANDRA-7913)
 + * Add metrics for tracking PreparedStatement use (CASSANDRA-7719)
 + * (cqlsh) tab-completion for triggers (CASSANDRA-7824)
 + * (cqlsh) Support for query paging (CASSANDRA-7514)
 + * (cqlsh) Show progress of COPY operations (CASSANDRA-7789)
 + * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
 + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
 + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
 + * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
 + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
 + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
 +   output (CASSANDRA-7659)
 + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
 + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
 + * Support list index operations with conditions (CASSANDRA-7499)
 + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
 + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
 + * (cqlsh) Error when tracing query (CASSANDRA-7613)
 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569)
 + * SSTableExport

[jira] [Commented] (CASSANDRA-8111) Create backup directories for commitlog archiving during startup


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170922#comment-14170922
 ] 

Jason Brown commented on CASSANDRA-8111:


+1. committed to 2.0, 2.1, and trunk

Thanks!

 Create backup directories for commitlog archiving during startup
 

 Key: CASSANDRA-8111
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8111
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jan Karlsson
Priority: Trivial
  Labels: archive, commitlog
 Fix For: 2.0.11

 Attachments: CASSANDRA-8111.patch


 Cassandra currently crashes if the recovery directory in commitlog_archiving 
 does not exist (or cannot be listed). I would like to propose that Cassandra 
 creates this directory if it does not exist. This will mimic the behavior of 
 creating data, commitlog .. etc. directories during startup. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] git commit: Create backup directories for commitlog archiving during startup

Repository: cassandra
Updated Branches:
  refs/heads/trunk cd601294d - 25821864f


Create backup directories for commitlog archiving during startup

patch by Jan Karlsson, reviewed by jasobrown for CASSANDRA-8111


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d828643f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d828643f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d828643f

Branch: refs/heads/trunk
Commit: d828643f555aac14be59e7f8b35ac67cb06da20b
Parents: 7497399
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 06:31:20 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 06:31:20 2014 -0700

--
 CHANGES.txt |  1 +
 .../cassandra/db/commitlog/CommitLogArchiver.java   | 16 ++--
 2 files changed, 15 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d828643f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 84bbe75..2faea63 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * Create backup directories for commitlog archiving during startup 
(CASSANDRA-8111)
  * Reduce totalBlockFor() for LOCAL_* consistency levels (CASSANDRA-8058)
  * Fix merging schemas with re-dropped keyspaces (CASSANDRA-7256)
  * Fix counters in supercolumns during live upgrades from 1.2 (CASSANDRA-7188)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d828643f/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
index f020182..e30443e 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
@@ -45,6 +45,7 @@ public class CommitLogArchiver
 {
 private static final Logger logger = 
LoggerFactory.getLogger(CommitLogArchiver.class);
 public static final SimpleDateFormat format = new 
SimpleDateFormat(:MM:dd HH:mm:ss);
+private static final String DELIMITER = ,;
 static
 {
 format.setTimeZone(TimeZone.getTimeZone(GMT));
@@ -81,6 +82,17 @@ public class CommitLogArchiver
 archiveCommand = 
commitlog_commands.getProperty(archive_command);
 restoreCommand = 
commitlog_commands.getProperty(restore_command);
 restoreDirectories = 
commitlog_commands.getProperty(restore_directories);
+for (String dir : restoreDirectories.split(DELIMITER))
+{
+File directory = new File(dir);
+if (!directory.exists())
+{
+if (!directory.mkdir())
+{
+throw new RuntimeException(Unable to create 
directory  + dir);
+}
+}
+}
 String targetTime = 
commitlog_commands.getProperty(restore_point_in_time);
 precision = 
TimeUnit.valueOf(commitlog_commands.getProperty(precision, MICROSECONDS));
 try
@@ -151,12 +163,12 @@ public class CommitLogArchiver
 if (Strings.isNullOrEmpty(restoreDirectories))
 return;
 
-for (String dir : restoreDirectories.split(,))
+for (String dir : restoreDirectories.split(DELIMITER))
 {
 File[] files = new File(dir).listFiles();
 if (files == null)
 {
-throw new RuntimeException(Unable to list director  + dir);
+throw new RuntimeException(Unable to list directory  + dir);
 }
 for (File fromFile : files)
 {

[jira] [Commented] (CASSANDRA-8111) Create backup directories for commitlog archiving during startup

2014-10-14 Thread Michael Shuler (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170933#comment-14170933
 ] 

Michael Shuler commented on CASSANDRA-8111:
---

This commit is causing all the unit tests to fail with an NPE in cassandra-2.0 
branch (will check other branches)
{noformat}
13:44:06 [junit] Testsuite: org.apache.cassandra.cache.CacheProviderTest
13:44:09 [junit] Tests run: 0, Failures: 0, Errors: 1, Skipped: 0, Time 
elapsed: 2.851 sec
13:44:09 [junit] 
13:44:09 [junit] Testcase: org.apache.cassandra.cache.CacheProviderTest:
Caused an ERROR
13:44:09 [junit] null
13:44:09 [junit] java.lang.ExceptionInInitializerError
13:44:09 [junit]at 
org.apache.cassandra.SchemaLoader.cleanupAndLeaveDirs(SchemaLoader.java:432)
13:44:09 [junit]at 
org.apache.cassandra.SchemaLoader.loadSchema(SchemaLoader.java:64)
13:44:09 [junit]at 
org.apache.cassandra.SchemaLoader.loadSchema(SchemaLoader.java:58)
13:44:09 [junit] Caused by: java.lang.NullPointerException
13:44:09 [junit]at 
org.apache.cassandra.db.commitlog.CommitLogArchiver.init(CommitLogArchiver.java:85)
13:44:09 [junit]at 
org.apache.cassandra.db.commitlog.CommitLog.init(CommitLog.java:59)
13:44:09 [junit]at 
org.apache.cassandra.db.commitlog.CommitLog.clinit(CommitLog.java:53)
13:44:09 [junit] 
13:44:09 [junit] 
13:44:09 [junit] Test org.apache.cassandra.cache.CacheProviderTest FAILED
{noformat}
http://cassci.datastax.com/job/cassandra-2.0_utest/621/

 Create backup directories for commitlog archiving during startup
 

 Key: CASSANDRA-8111
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8111
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jan Karlsson
Priority: Trivial
  Labels: archive, commitlog
 Fix For: 2.0.11

 Attachments: CASSANDRA-8111.patch


 Cassandra currently crashes if the recovery directory in commitlog_archiving 
 does not exist (or cannot be listed). I would like to propose that Cassandra 
 creates this directory if it does not exist. This will mimic the behavior of 
 creating data, commitlog .. etc. directories during startup. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: ninja fix to add null check on CommitLogArchiver startup

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 d828643f5 - cc8eeefe8


ninja fix to add null check on CommitLogArchiver startup


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cc8eeefe
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cc8eeefe
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cc8eeefe

Branch: refs/heads/cassandra-2.0
Commit: cc8eeefe8895e01a2a5f387788bb3772b4dabbc9
Parents: d828643
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 06:57:52 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 06:57:52 2014 -0700

--
 .../cassandra/db/commitlog/CommitLogArchiver.java  | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc8eeefe/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
index e30443e..8957643 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
@@ -82,14 +82,17 @@ public class CommitLogArchiver
 archiveCommand = 
commitlog_commands.getProperty(archive_command);
 restoreCommand = 
commitlog_commands.getProperty(restore_command);
 restoreDirectories = 
commitlog_commands.getProperty(restore_directories);
-for (String dir : restoreDirectories.split(DELIMITER))
+if (restoreDirectories != null  
!restoreDirectories.isEmpty())
 {
-File directory = new File(dir);
-if (!directory.exists())
+for (String dir : restoreDirectories.split(DELIMITER))
 {
-if (!directory.mkdir())
+File directory = new File(dir);
+if (!directory.exists())
 {
-throw new RuntimeException(Unable to create 
directory  + dir);
+if (!directory.mkdir())
+{
+throw new RuntimeException(Unable to create 
directory:  + dir);
+}
 }
 }
 }

[1/2] git commit: ninja fix to add null check on CommitLogArchiver startup

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 4b7161da4 - b76dc9727


ninja fix to add null check on CommitLogArchiver startup


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cc8eeefe
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cc8eeefe
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cc8eeefe

Branch: refs/heads/cassandra-2.1
Commit: cc8eeefe8895e01a2a5f387788bb3772b4dabbc9
Parents: d828643
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 06:57:52 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 06:57:52 2014 -0700

--
 .../cassandra/db/commitlog/CommitLogArchiver.java  | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc8eeefe/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
index e30443e..8957643 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
@@ -82,14 +82,17 @@ public class CommitLogArchiver
 archiveCommand = 
commitlog_commands.getProperty(archive_command);
 restoreCommand = 
commitlog_commands.getProperty(restore_command);
 restoreDirectories = 
commitlog_commands.getProperty(restore_directories);
-for (String dir : restoreDirectories.split(DELIMITER))
+if (restoreDirectories != null  
!restoreDirectories.isEmpty())
 {
-File directory = new File(dir);
-if (!directory.exists())
+for (String dir : restoreDirectories.split(DELIMITER))
 {
-if (!directory.mkdir())
+File directory = new File(dir);
+if (!directory.exists())
 {
-throw new RuntimeException(Unable to create 
directory  + dir);
+if (!directory.mkdir())
+{
+throw new RuntimeException(Unable to create 
directory:  + dir);
+}
 }
 }
 }

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/18912409
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/18912409
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/18912409

Branch: refs/heads/trunk
Commit: 18912409116e4ceb8cac037264e76c194fb879d6
Parents: 2582186 b76dc97
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 06:59:14 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 06:59:14 2014 -0700

--
 .../cassandra/db/commitlog/CommitLogArchiver.java  | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)
--

[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b76dc972
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b76dc972
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b76dc972

Branch: refs/heads/trunk
Commit: b76dc9727403c6f9ef2eaaf9f990d784b4d2a722
Parents: 4b7161d cc8eeef
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 06:58:46 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 06:58:46 2014 -0700

--
 .../cassandra/db/commitlog/CommitLogArchiver.java  | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b76dc972/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
--

[1/3] git commit: ninja fix to add null check on CommitLogArchiver startup

Repository: cassandra
Updated Branches:
  refs/heads/trunk 25821864f - 189124091


ninja fix to add null check on CommitLogArchiver startup


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cc8eeefe
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cc8eeefe
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cc8eeefe

Branch: refs/heads/trunk
Commit: cc8eeefe8895e01a2a5f387788bb3772b4dabbc9
Parents: d828643
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 06:57:52 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 06:57:52 2014 -0700

--
 .../cassandra/db/commitlog/CommitLogArchiver.java  | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc8eeefe/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
index e30443e..8957643 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
@@ -82,14 +82,17 @@ public class CommitLogArchiver
 archiveCommand = 
commitlog_commands.getProperty(archive_command);
 restoreCommand = 
commitlog_commands.getProperty(restore_command);
 restoreDirectories = 
commitlog_commands.getProperty(restore_directories);
-for (String dir : restoreDirectories.split(DELIMITER))
+if (restoreDirectories != null  
!restoreDirectories.isEmpty())
 {
-File directory = new File(dir);
-if (!directory.exists())
+for (String dir : restoreDirectories.split(DELIMITER))
 {
-if (!directory.mkdir())
+File directory = new File(dir);
+if (!directory.exists())
 {
-throw new RuntimeException(Unable to create 
directory  + dir);
+if (!directory.mkdir())
+{
+throw new RuntimeException(Unable to create 
directory:  + dir);
+}
 }
 }
 }

[jira] [Commented] (CASSANDRA-8111) Create backup directories for commitlog archiving during startup

2014-10-14 Thread Jan Karlsson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170972#comment-14170972
 ] 

Jan Karlsson commented on CASSANDRA-8111:
-

I believe 
{code}
if(Strings.isNullOrEmpty(restoreDirectories))
{code}
should be added before looping on restoreDirectories

 Create backup directories for commitlog archiving during startup
 

 Key: CASSANDRA-8111
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8111
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jan Karlsson
Priority: Trivial
  Labels: archive, commitlog
 Fix For: 2.0.11

 Attachments: CASSANDRA-8111.patch


 Cassandra currently crashes if the recovery directory in commitlog_archiving 
 does not exist (or cannot be listed). I would like to propose that Cassandra 
 creates this directory if it does not exist. This will mimic the behavior of 
 creating data, commitlog .. etc. directories during startup. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8111) Create backup directories for commitlog archiving during startup


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170973#comment-14170973
 ] 

Jason Brown commented on CASSANDRA-8111:


pushed a null check fix in cc8eeefe8895e01a2a5f387788bb3772b4dabbc9

 Create backup directories for commitlog archiving during startup
 

 Key: CASSANDRA-8111
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8111
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jan Karlsson
Priority: Trivial
  Labels: archive, commitlog
 Fix For: 2.0.11

 Attachments: CASSANDRA-8111.patch


 Cassandra currently crashes if the recovery directory in commitlog_archiving 
 does not exist (or cannot be listed). I would like to propose that Cassandra 
 creates this directory if it does not exist. This will mimic the behavior of 
 creating data, commitlog .. etc. directories during startup. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b76dc972
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b76dc972
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b76dc972

Branch: refs/heads/cassandra-2.1
Commit: b76dc9727403c6f9ef2eaaf9f990d784b4d2a722
Parents: 4b7161d cc8eeef
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 06:58:46 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 06:58:46 2014 -0700

--
 .../cassandra/db/commitlog/CommitLogArchiver.java  | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b76dc972/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java
--

[jira] [Comment Edited] (CASSANDRA-8111) Create backup directories for commitlog archiving during startup

2014-10-14 Thread Jan Karlsson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170972#comment-14170972
 ] 

Jan Karlsson edited comment on CASSANDRA-8111 at 10/14/14 2:16 PM:
---

I believe I missed
{code}
if(Strings.isNullOrEmpty(restoreDirectories))
{code}
before looping on restoreDirectories


was (Author: jan karlsson):
I believe 
{code}
if(Strings.isNullOrEmpty(restoreDirectories))
{code}
should be added before looping on restoreDirectories

 Create backup directories for commitlog archiving during startup
 

 Key: CASSANDRA-8111
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8111
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jan Karlsson
Priority: Trivial
  Labels: archive, commitlog
 Fix For: 2.0.11, 2.1.1

 Attachments: CASSANDRA-8111.patch


 Cassandra currently crashes if the recovery directory in commitlog_archiving 
 does not exist (or cannot be listed). I would like to propose that Cassandra 
 creates this directory if it does not exist. This will mimic the behavior of 
 creating data, commitlog .. etc. directories during startup. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8109) Avoid constant boxing in ColumnStats.{Min/Max}Tracker


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8109:
---
Assignee: Rajanarayanan Thottuvaikkatumana

 Avoid constant boxing in ColumnStats.{Min/Max}Tracker
 -

 Key: CASSANDRA-8109
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8109
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Rajanarayanan Thottuvaikkatumana
Priority: Minor
  Labels: lhf
 Fix For: 3.0


 We use the {{ColumnStats.MinTracker}} and {{ColumnStats.MaxTracker}} to track 
 timestamps and deletion times in sstable. Those classes are generics but we 
 really ever use them for longs and integers. The consequence is that every 
 call to their {{update}} method (called for every cell during sstable write) 
 box it's argument (since we don't store the cell timestamps and deletion time 
 boxed). That feels like a waste that is easy to fix: we could just make those 
 work on longs only for instance and convert back to int at the end when 
 that's what we need.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8109) Avoid constant boxing in ColumnStats.{Min/Max}Tracker


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171025#comment-14171025
 ] 

Tyler Hobbs commented on CASSANDRA-8109:


[~rnamboodiri] I've assigned the ticket to you, but in general, yes, you can 
assign yourself a ticket and leave a comment stating that you intend to handle 
the ticket.

 Avoid constant boxing in ColumnStats.{Min/Max}Tracker
 -

 Key: CASSANDRA-8109
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8109
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Rajanarayanan Thottuvaikkatumana
Priority: Minor
  Labels: lhf
 Fix For: 3.0


 We use the {{ColumnStats.MinTracker}} and {{ColumnStats.MaxTracker}} to track 
 timestamps and deletion times in sstable. Those classes are generics but we 
 really ever use them for longs and integers. The consequence is that every 
 call to their {{update}} method (called for every cell during sstable write) 
 box it's argument (since we don't store the cell timestamps and deletion time 
 boxed). That feels like a waste that is easy to fix: we could just make those 
 work on longs only for instance and convert back to int at the end when 
 that's what we need.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8090) NullPointerException when using prepared statements

2014-10-14 Thread T Jake Luciani (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171069#comment-14171069
 ] 

T Jake Luciani commented on CASSANDRA-8090:
---

Actually any stress read now fails.  

{code}
➜  cassandra git:(trunk) ✗ ./tools/bin/cassandra-stress read n=100 -rate 
threads=50
Warming up READ with 5 iterations...
INFO  15:23:24 Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy 
(if this is incorrect, please provide the correct datacenter name with 
DCAwareRoundRobinPolicy constructor)
Connected to cluster: Test Cluster
INFO  15:23:24 New Cassandra host localhost/127.0.0.1:9042 added
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
java.io.IOException: Operation x0 on key(s) [4f4c344d323535353430]: Data 
returned was not validated

at org.apache.cassandra.stress.Operation.error(Operation.java:153)
at 
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:125)
at 
org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:99)
at 
org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:107)
at 
org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:281)
at 
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:320)
{code}

 NullPointerException when using prepared statements
 ---

 Key: CASSANDRA-8090
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8090
 Project: Cassandra
  Issue Type: Bug
Reporter: Carl Yeksigian
Assignee: Benjamin Lerer

 Due to the changes in CASSANDRA-4914, using a prepared statement from 
 multiple threads leads to a race condition where the simple selection may be 
 reset from a different thread, causing the following NPE:
 {noformat}
 java.lang.NullPointerException: null
   at org.apache.cassandra.cql3.ResultSet.addRow(ResultSet.java:63) 
 ~[main/:na]
   at 
 org.apache.cassandra.cql3.statements.Selection$ResultSetBuilder.build(Selection.java:372)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:1120)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:283)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:260)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:213)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:63)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:226)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:481)
  ~[main/:na]
   at 
 org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:133)
  ~[main/:na]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:438)
  [main/:na]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:334)
  [main/:na]
   at 
 io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_67]
   at 
 org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163)
  [main/:na]
   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) 
 [main/:na]
   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
 {noformat}
 Reproduced this using the stress tool:
 {noformat}
  ./tools/bin/cassandra-stress user profile=tools/cqlstress-example.yaml 
 ops\(insert=1,simple1=1\)
 {noformat}
 You'll need to change the {noformat}select:{noformat} line to be /1000 to 
 prevent the illegal query exceptions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7446) Batchlog should be streamed to a different node on decom


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-7446:
-
Reviewer: Aleksey Yeschenko

 Batchlog should be streamed to a different node on decom
 

 Key: CASSANDRA-7446
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7446
 Project: Cassandra
  Issue Type: Bug
Reporter: Aleksey Yeschenko
Assignee: Branimir Lambov

 Just like we stream hints on decom, we should also stream the contents of the 
 batchlog - even though we do replicate the batch to at least two nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: remove empty static constructor

Repository: cassandra
Updated Branches:
  refs/heads/trunk 6801cb041 - c55cbc763


remove empty static constructor


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c55cbc76
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c55cbc76
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c55cbc76

Branch: refs/heads/trunk
Commit: c55cbc763506d8c48ae251fc806316ac055de696
Parents: 6801cb0
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Oct 14 10:40:29 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Oct 14 10:40:29 2014 -0500

--
 src/java/org/apache/cassandra/io/sstable/SSTableLoader.java | 4 
 1 file changed, 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c55cbc76/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java 
b/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
index 68bc6ab..6cb7f03 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
@@ -55,10 +55,6 @@ public class SSTableLoader implements StreamEventHandler
 private final ListSSTableReader sstables = new ArrayList();
 private final MultimapInetAddress, 
StreamSession.SSTableStreamingSections streamingDetails = 
HashMultimap.create();
 
-static
-{
-}
-
 public SSTableLoader(File directory, Client client, OutputHandler 
outputHandler)
 {
 this(directory, client, outputHandler, 1);

[jira] [Updated] (CASSANDRA-7446) Batchlog should be streamed to a different node on decom


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-7446:
-
Fix Version/s: 2.1.1
   2.0.11

 Batchlog should be streamed to a different node on decom
 

 Key: CASSANDRA-7446
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7446
 Project: Cassandra
  Issue Type: Bug
Reporter: Aleksey Yeschenko
Assignee: Branimir Lambov
 Fix For: 2.0.11, 2.1.1


 Just like we stream hints on decom, we should also stream the contents of the 
 batchlog - even though we do replicate the batch to at least two nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7446) Batchlog should be streamed to a different node on decom


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171094#comment-14171094
 ] 

Aleksey Yeschenko commented on CASSANDRA-7446:
--

bq. Is this understanding correct?

Yup. And the commit looks good so far.

 Batchlog should be streamed to a different node on decom
 

 Key: CASSANDRA-7446
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7446
 Project: Cassandra
  Issue Type: Bug
Reporter: Aleksey Yeschenko
Assignee: Branimir Lambov
 Fix For: 2.0.11, 2.1.1


 Just like we stream hints on decom, we should also stream the contents of the 
 batchlog - even though we do replicate the batch to at least two nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Remove fat client mode

Repository: cassandra
Updated Branches:
  refs/heads/trunk 189124091 - 6801cb041


Remove fat client mode

Patch by brandonwilliams, reviewed by Joshua McKenzie for CASSANDRA-7820


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6801cb04
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6801cb04
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6801cb04

Branch: refs/heads/trunk
Commit: 6801cb0415f40d6be1333262ae95befd8abd9175
Parents: 1891240
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Oct 14 10:36:04 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Oct 14 10:36:04 2014 -0500

--
 NEWS.txt|   1 +
 examples/client_only/README.txt |  49 --
 examples/client_only/bin/client_only|  56 --
 examples/client_only/build.xml  |  70 ---
 examples/client_only/conf/cassandra.yaml| 625 ---
 examples/client_only/src/ClientOnlyExample.java | 114 
 .../org/apache/cassandra/config/Config.java |  12 -
 .../cassandra/config/DatabaseDescriptor.java|  14 +-
 .../org/apache/cassandra/db/DefsTables.java |  78 +--
 .../org/apache/cassandra/db/Directories.java|   5 -
 src/java/org/apache/cassandra/db/Keyspace.java  |   3 +-
 src/java/org/apache/cassandra/gms/Gossiper.java |   4 +-
 .../hadoop/AbstractBulkRecordWriter.java|   1 -
 .../cassandra/io/sstable/SSTableLoader.java |   1 -
 .../cassandra/io/sstable/SSTableReader.java |   5 +-
 .../cassandra/service/AbstractReadExecutor.java |   5 -
 .../cassandra/service/MigrationManager.java |   2 +-
 .../cassandra/service/StorageService.java   |  93 +--
 .../locator/DynamicEndpointSnitchTest.java  |   2 +-
 .../cassandra/service/InitClientTest.java   |  33 -
 .../service/StorageServiceClientTest.java   |  46 --
 21 files changed, 48 insertions(+), 1171 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6801cb04/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 99144ca..9e8f726 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -25,6 +25,7 @@ New features
 
 Upgrading
 -
+   - Fat client support has been removed since we have push notifications to 
clients
- cassandra-cli has been removed. Please use cqlsh instead.
- YamlFileNetworkTopologySnitch has been removed; switch to
  GossipingPropertyFileSnitch instead.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6801cb04/examples/client_only/README.txt
--
diff --git a/examples/client_only/README.txt b/examples/client_only/README.txt
deleted file mode 100644
index 47e6a3d..000
--- a/examples/client_only/README.txt
+++ /dev/null
@@ -1,49 +0,0 @@
-The client_only example uses the fat client to insert data into and read
-data from Cassandra.
-
--- Warning --
-The method used in this example (the fat client) should generally
-not be used instead of the thrift interface because of possible
-instability of the internal Cassandra API.
-
--- Prerequisite --
-Build the Cassandra source in the current source tree. Also, if
-running the client_only code against a local Cassandra node, start
-the local node prior to running the client_only script. See the
-configuration below for more info.
-
--- Build --
-To build, run ant from the contrib/client_only directory. It will build
-the source, then jar up the compiled class, the conf/cassandra.yaml, and
-dependencies into build/client_only.jar.
-
--- Run --
-To run, from the contrib/client_only directory run:
-bin/client_only write
-or
-bin/client_only read
-
-'write' will create keyspace Keyspace1 and column family Standard1. If
-it is already there, it will error out. It will then write a bunch of
-data to the cluster it connects to.
-
-'read' will read the data that was written in the write step.
-
--- Configuration --
-The conf/cassandra.yaml is to start up the fat client. The fat client
-joins the gossip network but does not participate in storage. It
-needs to have the same configuration as the rest of the cluster except
-listen address and rpc address. If you are running your cluster just
-on your local machine, you'll need to use another address for this node.
-Therefore, your local full Cassandra node can be 127.0.0.1 and the fat 
-client can be 127.0.0.2. Such aliasing is enabled by default on linux.
-On Mac OS X, use the following command to use the second IP address:
-sudo ifconfig lo0 alias 127.0.0.2 up
-
-cassandra.yaml can be on the classpath as is done here, can be specified
-(by modifying the script) in a location within the classpath like this:
-java -Xmx1G

[jira] [Commented] (CASSANDRA-8113) Gossip should ignore generation numbers too far in the future


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171097#comment-14171097
 ] 

Brandon Williams commented on CASSANDRA-8113:
-

On the whole, +1, though I think MAX_VERSION_DIFFERENCE should be 
MAX_GENERATION_DIFFERENCE so it's more correct, since we have both generations 
and versions in gossip.

 Gossip should ignore generation numbers too far in the future
 -

 Key: CASSANDRA-8113
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8113
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Low
Assignee: Jason Brown
 Attachments: 8113-v1.txt, 8113-v2.txt


 If a node sends corrupted gossip, it could set the generation numbers for 
 other nodes to arbitrarily large values. This is dangerous since one bad node 
 (e.g. with bad memory) could in theory bring down the cluster. Nodes should 
 refuse to accept generation numbers that are too far in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8109) Avoid constant boxing in ColumnStats.{Min/Max}Tracker


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171099#comment-14171099
 ] 

Brandon Williams commented on CASSANDRA-8109:
-

Actually to be an assignee that means you have to be a contributor, so if we 
add someone who hasn't contributed before just to assign, then they never post 
a patch, we have go undo the addition which is painful.  Much better to just 
leave a comment, post a patch, then we'll assign on commit.

 Avoid constant boxing in ColumnStats.{Min/Max}Tracker
 -

 Key: CASSANDRA-8109
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8109
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Rajanarayanan Thottuvaikkatumana
Priority: Minor
  Labels: lhf
 Fix For: 3.0


 We use the {{ColumnStats.MinTracker}} and {{ColumnStats.MaxTracker}} to track 
 timestamps and deletion times in sstable. Those classes are generics but we 
 really ever use them for longs and integers. The consequence is that every 
 call to their {{update}} method (called for every cell during sstable write) 
 box it's argument (since we don't store the cell timestamps and deletion time 
 boxed). That feels like a waste that is easy to fix: we could just make those 
 work on longs only for instance and convert back to int at the end when 
 that's what we need.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8110) Make streaming backwards compatible


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171103#comment-14171103
 ] 

Aleksey Yeschenko commented on CASSANDRA-8110:
--

One direction is easy - and that's older nodes streaming *to* newer nodes, 
because newer nodes can handle previous-version sstables. That alone would be a 
significant improvement, and I suggest we start with that first.

The other direction is trickier, because new-format sstables would have to be 
converted to old-format sstables before streaming, or on the fly.

 Make streaming backwards compatible
 ---

 Key: CASSANDRA-8110
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8110
 Project: Cassandra
  Issue Type: Improvement
Reporter: Marcus Eriksson
 Fix For: 3.0


 To be able to seamlessly upgrade clusters we need to make it possible to 
 stream files between nodes with different StreamMessage.CURRENT_VERSION



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8097) Hi Team, I am new to cassandr seeing folloiwng errors in cassandra system.log file.NO_SOURCE_FILE (line 0) Thrift operation queue is full, discarding thrift operatio


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs resolved CASSANDRA-8097.

Resolution: Invalid
  Assignee: Tyler Hobbs

I'm not sure what's causing your problem, but I would check with the DataStax 
OpsCenter team through a StackOverflow post with proper tagging.

 Hi Team, I am new to cassandr seeing folloiwng errors in cassandra system.log 
 file.NO_SOURCE_FILE (line 0) Thrift operation queue is full, discarding 
 thrift operation
 --

 Key: CASSANDRA-8097
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8097
 Project: Cassandra
  Issue Type: Bug
  Components: Examples
Reporter: Swapnil Bhisey
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.0.10


 NO_SOURCE_FILE (line 0) Thrift operation queue is full, discarding thrift 
 operation
 NO_SOURCE_FILE (line 0) 919788 operations dropped so far.
 NO_SOURCE_FILE (line 0) Thrift operation queue is full, discarding thrift 
 operation
 NO_SOURCE_FILE (line 0) 919789 operations dropped so far.
 NO_SOURCE_FILE (line 0) Thrift operation queue is full, discarding thrift 
 operation
 NO_SOURCE_FILE (line 0) 919790 operations dropped so far.
 NO_SOURCE_FILE (line 0) Thrift operation queue is full, discarding thrift 
 operation
 NO_SOURCE_FILE (line 0) 919791 operations dropped so far.
 NO_SOURCE_FILE (line 0) Thrift operation queue is full, discarding thrift 
 operation
 NO_SOURCE_FILE (line 0) 919792 operations dropped so far.
 NO_SOURCE_FILE (line 0) Thrift operation queue is full, discarding thrift 
 operation
 NO_SOURCE_FILE (line 0) 919793 operations dropped so far.
 NO_SOURCE_FILE (line 0) Thrift operation queue is full, discarding thrift 
 operation
 NO_SOURCE_FILE (line 0) 919794 operations dropped so far.
 NO_SOURCE_FILE (line 0) Thrift operation queue is full, discarding thrift 
 operation
 NO_SOURCE_FILE (line 0) 919795 operations dropped so far.
 NO_SOURCE_FILE (line 0) Thrift operation queue is full, discarding thrift 
 operation
 NO_SOURCE_FILE (line 0) 919796 operations dropped so far.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8105) NPE for null embedded UDT inside set


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8105:
---
Reviewer: Tyler Hobbs

 NPE for null embedded UDT inside set
 

 Key: CASSANDRA-8105
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8105
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Andy Ketskes
Assignee: Robert Stupp
  Labels: cql
 Fix For: 2.1.1

 Attachments: 8105.txt


 An NPE is thrown parsing an INSERT statement when a embedded UDT inside 
 another UDT is set to null inside a set.
 This sounds very convoluted, but the examples below will hopefully make it 
 clear...
 With the following definitions:
 CREATE TYPE ut1 (a int, b int);
 CREATE TYPE ut2 (j frozenut1, k int);
 CREATE TYPE ut3 (i int, j frozenut1);
 CREATE TABLE tab1 (x int PRIMARY KEY, y setfrozenut2);
 CREATE TABLE tab2 (x int PRIMARY KEY, y listfrozenut2);
 CREATE TABLE tab3 (x int PRIMARY KEY, y setfrozenut3);
 This query throws a NullPointerException:
 INSERT INTO tab1 (x, y) VALUES (1, { { k: 1 } });
 These however doesn't:
 INSERT INTO tab2 (x, y) VALUES (1, [ { k: 1 } ]);
 INSERT INTO tab3 (x, y) VALUES (1, { { i: 1 } });
 So, the bug seems to be triggered only when the UDT is in a set, lists are 
 fine. If the embedded UDT is after the value specified in the query, the bug  
 doesn't seem to trigger.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8115) Windows install scripts fail to set logdir and datadir

2014-10-14 Thread Joshua McKenzie (JIRA)

Joshua McKenzie created CASSANDRA-8115:
--

 Summary: Windows install scripts fail to set logdir and datadir
 Key: CASSANDRA-8115
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8115
 Project: Cassandra
  Issue Type: Bug
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
 Fix For: 2.1.1


After CASSANDRA-7136, the install scripts to run Cassandra as a service fail on 
both the legacy and the powershell paths.  Looks like they need to have
{code}
++JvmOptions=-Dcassandra.logdir=%CASSANDRA_HOME%\logs ^
++JvmOptions=-Dcassandra.storagedir=%CASSANDRA_HOME%\data
{code}
added to function correctly.

We should take this opportunity to make sure the source of the java options is 
uniform for both running and installation to prevent mismatches like this in 
the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8116) HSHA fails with default rpc_max_threads setting

2014-10-14 Thread Mike Adamson (JIRA)

Mike Adamson created CASSANDRA-8116:
---

 Summary: HSHA fails with default rpc_max_threads setting
 Key: CASSANDRA-8116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8116
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Mike Adamson
Assignee: Mike Adamson
Priority: Minor
 Fix For: 2.0.11


The HSHA server fails with 'Out of heap space' error if the rpc_max_threads is 
left at its default setting (unlimited) in cassandra.yaml.

I'm not proposing any code change for this but have submitted a patch for a 
comment change in cassandra.yaml to indicate that rpc_max_threads needs to be 
changed if you use HSHA.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8116) HSHA fails with default rpc_max_threads setting

2014-10-14 Thread Mike Adamson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Adamson updated CASSANDRA-8116:

Attachment: 8116.txt

 HSHA fails with default rpc_max_threads setting
 ---

 Key: CASSANDRA-8116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8116
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Mike Adamson
Assignee: Mike Adamson
Priority: Minor
 Fix For: 2.0.11

 Attachments: 8116.txt


 The HSHA server fails with 'Out of heap space' error if the rpc_max_threads 
 is left at its default setting (unlimited) in cassandra.yaml.
 I'm not proposing any code change for this but have submitted a patch for a 
 comment change in cassandra.yaml to indicate that rpc_max_threads needs to be 
 changed if you use HSHA.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7979) Acceptable time skew for C*

2014-10-14 Thread sankalp kohli (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-7979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171251#comment-14171251
]

sankalp kohli commented on CASSANDRA-7979:
--

[~JoshuaMcKenzie] What do you think about about Histogram?

Acceptable time skew for C*
---

Key: CASSANDRA-7979
URL: https://issues.apache.org/jira/browse/CASSANDRA-7979
Project: Cassandra
Issue Type: Improvement
Reporter: sankalp kohli
Assignee: sankalp kohli
Priority: Minor
Attachments: 2.0_7979.diff, trunk_7979.diff

It is very hard to know the bounds on clock skew required for C* to work
properly. Since the resolution is based on time and is at thrift column
level, it depends on the application. How fast is the application updating
the same column. If you update a column say after 5 millisecond and the clock
skew is more than that, you might not see the updates in correct order.
In this JIRA, I am proposing a change which will answer this question: How
much clock skew is acceptable for a given application. This will help answer
the question whether the system needs some alternate NTP algorithms to keep
time in sync.
If we measure the time difference between two updates to the same column, we
will be able to answer the question on clock skew.
We can implement this in memtable(AtomicSortedColumns.addColumn). If we find
that a column is updated within say 100 millisecond, add the diff to a
histogram. Since this might have performance issues, we might want to have
some throttling like randomization or only enable it for a small time via
nodetool.
With this histogram, we will know what is an acceptable clock skew.
Also apart from column resolution, is there any other area which will be
affected by clock skew?
Note: For the sake of argument, I am not talking about back date deletes or
application modified timestamps.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[5/6] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e57d027c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e57d027c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e57d027c

Branch: refs/heads/cassandra-2.1
Commit: e57d027c94ad6e0c447cd3162e92517db017ee25
Parents: b76dc97 1d02d7b
Author: Mikhail Stepura mish...@apache.org
Authored: Tue Oct 14 10:38:30 2014 -0700
Committer: Mikhail Stepura mish...@apache.org
Committed: Tue Oct 14 10:38:30 2014 -0700

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e57d027c/CHANGES.txt
--
diff --cc CHANGES.txt
index bbe6982,8b462f1..74647c8
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,76 -1,5 +1,77 @@@
 -2.0.11:
 +2.1.1
 + * Fix high memory use due to tracking reads on incrementally opened sstable 
readers (CASSANDRA-8066)
 + * Fix EXECUTE request with skipMetadata=false returning no metadata
 +   (CASSANDRA-8054)
 + * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)
 + * Shutdown JVM on OOM (CASSANDRA-7507)
 + * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
 + * Don't duplicate sstables smaller than split size when using
 +   the sstablesplitter tool (CASSANDRA-7616)
 + * Avoid re-parsing already prepared statements (CASSANDRA-7923)
 + * Fix some Thrift slice deletions and updates of COMPACT STORAGE
 +   tables with some clustering columns omitted (CASSANDRA-7990)
 + * Fix filtering for CONTAINS on sets (CASSANDRA-8033)
 + * Properly track added size (CASSANDRA-7239)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Fix Assertion error on RangeTombstoneList diff (CASSANDRA-8013)
 + * Release references to overlapping sstables during compaction 
(CASSANDRA-7819)
 + * Send notification when opening compaction results early (CASSANDRA-8034)
 + * Make native server start block until properly bound (CASSANDRA-7885)
 + * (cqlsh) Fix IPv6 support (CASSANDRA-7988)
 + * Ignore fat clients when checking for endpoint collision (CASSANDRA-7939)
 + * Make sstablerepairedset take a list of files (CASSANDRA-7995)
 + * (cqlsh) Tab completeion for indexes on map keys (CASSANDRA-7972)
 + * (cqlsh) Fix UDT field selection in select clause (CASSANDRA-7891)
 + * Fix resource leak in event of corrupt sstable
 + * (cqlsh) Add command line option for cqlshrc file path (CASSANDRA-7131)
 + * Provide visibility into prepared statements churn (CASSANDRA-7921, 
CASSANDRA-7930)
 + * Invalidate prepared statements when their keyspace or table is
 +   dropped (CASSANDRA-7566)
 + * cassandra-stress: fix support for NetworkTopologyStrategy (CASSANDRA-7945)
 + * Fix saving caches when a table is dropped (CASSANDRA-7784)
 + * Add better error checking of new stress profile (CASSANDRA-7716)
 + * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom 
(CASSANDRA-7934)
 + * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
 + * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
 + * GCInspector more closely tracks GC; cassandra-stress and nodetool report 
it (CASSANDRA-7916)
 + * nodetool won't output bogus ownership info without a keyspace 
(CASSANDRA-7173)
 + * Add human readable option to nodetool commands (CASSANDRA-5433)
 + * Don't try to set repairedAt on old sstables (CASSANDRA-7913)
 + * Add metrics for tracking PreparedStatement use (CASSANDRA-7719)
 + * (cqlsh) tab-completion for triggers (CASSANDRA-7824)
 + * (cqlsh) Support for query paging (CASSANDRA-7514)
 + * (cqlsh) Show progress of COPY operations (CASSANDRA-7789)
 + * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
 + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
 + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
 + * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
 + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
 + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
 +   output (CASSANDRA-7659)
 + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
 + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
 + * Support list index operations with conditions (CASSANDRA-7499)
 + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
 + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
 + * (cqlsh) Error when tracing query (CASSANDRA-7613)
 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569)
 + * SSTableExport uses correct validator to create string

[1/6] git commit: Fix autocompletion for alter keyspace

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 cc8eeefe8 - 1d02d7b34
  refs/heads/cassandra-2.1 b76dc9727 - e57d027c9
  refs/heads/trunk c55cbc763 - 13c4719f7


Fix autocompletion for alter keyspace

patch by Rajanarayanan Thottuvaikkatumana; reviewed by Mikhail Stepura for 
CASSANDRA-8021


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1d02d7b3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1d02d7b3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1d02d7b3

Branch: refs/heads/cassandra-2.0
Commit: 1d02d7b34eafd0594ffc87608ac816fb5af39b89
Parents: cc8eeef
Author: Rajanarayanan Thottuvaikkatumana rnambood...@gmail.com
Authored: Tue Oct 14 10:36:53 2014 -0700
Committer: Mikhail Stepura mish...@apache.org
Committed: Tue Oct 14 10:36:53 2014 -0700

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d02d7b3/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2faea63..8b462f1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * (cqlsh) Fix autocompletion for alter keyspace (CASSANDRA-8021)
  * Create backup directories for commitlog archiving during startup 
(CASSANDRA-8111)
  * Reduce totalBlockFor() for LOCAL_* consistency levels (CASSANDRA-8058)
  * Fix merging schemas with re-dropped keyspaces (CASSANDRA-7256)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d02d7b3/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 01818ca..c08088a 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -993,7 +993,7 @@ def alter_table_col_completer(ctxt, cass):
 explain_completion('alterInstructions', 'newcol', 'new_column_name')
 
 syntax_rules += r'''
-alterKeyspaceStatement ::= ALTER ( KEYSPACE | SCHEMA ) 
ks=alterableKeyspaceName
+alterKeyspaceStatement ::= ALTER wat=( KEYSPACE | SCHEMA ) 
ks=alterableKeyspaceName
  WITH property ( AND property )*
;
 '''

[3/6] git commit: Fix autocompletion for alter keyspace

Fix autocompletion for alter keyspace

patch by Rajanarayanan Thottuvaikkatumana; reviewed by Mikhail Stepura for 
CASSANDRA-8021


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1d02d7b3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1d02d7b3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1d02d7b3

Branch: refs/heads/trunk
Commit: 1d02d7b34eafd0594ffc87608ac816fb5af39b89
Parents: cc8eeef
Author: Rajanarayanan Thottuvaikkatumana rnambood...@gmail.com
Authored: Tue Oct 14 10:36:53 2014 -0700
Committer: Mikhail Stepura mish...@apache.org
Committed: Tue Oct 14 10:36:53 2014 -0700

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d02d7b3/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2faea63..8b462f1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * (cqlsh) Fix autocompletion for alter keyspace (CASSANDRA-8021)
  * Create backup directories for commitlog archiving during startup 
(CASSANDRA-8111)
  * Reduce totalBlockFor() for LOCAL_* consistency levels (CASSANDRA-8058)
  * Fix merging schemas with re-dropped keyspaces (CASSANDRA-7256)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d02d7b3/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 01818ca..c08088a 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -993,7 +993,7 @@ def alter_table_col_completer(ctxt, cass):
 explain_completion('alterInstructions', 'newcol', 'new_column_name')
 
 syntax_rules += r'''
-alterKeyspaceStatement ::= ALTER ( KEYSPACE | SCHEMA ) 
ks=alterableKeyspaceName
+alterKeyspaceStatement ::= ALTER wat=( KEYSPACE | SCHEMA ) 
ks=alterableKeyspaceName
  WITH property ( AND property )*
;
 '''

[4/6] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e57d027c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e57d027c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e57d027c

Branch: refs/heads/trunk
Commit: e57d027c94ad6e0c447cd3162e92517db017ee25
Parents: b76dc97 1d02d7b
Author: Mikhail Stepura mish...@apache.org
Authored: Tue Oct 14 10:38:30 2014 -0700
Committer: Mikhail Stepura mish...@apache.org
Committed: Tue Oct 14 10:38:30 2014 -0700

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e57d027c/CHANGES.txt
--
diff --cc CHANGES.txt
index bbe6982,8b462f1..74647c8
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,76 -1,5 +1,77 @@@
 -2.0.11:
 +2.1.1
 + * Fix high memory use due to tracking reads on incrementally opened sstable 
readers (CASSANDRA-8066)
 + * Fix EXECUTE request with skipMetadata=false returning no metadata
 +   (CASSANDRA-8054)
 + * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)
 + * Shutdown JVM on OOM (CASSANDRA-7507)
 + * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
 + * Don't duplicate sstables smaller than split size when using
 +   the sstablesplitter tool (CASSANDRA-7616)
 + * Avoid re-parsing already prepared statements (CASSANDRA-7923)
 + * Fix some Thrift slice deletions and updates of COMPACT STORAGE
 +   tables with some clustering columns omitted (CASSANDRA-7990)
 + * Fix filtering for CONTAINS on sets (CASSANDRA-8033)
 + * Properly track added size (CASSANDRA-7239)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Fix Assertion error on RangeTombstoneList diff (CASSANDRA-8013)
 + * Release references to overlapping sstables during compaction 
(CASSANDRA-7819)
 + * Send notification when opening compaction results early (CASSANDRA-8034)
 + * Make native server start block until properly bound (CASSANDRA-7885)
 + * (cqlsh) Fix IPv6 support (CASSANDRA-7988)
 + * Ignore fat clients when checking for endpoint collision (CASSANDRA-7939)
 + * Make sstablerepairedset take a list of files (CASSANDRA-7995)
 + * (cqlsh) Tab completeion for indexes on map keys (CASSANDRA-7972)
 + * (cqlsh) Fix UDT field selection in select clause (CASSANDRA-7891)
 + * Fix resource leak in event of corrupt sstable
 + * (cqlsh) Add command line option for cqlshrc file path (CASSANDRA-7131)
 + * Provide visibility into prepared statements churn (CASSANDRA-7921, 
CASSANDRA-7930)
 + * Invalidate prepared statements when their keyspace or table is
 +   dropped (CASSANDRA-7566)
 + * cassandra-stress: fix support for NetworkTopologyStrategy (CASSANDRA-7945)
 + * Fix saving caches when a table is dropped (CASSANDRA-7784)
 + * Add better error checking of new stress profile (CASSANDRA-7716)
 + * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom 
(CASSANDRA-7934)
 + * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
 + * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
 + * GCInspector more closely tracks GC; cassandra-stress and nodetool report 
it (CASSANDRA-7916)
 + * nodetool won't output bogus ownership info without a keyspace 
(CASSANDRA-7173)
 + * Add human readable option to nodetool commands (CASSANDRA-5433)
 + * Don't try to set repairedAt on old sstables (CASSANDRA-7913)
 + * Add metrics for tracking PreparedStatement use (CASSANDRA-7719)
 + * (cqlsh) tab-completion for triggers (CASSANDRA-7824)
 + * (cqlsh) Support for query paging (CASSANDRA-7514)
 + * (cqlsh) Show progress of COPY operations (CASSANDRA-7789)
 + * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
 + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
 + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
 + * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
 + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
 + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
 +   output (CASSANDRA-7659)
 + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
 + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
 + * Support list index operations with conditions (CASSANDRA-7499)
 + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
 + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
 + * (cqlsh) Error when tracing query (CASSANDRA-7613)
 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569)
 + * SSTableExport uses correct validator to create string representation

[2/6] git commit: Fix autocompletion for alter keyspace

Fix autocompletion for alter keyspace

patch by Rajanarayanan Thottuvaikkatumana; reviewed by Mikhail Stepura for 
CASSANDRA-8021


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1d02d7b3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1d02d7b3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1d02d7b3

Branch: refs/heads/cassandra-2.1
Commit: 1d02d7b34eafd0594ffc87608ac816fb5af39b89
Parents: cc8eeef
Author: Rajanarayanan Thottuvaikkatumana rnambood...@gmail.com
Authored: Tue Oct 14 10:36:53 2014 -0700
Committer: Mikhail Stepura mish...@apache.org
Committed: Tue Oct 14 10:36:53 2014 -0700

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d02d7b3/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2faea63..8b462f1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * (cqlsh) Fix autocompletion for alter keyspace (CASSANDRA-8021)
  * Create backup directories for commitlog archiving during startup 
(CASSANDRA-8111)
  * Reduce totalBlockFor() for LOCAL_* consistency levels (CASSANDRA-8058)
  * Fix merging schemas with re-dropped keyspaces (CASSANDRA-7256)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d02d7b3/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 01818ca..c08088a 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -993,7 +993,7 @@ def alter_table_col_completer(ctxt, cass):
 explain_completion('alterInstructions', 'newcol', 'new_column_name')
 
 syntax_rules += r'''
-alterKeyspaceStatement ::= ALTER ( KEYSPACE | SCHEMA ) 
ks=alterableKeyspaceName
+alterKeyspaceStatement ::= ALTER wat=( KEYSPACE | SCHEMA ) 
ks=alterableKeyspaceName
  WITH property ( AND property )*
;
 '''

[6/6] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/13c4719f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/13c4719f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/13c4719f

Branch: refs/heads/trunk
Commit: 13c4719f772bc43d15a22f903f6c46353a0fc349
Parents: c55cbc7 e57d027
Author: Mikhail Stepura mish...@apache.org
Authored: Tue Oct 14 10:38:42 2014 -0700
Committer: Mikhail Stepura mish...@apache.org
Committed: Tue Oct 14 10:38:42 2014 -0700

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/13c4719f/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/13c4719f/pylib/cqlshlib/cql3handling.py
--

[jira] [Updated] (CASSANDRA-8073) Exception filtering data in a collection and compound primary key


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8073:
---
Reviewer: Aleksey Yeschenko  (was: Benjamin Lerer)

 Exception filtering data in a collection and compound primary key
 -

 Key: CASSANDRA-8073
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8073
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Mac OS, DataStax 2.1.0 Cassandra server
Reporter: Ozéias Sant'ana
Assignee: Tyler Hobbs
 Fix For: 2.1.1

 Attachments: 8073.txt, example.cql.sql


 I'm trying to query using compound primary key and then collection. Is 
 possible to use partition key in the query, but as soon as you include the 
 clustering column will receive a timeout.
 {code:sql}
 SELECT * FROM playlists WHERE user_id = 91d8eb60-4db1-11e4-b217-f96a8b2e7d2e 
 AND tags CONTAINS 'blues';
  user_id  | id   
 | song_order | album| artist | song_id  | 
 tags   | title
 --+--++--++--++---
  91d8eb60-4db1-11e4-b217-f96a8b2e7d2e | 62c36092-82a1-3a00-93d1-46196ee77204 
 |  1 | Tres Hombres | ZZ Top | a3e64f8f-bd44-4f28-b8d9-6938726e34d4 | 
 {'blues', 'boogie rock', 'rock', 'rock blues'} | La Grange
 (1 rows)
 {code}
 {code:sql}
 SELECT * FROM playlists WHERE user_id = 91d8eb60-4db1-11e4-b217-f96a8b2e7d2e 
 AND id = 62c36092-82a1-3a00-93d1-46196ee77204 AND tags CONTAINS 'blues';
 errors={}, last_host=127.0.0.1
 {code}
 See below the log:
 {code:none}
 WARN  [SharedPool-Worker-2] 2014-10-07 10:23:56,437 
 AbstractTracingAwareExecutorService.java:167 - Uncaught exception on thread 
 Thread[SharedPool-Worker-2,5,main]: {}
 java.lang.RuntimeException: java.lang.NullPointerException
   at 
 org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2047)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 ~[na:1.7.0_67]
   at 
 org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) 
 [apache-cassandra-2.1.0.jar:2.1.0]
   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
 Caused by: java.lang.NullPointerException: null
   at org.apache.cassandra.db.marshal.Int32Type.compare(Int32Type.java:38) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at org.apache.cassandra.db.marshal.Int32Type.compare(Int32Type.java:28) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.db.composites.AbstractCType.compare(AbstractCType.java:136)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.db.composites.AbstractCType.compare(AbstractCType.java:40)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.db.AtomicBTreeColumns$3.compare(AtomicBTreeColumns.java:234)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at org.apache.cassandra.utils.btree.BTree.find(BTree.java:277) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at org.apache.cassandra.utils.btree.Path.find(Path.java:122) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at org.apache.cassandra.utils.btree.Cursor._reset(Cursor.java:107) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at org.apache.cassandra.utils.btree.Cursor.reset(Cursor.java:91) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at org.apache.cassandra.utils.btree.BTree.slice(BTree.java:238) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.db.AtomicBTreeColumns.slice(AtomicBTreeColumns.java:453) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.db.AtomicBTreeColumns.iterator(AtomicBTreeColumns.java:283)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.db.filter.SliceQueryFilter.getColumnIterator(SliceQueryFilter.java:138)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.db.filter.QueryFilter.getIterator(QueryFilter.java:57) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:206)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:59)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1872)

[jira] [Created] (CASSANDRA-8117) Checksum all communication between Cassandra nodes

2014-10-14 Thread sankalp kohli (JIRA)

sankalp kohli created CASSANDRA-8117:


 Summary: Checksum all communication between Cassandra nodes
 Key: CASSANDRA-8117
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8117
 Project: Cassandra
  Issue Type: Improvement
Reporter: sankalp kohli


We have seen corruption in Gossip due to corruption happening in the n/w card 
between main memory which is ECC protected and TCP checksum. Same corruption 
can happen to other data flowing b/w machines. We should add checksum to all 
communication b/w machines in the application. 

A different JIRA could be to add checksums from the client driver for writes 
and back to the driver for reads. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Validate ascii and utf8 string literals in cql queries

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 1d02d7b34 - 781018cb2


Validate ascii and utf8 string literals in cql queries

Patch by Tyler Hobbs; review by Aleksey Yeschenko for CASSANDRA-8101


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/781018cb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/781018cb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/781018cb

Branch: refs/heads/cassandra-2.0
Commit: 781018cb2b8df32291a2ead0d17a814074ce
Parents: 1d02d7b
Author: Tyler Hobbs tylerho...@apache.org
Authored: Tue Oct 14 12:56:23 2014 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Tue Oct 14 12:57:20 2014 -0500

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/marshal/AsciiType.java  | 27 ++-
 .../org/apache/cassandra/transport/CBUtil.java  | 51 +---
 3 files changed, 71 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/781018cb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8b462f1..432750a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * Properly validate ascii and utf8 string literals in CQL queries 
(CASSANDRA-8101)
  * (cqlsh) Fix autocompletion for alter keyspace (CASSANDRA-8021)
  * Create backup directories for commitlog archiving during startup 
(CASSANDRA-8111)
  * Reduce totalBlockFor() for LOCAL_* consistency levels (CASSANDRA-8058)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/781018cb/src/java/org/apache/cassandra/db/marshal/AsciiType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/AsciiType.java 
b/src/java/org/apache/cassandra/db/marshal/AsciiType.java
index 74ae6be..fdccfcd 100644
--- a/src/java/org/apache/cassandra/db/marshal/AsciiType.java
+++ b/src/java/org/apache/cassandra/db/marshal/AsciiType.java
@@ -18,8 +18,13 @@
 package org.apache.cassandra.db.marshal;
 
 import java.nio.ByteBuffer;
+import java.nio.CharBuffer;
+import java.nio.charset.Charset;
+import java.nio.charset.CharsetEncoder;
+import java.nio.charset.CharacterCodingException;
 
 import org.apache.cassandra.cql3.CQL3Type;
+import org.apache.cassandra.serializers.MarshalException;
 import org.apache.cassandra.serializers.TypeSerializer;
 import org.apache.cassandra.serializers.AsciiSerializer;
 
@@ -29,6 +34,15 @@ public class AsciiType extends AbstractTypeString
 
 AsciiType() {} // singleton
 
+private final ThreadLocalCharsetEncoder encoder = new 
ThreadLocalCharsetEncoder()
+{
+@Override
+protected CharsetEncoder initialValue()
+{
+return Charset.forName(US-ASCII).newEncoder();
+}
+};
+
 public int compare(ByteBuffer o1, ByteBuffer o2)
 {
 return BytesType.bytesCompare(o1, o2);
@@ -36,7 +50,18 @@ public class AsciiType extends AbstractTypeString
 
 public ByteBuffer fromString(String source)
 {
-return decompose(source);
+// the encoder must be reset each time it's used, hence the 
thread-local storage
+CharsetEncoder theEncoder = encoder.get();
+theEncoder.reset();
+
+try
+{
+return theEncoder.encode(CharBuffer.wrap(source));
+}
+catch (CharacterCodingException exc)
+{
+throw new MarshalException(String.format(Invalid ASCII character 
in string literal: %s, exc));
+}
 }
 
 public CQL3Type asCQL3Type()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/781018cb/src/java/org/apache/cassandra/transport/CBUtil.java
--
diff --git a/src/java/org/apache/cassandra/transport/CBUtil.java 
b/src/java/org/apache/cassandra/transport/CBUtil.java
index dfd0439..eb32faa 100644
--- a/src/java/org/apache/cassandra/transport/CBUtil.java
+++ b/src/java/org/apache/cassandra/transport/CBUtil.java
@@ -21,7 +21,11 @@ import java.net.InetAddress;
 import java.net.InetSocketAddress;
 import java.net.UnknownHostException;
 import java.nio.ByteBuffer;
+import java.nio.CharBuffer;
 import java.nio.charset.CharacterCodingException;
+import java.nio.charset.Charset;
+import java.nio.charset.CharsetDecoder;
+import java.nio.charset.CoderResult;
 import java.util.ArrayList;
 import java.util.Collections;
 import java.util.HashMap;
@@ -36,6 +40,7 @@ import org.jboss.netty.util.CharsetUtil;
 import org.apache.cassandra.db.ConsistencyLevel;
 import org.apache.cassandra.db.TypeSizes;
 import org.apache.cassandra.utils.UUIDGen;
+import org.apache.cassandra.utils.ByteBufferUtil;
 
 /**
  * ChannelBuffer utility methods.
@@ -48,21

[jira] [Updated] (CASSANDRA-8113) Gossip should ignore generation numbers too far in the future


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-8113:
---
Attachment: 8113-v3.txt

v3 addresses both generation value being crazy, as well as the max version 
being crazy. I think the same constant value (numSecondsPerDay * 365) is a 
reasonable value for both checks, however, the name of the constant is still 
open to bikeshedding. 

 Gossip should ignore generation numbers too far in the future
 -

 Key: CASSANDRA-8113
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8113
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Low
Assignee: Jason Brown
 Attachments: 8113-v1.txt, 8113-v2.txt, 8113-v3.txt


 If a node sends corrupted gossip, it could set the generation numbers for 
 other nodes to arbitrarily large values. This is dangerous since one bad node 
 (e.g. with bad memory) could in theory bring down the cluster. Nodes should 
 refuse to accept generation numbers that are too far in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8113) Gossip should ignore generation numbers too far in the future


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171299#comment-14171299
 ] 

Brandon Williams commented on CASSANDRA-8113:
-

I'm more leery about the max version check since technically the version has no 
bounds, and possibly you could someday have something minor get out of whack 
like severity but it shouldn't effectively make the node be ignore.  Plus if 
you get a bit flipped in a version making it artificially high, simple 
restarting the node would fix it since it will have a superior generation after 
that.  What you can't do is easily fix the artificially generation as easily 
since you basically have to assassinate it at that point.  So, I would say drop 
the version check since there's some risk of a regression there and it's a 
little overly zealous since that's trivial to solve if it does happen. (and 
then rename max_version since it's not looking at versions)

 Gossip should ignore generation numbers too far in the future
 -

 Key: CASSANDRA-8113
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8113
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Low
Assignee: Jason Brown
 Attachments: 8113-v1.txt, 8113-v2.txt, 8113-v3.txt


 If a node sends corrupted gossip, it could set the generation numbers for 
 other nodes to arbitrarily large values. This is dangerous since one bad node 
 (e.g. with bad memory) could in theory bring down the cluster. Nodes should 
 refuse to accept generation numbers that are too far in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8113) Gossip should ignore generation numbers too far in the future


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171304#comment-14171304
 ] 

Brandon Williams commented on CASSANDRA-8113:
-

Also there's no real contract for how often you can update your gossip state if 
you want to, so we could have an accidental loop blow the version up and then 
run into further problems because now it's being ignored.

 Gossip should ignore generation numbers too far in the future
 -

 Key: CASSANDRA-8113
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8113
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Low
Assignee: Jason Brown
 Attachments: 8113-v1.txt, 8113-v2.txt, 8113-v3.txt


 If a node sends corrupted gossip, it could set the generation numbers for 
 other nodes to arbitrarily large values. This is dangerous since one bad node 
 (e.g. with bad memory) could in theory bring down the cluster. Nodes should 
 refuse to accept generation numbers that are too far in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/transport/CBUtil.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3a94b337
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3a94b337
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3a94b337

Branch: refs/heads/trunk
Commit: 3a94b3379ec9efb012d227af684c61ca19291b99
Parents: e57d027 781018c
Author: Tyler Hobbs tylerho...@apache.org
Authored: Tue Oct 14 13:22:42 2014 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Tue Oct 14 13:22:42 2014 -0500

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/marshal/AsciiType.java  | 27 ++-
 .../org/apache/cassandra/transport/CBUtil.java  | 51 +---
 3 files changed, 71 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3a94b337/CHANGES.txt
--
diff --cc CHANGES.txt
index 74647c8,432750a..47bea3a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,76 -1,5 +1,77 @@@
 -2.0.11:
 +2.1.1
 + * Fix high memory use due to tracking reads on incrementally opened sstable 
readers (CASSANDRA-8066)
 + * Fix EXECUTE request with skipMetadata=false returning no metadata
 +   (CASSANDRA-8054)
 + * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)
 + * Shutdown JVM on OOM (CASSANDRA-7507)
 + * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
 + * Don't duplicate sstables smaller than split size when using
 +   the sstablesplitter tool (CASSANDRA-7616)
 + * Avoid re-parsing already prepared statements (CASSANDRA-7923)
 + * Fix some Thrift slice deletions and updates of COMPACT STORAGE
 +   tables with some clustering columns omitted (CASSANDRA-7990)
 + * Fix filtering for CONTAINS on sets (CASSANDRA-8033)
 + * Properly track added size (CASSANDRA-7239)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Fix Assertion error on RangeTombstoneList diff (CASSANDRA-8013)
 + * Release references to overlapping sstables during compaction 
(CASSANDRA-7819)
 + * Send notification when opening compaction results early (CASSANDRA-8034)
 + * Make native server start block until properly bound (CASSANDRA-7885)
 + * (cqlsh) Fix IPv6 support (CASSANDRA-7988)
 + * Ignore fat clients when checking for endpoint collision (CASSANDRA-7939)
 + * Make sstablerepairedset take a list of files (CASSANDRA-7995)
 + * (cqlsh) Tab completeion for indexes on map keys (CASSANDRA-7972)
 + * (cqlsh) Fix UDT field selection in select clause (CASSANDRA-7891)
 + * Fix resource leak in event of corrupt sstable
 + * (cqlsh) Add command line option for cqlshrc file path (CASSANDRA-7131)
 + * Provide visibility into prepared statements churn (CASSANDRA-7921, 
CASSANDRA-7930)
 + * Invalidate prepared statements when their keyspace or table is
 +   dropped (CASSANDRA-7566)
 + * cassandra-stress: fix support for NetworkTopologyStrategy (CASSANDRA-7945)
 + * Fix saving caches when a table is dropped (CASSANDRA-7784)
 + * Add better error checking of new stress profile (CASSANDRA-7716)
 + * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom 
(CASSANDRA-7934)
 + * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
 + * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
 + * GCInspector more closely tracks GC; cassandra-stress and nodetool report 
it (CASSANDRA-7916)
 + * nodetool won't output bogus ownership info without a keyspace 
(CASSANDRA-7173)
 + * Add human readable option to nodetool commands (CASSANDRA-5433)
 + * Don't try to set repairedAt on old sstables (CASSANDRA-7913)
 + * Add metrics for tracking PreparedStatement use (CASSANDRA-7719)
 + * (cqlsh) tab-completion for triggers (CASSANDRA-7824)
 + * (cqlsh) Support for query paging (CASSANDRA-7514)
 + * (cqlsh) Show progress of COPY operations (CASSANDRA-7789)
 + * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
 + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
 + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
 + * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
 + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
 + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
 +   output (CASSANDRA-7659)
 + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
 + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
 + * Support list index operations with conditions (CASSANDRA-7499)
 + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
 + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
 + * (cqlsh) Error when

[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/transport/CBUtil.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3a94b337
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3a94b337
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3a94b337

Branch: refs/heads/cassandra-2.1
Commit: 3a94b3379ec9efb012d227af684c61ca19291b99
Parents: e57d027 781018c
Author: Tyler Hobbs tylerho...@apache.org
Authored: Tue Oct 14 13:22:42 2014 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Tue Oct 14 13:22:42 2014 -0500

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/marshal/AsciiType.java  | 27 ++-
 .../org/apache/cassandra/transport/CBUtil.java  | 51 +---
 3 files changed, 71 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3a94b337/CHANGES.txt
--
diff --cc CHANGES.txt
index 74647c8,432750a..47bea3a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,76 -1,5 +1,77 @@@
 -2.0.11:
 +2.1.1
 + * Fix high memory use due to tracking reads on incrementally opened sstable 
readers (CASSANDRA-8066)
 + * Fix EXECUTE request with skipMetadata=false returning no metadata
 +   (CASSANDRA-8054)
 + * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)
 + * Shutdown JVM on OOM (CASSANDRA-7507)
 + * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
 + * Don't duplicate sstables smaller than split size when using
 +   the sstablesplitter tool (CASSANDRA-7616)
 + * Avoid re-parsing already prepared statements (CASSANDRA-7923)
 + * Fix some Thrift slice deletions and updates of COMPACT STORAGE
 +   tables with some clustering columns omitted (CASSANDRA-7990)
 + * Fix filtering for CONTAINS on sets (CASSANDRA-8033)
 + * Properly track added size (CASSANDRA-7239)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Fix Assertion error on RangeTombstoneList diff (CASSANDRA-8013)
 + * Release references to overlapping sstables during compaction 
(CASSANDRA-7819)
 + * Send notification when opening compaction results early (CASSANDRA-8034)
 + * Make native server start block until properly bound (CASSANDRA-7885)
 + * (cqlsh) Fix IPv6 support (CASSANDRA-7988)
 + * Ignore fat clients when checking for endpoint collision (CASSANDRA-7939)
 + * Make sstablerepairedset take a list of files (CASSANDRA-7995)
 + * (cqlsh) Tab completeion for indexes on map keys (CASSANDRA-7972)
 + * (cqlsh) Fix UDT field selection in select clause (CASSANDRA-7891)
 + * Fix resource leak in event of corrupt sstable
 + * (cqlsh) Add command line option for cqlshrc file path (CASSANDRA-7131)
 + * Provide visibility into prepared statements churn (CASSANDRA-7921, 
CASSANDRA-7930)
 + * Invalidate prepared statements when their keyspace or table is
 +   dropped (CASSANDRA-7566)
 + * cassandra-stress: fix support for NetworkTopologyStrategy (CASSANDRA-7945)
 + * Fix saving caches when a table is dropped (CASSANDRA-7784)
 + * Add better error checking of new stress profile (CASSANDRA-7716)
 + * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom 
(CASSANDRA-7934)
 + * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
 + * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
 + * GCInspector more closely tracks GC; cassandra-stress and nodetool report 
it (CASSANDRA-7916)
 + * nodetool won't output bogus ownership info without a keyspace 
(CASSANDRA-7173)
 + * Add human readable option to nodetool commands (CASSANDRA-5433)
 + * Don't try to set repairedAt on old sstables (CASSANDRA-7913)
 + * Add metrics for tracking PreparedStatement use (CASSANDRA-7719)
 + * (cqlsh) tab-completion for triggers (CASSANDRA-7824)
 + * (cqlsh) Support for query paging (CASSANDRA-7514)
 + * (cqlsh) Show progress of COPY operations (CASSANDRA-7789)
 + * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
 + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
 + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
 + * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
 + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
 + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
 +   output (CASSANDRA-7659)
 + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
 + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
 + * Support list index operations with conditions (CASSANDRA-7499)
 + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
 + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
 + * (cqlsh)

[1/2] git commit: Validate ascii and utf8 string literals in cql queries

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 e57d027c9 - 3a94b3379


Validate ascii and utf8 string literals in cql queries

Patch by Tyler Hobbs; review by Aleksey Yeschenko for CASSANDRA-8101


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/781018cb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/781018cb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/781018cb

Branch: refs/heads/cassandra-2.1
Commit: 781018cb2b8df32291a2ead0d17a814074ce
Parents: 1d02d7b
Author: Tyler Hobbs tylerho...@apache.org
Authored: Tue Oct 14 12:56:23 2014 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Tue Oct 14 12:57:20 2014 -0500

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/marshal/AsciiType.java  | 27 ++-
 .../org/apache/cassandra/transport/CBUtil.java  | 51 +---
 3 files changed, 71 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/781018cb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8b462f1..432750a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * Properly validate ascii and utf8 string literals in CQL queries 
(CASSANDRA-8101)
  * (cqlsh) Fix autocompletion for alter keyspace (CASSANDRA-8021)
  * Create backup directories for commitlog archiving during startup 
(CASSANDRA-8111)
  * Reduce totalBlockFor() for LOCAL_* consistency levels (CASSANDRA-8058)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/781018cb/src/java/org/apache/cassandra/db/marshal/AsciiType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/AsciiType.java 
b/src/java/org/apache/cassandra/db/marshal/AsciiType.java
index 74ae6be..fdccfcd 100644
--- a/src/java/org/apache/cassandra/db/marshal/AsciiType.java
+++ b/src/java/org/apache/cassandra/db/marshal/AsciiType.java
@@ -18,8 +18,13 @@
 package org.apache.cassandra.db.marshal;
 
 import java.nio.ByteBuffer;
+import java.nio.CharBuffer;
+import java.nio.charset.Charset;
+import java.nio.charset.CharsetEncoder;
+import java.nio.charset.CharacterCodingException;
 
 import org.apache.cassandra.cql3.CQL3Type;
+import org.apache.cassandra.serializers.MarshalException;
 import org.apache.cassandra.serializers.TypeSerializer;
 import org.apache.cassandra.serializers.AsciiSerializer;
 
@@ -29,6 +34,15 @@ public class AsciiType extends AbstractTypeString
 
 AsciiType() {} // singleton
 
+private final ThreadLocalCharsetEncoder encoder = new 
ThreadLocalCharsetEncoder()
+{
+@Override
+protected CharsetEncoder initialValue()
+{
+return Charset.forName(US-ASCII).newEncoder();
+}
+};
+
 public int compare(ByteBuffer o1, ByteBuffer o2)
 {
 return BytesType.bytesCompare(o1, o2);
@@ -36,7 +50,18 @@ public class AsciiType extends AbstractTypeString
 
 public ByteBuffer fromString(String source)
 {
-return decompose(source);
+// the encoder must be reset each time it's used, hence the 
thread-local storage
+CharsetEncoder theEncoder = encoder.get();
+theEncoder.reset();
+
+try
+{
+return theEncoder.encode(CharBuffer.wrap(source));
+}
+catch (CharacterCodingException exc)
+{
+throw new MarshalException(String.format(Invalid ASCII character 
in string literal: %s, exc));
+}
 }
 
 public CQL3Type asCQL3Type()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/781018cb/src/java/org/apache/cassandra/transport/CBUtil.java
--
diff --git a/src/java/org/apache/cassandra/transport/CBUtil.java 
b/src/java/org/apache/cassandra/transport/CBUtil.java
index dfd0439..eb32faa 100644
--- a/src/java/org/apache/cassandra/transport/CBUtil.java
+++ b/src/java/org/apache/cassandra/transport/CBUtil.java
@@ -21,7 +21,11 @@ import java.net.InetAddress;
 import java.net.InetSocketAddress;
 import java.net.UnknownHostException;
 import java.nio.ByteBuffer;
+import java.nio.CharBuffer;
 import java.nio.charset.CharacterCodingException;
+import java.nio.charset.Charset;
+import java.nio.charset.CharsetDecoder;
+import java.nio.charset.CoderResult;
 import java.util.ArrayList;
 import java.util.Collections;
 import java.util.HashMap;
@@ -36,6 +40,7 @@ import org.jboss.netty.util.CharsetUtil;
 import org.apache.cassandra.db.ConsistencyLevel;
 import org.apache.cassandra.db.TypeSizes;
 import org.apache.cassandra.utils.UUIDGen;
+import org.apache.cassandra.utils.ByteBufferUtil;
 
 /**
  * ChannelBuffer utility methods.
@@ -48,21

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/caf8b705
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/caf8b705
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/caf8b705

Branch: refs/heads/trunk
Commit: caf8b70508fa95d0a47d3ba52830033512dfa033
Parents: 13c4719 3a94b33
Author: Tyler Hobbs tylerho...@apache.org
Authored: Tue Oct 14 13:23:25 2014 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Tue Oct 14 13:23:25 2014 -0500

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/marshal/AsciiType.java  | 27 ++-
 .../org/apache/cassandra/transport/CBUtil.java  | 51 +---
 3 files changed, 71 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/caf8b705/CHANGES.txt
--

[1/3] git commit: Validate ascii and utf8 string literals in cql queries

Repository: cassandra
Updated Branches:
  refs/heads/trunk 13c4719f7 - caf8b7050


Validate ascii and utf8 string literals in cql queries

Patch by Tyler Hobbs; review by Aleksey Yeschenko for CASSANDRA-8101


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/781018cb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/781018cb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/781018cb

Branch: refs/heads/trunk
Commit: 781018cb2b8df32291a2ead0d17a814074ce
Parents: 1d02d7b
Author: Tyler Hobbs tylerho...@apache.org
Authored: Tue Oct 14 12:56:23 2014 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Tue Oct 14 12:57:20 2014 -0500

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/marshal/AsciiType.java  | 27 ++-
 .../org/apache/cassandra/transport/CBUtil.java  | 51 +---
 3 files changed, 71 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/781018cb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8b462f1..432750a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * Properly validate ascii and utf8 string literals in CQL queries 
(CASSANDRA-8101)
  * (cqlsh) Fix autocompletion for alter keyspace (CASSANDRA-8021)
  * Create backup directories for commitlog archiving during startup 
(CASSANDRA-8111)
  * Reduce totalBlockFor() for LOCAL_* consistency levels (CASSANDRA-8058)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/781018cb/src/java/org/apache/cassandra/db/marshal/AsciiType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/AsciiType.java 
b/src/java/org/apache/cassandra/db/marshal/AsciiType.java
index 74ae6be..fdccfcd 100644
--- a/src/java/org/apache/cassandra/db/marshal/AsciiType.java
+++ b/src/java/org/apache/cassandra/db/marshal/AsciiType.java
@@ -18,8 +18,13 @@
 package org.apache.cassandra.db.marshal;
 
 import java.nio.ByteBuffer;
+import java.nio.CharBuffer;
+import java.nio.charset.Charset;
+import java.nio.charset.CharsetEncoder;
+import java.nio.charset.CharacterCodingException;
 
 import org.apache.cassandra.cql3.CQL3Type;
+import org.apache.cassandra.serializers.MarshalException;
 import org.apache.cassandra.serializers.TypeSerializer;
 import org.apache.cassandra.serializers.AsciiSerializer;
 
@@ -29,6 +34,15 @@ public class AsciiType extends AbstractTypeString
 
 AsciiType() {} // singleton
 
+private final ThreadLocalCharsetEncoder encoder = new 
ThreadLocalCharsetEncoder()
+{
+@Override
+protected CharsetEncoder initialValue()
+{
+return Charset.forName(US-ASCII).newEncoder();
+}
+};
+
 public int compare(ByteBuffer o1, ByteBuffer o2)
 {
 return BytesType.bytesCompare(o1, o2);
@@ -36,7 +50,18 @@ public class AsciiType extends AbstractTypeString
 
 public ByteBuffer fromString(String source)
 {
-return decompose(source);
+// the encoder must be reset each time it's used, hence the 
thread-local storage
+CharsetEncoder theEncoder = encoder.get();
+theEncoder.reset();
+
+try
+{
+return theEncoder.encode(CharBuffer.wrap(source));
+}
+catch (CharacterCodingException exc)
+{
+throw new MarshalException(String.format(Invalid ASCII character 
in string literal: %s, exc));
+}
 }
 
 public CQL3Type asCQL3Type()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/781018cb/src/java/org/apache/cassandra/transport/CBUtil.java
--
diff --git a/src/java/org/apache/cassandra/transport/CBUtil.java 
b/src/java/org/apache/cassandra/transport/CBUtil.java
index dfd0439..eb32faa 100644
--- a/src/java/org/apache/cassandra/transport/CBUtil.java
+++ b/src/java/org/apache/cassandra/transport/CBUtil.java
@@ -21,7 +21,11 @@ import java.net.InetAddress;
 import java.net.InetSocketAddress;
 import java.net.UnknownHostException;
 import java.nio.ByteBuffer;
+import java.nio.CharBuffer;
 import java.nio.charset.CharacterCodingException;
+import java.nio.charset.Charset;
+import java.nio.charset.CharsetDecoder;
+import java.nio.charset.CoderResult;
 import java.util.ArrayList;
 import java.util.Collections;
 import java.util.HashMap;
@@ -36,6 +40,7 @@ import org.jboss.netty.util.CharsetUtil;
 import org.apache.cassandra.db.ConsistencyLevel;
 import org.apache.cassandra.db.TypeSizes;
 import org.apache.cassandra.utils.UUIDGen;
+import org.apache.cassandra.utils.ByteBufferUtil;
 
 /**
  * ChannelBuffer utility methods.
@@ -48,21 +53,30 @@ public

[jira] [Created] (CASSANDRA-8118) Exceptions while decoding native protocol messages are not handled correctly

Tyler Hobbs created CASSANDRA-8118:
--

 Summary: Exceptions while decoding native protocol messages are 
not handled correctly
 Key: CASSANDRA-8118
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8118
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
 Fix For: 2.1.1
 Attachments: repro.py

If there's an exception during decoding of a native protocol message (such as 
the check for non-UTF-8 chars in CASSANDRA-8101), it gets wrapped by Netty and 
may not be handled correctly.  Additionally, the stream ID is not set for the 
error response, which makes it nearly impossible for a driver to handle 
correctly.

For an example, run the attached repro.py with the latest cassandra-2.1.  
You'll see a stacktrace like this:

{noformat}
ERROR 18:36:25 Unexpected exception during request; channel = [id: 0xf7754f26, 
/127.0.0.1:56789 = /127.0.0.1:9042]
io.netty.handler.codec.DecoderException: 
org.apache.cassandra.transport.messages.ErrorMessage$WrappedException: 
org.apache.cassandra.transport.ProtocolException: Cannot decode string as UTF8: 
'c2012729'; java.nio.charset.MalformedInputException: Input length = 1
at 
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:99)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:163)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.epollInReady(EpollSocketChannel.java:722)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:326) 
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:264) 
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
Caused by: 
org.apache.cassandra.transport.messages.ErrorMessage$WrappedException: 
org.apache.cassandra.transport.ProtocolException: Cannot decode string as UTF8: 
'c2012729'; java.nio.charset.MalformedInputException: Input length = 1
at 
org.apache.cassandra.transport.messages.ErrorMessage.wrap(ErrorMessage.java:243)
 ~[main/:na]
at 
org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:273) 
~[main/:na]
at 
org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:235) 
~[main/:na]
at 
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
... 15 common frames omitted
Caused by: org.apache.cassandra.transport.ProtocolException: Cannot decode 
string as UTF8: 'c2012729'; java.nio.charset.MalformedInputException: Input 
length = 1
at org.apache.cassandra.transport.CBUtil.readString(CBUtil.java:81) 
~[main/:na]
at 
org.apache.cassandra.transport.CBUtil.readLongString(CBUtil.java:138) 
~[main/:na]
at 
org.apache.cassandra.transport.messages.QueryMessage$1.decode(QueryMessage.java:45)
 ~[main/:na]
at 
org.apache.cassandra.transport.messages.QueryMessage$1.decode(QueryMessage.java:42)
 ~[main/:na]
at

[jira] [Commented] (CASSANDRA-7280) Hadoop support not respecting cassandra.input.split.size

2014-10-14 Thread Alex Liu (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171391#comment-14171391
]

Alex Liu commented on CASSANDRA-7280:
-

cassandra.input.split.size is used to partition rows by partitioning key. It
doesn't affect native paging. Native internal paging has a page size which can
be set by cassandra.input.page.row.size

Hadoop support not respecting cassandra.input.split.size

Key: CASSANDRA-7280
URL: https://issues.apache.org/jira/browse/CASSANDRA-7280
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Jeremy Hanna

Long ago (0.7), I tried to set the cassandra.input.split.size property and
never really got it to respect that property. However the batch size was
useful for what I needed to affect the timeouts.
Now with the cql record reader and the native paging, users can specify
queries potentially using allow filtering clauses. The input split size is
more important because the server may have to scan through many many records
to get matching records. If the user can effectively set the input split
size, then that gives a hard limit on how many records it will traverse.
Currently it appears to be overriding the property, perhaps in the
client.describe_splits_ex method on the server side.
It can be argued that users shouldn't be using allow filtering clauses in
their cql in the first place. However it is still a bug that the input split
size is not honored.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8104) frozen broken / frozen collections in frozen tuple type


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8104:
---
Reviewer: Tyler Hobbs

 frozen broken / frozen collections in frozen tuple type
 ---

 Key: CASSANDRA-8104
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8104
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
 Attachments: frozen-broken-patch.txt


 {{CREATE TABLE foo (key int primary key, tup frozentupledouble, 
 listdouble, settext, mapint, boolean)}}
 Produces an NPE in {{CQL3Type.freeze()}}, because lists and sets have no keys.
 {{CREATE TABLE bar (key int primary key, tup frozentupledouble, 
 frozenlistdouble, frozensettext, frozenmapint, boolean)}}
 Produces some NPEs that prevent the correct error message being printed.
 Simple patch attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-8108) ClassCastException in AbstractCellNameType


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs reassigned CASSANDRA-8108:
--

Assignee: Tyler Hobbs

 ClassCastException in AbstractCellNameType
 --

 Key: CASSANDRA-8108
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8108
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra 2.1.0
 Datastax AMI on EC2 (Ubuntu Linux)
Reporter: David Hearnden
Assignee: Tyler Hobbs

 java.lang.ClassCastException: 
 org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast 
 to org.apache.cassandra.db.composites.CellName
   at 
 org.apache.cassandra.db.composites.AbstractCellNameType.cellFromByteBuffer(AbstractCellNameType.java:170)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.SliceQueryPager.init(SliceQueryPager.java:57)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.MultiPartitionPager.makePager(MultiPartitionPager.java:84)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.MultiPartitionPager.init(MultiPartitionPager.java:68)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:101) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:125) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:215)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:60)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:187)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:413)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:133)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:422)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:318)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at 
 io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:103)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:31)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:323)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_51]
   at 
 org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) 
 [apache-cassandra-2.1.0.jar:2.1.0]
   at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8115) Windows install scripts fail to set logdir and datadir

2014-10-14 Thread Joshua McKenzie (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-8115:
---
Attachment: 8115_v1.txt

Passing in multiple semicolon delimited options to ++JvmOptions in procrunsrv 
is supposed to work as documented 
[here|http://commons.apache.org/proper/commons-daemon/procrun.html], however 
that is failing and is causing the errors that led to the creation of this 
ticket.

I've attached a v1 that tokenizes JAVA_OPTS and CASSANDRA_PARAMS and appends a 
++JvmOptions=param for each entry. Tested installation and running the 
service and it works on both legacy and powershell paths, run from either the 
command prompt or powershell itself, from within a directory that had a space 
in the name.

I also moved the logdir and storagedir declarations into the -env file and 
cleaned up the quoting in the .bat file for the log and storage dir.

 Windows install scripts fail to set logdir and datadir
 --

 Key: CASSANDRA-8115
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8115
 Project: Cassandra
  Issue Type: Bug
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
  Labels: Windows
 Fix For: 2.1.1

 Attachments: 8115_v1.txt


 After CASSANDRA-7136, the install scripts to run Cassandra as a service fail 
 on both the legacy and the powershell paths.  Looks like they need to have
 {code}
 ++JvmOptions=-Dcassandra.logdir=%CASSANDRA_HOME%\logs ^
 ++JvmOptions=-Dcassandra.storagedir=%CASSANDRA_HOME%\data
 {code}
 added to function correctly.
 We should take this opportunity to make sure the source of the java options 
 is uniform for both running and installation to prevent mismatches like this 
 in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-8118) Exceptions while decoding native protocol messages are not handled correctly


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs reassigned CASSANDRA-8118:
--

Assignee: Tyler Hobbs

 Exceptions while decoding native protocol messages are not handled correctly
 

 Key: CASSANDRA-8118
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8118
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
 Fix For: 2.1.1

 Attachments: repro.py


 If there's an exception during decoding of a native protocol message (such as 
 the check for non-UTF-8 chars in CASSANDRA-8101), it gets wrapped by Netty 
 and may not be handled correctly.  Additionally, the stream ID is not set for 
 the error response, which makes it nearly impossible for a driver to handle 
 correctly.
 For an example, run the attached repro.py with the latest cassandra-2.1.  
 You'll see a stacktrace like this:
 {noformat}
 ERROR 18:36:25 Unexpected exception during request; channel = [id: 
 0xf7754f26, /127.0.0.1:56789 = /127.0.0.1:9042]
 io.netty.handler.codec.DecoderException: 
 org.apache.cassandra.transport.messages.ErrorMessage$WrappedException: 
 org.apache.cassandra.transport.ProtocolException: Cannot decode string as 
 UTF8: 'c2012729'; java.nio.charset.MalformedInputException: Input length = 1
   at 
 io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:99)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:163)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.epollInReady(EpollSocketChannel.java:722)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:326) 
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:264) 
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
 Caused by: 
 org.apache.cassandra.transport.messages.ErrorMessage$WrappedException: 
 org.apache.cassandra.transport.ProtocolException: Cannot decode string as 
 UTF8: 'c2012729'; java.nio.charset.MalformedInputException: Input length = 1
   at 
 org.apache.cassandra.transport.messages.ErrorMessage.wrap(ErrorMessage.java:243)
  ~[main/:na]
   at 
 org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:273)
  ~[main/:na]
   at 
 org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:235)
  ~[main/:na]
   at 
 io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   ... 15 common frames omitted
 Caused by: org.apache.cassandra.transport.ProtocolException: Cannot decode 
 string as UTF8: 'c2012729'; java.nio.charset.MalformedInputException: Input 
 length = 1
   at org.apache.cassandra.transport.CBUtil.readString(CBUtil.java:81) 
 ~[main/:na]
   at 
 org.apache.cassandra.transport.CBUtil.readLongString(CBUtil.java:138)

[jira] [Updated] (CASSANDRA-8118) Exceptions while decoding native protocol messages are not handled correctly


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8118:
---
Attachment: 8118.txt

The attached patch extracts the underlying exception when suitable and sets the 
stream ID for the response.  The dtest for CASSANDRA-8101 triggers this in 2.1, 
so I didn't add an extra test for this.  (I'll push that test once this is 
committed.)

 Exceptions while decoding native protocol messages are not handled correctly
 

 Key: CASSANDRA-8118
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8118
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
 Fix For: 2.1.1

 Attachments: 8118.txt, repro.py


 If there's an exception during decoding of a native protocol message (such as 
 the check for non-UTF-8 chars in CASSANDRA-8101), it gets wrapped by Netty 
 and may not be handled correctly.  Additionally, the stream ID is not set for 
 the error response, which makes it nearly impossible for a driver to handle 
 correctly.
 For an example, run the attached repro.py with the latest cassandra-2.1.  
 You'll see a stacktrace like this:
 {noformat}
 ERROR 18:36:25 Unexpected exception during request; channel = [id: 
 0xf7754f26, /127.0.0.1:56789 = /127.0.0.1:9042]
 io.netty.handler.codec.DecoderException: 
 org.apache.cassandra.transport.messages.ErrorMessage$WrappedException: 
 org.apache.cassandra.transport.ProtocolException: Cannot decode string as 
 UTF8: 'c2012729'; java.nio.charset.MalformedInputException: Input length = 1
   at 
 io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:99)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:163)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.epollInReady(EpollSocketChannel.java:722)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:326) 
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:264) 
 ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
 Caused by: 
 org.apache.cassandra.transport.messages.ErrorMessage$WrappedException: 
 org.apache.cassandra.transport.ProtocolException: Cannot decode string as 
 UTF8: 'c2012729'; java.nio.charset.MalformedInputException: Input length = 1
   at 
 org.apache.cassandra.transport.messages.ErrorMessage.wrap(ErrorMessage.java:243)
  ~[main/:na]
   at 
 org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:273)
  ~[main/:na]
   at 
 org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:235)
  ~[main/:na]
   at 
 io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
   ... 15 common frames omitted
 Caused by: org.apache.cassandra.transport.ProtocolException: Cannot decode 
 string as UTF8:

[jira] [Updated] (CASSANDRA-8108) ClassCastException in AbstractCellNameType


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8108:
---
Description: 
{noformat}
java.lang.ClassCastException: 
org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast to 
org.apache.cassandra.db.composites.CellName
at 
org.apache.cassandra.db.composites.AbstractCellNameType.cellFromByteBuffer(AbstractCellNameType.java:170)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.service.pager.SliceQueryPager.init(SliceQueryPager.java:57)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.service.pager.MultiPartitionPager.makePager(MultiPartitionPager.java:84)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.service.pager.MultiPartitionPager.init(MultiPartitionPager.java:68)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:101) 
~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:125) 
~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:215)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:60)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:187)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:413)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:133)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:422)
 [apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:318)
 [apache-cassandra-2.1.0.jar:2.1.0]
at 
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:103)
 [netty-all-4.0.20.Final.jar:4.0.20.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
 [netty-all-4.0.20.Final.jar:4.0.20.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:31)
 [netty-all-4.0.20.Final.jar:4.0.20.Final]
at 
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:323)
 [netty-all-4.0.20.Final.jar:4.0.20.Final]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
[na:1.7.0_51]
at 
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163)
 [apache-cassandra-2.1.0.jar:2.1.0]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) 
[apache-cassandra-2.1.0.jar:2.1.0]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
{noformat}

  was:
java.lang.ClassCastException: 
org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast to 
org.apache.cassandra.db.composites.CellName
at 
org.apache.cassandra.db.composites.AbstractCellNameType.cellFromByteBuffer(AbstractCellNameType.java:170)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.service.pager.SliceQueryPager.init(SliceQueryPager.java:57)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.service.pager.MultiPartitionPager.makePager(MultiPartitionPager.java:84)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.service.pager.MultiPartitionPager.init(MultiPartitionPager.java:68)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:101) 
~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:125) 
~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:215)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:60)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:187)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:413)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:133)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:422)
 [apache-cassandra-2.1.0.jar:2.1.0]
at

[jira] [Commented] (CASSANDRA-4762) Support IN clause for any clustering column


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171446#comment-14171446
 ] 

Tyler Hobbs commented on CASSANDRA-4762:


bq. So does 3.0 drop this back another 6 months?

Yes.  I cannot make any guarantees about deadlines, but we should at least have 
a beta or release candidate available in 6 months.

 Support IN clause for any clustering column
 ---

 Key: CASSANDRA-4762
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4762
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: T Jake Luciani
Assignee: Benjamin Lerer
  Labels: cql, docs
 Fix For: 3.0

 Attachments: 4762-1.txt


 Given CASSANDRA-3885
 It seems it should be possible to store multiple ranges for many predicates 
 even the inner parts of a composite column.
 They could be expressed as a expanded set of filter queries.
 example:
 {code}
 CREATE TABLE test (
name text,
tdate timestamp,
tdate2 timestamp,
tdate3 timestamp,
num double,
PRIMARY KEY(name,tdate,tdate2,tdate3)
  ) WITH COMPACT STORAGE;
 SELECT * FROM test WHERE 
   name IN ('a','b') and
   tdate IN ('2010-01-01','2011-01-01') and
   tdate2 IN ('2010-01-01','2011-01-01') and
   tdate3 IN ('2010-01-01','2011-01-01') 
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8104) frozen broken / frozen collections in frozen tuple type


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8104:
---
Fix Version/s: 2.1.1

 frozen broken / frozen collections in frozen tuple type
 ---

 Key: CASSANDRA-8104
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8104
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
 Fix For: 2.1.1

 Attachments: frozen-broken-patch.txt


 {{CREATE TABLE foo (key int primary key, tup frozentupledouble, 
 listdouble, settext, mapint, boolean)}}
 Produces an NPE in {{CQL3Type.freeze()}}, because lists and sets have no keys.
 {{CREATE TABLE bar (key int primary key, tup frozentupledouble, 
 frozenlistdouble, frozensettext, frozenmapint, boolean)}}
 Produces some NPEs that prevent the correct error message being printed.
 Simple patch attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: bump base version to 3.0

Repository: cassandra
Updated Branches:
  refs/heads/trunk caf8b7050 - 3ef892001


bump base version to 3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3ef89200
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3ef89200
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3ef89200

Branch: refs/heads/trunk
Commit: 3ef892001425f0ec67aa9f232bec5ecd29520e12
Parents: caf8b70
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Oct 14 15:19:37 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Oct 14 15:19:37 2014 -0500

--
 build.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3ef89200/build.xml
--
diff --git a/build.xml b/build.xml
index b758e0a..07d0578 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information --
-property name=base.version value=2.1.1/
+property name=base.version value=3.0.0/
 property name=scm.connection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.developerConnection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.url 
value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/

[jira] [Updated] (CASSANDRA-8096) Make cache serializers pluggable


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-8096:
--
Reviewer: Aleksey Yeschenko
Priority: Minor  (was: Major)

 Make cache serializers pluggable
 

 Key: CASSANDRA-8096
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8096
 Project: Cassandra
  Issue Type: Improvement
Reporter: Blake Eggleston
Assignee: Blake Eggleston
Priority: Minor
 Fix For: 2.1.1

 Attachments: CASSANDRA-8096.patch


 Make cache serializers configurable via system properties.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Avoid NPE when creating table w/ frozen sets, lists

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 3a94b3379 - 932b5eb46


Avoid NPE when creating table w/ frozen sets, lists

Patch by Robert Stupp; reviewed by Tyler Hobbs for CASSANDRA-8104


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/932b5eb4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/932b5eb4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/932b5eb4

Branch: refs/heads/cassandra-2.1
Commit: 932b5eb46d413af94add6ac199d4ef043234e55e
Parents: 3a94b33
Author: Robert Stupp sn...@snazy.de
Authored: Tue Oct 14 15:28:38 2014 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Tue Oct 14 15:28:38 2014 -0500

--
 CHANGES.txt  | 4 +++-
 src/java/org/apache/cassandra/cql3/CQL3Type.java | 8 +---
 2 files changed, 8 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/932b5eb4/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 47bea3a..1c7ee38 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,7 @@
 2.1.1
- * Fix high memory use due to tracking reads on incrementally opened sstable 
readers (CASSANDRA-8066)
+ * Fix NPE when creating a table with frozen sets, lists (CASSANDRA-8104)
+ * Fix high memory use due to tracking reads on incrementally opened sstable
+   readers (CASSANDRA-8066)
  * Fix EXECUTE request with skipMetadata=false returning no metadata
(CASSANDRA-8054)
  * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/932b5eb4/src/java/org/apache/cassandra/cql3/CQL3Type.java
--
diff --git a/src/java/org/apache/cassandra/cql3/CQL3Type.java 
b/src/java/org/apache/cassandra/cql3/CQL3Type.java
index 1589d6a..6d55285 100644
--- a/src/java/org/apache/cassandra/cql3/CQL3Type.java
+++ b/src/java/org/apache/cassandra/cql3/CQL3Type.java
@@ -347,7 +347,7 @@ public interface CQL3Type
 public static Raw tuple(ListCQL3Type.Raw ts) throws 
InvalidRequestException
 {
 for (int i = 0; i  ts.size(); i++)
-if (ts.get(i).isCounter())
+if (ts.get(i) != null  ts.get(i).isCounter())
 throw new InvalidRequestException(counters are not 
allowed inside tuples);
 
 return new RawTuple(ts);
@@ -404,7 +404,8 @@ public interface CQL3Type
 
 public Raw freeze()
 {
-keys.freeze();
+if (keys != null)
+keys.freeze();
 values.freeze();
 return super.freeze();
 }
@@ -500,7 +501,8 @@ public interface CQL3Type
 public Raw freeze()
 {
 for (CQL3Type.Raw t : types)
-t.freeze();
+if (t != null)
+t.freeze();
 return super.freeze();
 }

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3a822ca2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3a822ca2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3a822ca2

Branch: refs/heads/trunk
Commit: 3a822ca230e8bb37d9a0bd5b82b546af2ed68b1b
Parents: 3ef8920 932b5eb
Author: Tyler Hobbs tylerho...@apache.org
Authored: Tue Oct 14 15:29:49 2014 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Tue Oct 14 15:29:49 2014 -0500

--
 CHANGES.txt  | 4 +++-
 src/java/org/apache/cassandra/cql3/CQL3Type.java | 8 +---
 2 files changed, 8 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3a822ca2/CHANGES.txt
--
diff --cc CHANGES.txt
index 435562b,1c7ee38..5dac7f4
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,35 -1,7 +1,37 @@@
 +3.0
 + * Keep sstable levels when bootstrapping (CASSANDRA-7460)
 + * Add Sigar library and perform basic OS settings check on startup 
(CASSANDRA-7838)
 + * Support for aggregation functions (CASSANDRA-4914)
 + * Remove cassandra-cli (CASSANDRA-7920)
 + * Accept dollar quoted strings in CQL (CASSANDRA-7769)
 + * Make assassinate a first class command (CASSANDRA-7935)
 + * Support IN clause on any clustering column (CASSANDRA-4762)
 + * Improve compaction logging (CASSANDRA-7818)
 + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917)
 + * Do anticompaction in groups (CASSANDRA-6851)
 + * Support pure user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 
7781, 7929,
 +   7924, 7812, 8063)
 + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416)
 + * Move sstable RandomAccessReader to nio2, which allows using the
 +   FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050)
 + * Remove CQL2 (CASSANDRA-5918)
 + * Add Thrift get_multi_slice call (CASSANDRA-6757)
 + * Optimize fetching multiple cells by name (CASSANDRA-6933)
 + * Allow compilation in java 8 (CASSANDRA-7028)
 + * Make incremental repair default (CASSANDRA-7250)
 + * Enable code coverage thru JaCoCo (CASSANDRA-7226)
 + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) 
 + * Shorten SSTable path (CASSANDRA-6962)
 + * Use unsafe mutations for most unit tests (CASSANDRA-6969)
 + * Fix race condition during calculation of pending ranges (CASSANDRA-7390)
 + * Fail on very large batch sizes (CASSANDRA-8011)
 + * improve concurrency of repair (CASSANDRA-6455)
 +
 +
  2.1.1
-  * Fix high memory use due to tracking reads on incrementally opened sstable 
readers (CASSANDRA-8066)
+  * Fix NPE when creating a table with frozen sets, lists (CASSANDRA-8104)
+  * Fix high memory use due to tracking reads on incrementally opened sstable
+readers (CASSANDRA-8066)
   * Fix EXECUTE request with skipMetadata=false returning no metadata
 (CASSANDRA-8054)
   * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)

[1/2] git commit: Avoid NPE when creating table w/ frozen sets, lists

Repository: cassandra
Updated Branches:
  refs/heads/trunk 3ef892001 - 3a822ca23


Avoid NPE when creating table w/ frozen sets, lists

Patch by Robert Stupp; reviewed by Tyler Hobbs for CASSANDRA-8104


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/932b5eb4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/932b5eb4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/932b5eb4

Branch: refs/heads/trunk
Commit: 932b5eb46d413af94add6ac199d4ef043234e55e
Parents: 3a94b33
Author: Robert Stupp sn...@snazy.de
Authored: Tue Oct 14 15:28:38 2014 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Tue Oct 14 15:28:38 2014 -0500

--
 CHANGES.txt  | 4 +++-
 src/java/org/apache/cassandra/cql3/CQL3Type.java | 8 +---
 2 files changed, 8 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/932b5eb4/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 47bea3a..1c7ee38 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,7 @@
 2.1.1
- * Fix high memory use due to tracking reads on incrementally opened sstable 
readers (CASSANDRA-8066)
+ * Fix NPE when creating a table with frozen sets, lists (CASSANDRA-8104)
+ * Fix high memory use due to tracking reads on incrementally opened sstable
+   readers (CASSANDRA-8066)
  * Fix EXECUTE request with skipMetadata=false returning no metadata
(CASSANDRA-8054)
  * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/932b5eb4/src/java/org/apache/cassandra/cql3/CQL3Type.java
--
diff --git a/src/java/org/apache/cassandra/cql3/CQL3Type.java 
b/src/java/org/apache/cassandra/cql3/CQL3Type.java
index 1589d6a..6d55285 100644
--- a/src/java/org/apache/cassandra/cql3/CQL3Type.java
+++ b/src/java/org/apache/cassandra/cql3/CQL3Type.java
@@ -347,7 +347,7 @@ public interface CQL3Type
 public static Raw tuple(ListCQL3Type.Raw ts) throws 
InvalidRequestException
 {
 for (int i = 0; i  ts.size(); i++)
-if (ts.get(i).isCounter())
+if (ts.get(i) != null  ts.get(i).isCounter())
 throw new InvalidRequestException(counters are not 
allowed inside tuples);
 
 return new RawTuple(ts);
@@ -404,7 +404,8 @@ public interface CQL3Type
 
 public Raw freeze()
 {
-keys.freeze();
+if (keys != null)
+keys.freeze();
 values.freeze();
 return super.freeze();
 }
@@ -500,7 +501,8 @@ public interface CQL3Type
 public Raw freeze()
 {
 for (CQL3Type.Raw t : types)
-t.freeze();
+if (t != null)
+t.freeze();
 return super.freeze();
 }

[jira] [Commented] (CASSANDRA-8091) Stress tool creates too large batches


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171471#comment-14171471
 ] 

Jonathan Ellis commented on CASSANDRA-8091:
---

Default should be no batching at all unless you're trying to build large 
partitions...

 Stress tool creates too large batches
 -

 Key: CASSANDRA-8091
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8091
 Project: Cassandra
  Issue Type: Bug
Reporter: Carl Yeksigian
  Labels: stress

 With CASSANDRA-8011, the stress tool now gets exceptions because its batches 
 are too large. We should change the default behaviour to not create batches 
 so large, probably using individual inserts instead of an unlogged batch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8099) Refactor and modernize the storage engine


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171474#comment-14171474
 ] 

Jonathan Ellis commented on CASSANDRA-8099:
---

/cc [~jasobrown] [~xedin] [~iamaleksey] [~krummas] [~kohlisankalp] [~tjake]

 Refactor and modernize the storage engine
 -

 Key: CASSANDRA-8099
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8099
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 3.0


 The current storage engine (which for this ticket I'll loosely define as the 
 code implementing the read/write path) is suffering from old age. One of the 
 main problem is that the only structure it deals with is the cell, which 
 completely ignores the more high level CQL structure that groups cell into 
 (CQL) rows.
 This leads to many inefficiencies, like the fact that during a reads we have 
 to group cells multiple times (to count on replica, then to count on the 
 coordinator, then to produce the CQL resultset) because we forget about the 
 grouping right away each time (so lots of useless cell names comparisons in 
 particular). But outside inefficiencies, having to manually recreate the CQL 
 structure every time we need it for something is hindering new features and 
 makes the code more complex that it should be.
 Said storage engine also has tons of technical debt. To pick an example, the 
 fact that during range queries we update {{SliceQueryFilter.count}} is pretty 
 hacky and error prone. Or the overly complex ways {{AbstractQueryPager}} has 
 to go into to simply remove the last query result.
 So I want to bite the bullet and modernize this storage engine. I propose to 
 do 2 main things:
 # Make the storage engine more aware of the CQL structure. In practice, 
 instead of having partitions be a simple iterable map of cells, it should be 
 an iterable list of row (each being itself composed of per-column cells, 
 though obviously not exactly the same kind of cell we have today).
 # Make the engine more iterative. What I mean here is that in the read path, 
 we end up reading all cells in memory (we put them in a ColumnFamily object), 
 but there is really no reason to. If instead we were working with iterators 
 all the way through, we could get to a point where we're basically 
 transferring data from disk to the network, and we should be able to reduce 
 GC substantially.
 Please note that such refactor should provide some performance improvements 
 right off the bat but it's not it's primary goal either. It's primary goal is 
 to simplify the storage engine and adds abstraction that are better suited to 
 further optimizations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-5483) Repair tracing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171479#comment-14171479
 ] 

Jonathan Ellis commented on CASSANDRA-5483:
---

That should be it!

 Repair tracing
 --

 Key: CASSANDRA-5483
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5483
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Yuki Morishita
Assignee: Ben Chan
Priority: Minor
  Labels: repair
 Fix For: 3.0

 Attachments: 5483-full-trunk.txt, 
 5483-v06-04-Allow-tracing-ttl-to-be-configured.patch, 
 5483-v06-05-Add-a-command-column-to-system_traces.events.patch, 
 5483-v06-06-Fix-interruption-in-tracestate-propagation.patch, 
 5483-v07-07-Better-constructor-parameters-for-DebuggableThreadPoolExecutor.patch,
  5483-v07-08-Fix-brace-style.patch, 
 5483-v07-09-Add-trace-option-to-a-more-complete-set-of-repair-functions.patch,
  5483-v07-10-Correct-name-of-boolean-repairedAt-to-fullRepair.patch, 
 5483-v08-11-Shorten-trace-messages.-Use-Tracing-begin.patch, 
 5483-v08-12-Trace-streaming-in-Differencer-StreamingRepairTask.patch, 
 5483-v08-13-sendNotification-of-local-traces-back-to-nodetool.patch, 
 5483-v08-14-Poll-system_traces.events.patch, 
 5483-v08-15-Limit-trace-notifications.-Add-exponential-backoff.patch, 
 5483-v09-16-Fix-hang-caused-by-incorrect-exit-code.patch, 
 5483-v10-17-minor-bugfixes-and-changes.patch, 
 5483-v10-rebased-and-squashed-471f5cc.patch, 5483-v11-01-squashed.patch, 
 5483-v11-squashed-nits.patch, 5483-v12-02-cassandra-yaml-ttl-doc.patch, 
 5483-v13-608fb03-May-14-trace-formatting-changes.patch, 
 5483-v14-01-squashed.patch, 
 5483-v15-02-Hook-up-exponential-backoff-functionality.patch, 
 5483-v15-03-Exact-doubling-for-exponential-backoff.patch, 
 5483-v15-04-Re-add-old-StorageService-JMX-signatures.patch, 
 5483-v15-05-Move-command-column-to-system_traces.sessions.patch, 
 5483-v15.patch, ccm-repair-test, cqlsh-left-justify-text-columns.patch, 
 prerepair-vs-postbuggedrepair.diff, test-5483-system_traces-events.txt, 
 trunk@4620823-5483-v02-0001-Trace-filtering-and-tracestate-propagation.patch, 
 trunk@4620823-5483-v02-0002-Put-a-few-traces-parallel-to-the-repair-logging.patch,
  tr...@8ebeee1-5483-v01-001-trace-filtering-and-tracestate-propagation.txt, 
 tr...@8ebeee1-5483-v01-002-simple-repair-tracing.txt, 
 v02p02-5483-v03-0003-Make-repair-tracing-controllable-via-nodetool.patch, 
 v02p02-5483-v04-0003-This-time-use-an-EnumSet-to-pass-boolean-repair-options.patch,
  v02p02-5483-v05-0003-Use-long-instead-of-EnumSet-to-work-with-JMX.patch


 I think it would be nice to log repair stats and results like query tracing 
 stores traces to system keyspace. With it, you don't have to lookup each log 
 file to see what was the status and how it performed the repair you invoked. 
 Instead, you can query the repair log with session ID to see the state and 
 stats of all nodes involved in that repair session.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8094) Heavy writes in RangeSlice read requests


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171485#comment-14171485
 ] 

Jonathan Ellis commented on CASSANDRA-8094:
---

Yes.

 Heavy writes in RangeSlice read  requests 
 --

 Key: CASSANDRA-8094
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8094
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Minh Do
Assignee: Minh Do
 Fix For: 2.0.11


 RangeSlice requests always do a scheduled read repair when coordinators try 
 to resolve replicas' responses no matter read_repair_chance is set or not.
 Because of this, in low writes and high reads clusters, there are very high 
 write requests going on between nodes.  
 We should have an option to turn this off and this can be different than the 
 read_repair_chance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/6] git commit: Add the ability to cleanup files after CqlBulkRecordWriter succeeds

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 781018cb2 - 6c0ee30ea
  refs/heads/cassandra-2.1 932b5eb46 - ae9c06692
  refs/heads/trunk 3a822ca23 - 6630b8786


Add the ability to cleanup files after CqlBulkRecordWriter succeeds

Patch by Paul Pak, reviewed by Piotr KoÅaczkowski for CASSANDRA-


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6c0ee30e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6c0ee30e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6c0ee30e

Branch: refs/heads/cassandra-2.0
Commit: 6c0ee30ea7c57066aa15c25a40a217ca83aff248
Parents: 781018c
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Oct 14 15:42:43 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Oct 14 15:42:43 2014 -0500

--
 .../cassandra/hadoop/cql3/CqlBulkOutputFormat.java | 11 +++
 .../cassandra/hadoop/cql3/CqlBulkRecordWriter.java | 13 -
 2 files changed, 23 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c0ee30e/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java
index 58e05b6..887fe8e 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java
@@ -53,6 +53,7 @@ public class CqlBulkOutputFormat extends 
AbstractBulkOutputFormatObject, ListB
   
 private static final String OUTPUT_CQL_SCHEMA_PREFIX = 
cassandra.columnfamily.schema.;
 private static final String OUTPUT_CQL_INSERT_PREFIX = 
cassandra.columnfamily.insert.;
+private static final String DELETE_SOURCE = 
cassandra.output.delete.source;
   
 /** Fills the deprecated OutputFormat interface for streaming. */
 @Deprecated
@@ -103,4 +104,14 @@ public class CqlBulkOutputFormat extends 
AbstractBulkOutputFormatObject, ListB
 }
 return insert;
 }
+
+public static void setDeleteSourceOnSuccess(Configuration conf, boolean 
deleteSrc)
+{
+conf.setBoolean(DELETE_SOURCE, deleteSrc);
+}
+
+public static boolean getDeleteSourceOnSuccess(Configuration conf)
+{
+return conf.getBoolean(DELETE_SOURCE, false);
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c0ee30e/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
index 7a75bb4..43e3a12 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
@@ -32,6 +32,8 @@ import org.apache.cassandra.hadoop.ConfigHelper;
 import org.apache.cassandra.hadoop.HadoopCompat;
 import org.apache.cassandra.io.sstable.CQLSSTableWriter;
 import org.apache.cassandra.io.sstable.SSTableLoader;
+import org.apache.cassandra.io.util.FileUtils;
+import org.apache.cassandra.streaming.StreamState;
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.mapreduce.TaskAttemptContext;
 import org.apache.hadoop.util.Progressable;
@@ -57,6 +59,7 @@ public class CqlBulkRecordWriter extends 
AbstractBulkRecordWriterObject, ListB
 private String schema;
 private String insertStatement;
 private File outputDir;
+private boolean deleteSrc;
 
 CqlBulkRecordWriter(TaskAttemptContext context) throws IOException
 {
@@ -84,6 +87,7 @@ public class CqlBulkRecordWriter extends 
AbstractBulkRecordWriterObject, ListB
 schema = CqlBulkOutputFormat.getColumnFamilySchema(conf, columnFamily);
 insertStatement = 
CqlBulkOutputFormat.getColumnFamilyInsertStatement(conf, columnFamily);
 outputDir = getColumnFamilyDirectory();
+deleteSrc = CqlBulkOutputFormat.getDeleteSourceOnSuccess(conf);
 }
 
 
@@ -107,7 +111,14 @@ public class CqlBulkRecordWriter extends 
AbstractBulkRecordWriterObject, ListB
 
 externalClient.addKnownCfs(keyspace, schema);
 
-this.loader = new SSTableLoader(outputDir, externalClient, new 
BulkRecordWriter.NullOutputHandler());
+this.loader = new SSTableLoader(outputDir, externalClient, new 
BulkRecordWriter.NullOutputHandler()) {
+@Override
+public void onSuccess(StreamState finalState)
+{
+if (deleteSrc)
+

[5/6] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ae9c0669
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ae9c0669
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ae9c0669

Branch: refs/heads/cassandra-2.1
Commit: ae9c06692a405e484dff1d848e46b60476cb5b5b
Parents: 932b5eb 6c0ee30
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Oct 14 15:44:00 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Oct 14 15:44:00 2014 -0500

--
 .../cassandra/hadoop/cql3/CqlBulkOutputFormat.java | 11 +++
 .../cassandra/hadoop/cql3/CqlBulkRecordWriter.java | 13 -
 2 files changed, 23 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae9c0669/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
--

[3/6] git commit: Add the ability to cleanup files after CqlBulkRecordWriter succeeds

Add the ability to cleanup files after CqlBulkRecordWriter succeeds

Patch by Paul Pak, reviewed by Piotr KoÅaczkowski for CASSANDRA-


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6c0ee30e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6c0ee30e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6c0ee30e

Branch: refs/heads/trunk
Commit: 6c0ee30ea7c57066aa15c25a40a217ca83aff248
Parents: 781018c
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Oct 14 15:42:43 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Oct 14 15:42:43 2014 -0500

--
 .../cassandra/hadoop/cql3/CqlBulkOutputFormat.java | 11 +++
 .../cassandra/hadoop/cql3/CqlBulkRecordWriter.java | 13 -
 2 files changed, 23 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c0ee30e/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java
index 58e05b6..887fe8e 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java
@@ -53,6 +53,7 @@ public class CqlBulkOutputFormat extends 
AbstractBulkOutputFormatObject, ListB
   
 private static final String OUTPUT_CQL_SCHEMA_PREFIX = 
cassandra.columnfamily.schema.;
 private static final String OUTPUT_CQL_INSERT_PREFIX = 
cassandra.columnfamily.insert.;
+private static final String DELETE_SOURCE = 
cassandra.output.delete.source;
   
 /** Fills the deprecated OutputFormat interface for streaming. */
 @Deprecated
@@ -103,4 +104,14 @@ public class CqlBulkOutputFormat extends 
AbstractBulkOutputFormatObject, ListB
 }
 return insert;
 }
+
+public static void setDeleteSourceOnSuccess(Configuration conf, boolean 
deleteSrc)
+{
+conf.setBoolean(DELETE_SOURCE, deleteSrc);
+}
+
+public static boolean getDeleteSourceOnSuccess(Configuration conf)
+{
+return conf.getBoolean(DELETE_SOURCE, false);
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c0ee30e/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
index 7a75bb4..43e3a12 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
@@ -32,6 +32,8 @@ import org.apache.cassandra.hadoop.ConfigHelper;
 import org.apache.cassandra.hadoop.HadoopCompat;
 import org.apache.cassandra.io.sstable.CQLSSTableWriter;
 import org.apache.cassandra.io.sstable.SSTableLoader;
+import org.apache.cassandra.io.util.FileUtils;
+import org.apache.cassandra.streaming.StreamState;
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.mapreduce.TaskAttemptContext;
 import org.apache.hadoop.util.Progressable;
@@ -57,6 +59,7 @@ public class CqlBulkRecordWriter extends 
AbstractBulkRecordWriterObject, ListB
 private String schema;
 private String insertStatement;
 private File outputDir;
+private boolean deleteSrc;
 
 CqlBulkRecordWriter(TaskAttemptContext context) throws IOException
 {
@@ -84,6 +87,7 @@ public class CqlBulkRecordWriter extends 
AbstractBulkRecordWriterObject, ListB
 schema = CqlBulkOutputFormat.getColumnFamilySchema(conf, columnFamily);
 insertStatement = 
CqlBulkOutputFormat.getColumnFamilyInsertStatement(conf, columnFamily);
 outputDir = getColumnFamilyDirectory();
+deleteSrc = CqlBulkOutputFormat.getDeleteSourceOnSuccess(conf);
 }
 
 
@@ -107,7 +111,14 @@ public class CqlBulkRecordWriter extends 
AbstractBulkRecordWriterObject, ListB
 
 externalClient.addKnownCfs(keyspace, schema);
 
-this.loader = new SSTableLoader(outputDir, externalClient, new 
BulkRecordWriter.NullOutputHandler());
+this.loader = new SSTableLoader(outputDir, externalClient, new 
BulkRecordWriter.NullOutputHandler()) {
+@Override
+public void onSuccess(StreamState finalState)
+{
+if (deleteSrc)
+FileUtils.deleteRecursive(outputDir);
+}
+};
 }
 }
 catch (Exception e)

[jira] [Commented] (CASSANDRA-8099) Refactor and modernize the storage engine

2014-10-14 Thread Pavel Yaskevich (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171492#comment-14171492
]

Pavel Yaskevich commented on CASSANDRA-8099:

{color:red}
_No-no-no, I think we should go all the way and re-write everything, gives
us all the benefits in the single shot, plus who cares about supporting all the
features anyway, right?_
{color}

Seriously tho, this sounds like a good first step.

Refactor and modernize the storage engine
-

Key: CASSANDRA-8099
URL: https://issues.apache.org/jira/browse/CASSANDRA-8099
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 3.0

The current storage engine (which for this ticket I'll loosely define as the
code implementing the read/write path) is suffering from old age. One of the
main problem is that the only structure it deals with is the cell, which
completely ignores the more high level CQL structure that groups cell into
(CQL) rows.
This leads to many inefficiencies, like the fact that during a reads we have
to group cells multiple times (to count on replica, then to count on the
coordinator, then to produce the CQL resultset) because we forget about the
grouping right away each time (so lots of useless cell names comparisons in
particular). But outside inefficiencies, having to manually recreate the CQL
structure every time we need it for something is hindering new features and
makes the code more complex that it should be.
Said storage engine also has tons of technical debt. To pick an example, the
fact that during range queries we update {{SliceQueryFilter.count}} is pretty
hacky and error prone. Or the overly complex ways {{AbstractQueryPager}} has
to go into to simply remove the last query result.
So I want to bite the bullet and modernize this storage engine. I propose to
do 2 main things:
# Make the storage engine more aware of the CQL structure. In practice,
instead of having partitions be a simple iterable map of cells, it should be
an iterable list of row (each being itself composed of per-column cells,
though obviously not exactly the same kind of cell we have today).
# Make the engine more iterative. What I mean here is that in the read path,
we end up reading all cells in memory (we put them in a ColumnFamily object),
but there is really no reason to. If instead we were working with iterators
all the way through, we could get to a point where we're basically
transferring data from disk to the network, and we should be able to reduce
GC substantially.
Please note that such refactor should provide some performance improvements
right off the bat but it's not it's primary goal either. It's primary goal is
to simplify the storage engine and adds abstraction that are better suited to
further optimizations.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/6] git commit: Add the ability to cleanup files after CqlBulkRecordWriter succeeds

Add the ability to cleanup files after CqlBulkRecordWriter succeeds

Patch by Paul Pak, reviewed by Piotr KoÅaczkowski for CASSANDRA-


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6c0ee30e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6c0ee30e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6c0ee30e

Branch: refs/heads/cassandra-2.1
Commit: 6c0ee30ea7c57066aa15c25a40a217ca83aff248
Parents: 781018c
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Oct 14 15:42:43 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Oct 14 15:42:43 2014 -0500

--
 .../cassandra/hadoop/cql3/CqlBulkOutputFormat.java | 11 +++
 .../cassandra/hadoop/cql3/CqlBulkRecordWriter.java | 13 -
 2 files changed, 23 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c0ee30e/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java
index 58e05b6..887fe8e 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java
@@ -53,6 +53,7 @@ public class CqlBulkOutputFormat extends 
AbstractBulkOutputFormatObject, ListB
   
 private static final String OUTPUT_CQL_SCHEMA_PREFIX = 
cassandra.columnfamily.schema.;
 private static final String OUTPUT_CQL_INSERT_PREFIX = 
cassandra.columnfamily.insert.;
+private static final String DELETE_SOURCE = 
cassandra.output.delete.source;
   
 /** Fills the deprecated OutputFormat interface for streaming. */
 @Deprecated
@@ -103,4 +104,14 @@ public class CqlBulkOutputFormat extends 
AbstractBulkOutputFormatObject, ListB
 }
 return insert;
 }
+
+public static void setDeleteSourceOnSuccess(Configuration conf, boolean 
deleteSrc)
+{
+conf.setBoolean(DELETE_SOURCE, deleteSrc);
+}
+
+public static boolean getDeleteSourceOnSuccess(Configuration conf)
+{
+return conf.getBoolean(DELETE_SOURCE, false);
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c0ee30e/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
index 7a75bb4..43e3a12 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
@@ -32,6 +32,8 @@ import org.apache.cassandra.hadoop.ConfigHelper;
 import org.apache.cassandra.hadoop.HadoopCompat;
 import org.apache.cassandra.io.sstable.CQLSSTableWriter;
 import org.apache.cassandra.io.sstable.SSTableLoader;
+import org.apache.cassandra.io.util.FileUtils;
+import org.apache.cassandra.streaming.StreamState;
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.mapreduce.TaskAttemptContext;
 import org.apache.hadoop.util.Progressable;
@@ -57,6 +59,7 @@ public class CqlBulkRecordWriter extends 
AbstractBulkRecordWriterObject, ListB
 private String schema;
 private String insertStatement;
 private File outputDir;
+private boolean deleteSrc;
 
 CqlBulkRecordWriter(TaskAttemptContext context) throws IOException
 {
@@ -84,6 +87,7 @@ public class CqlBulkRecordWriter extends 
AbstractBulkRecordWriterObject, ListB
 schema = CqlBulkOutputFormat.getColumnFamilySchema(conf, columnFamily);
 insertStatement = 
CqlBulkOutputFormat.getColumnFamilyInsertStatement(conf, columnFamily);
 outputDir = getColumnFamilyDirectory();
+deleteSrc = CqlBulkOutputFormat.getDeleteSourceOnSuccess(conf);
 }
 
 
@@ -107,7 +111,14 @@ public class CqlBulkRecordWriter extends 
AbstractBulkRecordWriterObject, ListB
 
 externalClient.addKnownCfs(keyspace, schema);
 
-this.loader = new SSTableLoader(outputDir, externalClient, new 
BulkRecordWriter.NullOutputHandler());
+this.loader = new SSTableLoader(outputDir, externalClient, new 
BulkRecordWriter.NullOutputHandler()) {
+@Override
+public void onSuccess(StreamState finalState)
+{
+if (deleteSrc)
+FileUtils.deleteRecursive(outputDir);
+}
+};
 }
 }
 catch (Exception e)

[6/6] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6630b878
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6630b878
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6630b878

Branch: refs/heads/trunk
Commit: 6630b8786d8e51703544a7cbc4d6cba596a10353
Parents: 3a822ca ae9c066
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Oct 14 15:44:10 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Oct 14 15:44:10 2014 -0500

--
 .../cassandra/hadoop/cql3/CqlBulkOutputFormat.java | 11 +++
 .../cassandra/hadoop/cql3/CqlBulkRecordWriter.java | 13 -
 2 files changed, 23 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6630b878/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkOutputFormat.java
--

[4/6] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ae9c0669
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ae9c0669
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ae9c0669

Branch: refs/heads/trunk
Commit: ae9c06692a405e484dff1d848e46b60476cb5b5b
Parents: 932b5eb 6c0ee30
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Oct 14 15:44:00 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Oct 14 15:44:00 2014 -0500

--
 .../cassandra/hadoop/cql3/CqlBulkOutputFormat.java | 11 +++
 .../cassandra/hadoop/cql3/CqlBulkRecordWriter.java | 13 -
 2 files changed, 23 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae9c0669/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
--

[jira] [Assigned] (CASSANDRA-4476) Support 2ndary index queries with only non-EQ clauses

2014-10-14 Thread Robbie Strickland (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robbie Strickland reassigned CASSANDRA-4476:


Assignee: Robbie Strickland

 Support 2ndary index queries with only non-EQ clauses
 -

 Key: CASSANDRA-4476
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4476
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Core
Reporter: Sylvain Lebresne
Assignee: Robbie Strickland
Priority: Minor
  Labels: cql
 Fix For: 2.1.1


 Currently, a query that uses 2ndary indexes must have at least one EQ clause 
 (on an indexed column). Given that indexed CFs are local (and use 
 LocalPartitioner that order the row by the type of the indexed column), we 
 should extend 2ndary indexes to allow querying indexed columns even when no 
 EQ clause is provided.
 As far as I can tell, the main problem to solve for this is to update 
 KeysSearcher.highestSelectivityPredicate(). I.e. how do we estimate the 
 selectivity of non-EQ clauses? I note however that if we can do that estimate 
 reasonably accurately, this might provide better performance even for index 
 queries that both EQ and non-EQ clauses, because some non-EQ clauses may have 
 a much better selectivity than EQ ones (say you index both the user country 
 and birth date, for SELECT * FROM users WHERE country = 'US' AND birthdate  
 'Jan 2009' AND birtdate  'July 2009', you'd better use the birthdate index 
 first).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8119) More Expressive Consistency Levels

Tyler Hobbs created CASSANDRA-8119:
--

 Summary: More Expressive Consistency Levels
 Key: CASSANDRA-8119
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8119
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Tyler Hobbs


For some multi-datacenter environments, the current set of consistency levels 
are too restrictive.  For example, the following consistency requirements 
cannot be expressed:
* LOCAL_QUORUM in two specific DCs
* LOCAL_QUORUM in the local DC plus LOCAL_QUORUM in at least one other DC
* LOCAL_QUORUM in the local DC plus N remote replicas in any DC

I propose that we add a new consistency level: CUSTOM.  In the v4 (or v5) 
protocol, this would be accompanied by an additional map argument.  A map of 
{DC: CL} or a map of {DC: int} is sufficient to cover the first example.  If we 
accept a special keys to represent any datacenter, the second case can be 
handled.  A similar technique could be used for any other nodes.

I'm not in love with the special keys, so if anybody has ideas for something 
more elegant, feel free to propose them.  The main idea is that we want to be 
flexible enough to cover any reasonable consistency or durability requirements.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8108) ClassCastException in AbstractCellNameType


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171540#comment-14171540
 ] 

Tyler Hobbs commented on CASSANDRA-8108:


[~hearndencanva] can you paste your schema for this table and the query that 
results in this error?

 ClassCastException in AbstractCellNameType
 --

 Key: CASSANDRA-8108
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8108
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra 2.1.0
 Datastax AMI on EC2 (Ubuntu Linux)
Reporter: David Hearnden
Assignee: Tyler Hobbs

 {noformat}
 java.lang.ClassCastException: 
 org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast 
 to org.apache.cassandra.db.composites.CellName
   at 
 org.apache.cassandra.db.composites.AbstractCellNameType.cellFromByteBuffer(AbstractCellNameType.java:170)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.SliceQueryPager.init(SliceQueryPager.java:57)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.MultiPartitionPager.makePager(MultiPartitionPager.java:84)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.MultiPartitionPager.init(MultiPartitionPager.java:68)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:101) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:125) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:215)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:60)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:187)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:413)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:133)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:422)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:318)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at 
 io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:103)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:31)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:323)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_51]
   at 
 org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) 
 [apache-cassandra-2.1.0.jar:2.1.0]
   at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-4476) Support 2ndary index queries with only non-EQ clauses

2014-10-14 Thread Jeremiah Jordan (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Jordan updated CASSANDRA-4476:
---
Assignee: (was: Robbie Strickland)

 Support 2ndary index queries with only non-EQ clauses
 -

 Key: CASSANDRA-4476
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4476
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Core
Reporter: Sylvain Lebresne
Priority: Minor
  Labels: cql
 Fix For: 2.1.1


 Currently, a query that uses 2ndary indexes must have at least one EQ clause 
 (on an indexed column). Given that indexed CFs are local (and use 
 LocalPartitioner that order the row by the type of the indexed column), we 
 should extend 2ndary indexes to allow querying indexed columns even when no 
 EQ clause is provided.
 As far as I can tell, the main problem to solve for this is to update 
 KeysSearcher.highestSelectivityPredicate(). I.e. how do we estimate the 
 selectivity of non-EQ clauses? I note however that if we can do that estimate 
 reasonably accurately, this might provide better performance even for index 
 queries that both EQ and non-EQ clauses, because some non-EQ clauses may have 
 a much better selectivity than EQ ones (say you index both the user country 
 and birth date, for SELECT * FROM users WHERE country = 'US' AND birthdate  
 'Jan 2009' AND birtdate  'July 2009', you'd better use the birthdate index 
 first).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8120) Failed creating a merkle tree during repair.

2014-10-14 Thread jason milliron (JIRA)

jason milliron created CASSANDRA-8120:
-

 Summary: Failed creating a merkle tree during repair.
 Key: CASSANDRA-8120
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8120
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Ubuntu 12.04.5 LTS.  3.2.0-69-virtual.
Reporter: jason milliron


Seeing this while running nodetool repair on several nodes of our new (not an 
upgrade) 2.1.0 cluster.  Multiple keyspaces.  Doesn't hang the repair job.  
Storage is SSD ephemeral (EC2).  No storage errors in syslog.  

ERROR [ValidationExecutor:39] 2014-10-13 22:20:51,140 Validator.java:242 - 
Failed creating a merkle tree for [repair #1523dfa0-5327-11e4-925c-65a001aa6d80 
on censored/censored, (-4468426774357114003,-4433737571269222163]], 
/10.37.666.175 (see log for details)
INFO  [AntiEntropySessions:37] 2014-10-13 22:20:51,141 RepairSession.java:260 - 
[repair #308e2750-5327-11e4-925c-65a001aa6d80] new session: will sync 
ip-10-37-666-175.ec2.internal/10.37.666.175, /10.167.666.137, /10.164.666.81 on 
range (6870883176570587821,6871062738154464368] for censored.[censored]
INFO  [AntiEntropyStage:1] 2014-10-13 22:20:51,142 ColumnFamilyStore.java:856 - 
Enqueuing flush of censored: 3227760 (0%) on-heap, 0 (0%) off-heap
INFO  [MemtableFlushWriter:2295] 2014-10-13 22:20:51,145 Memtable.java:326 - 
Writing Memtable-censored@2052537453(380369 serialized bytes, 8235 ops, 0%/0% 
of on/off-heap limit)
ERROR [AntiEntropySessions:34] 2014-10-13 22:20:51,148 RepairSession.java:303 - 
[repair #1523dfa0-5327-11e4-925c-65a001aa6d80] session completed with the 
following error
org.apache.cassandra.exceptions.RepairException: [repair 
#1523dfa0-5327-11e4-925c-65a001aa6d80 on censored/censored, 
(-4468426774357114003,-4433737571269222163]] Validation failed in /10.37.666.175
at 
org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:166)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:384)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:123)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62) 
~[apache-cassandra-2.1.0.jar:2.1.0]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
[na:1.7.0_67]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_67]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
ERROR [ValidationExecutor:39] 2014-10-13 22:20:51,148 CassandraDaemon.java:166 
- Exception in thread Thread[ValidationExecutor:39,1,main]
java.lang.AssertionError: row DecoratedKey(-4468391015221140553, 
0004000108005abc2d00) received out of order wrt 
DecoratedKey(-4436255891196926644, 000400010800617f4400)
at org.apache.cassandra.repair.Validator.add(Validator.java:136) 
~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:916)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:100)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:560)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
~[na:1.7.0_67]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
~[na:1.7.0_67]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_67]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
ERROR [AntiEntropySessions:34] 2014-10-13 22:20:51,149 CassandraDaemon.java:166 
- Exception in thread Thread[AntiEntropySessions:34,5,RMI Runtime]
java.lang.RuntimeException: org.apache.cassandra.exceptions.RepairException: 
[repair #1523dfa0-5327-11e4-925c-65a001aa6d80 on censored/censored, 
(-4468426774357114003,-4433737571269222163]] Validation failed in /10.37.666.175
at com.google.common.base.Throwables.propagate(Throwables.java:160) 
~[guava-16.0.jar:na]
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32) 
~[apache-cassandra-2.1.0.jar:2.1.0]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
~[na:1.7.0_67]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
~[na:1.7.0_67]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
~[na:1.7.0_67]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_67]

[jira] [Commented] (CASSANDRA-8108) ClassCastException in AbstractCellNameType

2014-10-14 Thread David Hearnden (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171556#comment-14171556
 ] 

David Hearnden commented on CASSANDRA-8108:
---

{noformat}
CREATE TABLE comments (
object ascii,
id timeuuid,
...
latest_content text static,
...
PRIMARY KEY (object, id)
) WITH CLUSTERING ORDER BY (id DESC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
{noformat}

The query that triggers the exception:
{noformat}
SELECT DISTINCT object, latest_content 
FROM comments
WHERE object IN (...);
{noformat}

I only observe this error running Cassandra 2.1.0 via the Datastax AMI 2.5.1 
(Ubuntu Linux 12.04) on EC2 (ami-ada2b6c4).
I don't observe this error running Cassandra 2.1.0 on 0S X 10.9.
I'm about to test running Cassandra 2.1.0 on Ubuntu Linux 14.04 on EC2.

 ClassCastException in AbstractCellNameType
 --

 Key: CASSANDRA-8108
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8108
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra 2.1.0
 Datastax AMI on EC2 (Ubuntu Linux)
Reporter: David Hearnden
Assignee: Tyler Hobbs

 {noformat}
 java.lang.ClassCastException: 
 org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast 
 to org.apache.cassandra.db.composites.CellName
   at 
 org.apache.cassandra.db.composites.AbstractCellNameType.cellFromByteBuffer(AbstractCellNameType.java:170)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.SliceQueryPager.init(SliceQueryPager.java:57)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.MultiPartitionPager.makePager(MultiPartitionPager.java:84)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.MultiPartitionPager.init(MultiPartitionPager.java:68)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:101) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:125) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:215)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:60)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:187)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:413)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:133)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:422)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:318)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at 
 io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:103)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:31)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:323)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_51]
   at 
 org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) 
 [apache-cassandra-2.1.0.jar:2.1.0]
   at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7280) Hadoop support not respecting cassandra.input.split.size

2014-10-14 Thread Jeremy Hanna (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171555#comment-14171555
 ] 

Jeremy Hanna commented on CASSANDRA-7280:
-

The page size is for how many to fetch at a time. The input split size is how 
much is in the overall split.

 Hadoop support not respecting cassandra.input.split.size
 

 Key: CASSANDRA-7280
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7280
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Reporter: Jeremy Hanna

 Long ago (0.7), I tried to set the cassandra.input.split.size property and 
 never really got it to respect that property.  However the batch size was 
 useful for what I needed to affect the timeouts.
 Now with the cql record reader and the native paging, users can specify 
 queries potentially using allow filtering clauses.  The input split size is 
 more important because the server may have to scan through many many records 
 to get matching records.  If the user can effectively set the input split 
 size, then that gives a hard limit on how many records it will traverse.
 Currently it appears to be overriding the property, perhaps in the 
 client.describe_splits_ex method on the server side.
 It can be argued that users shouldn't be using allow filtering clauses in 
 their cql in the first place.  However it is still a bug that the input split 
 size is not honored.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8108) ClassCastException in AbstractCellNameType


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8108:
---
  Component/s: Core
Reproduced In: 2.1.0
Fix Version/s: 2.1.1

 ClassCastException in AbstractCellNameType
 --

 Key: CASSANDRA-8108
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8108
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1.0
 Datastax AMI on EC2 (Ubuntu Linux)
Reporter: David Hearnden
Assignee: Tyler Hobbs
 Fix For: 2.1.1


 {noformat}
 java.lang.ClassCastException: 
 org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast 
 to org.apache.cassandra.db.composites.CellName
   at 
 org.apache.cassandra.db.composites.AbstractCellNameType.cellFromByteBuffer(AbstractCellNameType.java:170)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.SliceQueryPager.init(SliceQueryPager.java:57)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.MultiPartitionPager.makePager(MultiPartitionPager.java:84)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.MultiPartitionPager.init(MultiPartitionPager.java:68)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:101) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:125) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:215)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:60)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:187)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:413)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:133)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:422)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:318)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at 
 io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:103)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:31)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:323)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_51]
   at 
 org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) 
 [apache-cassandra-2.1.0.jar:2.1.0]
   at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8113) Gossip should ignore generation numbers too far in the future


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-8113:
---
Attachment: 8113-v4.txt

v4 removes check on version, and only checks generation. 

 Gossip should ignore generation numbers too far in the future
 -

 Key: CASSANDRA-8113
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8113
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Low
Assignee: Jason Brown
 Attachments: 8113-v1.txt, 8113-v2.txt, 8113-v3.txt, 8113-v4.txt


 If a node sends corrupted gossip, it could set the generation numbers for 
 other nodes to arbitrarily large values. This is dangerous since one bad node 
 (e.g. with bad memory) could in theory bring down the cluster. Nodes should 
 refuse to accept generation numbers that are too far in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8113) Gossip should ignore generation numbers too far in the future


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171576#comment-14171576
 ] 

Brandon Williams commented on CASSANDRA-8113:
-

+1 with minor nit: the continue used in the generation isn't needed

 Gossip should ignore generation numbers too far in the future
 -

 Key: CASSANDRA-8113
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8113
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Low
Assignee: Jason Brown
 Attachments: 8113-v1.txt, 8113-v2.txt, 8113-v3.txt, 8113-v4.txt


 If a node sends corrupted gossip, it could set the generation numbers for 
 other nodes to arbitrarily large values. This is dangerous since one bad node 
 (e.g. with bad memory) could in theory bring down the cluster. Nodes should 
 refuse to accept generation numbers that are too far in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8108) ClassCastException in AbstractCellNameType


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171577#comment-14171577
 ] 

Tyler Hobbs commented on CASSANDRA-8108:


Thanks, I can reproduce the issue.

As a workaround in the meantime, you can disable paging at the driver level.  
There should be a setFetchSize() command somewhere to do that.  Of course, if 
you're fetching a lot of keys at once, you'll want to break that up.

 ClassCastException in AbstractCellNameType
 --

 Key: CASSANDRA-8108
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8108
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1.0
 Datastax AMI on EC2 (Ubuntu Linux)
Reporter: David Hearnden
Assignee: Tyler Hobbs
 Fix For: 2.1.1


 {noformat}
 java.lang.ClassCastException: 
 org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast 
 to org.apache.cassandra.db.composites.CellName
   at 
 org.apache.cassandra.db.composites.AbstractCellNameType.cellFromByteBuffer(AbstractCellNameType.java:170)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.SliceQueryPager.init(SliceQueryPager.java:57)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.MultiPartitionPager.makePager(MultiPartitionPager.java:84)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.MultiPartitionPager.init(MultiPartitionPager.java:68)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:101) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:125) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:215)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:60)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:187)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:413)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:133)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:422)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:318)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at 
 io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:103)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:31)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:323)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_51]
   at 
 org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163)
  [apache-cassandra-2.1.0.jar:2.1.0]
   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) 
 [apache-cassandra-2.1.0.jar:2.1.0]
   at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8113) Gossip should ignore generation numbers too far in the future


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171576#comment-14171576
 ] 

Brandon Williams edited comment on CASSANDRA-8113 at 10/14/14 9:44 PM:
---

+1 with minor nit: the continue used in the generation check isn't needed


was (Author: brandon.williams):
+1 with minor nit: the continue used in the generation isn't needed

 Gossip should ignore generation numbers too far in the future
 -

 Key: CASSANDRA-8113
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8113
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Low
Assignee: Jason Brown
 Attachments: 8113-v1.txt, 8113-v2.txt, 8113-v3.txt, 8113-v4.txt


 If a node sends corrupted gossip, it could set the generation numbers for 
 other nodes to arbitrarily large values. This is dangerous since one bad node 
 (e.g. with bad memory) could in theory bring down the cluster. Nodes should 
 refuse to accept generation numbers that are too far in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8113) Gossip should ignore generation numbers too far in the future


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171584#comment-14171584
 ] 

Jason Brown commented on CASSANDRA-8113:


ahh, will remove that continue. I'm planning on applying this to 2.0 and up, 
wdyt?

 Gossip should ignore generation numbers too far in the future
 -

 Key: CASSANDRA-8113
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8113
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Low
Assignee: Jason Brown
 Attachments: 8113-v1.txt, 8113-v2.txt, 8113-v3.txt, 8113-v4.txt


 If a node sends corrupted gossip, it could set the generation numbers for 
 other nodes to arbitrarily large values. This is dangerous since one bad node 
 (e.g. with bad memory) could in theory bring down the cluster. Nodes should 
 refuse to accept generation numbers that are too far in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8113) Gossip should ignore generation numbers too far in the future


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171590#comment-14171590
 ] 

Brandon Williams commented on CASSANDRA-8113:
-

I was thinking 2.1.1, since this is an improvement rather than a bug.

 Gossip should ignore generation numbers too far in the future
 -

 Key: CASSANDRA-8113
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8113
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Low
Assignee: Jason Brown
 Attachments: 8113-v1.txt, 8113-v2.txt, 8113-v3.txt, 8113-v4.txt


 If a node sends corrupted gossip, it could set the generation numbers for 
 other nodes to arbitrarily large values. This is dangerous since one bad node 
 (e.g. with bad memory) could in theory bring down the cluster. Nodes should 
 refuse to accept generation numbers that are too far in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Gossip should ignore generation numbers too far in the future

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 ae9c06692 - 8be8a086c


Gossip should ignore generation numbers too far in the future

patch by jasobrown; reviewed by driftx for CASSANDRA-8113


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8be8a086
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8be8a086
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8be8a086

Branch: refs/heads/cassandra-2.1
Commit: 8be8a086cb40f3280e9af5543d7b170241b268d8
Parents: ae9c066
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 14:55:42 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 14:55:42 2014 -0700

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/gms/Gossiper.java | 11 ++-
 2 files changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8be8a086/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1c7ee38..6d9d221 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.1
+ * Gossip should ignore generation numbers too far in the future 
(CASSANDRA-8113)
  * Fix NPE when creating a table with frozen sets, lists (CASSANDRA-8104)
  * Fix high memory use due to tracking reads on incrementally opened sstable
readers (CASSANDRA-8066)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8be8a086/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java 
b/src/java/org/apache/cassandra/gms/Gossiper.java
index bedc8d7..ae8bf32 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -82,6 +82,9 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
 public static final Gossiper instance = new Gossiper();
 
 public static final long aVeryLongTime = 259200 * 1000; // 3 days
+
+/** Maximimum difference in generation and version values we are willing 
to accept about a peer */
+private static final long MAX_GENERATION_DIFFERENCE = 86400 * 365;
 private long FatClientTimeout;
 private final Random random = new Random();
 private final ComparatorInetAddress inetcomparator = new 
ComparatorInetAddress()
@@ -984,7 +987,13 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
 if (logger.isTraceEnabled())
 logger.trace(ep + local generation  + localGeneration + 
, remote generation  + remoteGeneration);
 
-if (remoteGeneration  localGeneration)
+if (remoteGeneration  localGeneration + 
MAX_GENERATION_DIFFERENCE)
+{
+// assume some peer has corrupted memory and is 
broadcasting an unbelievable generation about another peer (or itself)
+logger.warn(received an invalid gossip generation for 
peer {}; local generation = {}, received generation = {}, ep, localGeneration, 
remoteGeneration);
+continue;
+}
+else if (remoteGeneration  localGeneration)
 {
 if (logger.isTraceEnabled())
 logger.trace(Updating heartbeat state generation to  
+ remoteGeneration +  from  + localGeneration +  for  + ep);

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/18549038
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/18549038
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/18549038

Branch: refs/heads/trunk
Commit: 18549038a1ea76a48b2bb4a3eede642978ef966d
Parents: 6630b87 8be8a08
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 14:56:33 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 14:56:33 2014 -0700

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/gms/Gossiper.java | 11 ++-
 2 files changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/18549038/CHANGES.txt
--
diff --cc CHANGES.txt
index 5dac7f4,6d9d221..be84b11
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,34 -1,5 +1,35 @@@
 +3.0
 + * Keep sstable levels when bootstrapping (CASSANDRA-7460)
 + * Add Sigar library and perform basic OS settings check on startup 
(CASSANDRA-7838)
 + * Support for aggregation functions (CASSANDRA-4914)
 + * Remove cassandra-cli (CASSANDRA-7920)
 + * Accept dollar quoted strings in CQL (CASSANDRA-7769)
 + * Make assassinate a first class command (CASSANDRA-7935)
 + * Support IN clause on any clustering column (CASSANDRA-4762)
 + * Improve compaction logging (CASSANDRA-7818)
 + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917)
 + * Do anticompaction in groups (CASSANDRA-6851)
 + * Support pure user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 
7781, 7929,
 +   7924, 7812, 8063)
 + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416)
 + * Move sstable RandomAccessReader to nio2, which allows using the
 +   FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050)
 + * Remove CQL2 (CASSANDRA-5918)
 + * Add Thrift get_multi_slice call (CASSANDRA-6757)
 + * Optimize fetching multiple cells by name (CASSANDRA-6933)
 + * Allow compilation in java 8 (CASSANDRA-7028)
 + * Make incremental repair default (CASSANDRA-7250)
 + * Enable code coverage thru JaCoCo (CASSANDRA-7226)
 + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) 
 + * Shorten SSTable path (CASSANDRA-6962)
 + * Use unsafe mutations for most unit tests (CASSANDRA-6969)
 + * Fix race condition during calculation of pending ranges (CASSANDRA-7390)
 + * Fail on very large batch sizes (CASSANDRA-8011)
 + * improve concurrency of repair (CASSANDRA-6455)
 +
 +
  2.1.1
+  * Gossip should ignore generation numbers too far in the future 
(CASSANDRA-8113)
   * Fix NPE when creating a table with frozen sets, lists (CASSANDRA-8104)
   * Fix high memory use due to tracking reads on incrementally opened sstable
 readers (CASSANDRA-8066)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/18549038/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --cc src/java/org/apache/cassandra/gms/Gossiper.java
index c488b09,ae8bf32..364837d
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@@ -990,12 -985,18 +993,18 @@@ public class Gossiper implements IFailu
  int localGeneration = 
localEpStatePtr.getHeartBeatState().getGeneration();
  int remoteGeneration = 
remoteState.getHeartBeatState().getGeneration();
  if (logger.isTraceEnabled())
 -logger.trace(ep + local generation  + localGeneration + 
, remote generation  + remoteGeneration);
 +logger.trace({} local generation {}, remote generation 
{}, ep, localGeneration, remoteGeneration);
  
- if (remoteGeneration  localGeneration)
+ if (remoteGeneration  localGeneration + 
MAX_GENERATION_DIFFERENCE)
+ {
+ // assume some peer has corrupted memory and is 
broadcasting an unbelievable generation about another peer (or itself)
+ logger.warn(received an invalid gossip generation for 
peer {}; local generation = {}, received generation = {}, ep, localGeneration, 
remoteGeneration);
+ continue;
+ }
+ else if (remoteGeneration  localGeneration)
  {
  if (logger.isTraceEnabled())
 -logger.trace(Updating heartbeat state generation to 
 + remoteGeneration +  from  + localGeneration +  for  + ep);
 +logger.trace(Updating heartbeat state generation to 
{} from {} for {}, remoteGeneration, localGeneration, ep);
  // major state change will handle the update by inserting 
the remote

[1/2] git commit: Gossip should ignore generation numbers too far in the future

Repository: cassandra
Updated Branches:
  refs/heads/trunk 6630b8786 - 18549038a


Gossip should ignore generation numbers too far in the future

patch by jasobrown; reviewed by driftx for CASSANDRA-8113


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8be8a086
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8be8a086
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8be8a086

Branch: refs/heads/trunk
Commit: 8be8a086cb40f3280e9af5543d7b170241b268d8
Parents: ae9c066
Author: Jason Brown jasedbr...@gmail.com
Authored: Tue Oct 14 14:55:42 2014 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Tue Oct 14 14:55:42 2014 -0700

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/gms/Gossiper.java | 11 ++-
 2 files changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8be8a086/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1c7ee38..6d9d221 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.1
+ * Gossip should ignore generation numbers too far in the future 
(CASSANDRA-8113)
  * Fix NPE when creating a table with frozen sets, lists (CASSANDRA-8104)
  * Fix high memory use due to tracking reads on incrementally opened sstable
readers (CASSANDRA-8066)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8be8a086/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java 
b/src/java/org/apache/cassandra/gms/Gossiper.java
index bedc8d7..ae8bf32 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -82,6 +82,9 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
 public static final Gossiper instance = new Gossiper();
 
 public static final long aVeryLongTime = 259200 * 1000; // 3 days
+
+/** Maximimum difference in generation and version values we are willing 
to accept about a peer */
+private static final long MAX_GENERATION_DIFFERENCE = 86400 * 365;
 private long FatClientTimeout;
 private final Random random = new Random();
 private final ComparatorInetAddress inetcomparator = new 
ComparatorInetAddress()
@@ -984,7 +987,13 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
 if (logger.isTraceEnabled())
 logger.trace(ep + local generation  + localGeneration + 
, remote generation  + remoteGeneration);
 
-if (remoteGeneration  localGeneration)
+if (remoteGeneration  localGeneration + 
MAX_GENERATION_DIFFERENCE)
+{
+// assume some peer has corrupted memory and is 
broadcasting an unbelievable generation about another peer (or itself)
+logger.warn(received an invalid gossip generation for 
peer {}; local generation = {}, received generation = {}, ep, localGeneration, 
remoteGeneration);
+continue;
+}
+else if (remoteGeneration  localGeneration)
 {
 if (logger.isTraceEnabled())
 logger.trace(Updating heartbeat state generation to  
+ remoteGeneration +  from  + localGeneration +  for  + ep);

[jira] [Commented] (CASSANDRA-8113) Gossip should ignore generation numbers too far in the future


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171598#comment-14171598
 ] 

Jason Brown commented on CASSANDRA-8113:


wfm, committed to 2.1 and trunk

 Gossip should ignore generation numbers too far in the future
 -

 Key: CASSANDRA-8113
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8113
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Low
Assignee: Jason Brown
 Fix For: 2.1.1

 Attachments: 8113-v1.txt, 8113-v2.txt, 8113-v3.txt, 8113-v4.txt


 If a node sends corrupted gossip, it could set the generation numbers for 
 other nodes to arbitrarily large values. This is dangerous since one bad node 
 (e.g. with bad memory) could in theory bring down the cluster. Nodes should 
 refuse to accept generation numbers that are too far in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: ninja-remove unneeded continue statement