date:20150508

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4190c9dc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4190c9dc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4190c9dc

Branch: refs/heads/trunk
Commit: 4190c9dc2ff6181ae68f88208b559623b3a7aad8
Parents: 36958f3 e2ad784
Author: Marcus Eriksson marc...@apache.org
Authored: Fri May 8 10:45:16 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Fri May 8 10:45:16 2015 +0200

--
 CHANGES.txt  | 1 +
 src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 2 ++
 2 files changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4190c9dc/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4190c9dc/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--

[jira] [Commented] (CASSANDRA-8574) Gracefully degrade SELECT when there are lots of tombstones

2015-05-08 Thread Jens Rantil (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534029#comment-14534029
 ] 

Jens Rantil commented on CASSANDRA-8574:


I apologize for my lack of knowledge - what does TOE stand for? Theory of 
everything sounds out of place ;)

 Gracefully degrade SELECT when there are lots of tombstones
 ---

 Key: CASSANDRA-8574
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8574
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jens Rantil
 Fix For: 3.x


 *Background:* There's lots of tooling out there to do BigData analysis on 
 Cassandra clusters. Examples are Spark and Hadoop, which is offered by DSE. 
 The problem with both of these so far, is that a single partition key with 
 too many tombstones can make the query job fail hard.
 The described scenario happens despite the user setting a rather small 
 FetchSize. I assume this is a common scenario if you have larger rows.
 *Proposal:* To allow a CQL SELECT to gracefully degrade to only return a 
 smaller batch of results if there are too many tombstones. The tombstones are 
 ordered according to clustering key and one should be able to page through 
 them. Potentially:
 SELECT * FROM mytable LIMIT 1000 TOMBSTONES;
 would page through maximum 1000 tombstones, _or_ 1000 (CQL) rows.
 I understand that this obviously would degrade performance, but it would at 
 least yield a result.
 *Additional comment:* I haven't dug into Cassandra code, but conceptually I 
 guess this would be doable. Let me know what you think.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e2ad784c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e2ad784c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e2ad784c

Branch: refs/heads/cassandra-2.1
Commit: e2ad784c73f6084de072b01d4652812c226b19be
Parents: 09ccf60 917068d
Author: Marcus Eriksson marc...@apache.org
Authored: Fri May 8 10:44:34 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Fri May 8 10:44:34 2015 +0200

--
 CHANGES.txt  | 1 +
 src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 2 ++
 2 files changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e2ad784c/CHANGES.txt
--
diff --cc CHANGES.txt
index 098cfe0,1540d28..37f0117
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,21 -1,5 +1,22 @@@
 -2.0.15:
 +2.1.6
 + * Delete processed sstables in sstablesplit/sstableupgrade (CASSANDRA-8606)
 + * Improve sstable exclusion from partition tombstones (CASSANDRA-9298)
 + * Validate the indexed column rather than the cell's contents for 2i 
(CASSANDRA-9057)
 + * Add support for top-k custom 2i queries (CASSANDRA-8717)
 + * Fix error when dropping table during compaction (CASSANDRA-9251)
 + * cassandra-stress supports validation operations over user profiles 
(CASSANDRA-8773)
 + * Add support for rate limiting log messages (CASSANDRA-9029)
 + * Log the partition key with tombstone warnings (CASSANDRA-8561)
 + * Reduce runWithCompactionsDisabled poll interval to 1ms (CASSANDRA-9271)
 + * Fix PITR commitlog replay (CASSANDRA-9195)
 + * GCInspector logs very different times (CASSANDRA-9124)
 + * Fix deleting from an empty list (CASSANDRA-9198)
 + * Update tuple and collection types that use a user-defined type when that 
UDT
 +   is modified (CASSANDRA-9148, CASSANDRA-9192)
 + * Use higher timeout for prepair and snapshot in repair (CASSANDRA-9261)
 + * Fix anticompaction blocking ANTI_ENTROPY stage (CASSANDRA-9151)
 +Merged from 2.0:
+  * Avoid potential overlap in LCS with single-partition sstables 
(CASSANDRA-9322)
   * Log warning message when a table is queried before the schema has fully
 propagated (CASSANDRA-9136)
   * Overload SecondaryIndex#indexes to accept the column definition 
(CASSANDRA-9314)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e2ad784c/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--

[1/3] cassandra git commit: Avoid potential overlap in LCS with single-partition sstables

Repository: cassandra
Updated Branches:
  refs/heads/trunk 36958f3ca - 4190c9dc2


Avoid potential overlap in LCS with single-partition sstables

Patch by marcuse; reviewed by yukim for CASSANDRA-9322


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/917068dc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/917068dc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/917068dc

Branch: refs/heads/trunk
Commit: 917068dcbfb9909b5f1695dc18e9a7b250264b42
Parents: 58de86b
Author: Marcus Eriksson marc...@apache.org
Authored: Thu May 7 13:56:51 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Fri May 8 10:40:15 2015 +0200

--
 CHANGES.txt  | 1 +
 src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 2 ++
 2 files changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/917068dc/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8473777..1540d28 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.15:
+ * Avoid potential overlap in LCS with single-partition sstables 
(CASSANDRA-9322)
  * Log warning message when a table is queried before the schema has fully
propagated (CASSANDRA-9136)
  * Overload SecondaryIndex#indexes to accept the column definition 
(CASSANDRA-9314)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/917068dc/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index 867b5ce..81c65e4 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -353,6 +353,8 @@ public class LeveledManifest
 if (max == null || candidate.last.compareTo(max)  0)
 max = candidate.last;
 }
+if (min == null || max == null || min.equals(max)) // 
single partition sstables - we cannot include a high level sstable.
+return candidates;
 SetSSTableReader compacting = 
cfs.getDataTracker().getCompacting();
 RangeRowPosition boundaries = new Range(min, max);
 for (SSTableReader sstable : getLevel(i))

cassandra git commit: Avoid potential overlap in LCS with single-partition sstables

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 58de86bff - 917068dcb


Avoid potential overlap in LCS with single-partition sstables

Patch by marcuse; reviewed by yukim for CASSANDRA-9322


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/917068dc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/917068dc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/917068dc

Branch: refs/heads/cassandra-2.0
Commit: 917068dcbfb9909b5f1695dc18e9a7b250264b42
Parents: 58de86b
Author: Marcus Eriksson marc...@apache.org
Authored: Thu May 7 13:56:51 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Fri May 8 10:40:15 2015 +0200

--
 CHANGES.txt  | 1 +
 src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 2 ++
 2 files changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/917068dc/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8473777..1540d28 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.15:
+ * Avoid potential overlap in LCS with single-partition sstables 
(CASSANDRA-9322)
  * Log warning message when a table is queried before the schema has fully
propagated (CASSANDRA-9136)
  * Overload SecondaryIndex#indexes to accept the column definition 
(CASSANDRA-9314)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/917068dc/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index 867b5ce..81c65e4 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -353,6 +353,8 @@ public class LeveledManifest
 if (max == null || candidate.last.compareTo(max)  0)
 max = candidate.last;
 }
+if (min == null || max == null || min.equals(max)) // 
single partition sstables - we cannot include a high level sstable.
+return candidates;
 SetSSTableReader compacting = 
cfs.getDataTracker().getCompacting();
 RangeRowPosition boundaries = new Range(min, max);
 for (SSTableReader sstable : getLevel(i))

[1/2] cassandra git commit: Avoid potential overlap in LCS with single-partition sstables

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 09ccf60ed - e2ad784c7


Avoid potential overlap in LCS with single-partition sstables

Patch by marcuse; reviewed by yukim for CASSANDRA-9322


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/917068dc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/917068dc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/917068dc

Branch: refs/heads/cassandra-2.1
Commit: 917068dcbfb9909b5f1695dc18e9a7b250264b42
Parents: 58de86b
Author: Marcus Eriksson marc...@apache.org
Authored: Thu May 7 13:56:51 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Fri May 8 10:40:15 2015 +0200

--
 CHANGES.txt  | 1 +
 src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 2 ++
 2 files changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/917068dc/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8473777..1540d28 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.15:
+ * Avoid potential overlap in LCS with single-partition sstables 
(CASSANDRA-9322)
  * Log warning message when a table is queried before the schema has fully
propagated (CASSANDRA-9136)
  * Overload SecondaryIndex#indexes to accept the column definition 
(CASSANDRA-9314)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/917068dc/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index 867b5ce..81c65e4 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -353,6 +353,8 @@ public class LeveledManifest
 if (max == null || candidate.last.compareTo(max)  0)
 max = candidate.last;
 }
+if (min == null || max == null || min.equals(max)) // 
single partition sstables - we cannot include a high level sstable.
+return candidates;
 SetSSTableReader compacting = 
cfs.getDataTracker().getCompacting();
 RangeRowPosition boundaries = new Range(min, max);
 for (SSTableReader sstable : getLevel(i))

[jira] [Commented] (CASSANDRA-8574) Gracefully degrade SELECT when there are lots of tombstones

2015-05-08 Thread Christian Spriegel (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534110#comment-14534110
 ] 

Christian Spriegel commented on CASSANDRA-8574:
---

[~ztyx] TOE = TombstoneOverwhelmingException

 Gracefully degrade SELECT when there are lots of tombstones
 ---

 Key: CASSANDRA-8574
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8574
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jens Rantil
 Fix For: 3.x


 *Background:* There's lots of tooling out there to do BigData analysis on 
 Cassandra clusters. Examples are Spark and Hadoop, which is offered by DSE. 
 The problem with both of these so far, is that a single partition key with 
 too many tombstones can make the query job fail hard.
 The described scenario happens despite the user setting a rather small 
 FetchSize. I assume this is a common scenario if you have larger rows.
 *Proposal:* To allow a CQL SELECT to gracefully degrade to only return a 
 smaller batch of results if there are too many tombstones. The tombstones are 
 ordered according to clustering key and one should be able to page through 
 them. Potentially:
 SELECT * FROM mytable LIMIT 1000 TOMBSTONES;
 would page through maximum 1000 tombstones, _or_ 1000 (CQL) rows.
 I understand that this obviously would degrade performance, but it would at 
 least yield a result.
 *Additional comment:* I haven't dug into Cassandra code, but conceptually I 
 guess this would be doable. Let me know what you think.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9231) Support Routing Key as part of Partition Key

2015-05-08 Thread Sylvain Lebresne (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534202#comment-14534202
 ] 

Sylvain Lebresne commented on CASSANDRA-9231:
-

What I'm talking about is basically the idea of CASSANDRA-5054. Or to put it 
another way, we could use a function like:
{noformat}
CREATE FUNCTION myTokenFct(a int, b int) RETURNS bigint AS 
$$
long high = murmur3(a);
long low = murmur3(b);
return (high  0x) | (low  0x);
$$;
{noformat}
The goal being to make it likely that partitions with the same value for {{a}} 
are on a small amount of nodes but without forcing everything on the same node 
(the latter having a fair amount of foot-shooting potential). But that's really 
just an example. You could imagine to actually have a specific table that is 
ordered (in a predictable way) without having to use {{ByteOrderPartitioner}} 
for the whole cluster:
{noformat}
CREATE FUNCTION myOrderedTokenFct(a bigint) RETURNS bigint AS 'return a';
CREATE TABLE t (
   a int PRIMARY KEY,
   b text,
   c text
) with tokenizer=myOrderedTokenFct;
{noformat}

Basically, this gets you very close to a per-table partitioner. The actual 
partitioner would just define the domain of the tokens and how they sort, but 
the actual computation would be per-table. And this for very, very little 
change to the syntax and barely more complexity code-wise than the routing 
key idea.

Of course, this will be an advanced feature that people should use at their own 
risk.  But that's true of the routing key idea too: we'd better label it as 
an advanced feature or I'm certain people will misuse it and shoot themselves 
in the foot more often than not. This is also why I'm not too worried about the 
drivers parts: it's simple to say that if you use a custom token function, 
which will be rare in the first place, then you have to provide it to the 
driver too to get token awareness (which is not saying that this isn't a small 
downside, but it's a very small one in practice and given the context).

Perhaps more importantly, I think the function idea is conceptually *simpler* 
than the routing key idea. All that you basically have to say is that we allow 
you to define the {{token}} function on a per-table basis, the exact same 
function that already exists and can be used in {{SELECT}}.

While the routing key concept (or whatever name we would pick) is imo more 
confusing. You have to explain that on top of the _primary key_ having a 
subpart that is the _partition key_, you also have a subpart of the latter 
which is now the _routing key_. And how do you define what the _partition key_ 
is now in simple terms? Well, I don't know, because once you have a routing key 
that is different from the partition key, the partition key start to be kind of 
an implementation detail. It's the thing that don't really determine where 
the row is distributed, but is not part of the clustering so you can't query it 
like a clustering column because ... because?

Honestly, allowing to provide custom {{token}} function per table is 1) more 
powerful and 2) imo way more easy to explain conceptually and this without 
fuzzing existing concept. So I'm a -1 on the routing key concept unless I'm 
proved that the custom {{token}} function idea doesn't work, is substantially 
more complex to implement or has fundamental flaws I have missed. I would hate 
to add the routing key idea to realize that some other user has a clever 
routing idea that is just not handled by the routing key (and having to add 
some new custom concept).

bq. the distinct concept of token (which is more an implementation detail, 
IMO)

Your opinion are your own, but the token is most definitively *not* an 
implementation detail since 1) we have a {{token}} function in CQL to compute 
it and 2) we reference it all the time in the documentation, have scores of 
options that mention it, it's exposed by drivers, etc... Actually, the fact 
that we would use the token concept rather than adding a new custom one is part 
of why I'm convinced it's conceptually simpler: everyone that knows Cassandra 
knows of tokens.


 Support Routing Key as part of Partition Key
 

 Key: CASSANDRA-9231
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9231
 Project: Cassandra
  Issue Type: Wish
  Components: Core
Reporter: Matthias Broecheler
 Fix For: 3.x


 Provide support for sub-dividing the partition key into a routing key and a 
 non-routing key component. Currently, all columns that make up the partition 
 key of the primary key are also routing keys, i.e. they determine which nodes 
 store the data. This proposal would give the data modeler the ability to 
 designate only a subset of the columns that comprise the partition

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534260#comment-14534260
 ] 

Benedict commented on CASSANDRA-9318:
-

bq. In my mind in-flight means that until the response is sent back to the 
client the request counts against the in-flight limit.
bq. We already keep the original request around until we either get acks from 
all replicas, or they time out (and we write a hint).

Perhaps we need to clarify more clearly what this ticket is proposing. I 
understood it to mean what Ariel commented here, whereas it sounds like 
Jonathan is suggesting we simply prune our ExpiringMap based on bytes tracked 
as well as time?

The ExpiringMap requests are already in-flight and cannot be cancelled, so 
their effect on other nodes cannot be rescinded, and imposing a limit does not 
stop us issuing more requests to the nodes in the cluster that are failing to 
keep up and respond to us. It _might_ be sensible if introducing more 
aggressive shedding at processing nodes to also shed the response handlers more 
aggressively locally, but I'm not convinced it would have a significant impact 
on cluster health by itself; cluster instability spreads out from problematic 
nodes, and this scheme still permits us to inundate those nodes with requests 
they cannot keep up with.

Alternatively, the approach of forbidding new requests if you have items in the 
ExpiringMap causes the collapse of other nodes to spread throughout the 
cluster, as rapidly (especially on small clusters) all requests on the system 
are destined for the collapsing node, and every coordinator stops accepting 
requests. The system seizes up, and that node is still failing since it's got 
requests from the entire cluster queued up with it. In general on the 
coordinator there's no way of distinguishing between a failed node, network 
partition, or just struggling, so we don't know if we should wait.

Some mix of the two might be possible, if we were to wait while a node is just 
slow, then drop our response handlers for the node if it's marked as down. This 
latter may not be a bad thing to do anyway, but I would not want to depend on 
this behaviour to maintain our precious A

It still seems the simplest and most robust solution is to make our work queues 
leaky, since this insulates the processing nodes from cluster-wide inundation, 
which the coordinator approach cannot (even with the loss of A and cessation 
of processing, there is a whole cluster vs a potentially single node; with hash 
partition it doesn't take long for all processing to begin involving a single 
failing node). We can do this just on the number of requests and still be much 
better than we are currently. 

We could also pair this with coordinator-level dropping of handlers for down 
nodes, and above a threshold. This latter, though, could result in widespread 
uncoordinated dropping of requests, which may leave us open to a multiplying 
effect of cluster overload, with each node dropping different requests, 
possibly leading to only a tiny fraction of requests being serviced to their 
required CL across the cluster. I'm not sure how we can best model this risk, 
or avoid it without notifying coordinators of the drop of a message, and I 
don't see that being delivered for 2.1

 Bound the number of in-flight requests at the coordinator
 -

 Key: CASSANDRA-9318
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
 Project: Cassandra
  Issue Type: Improvement
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 2.1.x


 It's possible to somewhat bound the amount of load accepted into the cluster 
 by bounding the number of in-flight requests and request bytes.
 An implementation might do something like track the number of outstanding 
 bytes and requests and if it reaches a high watermark disable read on client 
 connections until it goes back below some low watermark.
 Need to make sure that disabling read on the client connection won't 
 introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9231) Support Routing Key as part of Partition Key

[
https://issues.apache.org/jira/browse/CASSANDRA-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534220#comment-14534220
]

Benedict commented on CASSANDRA-9231:
-

The token is an implementation detail for the _concept_ of routing, or fair
distribution. Perhaps we have different definitions of implementation detail,
but I stand by it under my nomenclature, and the presence of a {{token}}
function doesn't really change that.

My point is that from a data modelling perspective, being able to define the
values on which you distribute is the concept you care about. The token that is
ultimately used to deliver that is not important for you when modelling your
system.

Support Routing Key as part of Partition Key

Key: CASSANDRA-9231
URL: https://issues.apache.org/jira/browse/CASSANDRA-9231
Project: Cassandra
Issue Type: Wish
Components: Core
Reporter: Matthias Broecheler
Fix For: 3.x

Provide support for sub-dividing the partition key into a routing key and a
non-routing key component. Currently, all columns that make up the partition
key of the primary key are also routing keys, i.e. they determine which nodes
store the data. This proposal would give the data modeler the ability to
designate only a subset of the columns that comprise the partition key to be
routing keys. The non-routing key columns of the partition key identify the
partition but are not used to determine where to store the data.
Consider the following example table definition:
CREATE TABLE foo (
a int,
b int,
c int,
d int,
PRIMARY KEY (([a], b), c ) );
(a,b) is the partition key, c is the clustering key, and d is just a column.
In addition, the square brackets identify the routing key as column a. This
means that only the value of column a is used to determine the node for data
placement (i.e. only the value of column a is murmur3 hashed to compute the
token). In addition, column b is needed to identify the partition but does
not influence the placement.
This has the benefit that all rows with the same routing key (but potentially
different non-routing key columns of the partition key) are stored on the
same node and that knowledge of such co-locality can be exploited by
applications build on top of Cassandra.
Currently, the only way to achieve co-locality is within a partition.
However, this approach has the limitations that: a) there are theoretical and
(more importantly) practical limitations on the size of a partition and b)
rows within a partition are ordered and an index is build to exploit such
ordering. For large partitions that overhead is significant if ordering isn't
needed.
In other words, routing keys afford a simple means to achieve scalable
node-level co-locality without ordering while clustering keys afford
page-level co-locality with ordering. As such, they address different
co-locality needs giving the data modeler the flexibility to choose what is
needed for their application.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9326) Seeing tombstone warning message

2015-05-08 Thread Aleksey Yeschenko (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534360#comment-14534360
 ] 

Aleksey Yeschenko commented on CASSANDRA-9326:
--

CASSANDRA-7272 will eventually help you.

 Seeing tombstone warning message
 

 Key: CASSANDRA-9326
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9326
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: srinivasu gottipati
Priority: Minor
 Fix For: 2.0.x


 We deleted data for some of the rows in one of the column families. 
 After that we ran repair on all nodes, and followed by reducing 
 gc_grace_seconds that way compaction can remove all the tombstones upon 
 expiry of gc_grace_seconds time. 
 When we are querying the data now, seeing the following errors:
 WARN [ReadStage:1142] 2015-05-06 17:50:53,602 SliceQueryFilter.java (line 
 231) Read 1 live and 1487 tombstoned cells in  (see 
 tombstone_warn_threshold). 10001 columns was requested, slices=[-], 
 delInfo={deletedAt=-9223372036854775808, localDeletion=2147483647} 
 We deleted this data while back and for sure gc_grace_seconds is elapsed long 
 time back and we use leveled compaction. In the above, errors, localDeletion 
 points to some time in future (MAX INT VALUE) and that could be the reason 
 these are n't being purged. Any help (or) workaround is appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534260#comment-14534260
 ] 

Benedict edited comment on CASSANDRA-9318 at 5/8/15 11:24 AM:
--

bq. In my mind in-flight means that until the response is sent back to the 
client the request counts against the in-flight limit.
bq. We already keep the original request around until we either get acks from 
all replicas, or they time out (and we write a hint).

Perhaps we need to clarify more clearly what this ticket is proposing. I 
understood it to mean what Ariel commented here, whereas it sounds like 
Jonathan is suggesting we simply prune our ExpiringMap based on bytes tracked 
as well as time?

The ExpiringMap requests are already in-flight and cannot be cancelled, so 
their effect on other nodes cannot be rescinded, and imposing a limit does not 
stop us issuing more requests to the nodes in the cluster that are failing to 
keep up and respond to us. It _might_ be sensible if introducing more 
aggressive shedding at processing nodes to also shed the response handlers more 
aggressively locally, but I'm not convinced it would have a significant impact 
on cluster health by itself; cluster instability spreads out from problematic 
nodes, and this scheme still permits us to inundate those nodes with requests 
they cannot keep up with.

Alternatively, the approach of forbidding new requests if you have items in the 
ExpiringMap causes the collapse of other nodes to spread throughout the 
cluster, as rapidly (especially on small clusters) all requests on the system 
are destined for the collapsing node, and every coordinator stops accepting 
requests. The system seizes up, and that node is still failing since it's got 
requests from the entire cluster queued up with it. In general on the 
coordinator there's no way of distinguishing between a failed node, network 
partition, or just struggling, so we don't know if we should wait.

Some mix of the two might be possible, if we were to wait while a node is just 
slow, then drop our response handlers for the node if it's marked as down. This 
latter may not be a bad thing to do anyway, but I would not want to depend on 
this behaviour to maintain our precious A

It still seems the simplest and most robust solution is to make our work queues 
leaky, since this insulates the processing nodes from cluster-wide inundation, 
which the coordinator approach cannot (even with the loss of A and cessation 
of processing, there is a whole cluster vs a potentially single node; with hash 
partition it doesn't take long for all processing to begin involving a single 
failing node). We can do this just on the number of requests and still be much 
better than we are currently. 

We could also pair this with coordinator-level dropping of handlers for down 
nodes, and above a size/count threshold. This latter, though, could result in 
widespread uncoordinated dropping of requests, which may leave us open to a 
multiplying effect of cluster overload, with each node dropping different 
requests, possibly leading to only a tiny fraction of requests being serviced 
to their required CL across the cluster. I'm not sure how we can best model 
this risk, or avoid it without notifying coordinators of the drop of a message, 
and I don't see that being delivered for 2.1


was (Author: benedict):
bq. In my mind in-flight means that until the response is sent back to the 
client the request counts against the in-flight limit.
bq. We already keep the original request around until we either get acks from 
all replicas, or they time out (and we write a hint).

Perhaps we need to clarify more clearly what this ticket is proposing. I 
understood it to mean what Ariel commented here, whereas it sounds like 
Jonathan is suggesting we simply prune our ExpiringMap based on bytes tracked 
as well as time?

The ExpiringMap requests are already in-flight and cannot be cancelled, so 
their effect on other nodes cannot be rescinded, and imposing a limit does not 
stop us issuing more requests to the nodes in the cluster that are failing to 
keep up and respond to us. It _might_ be sensible if introducing more 
aggressive shedding at processing nodes to also shed the response handlers more 
aggressively locally, but I'm not convinced it would have a significant impact 
on cluster health by itself; cluster instability spreads out from problematic 
nodes, and this scheme still permits us to inundate those nodes with requests 
they cannot keep up with.

Alternatively, the approach of forbidding new requests if you have items in the 
ExpiringMap causes the collapse of other nodes to spread throughout the 
cluster, as rapidly (especially on small clusters) all requests on the system 
are destined for the collapsing node, and every coordinator stops accepting 
requests. The system seizes up, and that node

[jira] [Updated] (CASSANDRA-9333) Edge case - Empty of blank password for JMX authentication not handled properly in nodetool commands

2015-05-08 Thread Philip Thompson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-9333:
---
Fix Version/s: 2.1.x

 Edge case - Empty of blank password for JMX authentication not handled 
 properly in nodetool commands
 

 Key: CASSANDRA-9333
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9333
 Project: Cassandra
  Issue Type: Bug
  Components: Core, Tools
 Environment: Apache Cassandra 2.1.2
Reporter: Sumod Pawgi
Priority: Minor
  Labels: security
 Fix For: 2.1.x


 While setting up JMX authentication for Apache Cassandra, if we set the 
 password blank (in the file - jmxremote.password), nodetool commands do not 
 work
 example creds are cassandra cassandra. In this case, for a secured cluster, 
 we run the nodetool command as - nodetool -u cassandra -pw cassandra status
 But if the password is kept as blank then we cannot execute nodetool command. 
 However, I believe that if a third party software used JMX authentication via 
 API, then they can use blank password for the operations. So this behavior 
 needs to be clarified and be consistent for this edge case scenario.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9332) NPE when creating column family via thrift


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-9332:
--
Assignee: Tyler Hobbs

 NPE when creating column family via thrift
 --

 Key: CASSANDRA-9332
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9332
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Oracle JDK 1.7.0_79
 Casandra 2.0.6 in single node
 Ubuntu 14.04
Reporter: Colin Kuo
Assignee: Tyler Hobbs
  Labels: thrift
 Fix For: 2.0.x


 When triggering unit test testAddDropColumnFamily() in 
 https://github.com/hector-client/hector/blob/master/core/src/test/java/me/prettyprint/cassandra/service/CassandraClusterTest.java
  
 It occurs NPE when using *Cassandra 2.0.6* or later version.
 {noformat}
 11:42:39,173 [Thrift:1] ERROR CustomTThreadPoolServer:212 - Error occurred 
 during processing of message.
 java.lang.NullPointerException
   at org.apache.cassandra.db.RowMutation.add(RowMutation.java:112)
   at 
 org.apache.cassandra.service.MigrationManager.addSerializedKeyspace(MigrationManager.java:265)
   at 
 org.apache.cassandra.service.MigrationManager.announceNewColumnFamily(MigrationManager.java:213)
   at 
 org.apache.cassandra.thrift.CassandraServer.system_add_column_family(CassandraServer.java:1521)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:4300)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:4284)
   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
   at 
 org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:194)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
 {noformat}
 It seems that was introduced by fix of CASSANDRA-5631.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: use cross-platform %n

use cross-platform %n


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/135151e5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/135151e5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/135151e5

Branch: refs/heads/trunk
Commit: 135151e53e8e9dce7b80ada74edbabaf67a45f43
Parents: e2ad784
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri May 8 09:29:31 2015 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri May 8 09:29:31 2015 -0500

--
 src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/135151e5/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java
--
diff --git a/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java 
b/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java
index 22c57f7..9057998 100644
--- a/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java
+++ b/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java
@@ -106,7 +106,7 @@ public class StandaloneUpgrader
 if (!options.keepSource)
 {
 // Remove the sstable (it's been copied by upgrade)
-System.out.format(Deleting table %s.\n, 
sstable.descriptor.baseFilename());
+System.out.format(Deleting table %s.%n, 
sstable.descriptor.baseFilename());
 sstable.markObsolete();
 sstable.selfRef().release();
 }

[jira] [Commented] (CASSANDRA-6477) Global indexes

2015-05-08 Thread Aleksey Yeschenko (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534635#comment-14534635
 ] 

Aleksey Yeschenko commented on CASSANDRA-6477:
--

bq.  having the underlying table being directly exposed make it clear that you 
can only access what you denormalize (this is obvious for MV, less with a GI) 
and simplify the read path (the CQL parts of the read path).

That's one of the main reasons I want this to be fully MV, too. So, reiterating 
my +1.

 Global indexes
 --

 Key: CASSANDRA-6477
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Carl Yeksigian
  Labels: cql
 Fix For: 3.x


 Local indexes are suitable for low-cardinality data, where spreading the 
 index across the cluster is a Good Thing.  However, for high-cardinality 
 data, local indexes require querying most nodes in the cluster even if only a 
 handful of rows is returned.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] cassandra git commit: use cross-platform %n

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 e2ad784c7 - 135151e53
  refs/heads/trunk 3ef932af7 - c7db18be8


use cross-platform %n


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/135151e5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/135151e5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/135151e5

Branch: refs/heads/cassandra-2.1
Commit: 135151e53e8e9dce7b80ada74edbabaf67a45f43
Parents: e2ad784
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri May 8 09:29:31 2015 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri May 8 09:29:31 2015 -0500

--
 src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/135151e5/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java
--
diff --git a/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java 
b/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java
index 22c57f7..9057998 100644
--- a/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java
+++ b/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java
@@ -106,7 +106,7 @@ public class StandaloneUpgrader
 if (!options.keepSource)
 {
 // Remove the sstable (it's been copied by upgrade)
-System.out.format(Deleting table %s.\n, 
sstable.descriptor.baseFilename());
+System.out.format(Deleting table %s.%n, 
sstable.descriptor.baseFilename());
 sstable.markObsolete();
 sstable.selfRef().release();
 }

[jira] [Resolved] (CASSANDRA-9331) Too many Compaction pending Tasks

[
https://issues.apache.org/jira/browse/CASSANDRA-9331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Ellis resolved CASSANDRA-9331.
---
Resolution: Invalid
Fix Version/s: (was: 2.1.x)

Please try the users list for troubleshooting help.

Too many Compaction pending Tasks
-

Key: CASSANDRA-9331
URL: https://issues.apache.org/jira/browse/CASSANDRA-9331
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Ubuntu
Reporter: Paddy Krishnamoorthy
Priority: Critical

We have a 9 node cluster that has been running for about 8 months now. Lately
the selects seems to take longer. Also I noticed that the compaction jobs are
always running. Last time when I checked every node has average 600+ pending
compaction tasks. Two days ago I posted this issue on IRC chat and someone
recommended to upgrade from 2.1.2 to current release. I updated one of the
nodes to 2.1.4 and I still noticed that newly added node has about 600+
compaction tasks. We also noticed that nodetool rebuild takes more than
24 hours to complete. Wondering if there is an issue with our configuration
or is it a bug?
Hardware and other config details:
The current storage size per node is approximately 100GB.
All our disks are spinning disks.
The servers are hosted in out own data centre.
We use VNODES and num_tokens set to 256
Here is the out from desc table daily_index command for one of our table
that has 10s of millions of records.
CREATE TABLE daily_indexes.daily_index ( indexday int, pid text, docid text,
author text, e mapint, decimal, hit boolean, p decimal, s int,
PRIMARY KEY ((indexday, pid), docid) ) WITH CLUSTERING ORDER BY (docid ASC)
AND bloom_filter_fp_chance = 0.01 AND caching = '{keys:ALL,
rows_per_partition:NONE}' AND comment = '' AND compaction =
{'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.LeveledCompactionStrategy',
'max_threshold': '32'} AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'} AND
dclocal_read_repair_chance = 0.1 AND default_time_to_live = 0 AND
gc_grace_seconds = 86400 AND max_index_interval = 2048 AND
memtable_flush_period_in_ms = 0 AND min_index_interval = 128 AND
read_repair_chance = 0.0 AND speculative_retry = '99.0PERCENTILE';

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9286) Add Keyspace/Table details to CollectionType.java error message

2015-05-08 Thread Carl Yeksigian (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-9286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Carl Yeksigian updated CASSANDRA-9286:
--
Attachment: 9286-2.1-v2.txt
9286-2.0-v2.txt

Since we aren't throwing an exception, we won't get an invalid message; we have
to look at the logs to find the exception. I'll look into adding a dtest to
test this.

Attaching update patches that use 'table' instead of 'CF'.

Add Keyspace/Table details to CollectionType.java error message
---

Key: CASSANDRA-9286
URL: https://issues.apache.org/jira/browse/CASSANDRA-9286
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: sequoyha pelletier
Assignee: Carl Yeksigian
Priority: Minor
Fix For: 2.1.x, 2.0.x

Attachments: 9286-2.0-v2.txt, 9286-2.0.txt, 9286-2.1-v2.txt,
9286-2.1.txt

The error message for too many element in a collection does not give keyspace
or column family information. This makes it a pain point to try to determine
which table is the offending table.
Example Error message:
{noformat}
ERROR [Native-Transport-Requests:809453] 2015-04-23 22:48:21,189
CollectionType.java (line 116) Detected collection with 136234 elements, more
than the 65535 limit. Only the first 65535 elements will be returned to the
client. Please see http://cassandra.apache.org/doc/cql3/CQL.html#collections
for more details.
{noformat}
Currently, to try to pinpoint the table in question. We need to trace all
requests and then try to match up the timestamps in the CQL tracing session
with the log timestamps to try and match. If prepared statements are used,
this is a dead end due to the logged tracing information missing the query.
In which case, we have to look at other 3rd party methods for capturing the
queries to try and match up. This is extremely tedious when many tables have
collections and a high number of ops against them.
Requesting that the error contain the keyspace.table name in the error
message.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9334) Returning null from a trigger does not abort the write

2015-05-08 Thread Brandon Williams (JIRA)

Brandon Williams created CASSANDRA-9334:
---

 Summary: Returning null from a trigger does not abort the write
 Key: CASSANDRA-9334
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9334
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Brandon Williams
Assignee: Sam Tunnicliffe
 Fix For: 2.0.x


The javadoc for ITrigger seems to indicate that returning null means no 
mutations for the write will be applied at all:

{noformat}
 * Called exactly once per CF update, returned mutations are atomically 
updated.
 *
 * @param key - Row Key for the update.
 * @param update - Update received for the CF
 * @return modifications to be applied, null if no action to be performed.
{noformat}

and indeed, TriggerExecutor.execute seems to have code for this:

{noformat}
if (augmentedMutations == null)
return null;
{noformat}

but it never fires.  In testing with a trigger that always returns null, the 
original mutation is still applied.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6477) Global indexes


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534585#comment-14534585
 ] 

Jonathan Ellis commented on CASSANDRA-6477:
---

It also means you don't give up token-aware routing (CASSANDRA-8517).

 Global indexes
 --

 Key: CASSANDRA-6477
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Carl Yeksigian
  Labels: cql
 Fix For: 3.x


 Local indexes are suitable for low-cardinality data, where spreading the 
 index across the cluster is a Good Thing.  However, for high-cardinality 
 data, local indexes require querying most nodes in the cluster even if only a 
 handful of rows is returned.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9097) Repeated incremental nodetool repair results in failed repairs due to running anticompaction

2015-05-08 Thread Marcus Eriksson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534880#comment-14534880
 ] 

Marcus Eriksson commented on CASSANDRA-9097:


+1

 Repeated incremental nodetool repair results in failed repairs due to running 
 anticompaction
 

 Key: CASSANDRA-9097
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9097
 Project: Cassandra
  Issue Type: Bug
Reporter: Gustav Munkby
Assignee: Yuki Morishita
Priority: Minor
 Fix For: 2.1.x

 Attachments: 0001-Wait-for-anticompaction-to-finish.patch


 I'm trying to synchronize incremental repairs over multiple nodes in a 
 Cassandra cluster, and it does not seem to easily achievable.
 In principle, the process iterates through the nodes of the cluster and 
 performs `nodetool -h $NODE repair --incremental`, but that sometimes fails 
 on subsequent nodes. The reason for failing seems to be that the repair 
 returns as soon as the repair and the _local_ anticompaction has completed, 
 but does not guarantee that remote anticompactions are complete. If I 
 subsequently try to issue another repair command, they fail to start (and 
 terminate with failure after about one minute). It usually isn't a problem, 
 as the local anticompaction typically involves as much (or more) data as the 
 remote ones, but sometimes not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: clarify 6757/7027 relationship

clarify 6757/7027 relationship


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ef8dc671
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ef8dc671
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ef8dc671

Branch: refs/heads/trunk
Commit: ef8dc6715655c3d37583f2f6477b71449809ec55
Parents: 135151e
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri May 8 12:08:36 2015 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri May 8 12:08:36 2015 -0500

--
 CHANGES.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ef8dc671/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 37f0117..2b7a355 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -779,7 +779,7 @@ Merged from 1.2:
  * Fix broken paging state with prepared statement (CASSANDRA-7120)
  * Fix IllegalArgumentException in CqlStorage (CASSANDRA-7287)
  * Allow nulls/non-existant fields in UDT (CASSANDRA-7206)
- * Backport Thrift MultiSliceRequest (CASSANDRA-7027)
+ * Add Thrift MultiSliceRequest (CASSANDRA-6757, CASSANDRA-7027)
  * Handle overlapping MultiSlices (CASSANDRA-7279)
  * Fix DataOutputTest on Windows (CASSANDRA-7265)
  * Embedded sets in user defined data-types are not updating (CASSANDRA-7267)

[1/3] cassandra git commit: clarify 6757/7027 relationship

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 135151e53 - ef8dc6715
  refs/heads/trunk f45e4a556 - 86f481ef9


clarify 6757/7027 relationship


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ef8dc671
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ef8dc671
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ef8dc671

Branch: refs/heads/cassandra-2.1
Commit: ef8dc6715655c3d37583f2f6477b71449809ec55
Parents: 135151e
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri May 8 12:08:36 2015 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri May 8 12:08:36 2015 -0500

--
 CHANGES.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ef8dc671/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 37f0117..2b7a355 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -779,7 +779,7 @@ Merged from 1.2:
  * Fix broken paging state with prepared statement (CASSANDRA-7120)
  * Fix IllegalArgumentException in CqlStorage (CASSANDRA-7287)
  * Allow nulls/non-existant fields in UDT (CASSANDRA-7206)
- * Backport Thrift MultiSliceRequest (CASSANDRA-7027)
+ * Add Thrift MultiSliceRequest (CASSANDRA-6757, CASSANDRA-7027)
  * Handle overlapping MultiSlices (CASSANDRA-7279)
  * Fix DataOutputTest on Windows (CASSANDRA-7265)
  * Embedded sets in user defined data-types are not updating (CASSANDRA-7267)

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/86f481ef
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/86f481ef
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/86f481ef

Branch: refs/heads/trunk
Commit: 86f481ef93bc35649c77fd26ad6da337fc54db6a
Parents: f45e4a5 ef8dc67
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri May 8 12:08:46 2015 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri May 8 12:08:46 2015 -0500

--
 CHANGES.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/86f481ef/CHANGES.txt
--

[jira] [Resolved] (CASSANDRA-6757) SliceQueryFilter can accept multiple ColumnSlice but not exposed to thrift


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-6757.
---
   Resolution: Fixed
Fix Version/s: 2.1 rc1

My mistake, this was backported in separate ticket CASSANDRA-7027.  Reverted my 
revert, and updated CHANGES to clarify.

 SliceQueryFilter can accept multiple ColumnSlice but not exposed to thrift
 --

 Key: CASSANDRA-6757
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6757
 Project: Cassandra
  Issue Type: New Feature
Reporter: Edward Capriolo
Assignee: Edward Capriolo
 Fix For: 2.1 rc1

 Attachments: Cassandra-6757.1.diff


 Thrift has getSlice() which takes a key and a predicate. SliceQueryFilter can 
 accept multiple ColumnSlice(s). Users should have access to this capability 
 already found in Cassandra.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7317) Repair range validation is too strict


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534858#comment-14534858
 ] 

Kenneth Failbus commented on CASSANDRA-7317:


Folks,

I am seeing this error again in 2.0.9 release. I have vnodes in my cluster 
enabled.

 Repair range validation is too strict
 -

 Key: CASSANDRA-7317
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7317
 Project: Cassandra
  Issue Type: Bug
Reporter: Nick Bailey
Assignee: Yuki Morishita
 Fix For: 2.0.9

 Attachments: 7317-2.0.txt, Untitled Diagram(1).png


 From what I can tell the calculation (using the -pr option) and validation of 
 tokens for repairing ranges is broken. Or at least should be improved. Using 
 an example with ccm:
 Nodetool ring:
 {noformat}
 Datacenter: dc1
 ==
 AddressRackStatus State   LoadOwns
 Token
   -10
 127.0.0.1  r1  Up Normal  188.96 KB   50.00%  
 -9223372036854775808
 127.0.0.2  r1  Up Normal  194.77 KB   50.00%  -10
 Datacenter: dc2
 ==
 AddressRackStatus State   LoadOwns
 Token
   0
 127.0.0.4  r1  Up Normal  160.58 KB   0.00%   
 -9223372036854775798
 127.0.0.3  r1  Up Normal  139.46 KB   0.00%   0
 {noformat}
 Schema:
 {noformat}
 CREATE KEYSPACE system_traces WITH replication = {
   'class': 'NetworkTopologyStrategy',
   'dc2': '2',
   'dc1': '2'
 };
 {noformat}
 Repair -pr:
 {noformat}
 [Nicks-MacBook-Pro:21:35:58 cassandra-2.0] cassandra$ bin/nodetool -p 7100 
 repair -pr system_traces
 [2014-05-28 21:36:01,977] Starting repair command #12, repairing 1 ranges for 
 keyspace system_traces
 [2014-05-28 21:36:02,207] Repair session f984d290-e6d9-11e3-9edc-5f8011daec21 
 for range (0,-9223372036854775808] finished
 [2014-05-28 21:36:02,207] Repair command #12 finished
 [Nicks-MacBook-Pro:21:36:02 cassandra-2.0] cassandra$ bin/nodetool -p 7200 
 repair -pr system_traces
 [2014-05-28 21:36:14,086] Starting repair command #1, repairing 1 ranges for 
 keyspace system_traces
 [2014-05-28 21:36:14,406] Repair session 00bd45b0-e6da-11e3-98fc-5f8011daec21 
 for range (-9223372036854775798,-10] finished
 [2014-05-28 21:36:14,406] Repair command #1 finished
 {noformat}
 Note that repairing both nodes in dc1, leaves very small ranges unrepaired. 
 For example (-10,0]. Repairing the 'primary range' in dc2 will repair those 
 small ranges. Maybe that is the behavior we want but it seems 
 counterintuitive.
 The behavior when manually trying to repair the full range of 127.0.0.01 
 definitely needs improvement though.
 Repair command:
 {noformat}
 [Nicks-MacBook-Pro:21:50:44 cassandra-2.0] cassandra$ bin/nodetool -p 7100 
 repair -st -10 -et -9223372036854775808 system_traces
 [2014-05-28 21:50:55,803] Starting repair command #17, repairing 1 ranges for 
 keyspace system_traces
 [2014-05-28 21:50:55,804] Starting repair command #17, repairing 1 ranges for 
 keyspace system_traces
 [2014-05-28 21:50:55,804] Repair command #17 finished
 [Nicks-MacBook-Pro:21:50:56 cassandra-2.0] cassandra$ echo $?
 1
 {noformat}
 system.log:
 {noformat}
 ERROR [Thread-96] 2014-05-28 21:40:05,921 StorageService.java (line 2621) 
 Repair session failed:
 java.lang.IllegalArgumentException: Requested range intersects a local range 
 but is not fully contained in one; this would lead to imprecise repair
 {noformat}
 * The actual output of the repair command doesn't really indicate that there 
 was an issue. Although the command does return with a non zero exit status.
 * The error here is invisible if you are using the synchronous jmx repair 
 api. It will appear as though the repair completed successfully.
 * Personally, I believe that should be a valid repair command. For the 
 system_traces keyspace, 127.0.0.1 is responsible for this range (and I would 
 argue the 'primary range' of the node).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-6477) Materialized Views

2015-05-08 Thread Aleksey Yeschenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-6477:
-
Summary: Materialized Views  (was: Global indexes)

 Materialized Views
 --

 Key: CASSANDRA-6477
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Carl Yeksigian
  Labels: cql
 Fix For: 3.x


 Local indexes are suitable for low-cardinality data, where spreading the 
 index across the cluster is a Good Thing.  However, for high-cardinality 
 data, local indexes require querying most nodes in the cluster even if only a 
 handful of rows is returned.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-6757) SliceQueryFilter can accept multiple ColumnSlice but not exposed to thrift


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reopened CASSANDRA-6757:
---

 SliceQueryFilter can accept multiple ColumnSlice but not exposed to thrift
 --

 Key: CASSANDRA-6757
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6757
 Project: Cassandra
  Issue Type: New Feature
Reporter: Edward Capriolo
Assignee: Edward Capriolo
 Attachments: Cassandra-6757.1.diff


 Thrift has getSlice() which takes a key and a predicate. SliceQueryFilter can 
 accept multiple ColumnSlice(s). Users should have access to this capability 
 already found in Cassandra.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-6477) Materialized Views (was: Global Indexes)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-6477:
--
Summary: Materialized Views (was: Global Indexes)  (was: Materialized Views)

 Materialized Views (was: Global Indexes)
 

 Key: CASSANDRA-6477
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Carl Yeksigian
  Labels: cql
 Fix For: 3.x


 Local indexes are suitable for low-cardinality data, where spreading the 
 index across the cluster is a Good Thing.  However, for high-cardinality 
 data, local indexes require querying most nodes in the cluster even if only a 
 handful of rows is returned.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9334) Returning null from a trigger does not abort the write

2015-05-08 Thread Brandon Williams (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534817#comment-14534817
 ] 

Brandon Williams commented on CASSANDRA-9334:
-

I can think of a few instances where it could be useful.  For instance, if you 
want to run a multi-tenant cluster where you don't necessarily control the 
applications, but you want to forbid a 'badwords' list in any field.  Or your 
trigger takes some action and if that fails, you want the write to fail as 
well.  I guess to do this properly though we'd need to let augment throw IRE or 
something back to the client and break the interface in 3.x.

 Returning null from a trigger does not abort the write
 --

 Key: CASSANDRA-9334
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9334
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Brandon Williams
Assignee: Sam Tunnicliffe
 Fix For: 2.0.x


 The javadoc for ITrigger seems to indicate that returning null means no 
 mutations for the write will be applied at all:
 {noformat}
  * Called exactly once per CF update, returned mutations are atomically 
 updated.
  *
  * @param key - Row Key for the update.
  * @param update - Update received for the CF
  * @return modifications to be applied, null if no action to be performed.
 {noformat}
 and indeed, TriggerExecutor.execute seems to have code for this:
 {noformat}
 if (augmentedMutations == null)
 return null;
 {noformat}
 but it never fires.  In testing with a trigger that always returns null, the 
 original mutation is still applied.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9335) JVM_EXTRA_OPTS not getting picked up by windows startup environment

2015-05-08 Thread Philip Thompson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535025#comment-14535025
 ] 

Philip Thompson commented on CASSANDRA-9335:


+1

 JVM_EXTRA_OPTS not getting picked up by windows startup environment
 ---

 Key: CASSANDRA-9335
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9335
 Project: Cassandra
  Issue Type: Bug
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
  Labels: windows
 Fix For: 3.x, 2.1.x

 Attachments: fix_extra_opts.txt


 -$env:JVM_OPTS=$env:JVM_OPTS $JVM_EXTRA_OPTS
 +$env:JVM_OPTS=$env:JVM_OPTS $env:JVM_EXTRA_OPTS
 Missing env: on front of JVM_EXTRA_OPTS in conf\cassandra-env.ps1
 Attaching trivial patch to fix this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9335) JVM_EXTRA_OPTS not getting picked up by windows startup environment

2015-05-08 Thread Joshua McKenzie (JIRA)

Joshua McKenzie created CASSANDRA-9335:
--

 Summary: JVM_EXTRA_OPTS not getting picked up by windows startup 
environment
 Key: CASSANDRA-9335
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9335
 Project: Cassandra
  Issue Type: Bug
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
 Fix For: 3.x, 2.1.x
 Attachments: fix_extra_opts.txt

-$env:JVM_OPTS=$env:JVM_OPTS $JVM_EXTRA_OPTS
+$env:JVM_OPTS=$env:JVM_OPTS $env:JVM_EXTRA_OPTS

Missing env: on front of JVM_EXTRA_OPTS in conf\cassandra-env.ps1

Attaching trivial patch to fix this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-9335) JVM_EXTRA_OPTS not getting picked up by windows startup environment

2015-05-08 Thread Joshua McKenzie (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie resolved CASSANDRA-9335.

   Resolution: Fixed
Fix Version/s: (was: 2.1.x)
   (was: 3.x)
   2.1.6
   3.0

Committed to 2.1 and 3.0.

 JVM_EXTRA_OPTS not getting picked up by windows startup environment
 ---

 Key: CASSANDRA-9335
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9335
 Project: Cassandra
  Issue Type: Bug
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
  Labels: windows
 Fix For: 3.0, 2.1.6

 Attachments: fix_extra_opts.txt


 -$env:JVM_OPTS=$env:JVM_OPTS $JVM_EXTRA_OPTS
 +$env:JVM_OPTS=$env:JVM_OPTS $env:JVM_EXTRA_OPTS
 Missing env: on front of JVM_EXTRA_OPTS in conf\cassandra-env.ps1
 Attaching trivial patch to fix this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7317) Repair range validation is too strict


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534858#comment-14534858
 ] 

Kenneth Failbus edited comment on CASSANDRA-7317 at 5/8/15 4:49 PM:


Folks,

I am seeing this error again in 2.0.9 release. I have vnodes in my cluster 
enabled.
{code}
2015-05-08 15:01:56,021 [AntiEntropyStage:1] INFO Validator [repair 
#254edb00-f593-11e4-9397-51babce9f892] Sending completed merkle tree to 
/10.22.168.35 for CF1/Sequence
2015-05-08 15:01:58,518 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/permissions
2015-05-08 15:01:58,791 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/credentials
2015-05-08 15:01:58,980 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/users
2015-05-08 15:02:00,640 [AntiEntropyStage:1] INFO Validator [repair 
#e0d31e00-f592-11e4-993e-c9cc22925782] Sending completed merkle tree to 
/10.22.168.97 for system_auth/credentials
2015-05-08 15:02:01,345 [AntiEntropyStage:1] INFO Validator [repair 
#e0d31e00-f592-11e4-993e-c9cc22925782] Sending completed merkle tree to 
/10.22.168.97 for system_auth/users
2015-05-08 15:02:01,577 [AntiEntropyStage:1] INFO Validator [repair 
#e0d31e00-f592-11e4-993e-c9cc22925782] Sending completed merkle tree to 
/10.22.168.97 for system_auth/permissions
2015-05-08 15:02:01,753 [AntiEntropyStage:1] INFO Validator [repair 
#27dba060-f593-11e4-873b-9d346bbba08e] Sending completed merkle tree to 
/10.22.168.87 for CF1/Sequence
2015-05-08 15:02:02,622 [AntiEntropyStage:1] INFO Validator [repair 
#dba213a0-f592-11e4-b745-192986bd7af2] Sending completed merkle tree to 
/10.22.168.117 for system_auth/credentials
2015-05-08 15:02:02,873 [AntiEntropyStage:1] INFO Validator [repair 
#dba213a0-f592-11e4-b745-192986bd7af2] Sending completed merkle tree to 
/10.22.168.117 for system_auth/users
2015-05-08 15:02:03,508 [AntiEntropyStage:1] INFO Validator [repair 
#dba213a0-f592-11e4-b745-192986bd7af2] Sending completed merkle tree to 
/10.22.168.117 for system_auth/permissions
2015-05-08 15:02:03,988 [AntiEntropyStage:1] INFO Validator [repair 
#d0a2ad70-f592-11e4-a5a2-b73fe73dbe79] Sending completed merkle tree to 
/10.22.168.109 for system_auth/credentials
2015-05-08 15:02:04,759 [AntiEntropyStage:1] INFO Validator [repair 
#d0a2ad70-f592-11e4-a5a2-b73fe73dbe79] Sending completed merkle tree to 
/10.22.168.109 for system_auth/users
2015-05-08 15:02:05,066 [AntiEntropyStage:1] INFO Validator [repair 
#d0a2ad70-f592-11e4-a5a2-b73fe73dbe79] Sending completed merkle tree to 
/10.22.168.109 for system_auth/permissions
2015-05-08 15:02:05,200 [Thread-227856] ERROR StorageService Repair session 
failed:
java.lang.IllegalArgumentException: Requested range intersects a local range 
but is not fully contained in one; this would lead to imprecise repair
at 
org.apache.cassandra.service.ActiveRepairService.getNeighbors(ActiveRepairService.java:161)
at 
org.apache.cassandra.repair.RepairSession.init(RepairSession.java:130)
at 
org.apache.cassandra.repair.RepairSession.init(RepairSession.java:119)
at 
org.apache.cassandra.service.ActiveRepairService.submitRepairSession(ActiveRepairService.java:97)
at 
org.apache.cassandra.service.StorageService.forceKeyspaceRepair(StorageService.java:2628)
at 
org.apache.cassandra.service.StorageService$4.runMayThrow(StorageService.java:2564)
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.lang.Thread.run(Thread.java:744)
{code}


was (Author: kenfailbus):
Folks,

I am seeing this error again in 2.0.9 release. I have vnodes in my cluster 
enabled.
{code}
2015-05-08 15:01:56,021 [AntiEntropyStage:1] INFO Validator [repair 
#254edb00-f593-11e4-9397-51babce9f892] Sending completed merkle tree to 
/10.22.168.35 for CF1/Sequence
2015-05-08 15:01:58,518 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/permissions
2015-05-08 15:01:58,791 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/credentials
2015-05-08 15:01:58,980 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/users
2015-05-08 15:02:00,640 [AntiEntropyStage:1] INFO Validator [repair

[2/3] cassandra git commit: Revert Revert add Thrift get_multi_slice call

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f45e4a55/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
diff --git 
a/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java 
b/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
index f29805b..bf15559 100644
--- a/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
+++ b/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
@@ -248,6 +248,13 @@ public class Cassandra {
 public void truncate(String cfname) throws InvalidRequestException, 
UnavailableException, TimedOutException, org.apache.thrift.TException;
 
 /**
+ * Select multiple slices of a key in a single RPC operation
+ * 
+ * @param request
+ */
+public ListColumnOrSuperColumn get_multi_slice(MultiSliceRequest 
request) throws InvalidRequestException, UnavailableException, 
TimedOutException, org.apache.thrift.TException;
+
+/**
  * for each schema version present in the cluster, returns a list of nodes 
at that version.
  * hosts that do not respond will be under the key 
DatabaseDescriptor.INITIAL_VERSION.
  * the cluster is all on the same version if the size of the map is 1.
@@ -480,6 +487,8 @@ public class Cassandra {
 
 public void truncate(String cfname, 
org.apache.thrift.async.AsyncMethodCallback resultHandler) throws 
org.apache.thrift.TException;
 
+public void get_multi_slice(MultiSliceRequest request, 
org.apache.thrift.async.AsyncMethodCallback resultHandler) throws 
org.apache.thrift.TException;
+
 public void 
describe_schema_versions(org.apache.thrift.async.AsyncMethodCallback 
resultHandler) throws org.apache.thrift.TException;
 
 public void describe_keyspaces(org.apache.thrift.async.AsyncMethodCallback 
resultHandler) throws org.apache.thrift.TException;
@@ -1138,6 +1147,38 @@ public class Cassandra {
   return;
 }
 
+public ListColumnOrSuperColumn get_multi_slice(MultiSliceRequest 
request) throws InvalidRequestException, UnavailableException, 
TimedOutException, org.apache.thrift.TException
+{
+  send_get_multi_slice(request);
+  return recv_get_multi_slice();
+}
+
+public void send_get_multi_slice(MultiSliceRequest request) throws 
org.apache.thrift.TException
+{
+  get_multi_slice_args args = new get_multi_slice_args();
+  args.setRequest(request);
+  sendBase(get_multi_slice, args);
+}
+
+public ListColumnOrSuperColumn recv_get_multi_slice() throws 
InvalidRequestException, UnavailableException, TimedOutException, 
org.apache.thrift.TException
+{
+  get_multi_slice_result result = new get_multi_slice_result();
+  receiveBase(result, get_multi_slice);
+  if (result.isSetSuccess()) {
+return result.success;
+  }
+  if (result.ire != null) {
+throw result.ire;
+  }
+  if (result.ue != null) {
+throw result.ue;
+  }
+  if (result.te != null) {
+throw result.te;
+  }
+  throw new 
org.apache.thrift.TApplicationException(org.apache.thrift.TApplicationException.MISSING_RESULT,
 get_multi_slice failed: unknown result);
+}
+
 public MapString,ListString describe_schema_versions() throws 
InvalidRequestException, org.apache.thrift.TException
 {
   send_describe_schema_versions();
@@ -2576,6 +2617,38 @@ public class Cassandra {
   }
 }
 
+public void get_multi_slice(MultiSliceRequest request, 
org.apache.thrift.async.AsyncMethodCallback resultHandler) throws 
org.apache.thrift.TException {
+  checkReady();
+  get_multi_slice_call method_call = new get_multi_slice_call(request, 
resultHandler, this, ___protocolFactory, ___transport);
+  this.___currentMethod = method_call;
+  ___manager.call(method_call);
+}
+
+public static class get_multi_slice_call extends 
org.apache.thrift.async.TAsyncMethodCall {
+  private MultiSliceRequest request;
+  public get_multi_slice_call(MultiSliceRequest request, 
org.apache.thrift.async.AsyncMethodCallback resultHandler, 
org.apache.thrift.async.TAsyncClient client, 
org.apache.thrift.protocol.TProtocolFactory protocolFactory, 
org.apache.thrift.transport.TNonblockingTransport transport) throws 
org.apache.thrift.TException {
+super(client, protocolFactory, transport, resultHandler, false);
+this.request = request;
+  }
+
+  public void write_args(org.apache.thrift.protocol.TProtocol prot) throws 
org.apache.thrift.TException {
+prot.writeMessageBegin(new 
org.apache.thrift.protocol.TMessage(get_multi_slice, 
org.apache.thrift.protocol.TMessageType.CALL, 0));
+get_multi_slice_args args = new get_multi_slice_args();
+args.setRequest(request);
+args.write(prot);
+prot.writeMessageEnd();
+  }
+
+  public ListColumnOrSuperColumn getResult() throws 
InvalidRequestException,

[1/3] cassandra git commit: Revert Revert add Thrift get_multi_slice call

Repository: cassandra
Updated Branches:
  refs/heads/trunk 3f0702509 - f45e4a556


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f45e4a55/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnSlice.java
--
diff --git 
a/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnSlice.java 
b/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnSlice.java
new file mode 100644
index 000..67b88a3
--- /dev/null
+++ b/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnSlice.java
@@ -0,0 +1,551 @@
+/**
+ * Autogenerated by Thrift Compiler (0.9.1)
+ *
+ * DO NOT EDIT UNLESS YOU ARE SURE THAT YOU KNOW WHAT YOU ARE DOING
+ *  @generated
+ */
+package org.apache.cassandra.thrift;
+/*
+ * 
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * 
+ */
+
+
+import org.apache.commons.lang3.builder.HashCodeBuilder;
+import org.apache.thrift.scheme.IScheme;
+import org.apache.thrift.scheme.SchemeFactory;
+import org.apache.thrift.scheme.StandardScheme;
+
+import org.apache.thrift.scheme.TupleScheme;
+import org.apache.thrift.protocol.TTupleProtocol;
+import org.apache.thrift.protocol.TProtocolException;
+import org.apache.thrift.EncodingUtils;
+import org.apache.thrift.TException;
+import org.apache.thrift.async.AsyncMethodCallback;
+import org.apache.thrift.server.AbstractNonblockingServer.*;
+import java.util.List;
+import java.util.ArrayList;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.EnumMap;
+import java.util.Set;
+import java.util.HashSet;
+import java.util.EnumSet;
+import java.util.Collections;
+import java.util.BitSet;
+import java.nio.ByteBuffer;
+import java.util.Arrays;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * The ColumnSlice is used to select a set of columns from inside a row.
+ * If start or finish are unspecified they will default to the start-of
+ * end-of value.
+ * @param start. The start of the ColumnSlice inclusive
+ * @param finish. The end of the ColumnSlice inclusive
+ */
+public class ColumnSlice implements org.apache.thrift.TBaseColumnSlice, 
ColumnSlice._Fields, java.io.Serializable, Cloneable, ComparableColumnSlice {
+  private static final org.apache.thrift.protocol.TStruct STRUCT_DESC = new 
org.apache.thrift.protocol.TStruct(ColumnSlice);
+
+  private static final org.apache.thrift.protocol.TField START_FIELD_DESC = 
new org.apache.thrift.protocol.TField(start, 
org.apache.thrift.protocol.TType.STRING, (short)1);
+  private static final org.apache.thrift.protocol.TField FINISH_FIELD_DESC = 
new org.apache.thrift.protocol.TField(finish, 
org.apache.thrift.protocol.TType.STRING, (short)2);
+
+  private static final MapClass? extends IScheme, SchemeFactory schemes = 
new HashMapClass? extends IScheme, SchemeFactory();
+  static {
+schemes.put(StandardScheme.class, new ColumnSliceStandardSchemeFactory());
+schemes.put(TupleScheme.class, new ColumnSliceTupleSchemeFactory());
+  }
+
+  public ByteBuffer start; // optional
+  public ByteBuffer finish; // optional
+
+  /** The set of fields this struct contains, along with convenience methods 
for finding and manipulating them. */
+  public enum _Fields implements org.apache.thrift.TFieldIdEnum {
+START((short)1, start),
+FINISH((short)2, finish);
+
+private static final MapString, _Fields byName = new HashMapString, 
_Fields();
+
+static {
+  for (_Fields field : EnumSet.allOf(_Fields.class)) {
+byName.put(field.getFieldName(), field);
+  }
+}
+
+/**
+ * Find the _Fields constant that matches fieldId, or null if its not 
found.
+ */
+public static _Fields findByThriftId(int fieldId) {
+  switch(fieldId) {
+case 1: // START
+  return START;
+case 2: // FINISH
+  return FINISH;
+default:
+  return null;
+  }
+}
+
+/**
+ * Find the _Fields constant that matches fieldId, throwing an exception
+ * if it is not found.
+ */
+public static _Fields findByThriftIdOrThrow(int fieldId) {
+  _Fields fields = findByThriftId(fieldId);
+  if (fields == null) throw new IllegalArgumentException(Field  + 
fieldId +

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534997#comment-14534997
]

Ariel Weisberg commented on CASSANDRA-9318:
---

I looked at the issues you linked and didn't come away with something that
looks like leaky queues? Can you describe what that is? Is that shedding from
the queues based on resources? Makes sense to me mostly to prevent the initial
overload at processing nodes until the cluster can adapt to the disparity
between requested capacity and actual capacity. If leaked items resulted in an
error response that would aid in feedback to the coordinator and free up
resources there.

Given the contract of CL=1 (or even quorum) you are right there is nothing to
be gained by bounding the number of in-flight requests at a coordinator by not
reading requests from clients. At CL=1 and the way I hear people think about
availability in C* I think what you want is to get better at failing to hinting
before the coordinator or processing node overloads. Under overload conditions
CL=1 is basically synonymous with writing hints right?

bq. which may leave us open to a multiplying effect of cluster overload, with
each node dropping different requests, possibly leading to only a tiny fraction
of requests being serviced to their required CL across the cluster. I'm not
sure how we can best model this risk, or avoid it without notifying
coordinators of the drop of a message, and I don't see that being delivered for
2.1
Maybe this is a congestion control problem? If we piggybacked information in
responses on congestion issues maybe we could make better decisions about new
requests such as rejecting a %age or going straight to hints before resources
have been committed across the cluster?

Once something is hinted you can trickle out the load to match the actual
capacity of the thing being hinted. I know this conflicts with hints not being
fast, but hints are just a queue and could be very fast. I haven't looked at
the work being done to hints that is in progress.

Bound the number of in-flight requests at the coordinator
-

Key: CASSANDRA-9318
URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
Project: Cassandra
Issue Type: Improvement
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Fix For: 2.1.x

It's possible to somewhat bound the amount of load accepted into the cluster
by bounding the number of in-flight requests and request bytes.
An implementation might do something like track the number of outstanding
bytes and requests and if it reaches a high watermark disable read on client
connections until it goes back below some low watermark.
Need to make sure that disabling read on the client connection won't
introduce other issues.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Fix JVM_EXTRA_OPTS in Windows startup scripts

2015-05-08 Thread jmckenzie

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 ef8dc6715 - a431d846c


Fix JVM_EXTRA_OPTS in Windows startup scripts

Patch by jmckenzie; reviewed by pthompson for CASSANDRA-9335


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a431d846
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a431d846
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a431d846

Branch: refs/heads/cassandra-2.1
Commit: a431d846c191d29d49f98c53675bffcc7956d294
Parents: ef8dc67
Author: Josh McKenzie josh.mcken...@datastax.com
Authored: Fri May 8 12:30:17 2015 -0500
Committer: Josh McKenzie josh.mcken...@datastax.com
Committed: Fri May 8 12:30:17 2015 -0500

--
 conf/cassandra-env.ps1 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a431d846/conf/cassandra-env.ps1
--
diff --git a/conf/cassandra-env.ps1 b/conf/cassandra-env.ps1
index 6d9b8b1..35da856 100644
--- a/conf/cassandra-env.ps1
+++ b/conf/cassandra-env.ps1
@@ -411,7 +411,7 @@ Function SetCassandraEnvironment
 #$env:JVM_OPTS=$env:JVM_OPTS 
-Dcom.sun.management.jmxremote.password.file=C:/jmxremote.password
 $env:JVM_OPTS=$env:JVM_OPTS -Dcassandra.jmx.local.port=$JMX_PORT 
-XX:+DisableExplicitGC
 
-$env:JVM_OPTS=$env:JVM_OPTS $JVM_EXTRA_OPTS
+$env:JVM_OPTS=$env:JVM_OPTS $env:JVM_EXTRA_OPTS
 
 $env:JVM_OPTS = $env:JVM_OPTS 
-Dlog4j.configuration=log4j-server.properties
 }

[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

2015-05-08 Thread jmckenzie

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e9d1e799
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e9d1e799
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e9d1e799

Branch: refs/heads/trunk
Commit: e9d1e799434cc08f7d4e62da3aff6af89cb6e8d5
Parents: 86f481e a431d84
Author: Josh McKenzie josh.mcken...@datastax.com
Authored: Fri May 8 12:33:25 2015 -0500
Committer: Josh McKenzie josh.mcken...@datastax.com
Committed: Fri May 8 12:33:25 2015 -0500

--
 conf/cassandra-env.ps1 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9d1e799/conf/cassandra-env.ps1
--
diff --cc conf/cassandra-env.ps1
index e0287c5,35da856..dbf83a3
--- a/conf/cassandra-env.ps1
+++ b/conf/cassandra-env.ps1
@@@ -452,9 -411,7 +452,9 @@@ Function SetCassandraEnvironmen
  #$env:JVM_OPTS=$env:JVM_OPTS 
-Dcom.sun.management.jmxremote.password.file=C:/jmxremote.password
  $env:JVM_OPTS=$env:JVM_OPTS -Dcassandra.jmx.local.port=$JMX_PORT 
-XX:+DisableExplicitGC
  
- $env:JVM_OPTS=$env:JVM_OPTS $JVM_EXTRA_OPTS
+ $env:JVM_OPTS=$env:JVM_OPTS $env:JVM_EXTRA_OPTS
  
  $env:JVM_OPTS = $env:JVM_OPTS 
-Dlog4j.configuration=log4j-server.properties
 +
 +#$env:JVM_OPTS=$env:JVM_OPTS -XX:+UnlockCommercialFeatures 
-XX:+FlightRecorder
  }

[1/2] cassandra git commit: Fix JVM_EXTRA_OPTS in Windows startup scripts

2015-05-08 Thread jmckenzie

Repository: cassandra
Updated Branches:
  refs/heads/trunk 86f481ef9 - e9d1e7994


Fix JVM_EXTRA_OPTS in Windows startup scripts

Patch by jmckenzie; reviewed by pthompson for CASSANDRA-9335


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a431d846
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a431d846
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a431d846

Branch: refs/heads/trunk
Commit: a431d846c191d29d49f98c53675bffcc7956d294
Parents: ef8dc67
Author: Josh McKenzie josh.mcken...@datastax.com
Authored: Fri May 8 12:30:17 2015 -0500
Committer: Josh McKenzie josh.mcken...@datastax.com
Committed: Fri May 8 12:30:17 2015 -0500

--
 conf/cassandra-env.ps1 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a431d846/conf/cassandra-env.ps1
--
diff --git a/conf/cassandra-env.ps1 b/conf/cassandra-env.ps1
index 6d9b8b1..35da856 100644
--- a/conf/cassandra-env.ps1
+++ b/conf/cassandra-env.ps1
@@ -411,7 +411,7 @@ Function SetCassandraEnvironment
 #$env:JVM_OPTS=$env:JVM_OPTS 
-Dcom.sun.management.jmxremote.password.file=C:/jmxremote.password
 $env:JVM_OPTS=$env:JVM_OPTS -Dcassandra.jmx.local.port=$JMX_PORT 
-XX:+DisableExplicitGC
 
-$env:JVM_OPTS=$env:JVM_OPTS $JVM_EXTRA_OPTS
+$env:JVM_OPTS=$env:JVM_OPTS $env:JVM_EXTRA_OPTS
 
 $env:JVM_OPTS = $env:JVM_OPTS 
-Dlog4j.configuration=log4j-server.properties
 }

[jira] [Commented] (CASSANDRA-8984) Introduce Transactional API for behaviours that can corrupt system state


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535047#comment-14535047
 ] 

Benedict commented on CASSANDRA-8984:
-

[~JoshuaMcKenzie]: I've updated my branch with some testing to cover each of 
the SequentialWriter implementations and BigTableWriter. As a by-product I 
brought the semantics of each of SequentialWriter inline with expectation, i.e. 
it deletes its file on abort. I also updated BigTableWriter to delete both TMP 
and FINAL files, which was probably an actual bug (one that's been around 
indefinitely)

 Introduce Transactional API for behaviours that can corrupt system state
 

 Key: CASSANDRA-8984
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8984
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
 Fix For: 3.x

 Attachments: 8984_windows_timeout.txt


 As a penultimate (and probably final for 2.1, if we agree to introduce it 
 there) round of changes to the internals managing sstable writing, I've 
 introduced a new API called Transactional that I hope will make it much 
 easier to write correct behaviour. As things stand we conflate a lot of 
 behaviours into methods like close - the recent changes unpicked some of 
 these, but didn't go far enough. My proposal here introduces an interface 
 designed to support four actions (on top of their normal function):
 * prepareToCommit
 * commit
 * abort
 * cleanup
 In normal operation, once we have finished constructing a state change we 
 call prepareToCommit; once all such state changes are prepared, we call 
 commit. If at any point everything fails, abort is called. In _either_ case, 
 cleanup is called at the very last.
 These transactional objects are all AutoCloseable, with the behaviour being 
 to rollback any changes unless commit has completed successfully.
 The changes are actually less invasive than it might sound, since we did 
 recently introduce abort in some places, as well as have commit like methods. 
 This simply formalises the behaviour, and makes it consistent between all 
 objects that interact in this way. Much of the code change is boilerplate, 
 such as moving an object into a try-declaration, although the change is still 
 non-trivial. What it _does_ do is eliminate a _lot_ of special casing that we 
 have had since 2.1 was released. The data tracker API changes and compaction 
 leftover cleanups should finish the job with making this much easier to 
 reason about, but this change I think is worthwhile considering for 2.1, 
 since we've just overhauled this entire area (and not released these 
 changes), and this change is essentially just the finishing touches, so the 
 risk is minimal and the potential gains reasonably significant.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7317) Repair range validation is too strict


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534858#comment-14534858
 ] 

Kenneth Failbus edited comment on CASSANDRA-7317 at 5/8/15 4:48 PM:


Folks,

I am seeing this error again in 2.0.9 release. I have vnodes in my cluster 
enabled.
{code}
2015-05-08 15:01:56,021 [AntiEntropyStage:1] INFO Validator [repair 
#254edb00-f593-11e4-9397-51babce9f892] Sending completed merkle tree to 
/10.22.168.35 for CF1/Sequence
2015-05-08 15:01:58,518 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/permissions
2015-05-08 15:01:58,791 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/credentials
2015-05-08 15:01:58,980 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/users
2015-05-08 15:02:00,640 [AntiEntropyStage:1] INFO Validator [repair 
#e0d31e00-f592-11e4-993e-c9cc22925782] Sending completed merkle tree to 
/10.22.168.97 for system_auth/credentials
2015-05-08 15:02:01,345 [AntiEntropyStage:1] INFO Validator [repair 
#e0d31e00-f592-11e4-993e-c9cc22925782] Sending completed merkle tree to 
/10.22.168.97 for system_auth/users
2015-05-08 15:02:01,577 [AntiEntropyStage:1] INFO Validator [repair 
#e0d31e00-f592-11e4-993e-c9cc22925782] Sending completed merkle tree to 
/10.22.168.97 for system_auth/permissions
2015-05-08 15:02:01,753 [AntiEntropyStage:1] INFO Validator [repair 
#27dba060-f593-11e4-873b-9d346bbba08e] Sending completed merkle tree to 
/10.22.168.87 for CF1/Sequence
2015-05-08 15:02:02,622 [AntiEntropyStage:1] INFO Validator [repair 
#dba213a0-f592-11e4-b745-192986bd7af2] Sending completed merkle tree to 
/10.22.168.117 for system_auth/credentials
2015-05-08 15:02:02,873 [AntiEntropyStage:1] INFO Validator [repair 
#dba213a0-f592-11e4-b745-192986bd7af2] Sending completed merkle tree to 
/10.22.168.117 for system_auth/users
2015-05-08 15:02:03,508 [AntiEntropyStage:1] INFO Validator [repair 
#dba213a0-f592-11e4-b745-192986bd7af2] Sending completed merkle tree to 
/10.22.168.117 for system_auth/permissions
2015-05-08 15:02:03,988 [AntiEntropyStage:1] INFO Validator [repair 
#d0a2ad70-f592-11e4-a5a2-b73fe73dbe79] Sending completed merkle tree to 
/10.22.168.109 for system_auth/credentials
2015-05-08 15:02:04,759 [AntiEntropyStage:1] INFO Validator [repair 
#d0a2ad70-f592-11e4-a5a2-b73fe73dbe79] Sending completed merkle tree to 
/10.22.168.109 for system_auth/users
2015-05-08 15:02:05,066 [AntiEntropyStage:1] INFO Validator [repair 
#d0a2ad70-f592-11e4-a5a2-b73fe73dbe79] Sending completed merkle tree to 
/10.22.168.109 for system_auth/permissions
2015-05-08 15:02:05,200 [Thread-227856] ERROR StorageService Repair session 
failed:
java.lang.IllegalArgumentException: Requested range intersects a local range 
but is not fully contained in one; this would lead to imprecise repair
at 
org.apache.cassandra.service.ActiveRepairService.getNeighbors(ActiveRepairService.java:161)
at 
org.apache.cassandra.repair.RepairSession.init(RepairSession.java:130)
at 
org.apache.cassandra.repair.RepairSession.init(RepairSession.java:119)
at 
org.apache.cassandra.service.ActiveRepairService.submitRepairSession(ActiveRepairService.java:97)
at 
org.apache.cassandra.service.StorageService.forceKeyspaceRepair(StorageService.java:2628)
at 
org.apache.cassandra.service.StorageService$4.runMayThrow(StorageService.java:2564)
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.lang.Thread.run(Thread.java:744)



was (Author: kenfailbus):
Folks,

I am seeing this error again in 2.0.9 release. I have vnodes in my cluster 
enabled.

 Repair range validation is too strict
 -

 Key: CASSANDRA-7317
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7317
 Project: Cassandra
  Issue Type: Bug
Reporter: Nick Bailey
Assignee: Yuki Morishita
 Fix For: 2.0.9

 Attachments: 7317-2.0.txt, Untitled Diagram(1).png


 From what I can tell the calculation (using the -pr option) and validation of 
 tokens for repairing ranges is broken. Or at least should be improved. Using 
 an example with ccm:
 Nodetool ring:
 {noformat}
 Datacenter: dc1
 ==
 AddressRackStatus State   LoadOwns
 Token
   -10
 127.0.0.1

[jira] [Resolved] (CASSANDRA-6757) SliceQueryFilter can accept multiple ColumnSlice but not exposed to thrift


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-6757.
---
   Resolution: Won't Fix
Fix Version/s: (was: 3.0)

Reverting since this didn't ever get into 2.1, after which Thrift API was 
frozen.

 SliceQueryFilter can accept multiple ColumnSlice but not exposed to thrift
 --

 Key: CASSANDRA-6757
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6757
 Project: Cassandra
  Issue Type: New Feature
Reporter: Edward Capriolo
Assignee: Edward Capriolo
 Attachments: Cassandra-6757.1.diff


 Thrift has getSlice() which takes a key and a predicate. SliceQueryFilter can 
 accept multiple ColumnSlice(s). Users should have access to this capability 
 already found in Cassandra.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-6757) SliceQueryFilter can accept multiple ColumnSlice but not exposed to thrift


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reopened CASSANDRA-6757:
---

 SliceQueryFilter can accept multiple ColumnSlice but not exposed to thrift
 --

 Key: CASSANDRA-6757
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6757
 Project: Cassandra
  Issue Type: New Feature
Reporter: Edward Capriolo
Assignee: Edward Capriolo
 Attachments: Cassandra-6757.1.diff


 Thrift has getSlice() which takes a key and a predicate. SliceQueryFilter can 
 accept multiple ColumnSlice(s). Users should have access to this capability 
 already found in Cassandra.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8770) Support reversing map and set collation order


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-8770.
---
Resolution: Later

Resolving as Later since nobody seems to need it in the near future.

 Support reversing map and set collation order
 -

 Key: CASSANDRA-8770
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8770
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Jonathan Ellis
Priority: Minor





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7317) Repair range validation is too strict


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534858#comment-14534858
 ] 

Kenneth Failbus edited comment on CASSANDRA-7317 at 5/8/15 4:53 PM:


Folks,

I am seeing this error again in 2.0.9 release. I have vnodes in my cluster 
enabled. We run our repairs with -local option without -pr on our cluster.
{code}
2015-05-08 15:01:56,021 [AntiEntropyStage:1] INFO Validator [repair 
#254edb00-f593-11e4-9397-51babce9f892] Sending completed merkle tree to 
/10.22.168.35 for CF1/Sequence
2015-05-08 15:01:58,518 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/permissions
2015-05-08 15:01:58,791 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/credentials
2015-05-08 15:01:58,980 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/users
2015-05-08 15:02:00,640 [AntiEntropyStage:1] INFO Validator [repair 
#e0d31e00-f592-11e4-993e-c9cc22925782] Sending completed merkle tree to 
/10.22.168.97 for system_auth/credentials
2015-05-08 15:02:01,345 [AntiEntropyStage:1] INFO Validator [repair 
#e0d31e00-f592-11e4-993e-c9cc22925782] Sending completed merkle tree to 
/10.22.168.97 for system_auth/users
2015-05-08 15:02:01,577 [AntiEntropyStage:1] INFO Validator [repair 
#e0d31e00-f592-11e4-993e-c9cc22925782] Sending completed merkle tree to 
/10.22.168.97 for system_auth/permissions
2015-05-08 15:02:01,753 [AntiEntropyStage:1] INFO Validator [repair 
#27dba060-f593-11e4-873b-9d346bbba08e] Sending completed merkle tree to 
/10.22.168.87 for CF1/Sequence
2015-05-08 15:02:02,622 [AntiEntropyStage:1] INFO Validator [repair 
#dba213a0-f592-11e4-b745-192986bd7af2] Sending completed merkle tree to 
/10.22.168.117 for system_auth/credentials
2015-05-08 15:02:02,873 [AntiEntropyStage:1] INFO Validator [repair 
#dba213a0-f592-11e4-b745-192986bd7af2] Sending completed merkle tree to 
/10.22.168.117 for system_auth/users
2015-05-08 15:02:03,508 [AntiEntropyStage:1] INFO Validator [repair 
#dba213a0-f592-11e4-b745-192986bd7af2] Sending completed merkle tree to 
/10.22.168.117 for system_auth/permissions
2015-05-08 15:02:03,988 [AntiEntropyStage:1] INFO Validator [repair 
#d0a2ad70-f592-11e4-a5a2-b73fe73dbe79] Sending completed merkle tree to 
/10.22.168.109 for system_auth/credentials
2015-05-08 15:02:04,759 [AntiEntropyStage:1] INFO Validator [repair 
#d0a2ad70-f592-11e4-a5a2-b73fe73dbe79] Sending completed merkle tree to 
/10.22.168.109 for system_auth/users
2015-05-08 15:02:05,066 [AntiEntropyStage:1] INFO Validator [repair 
#d0a2ad70-f592-11e4-a5a2-b73fe73dbe79] Sending completed merkle tree to 
/10.22.168.109 for system_auth/permissions
2015-05-08 15:02:05,200 [Thread-227856] ERROR StorageService Repair session 
failed:
java.lang.IllegalArgumentException: Requested range intersects a local range 
but is not fully contained in one; this would lead to imprecise repair
at 
org.apache.cassandra.service.ActiveRepairService.getNeighbors(ActiveRepairService.java:161)
at 
org.apache.cassandra.repair.RepairSession.init(RepairSession.java:130)
at 
org.apache.cassandra.repair.RepairSession.init(RepairSession.java:119)
at 
org.apache.cassandra.service.ActiveRepairService.submitRepairSession(ActiveRepairService.java:97)
at 
org.apache.cassandra.service.StorageService.forceKeyspaceRepair(StorageService.java:2628)
at 
org.apache.cassandra.service.StorageService$4.runMayThrow(StorageService.java:2564)
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.lang.Thread.run(Thread.java:744)
{code}


was (Author: kenfailbus):
Folks,

I am seeing this error again in 2.0.9 release. I have vnodes in my cluster 
enabled.
{code}
2015-05-08 15:01:56,021 [AntiEntropyStage:1] INFO Validator [repair 
#254edb00-f593-11e4-9397-51babce9f892] Sending completed merkle tree to 
/10.22.168.35 for CF1/Sequence
2015-05-08 15:01:58,518 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/permissions
2015-05-08 15:01:58,791 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/credentials
2015-05-08 15:01:58,980 [AntiEntropyStage:1] INFO Validator [repair 
#e3ca16e0-f592-11e4-bce3-6f1b5fa480b1] Sending completed merkle tree to 
/10.22.168.105 for system_auth/users
2015-05-08 15:02:00,640

[3/3] cassandra git commit: Revert Revert add Thrift get_multi_slice call

Revert Revert add Thrift get_multi_slice call

This reverts commit 3f0702509b3f12cfc159fc33a114bcf52ecf9fc9.


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f45e4a55
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f45e4a55
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f45e4a55

Branch: refs/heads/trunk
Commit: f45e4a556a0e617f0d6a68da45bca889b49f6159
Parents: 3f07025
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri May 8 12:04:40 2015 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri May 8 12:05:13 2015 -0500

--
 interface/cassandra.thrift  |   37 +-
 .../org/apache/cassandra/thrift/Cassandra.java  | 3071 +-
 .../apache/cassandra/thrift/ColumnSlice.java|  551 
 .../cassandra/thrift/MultiSliceRequest.java | 1042 ++
 .../cassandra/thrift/cassandraConstants.java|2 +-
 .../cassandra/thrift/CassandraServer.java   |   72 +
 .../cassandra/db/ColumnFamilyStoreTest.java |   20 +-
 .../apache/cassandra/thrift/MultiSliceTest.java |  186 ++
 8 files changed, 4062 insertions(+), 919 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f45e4a55/interface/cassandra.thrift
--
diff --git a/interface/cassandra.thrift b/interface/cassandra.thrift
index 835fa8c..0af7e64 100644
--- a/interface/cassandra.thrift
+++ b/interface/cassandra.thrift
@@ -55,7 +55,7 @@ namespace rb CassandraThrift
 # An effort should be made not to break forward-client-compatibility either
 # (e.g. one should avoid removing obsolete fields from the IDL), but no
 # guarantees in this respect are made by the Cassandra project.
-const string VERSION = 20.0.0
+const string VERSION = 20.1.0
 
 
 #
@@ -571,6 +571,35 @@ struct CfSplit {
 3: required i64 row_count
 }
 
+/** The ColumnSlice is used to select a set of columns from inside a row. 
+ * If start or finish are unspecified they will default to the start-of
+ * end-of value.
+ * @param start. The start of the ColumnSlice inclusive
+ * @param finish. The end of the ColumnSlice inclusive
+ */
+struct ColumnSlice {
+1: optional binary start,
+2: optional binary finish
+}
+
+/**
+ * Used to perform multiple slices on a single row key in one rpc operation
+ * @param key. The row key to be multi sliced
+ * @param column_parent. The column family (super columns are unsupported)
+ * @param column_slices. 0 to many ColumnSlice objects each will be used to 
select columns
+ * @param reversed. Direction of slice
+ * @param count. Maximum number of columns
+ * @param consistency_level. Level to perform the operation at
+ */
+struct MultiSliceRequest {
+1: optional binary key,
+2: optional ColumnParent column_parent,
+3: optional listColumnSlice column_slices,
+4: optional bool reversed=false,
+5: optional i32 count=1000,
+6: optional ConsistencyLevel consistency_level=ConsistencyLevel.ONE
+}
+
 service Cassandra {
   # auth methods
   void login(1: required AuthenticationRequest auth_request) throws 
(1:AuthenticationException authnx, 2:AuthorizationException authzx),
@@ -749,7 +778,11 @@ service Cassandra {
   void truncate(1:required string cfname)
throws (1: InvalidRequestException ire, 2: UnavailableException ue, 3: 
TimedOutException te),
 
-
+  /**
+  * Select multiple slices of a key in a single RPC operation
+  */
+  listColumnOrSuperColumn get_multi_slice(1:required MultiSliceRequest 
request)
+   throws (1:InvalidRequestException ire, 2:UnavailableException ue, 
3:TimedOutException te),
 
   // Meta-APIs -- APIs to get information about the node or cluster,
   // rather than user data.  The nodeprobe program provides usage examples.

[1/3] cassandra git commit: Revert add Thrift get_multi_slice call

Repository: cassandra
Updated Branches:
  refs/heads/trunk c7db18be8 - 3f0702509


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3f070250/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnSlice.java
--
diff --git 
a/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnSlice.java 
b/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnSlice.java
deleted file mode 100644
index 67b88a3..000
--- a/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnSlice.java
+++ /dev/null
@@ -1,551 +0,0 @@
-/**
- * Autogenerated by Thrift Compiler (0.9.1)
- *
- * DO NOT EDIT UNLESS YOU ARE SURE THAT YOU KNOW WHAT YOU ARE DOING
- *  @generated
- */
-package org.apache.cassandra.thrift;
-/*
- * 
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * License); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- * 
- *   http://www.apache.org/licenses/LICENSE-2.0
- * 
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied.  See the License for the
- * specific language governing permissions and limitations
- * under the License.
- * 
- */
-
-
-import org.apache.commons.lang3.builder.HashCodeBuilder;
-import org.apache.thrift.scheme.IScheme;
-import org.apache.thrift.scheme.SchemeFactory;
-import org.apache.thrift.scheme.StandardScheme;
-
-import org.apache.thrift.scheme.TupleScheme;
-import org.apache.thrift.protocol.TTupleProtocol;
-import org.apache.thrift.protocol.TProtocolException;
-import org.apache.thrift.EncodingUtils;
-import org.apache.thrift.TException;
-import org.apache.thrift.async.AsyncMethodCallback;
-import org.apache.thrift.server.AbstractNonblockingServer.*;
-import java.util.List;
-import java.util.ArrayList;
-import java.util.Map;
-import java.util.HashMap;
-import java.util.EnumMap;
-import java.util.Set;
-import java.util.HashSet;
-import java.util.EnumSet;
-import java.util.Collections;
-import java.util.BitSet;
-import java.nio.ByteBuffer;
-import java.util.Arrays;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-/**
- * The ColumnSlice is used to select a set of columns from inside a row.
- * If start or finish are unspecified they will default to the start-of
- * end-of value.
- * @param start. The start of the ColumnSlice inclusive
- * @param finish. The end of the ColumnSlice inclusive
- */
-public class ColumnSlice implements org.apache.thrift.TBaseColumnSlice, 
ColumnSlice._Fields, java.io.Serializable, Cloneable, ComparableColumnSlice {
-  private static final org.apache.thrift.protocol.TStruct STRUCT_DESC = new 
org.apache.thrift.protocol.TStruct(ColumnSlice);
-
-  private static final org.apache.thrift.protocol.TField START_FIELD_DESC = 
new org.apache.thrift.protocol.TField(start, 
org.apache.thrift.protocol.TType.STRING, (short)1);
-  private static final org.apache.thrift.protocol.TField FINISH_FIELD_DESC = 
new org.apache.thrift.protocol.TField(finish, 
org.apache.thrift.protocol.TType.STRING, (short)2);
-
-  private static final MapClass? extends IScheme, SchemeFactory schemes = 
new HashMapClass? extends IScheme, SchemeFactory();
-  static {
-schemes.put(StandardScheme.class, new ColumnSliceStandardSchemeFactory());
-schemes.put(TupleScheme.class, new ColumnSliceTupleSchemeFactory());
-  }
-
-  public ByteBuffer start; // optional
-  public ByteBuffer finish; // optional
-
-  /** The set of fields this struct contains, along with convenience methods 
for finding and manipulating them. */
-  public enum _Fields implements org.apache.thrift.TFieldIdEnum {
-START((short)1, start),
-FINISH((short)2, finish);
-
-private static final MapString, _Fields byName = new HashMapString, 
_Fields();
-
-static {
-  for (_Fields field : EnumSet.allOf(_Fields.class)) {
-byName.put(field.getFieldName(), field);
-  }
-}
-
-/**
- * Find the _Fields constant that matches fieldId, or null if its not 
found.
- */
-public static _Fields findByThriftId(int fieldId) {
-  switch(fieldId) {
-case 1: // START
-  return START;
-case 2: // FINISH
-  return FINISH;
-default:
-  return null;
-  }
-}
-
-/**
- * Find the _Fields constant that matches fieldId, throwing an exception
- * if it is not found.
- */
-public static _Fields findByThriftIdOrThrow(int fieldId) {
-  _Fields fields = findByThriftId(fieldId);
-  if (fields == null) throw new IllegalArgumentException(Field  + 
fieldId +

[2/3] cassandra git commit: Revert add Thrift get_multi_slice call

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3f070250/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
diff --git 
a/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java 
b/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
index bf15559..f29805b 100644
--- a/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
+++ b/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
@@ -248,13 +248,6 @@ public class Cassandra {
 public void truncate(String cfname) throws InvalidRequestException, 
UnavailableException, TimedOutException, org.apache.thrift.TException;
 
 /**
- * Select multiple slices of a key in a single RPC operation
- * 
- * @param request
- */
-public ListColumnOrSuperColumn get_multi_slice(MultiSliceRequest 
request) throws InvalidRequestException, UnavailableException, 
TimedOutException, org.apache.thrift.TException;
-
-/**
  * for each schema version present in the cluster, returns a list of nodes 
at that version.
  * hosts that do not respond will be under the key 
DatabaseDescriptor.INITIAL_VERSION.
  * the cluster is all on the same version if the size of the map is 1.
@@ -487,8 +480,6 @@ public class Cassandra {
 
 public void truncate(String cfname, 
org.apache.thrift.async.AsyncMethodCallback resultHandler) throws 
org.apache.thrift.TException;
 
-public void get_multi_slice(MultiSliceRequest request, 
org.apache.thrift.async.AsyncMethodCallback resultHandler) throws 
org.apache.thrift.TException;
-
 public void 
describe_schema_versions(org.apache.thrift.async.AsyncMethodCallback 
resultHandler) throws org.apache.thrift.TException;
 
 public void describe_keyspaces(org.apache.thrift.async.AsyncMethodCallback 
resultHandler) throws org.apache.thrift.TException;
@@ -1147,38 +1138,6 @@ public class Cassandra {
   return;
 }
 
-public ListColumnOrSuperColumn get_multi_slice(MultiSliceRequest 
request) throws InvalidRequestException, UnavailableException, 
TimedOutException, org.apache.thrift.TException
-{
-  send_get_multi_slice(request);
-  return recv_get_multi_slice();
-}
-
-public void send_get_multi_slice(MultiSliceRequest request) throws 
org.apache.thrift.TException
-{
-  get_multi_slice_args args = new get_multi_slice_args();
-  args.setRequest(request);
-  sendBase(get_multi_slice, args);
-}
-
-public ListColumnOrSuperColumn recv_get_multi_slice() throws 
InvalidRequestException, UnavailableException, TimedOutException, 
org.apache.thrift.TException
-{
-  get_multi_slice_result result = new get_multi_slice_result();
-  receiveBase(result, get_multi_slice);
-  if (result.isSetSuccess()) {
-return result.success;
-  }
-  if (result.ire != null) {
-throw result.ire;
-  }
-  if (result.ue != null) {
-throw result.ue;
-  }
-  if (result.te != null) {
-throw result.te;
-  }
-  throw new 
org.apache.thrift.TApplicationException(org.apache.thrift.TApplicationException.MISSING_RESULT,
 get_multi_slice failed: unknown result);
-}
-
 public MapString,ListString describe_schema_versions() throws 
InvalidRequestException, org.apache.thrift.TException
 {
   send_describe_schema_versions();
@@ -2617,38 +2576,6 @@ public class Cassandra {
   }
 }
 
-public void get_multi_slice(MultiSliceRequest request, 
org.apache.thrift.async.AsyncMethodCallback resultHandler) throws 
org.apache.thrift.TException {
-  checkReady();
-  get_multi_slice_call method_call = new get_multi_slice_call(request, 
resultHandler, this, ___protocolFactory, ___transport);
-  this.___currentMethod = method_call;
-  ___manager.call(method_call);
-}
-
-public static class get_multi_slice_call extends 
org.apache.thrift.async.TAsyncMethodCall {
-  private MultiSliceRequest request;
-  public get_multi_slice_call(MultiSliceRequest request, 
org.apache.thrift.async.AsyncMethodCallback resultHandler, 
org.apache.thrift.async.TAsyncClient client, 
org.apache.thrift.protocol.TProtocolFactory protocolFactory, 
org.apache.thrift.transport.TNonblockingTransport transport) throws 
org.apache.thrift.TException {
-super(client, protocolFactory, transport, resultHandler, false);
-this.request = request;
-  }
-
-  public void write_args(org.apache.thrift.protocol.TProtocol prot) throws 
org.apache.thrift.TException {
-prot.writeMessageBegin(new 
org.apache.thrift.protocol.TMessage(get_multi_slice, 
org.apache.thrift.protocol.TMessageType.CALL, 0));
-get_multi_slice_args args = new get_multi_slice_args();
-args.setRequest(request);
-args.write(prot);
-prot.writeMessageEnd();
-  }
-
-  public ListColumnOrSuperColumn getResult() throws 
InvalidRequestException,

[3/3] cassandra git commit: Revert add Thrift get_multi_slice call

Revert add Thrift get_multi_slice call

This reverts commit 60fb923018a6fd2dabf04a1d4500f7b29a23a6f1.

Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/thrift/CassandraServer.java
test/system/test_thrift_server.py
test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
test/unit/org/apache/cassandra/thrift/MultiSliceTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3f070250
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3f070250
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3f070250

Branch: refs/heads/trunk
Commit: 3f0702509b3f12cfc159fc33a114bcf52ecf9fc9
Parents: c7db18b
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri May 8 11:59:57 2015 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri May 8 11:59:57 2015 -0500

--
 CHANGES.txt |2 +-
 interface/cassandra.thrift  |   37 +-
 .../org/apache/cassandra/thrift/Cassandra.java  | 3071 +-
 .../apache/cassandra/thrift/ColumnSlice.java|  551 
 .../cassandra/thrift/MultiSliceRequest.java | 1042 --
 .../cassandra/thrift/cassandraConstants.java|2 +-
 .../cassandra/thrift/CassandraServer.java   |   72 -
 .../cassandra/db/ColumnFamilyStoreTest.java |   20 +-
 .../apache/cassandra/thrift/MultiSliceTest.java |  186 --
 9 files changed, 920 insertions(+), 4063 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3f070250/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4639bff..675e46c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -80,7 +80,6 @@
  * Move sstable RandomAccessReader to nio2, which allows using the
FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050)
  * Remove CQL2 (CASSANDRA-5918)
- * Add Thrift get_multi_slice call (CASSANDRA-6757)
  * Optimize fetching multiple cells by name (CASSANDRA-6933)
  * Allow compilation in java 8 (CASSANDRA-7028)
  * Make incremental repair default (CASSANDRA-7250)
@@ -99,6 +98,7 @@
  * Save repair data to system table (CASSANDRA-5839)
  * fix nodetool names that reference column families (CASSANDRA-8872)
 
+
 2.1.6
  * Delete processed sstables in sstablesplit/sstableupgrade (CASSANDRA-8606)
  * Improve sstable exclusion from partition tombstones (CASSANDRA-9298)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3f070250/interface/cassandra.thrift
--
diff --git a/interface/cassandra.thrift b/interface/cassandra.thrift
index 0af7e64..835fa8c 100644
--- a/interface/cassandra.thrift
+++ b/interface/cassandra.thrift
@@ -55,7 +55,7 @@ namespace rb CassandraThrift
 # An effort should be made not to break forward-client-compatibility either
 # (e.g. one should avoid removing obsolete fields from the IDL), but no
 # guarantees in this respect are made by the Cassandra project.
-const string VERSION = 20.1.0
+const string VERSION = 20.0.0
 
 
 #
@@ -571,35 +571,6 @@ struct CfSplit {
 3: required i64 row_count
 }
 
-/** The ColumnSlice is used to select a set of columns from inside a row. 
- * If start or finish are unspecified they will default to the start-of
- * end-of value.
- * @param start. The start of the ColumnSlice inclusive
- * @param finish. The end of the ColumnSlice inclusive
- */
-struct ColumnSlice {
-1: optional binary start,
-2: optional binary finish
-}
-
-/**
- * Used to perform multiple slices on a single row key in one rpc operation
- * @param key. The row key to be multi sliced
- * @param column_parent. The column family (super columns are unsupported)
- * @param column_slices. 0 to many ColumnSlice objects each will be used to 
select columns
- * @param reversed. Direction of slice
- * @param count. Maximum number of columns
- * @param consistency_level. Level to perform the operation at
- */
-struct MultiSliceRequest {
-1: optional binary key,
-2: optional ColumnParent column_parent,
-3: optional listColumnSlice column_slices,
-4: optional bool reversed=false,
-5: optional i32 count=1000,
-6: optional ConsistencyLevel consistency_level=ConsistencyLevel.ONE
-}
-
 service Cassandra {
   # auth methods
   void login(1: required AuthenticationRequest auth_request) throws 
(1:AuthenticationException authnx, 2:AuthorizationException authzx),
@@ -778,11 +749,7 @@ service Cassandra {
   void truncate(1:required string cfname)
throws (1: InvalidRequestException ire, 2: UnavailableException ue, 3: 
TimedOutException te),
 
-  /**
-  * Select multiple slices of a key in a single RPC operation
-  */
-  listColumnOrSuperColumn get_multi_slice(1:required MultiSliceRequest 
request)
-

[jira] [Commented] (CASSANDRA-9334) Returning null from a trigger does not abort the write

2015-05-08 Thread Sam Tunnicliffe (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534830#comment-14534830
 ] 

Sam Tunnicliffe commented on CASSANDRA-9334:


I agree

 Returning null from a trigger does not abort the write
 --

 Key: CASSANDRA-9334
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9334
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Brandon Williams
Assignee: Sam Tunnicliffe
 Fix For: 2.0.x


 The javadoc for ITrigger seems to indicate that returning null means no 
 mutations for the write will be applied at all:
 {noformat}
  * Called exactly once per CF update, returned mutations are atomically 
 updated.
  *
  * @param key - Row Key for the update.
  * @param update - Update received for the CF
  * @return modifications to be applied, null if no action to be performed.
 {noformat}
 and indeed, TriggerExecutor.execute seems to have code for this:
 {noformat}
 if (augmentedMutations == null)
 return null;
 {noformat}
 but it never fires.  In testing with a trigger that always returns null, the 
 original mutation is still applied.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9286) Add Keyspace/Table details to CollectionType.java error message

2015-05-08 Thread Carl Yeksigian (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-9286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534856#comment-14534856
]

Carl Yeksigian commented on CASSANDRA-9286:
---

Pushed a [dtest
branch|https://github.com/carlyeks/cassandra-dtest/tree/ticket/9286] for this.

Add Keyspace/Table details to CollectionType.java error message
---

Attachments: 9286-2.0-v2.txt, 9286-2.0.txt, 9286-2.1-v2.txt,
9286-2.1.txt

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9317) Populate TokenMetadata earlier during startup


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535049#comment-14535049
 ] 

Tyler Hobbs commented on CASSANDRA-9317:


I'm going to hold off on review until we can get those dtests passing (working 
on that right now).

 Populate TokenMetadata earlier during startup
 -

 Key: CASSANDRA-9317
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9317
 Project: Cassandra
  Issue Type: Improvement
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson
 Fix For: 3.x

 Attachments: 0001-populate-tmd-early.patch


 For CASSANDRA-6696 we need to know the local tokens earlier when starting up, 
 this is due to the fact that we need to know how to split the local ranges 
 when we flush for example (we can flush during commitlog replay etc).
 Breaking out the TMD population into its own method makes it possible to use 
 for the offline tools post 6696 as well - we will need to know tokens when 
 doing scrub etc



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Add ks and table to err log about large collections

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 a431d846c - d8b79d5af


Add ks and table to err log about large collections

Patch by Carl Yeksigian; reviewed by Tyler Hobbs for CASSANDRA-9286


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/15235ee6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/15235ee6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/15235ee6

Branch: refs/heads/cassandra-2.1
Commit: 15235ee63be2d8f39486ff97dbbdd8ed24bb9812
Parents: 917068d
Author: Tyler Hobbs tylerho...@apache.org
Authored: Fri May 8 13:28:33 2015 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Fri May 8 13:28:33 2015 -0500

--
 CHANGES.txt  |  2 ++
 .../cassandra/cql3/statements/SelectStatement.java   |  2 +-
 .../org/apache/cassandra/db/marshal/CollectionType.java  | 11 +++
 src/java/org/apache/cassandra/db/marshal/ListType.java   |  5 +++--
 src/java/org/apache/cassandra/db/marshal/MapType.java|  5 +++--
 src/java/org/apache/cassandra/db/marshal/SetType.java|  5 +++--
 6 files changed, 19 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/15235ee6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1540d28..d3715c4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 2.0.15:
+ * Include keyspace and table name in error log for collections over the size
+   limit (CASSANDRA-9286)
  * Avoid potential overlap in LCS with single-partition sstables 
(CASSANDRA-9322)
  * Log warning message when a table is queried before the schema has fully
propagated (CASSANDRA-9136)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/15235ee6/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 6b3c781..8a4deb6 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -1399,7 +1399,7 @@ public class SelectStatement implements CQLStatement, 
MeasurableForPreparedCache
 if (name.type.isCollection())
 {
 ListPairByteBuffer, Column collection = 
group.getCollection(name.name.key);
-result.add(collection == null ? null : 
((CollectionType)name.type).serialize(collection));
+result.add(collection == null ? null : 
((CollectionType)name.type).serialize(name, collection));
 }
 else
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/15235ee6/src/java/org/apache/cassandra/db/marshal/CollectionType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/CollectionType.java 
b/src/java/org/apache/cassandra/db/marshal/CollectionType.java
index 29b77a0..d1ae130 100644
--- a/src/java/org/apache/cassandra/db/marshal/CollectionType.java
+++ b/src/java/org/apache/cassandra/db/marshal/CollectionType.java
@@ -23,6 +23,7 @@ import java.util.List;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import org.apache.cassandra.cql3.CFDefinition;
 import org.apache.cassandra.cql3.CQL3Type;
 import org.apache.cassandra.db.Column;
 import org.apache.cassandra.serializers.MarshalException;
@@ -58,7 +59,7 @@ public abstract class CollectionTypeT extends 
AbstractTypeT
 
 protected abstract void appendToStringBuilder(StringBuilder sb);
 
-public abstract ByteBuffer serialize(ListPairByteBuffer, Column 
columns);
+public abstract ByteBuffer serialize(CFDefinition.Name name, 
ListPairByteBuffer, Column columns);
 
 @Override
 public String toString()
@@ -108,13 +109,15 @@ public abstract class CollectionTypeT extends 
AbstractTypeT
 return (ByteBuffer)result.flip();
 }
 
-protected ListPairByteBuffer, Column 
enforceLimit(ListPairByteBuffer, Column columns)
+protected ListPairByteBuffer, Column enforceLimit(CFDefinition.Name 
name, ListPairByteBuffer, Column columns)
 {
 if (columns.size() = MAX_ELEMENTS)
 return columns;
 
-logger.error(Detected collection with {} elements, more than the {} 
limit. Only the first {} elements will be returned to the client. 
-   + Please see 
http://cassandra.apache.org/doc/cql3/CQL.html#collections for more details., 
columns.size(), MAX_ELEMENTS, MAX_ELEMENTS);
+logger.error(Detected collection for table {}.{} with {} elements, 
more than the {} limit. Only the first {}
+

[jira] [Commented] (CASSANDRA-8559) OOM caused by large tombstone warning.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535252#comment-14535252
 ] 

Ariel Weisberg commented on CASSANDRA-8559:
---

[~iamaleksey] What's the data model or queries that cause this? Are they worth 
including in our testing process?

 OOM caused by large tombstone warning.
 --

 Key: CASSANDRA-8559
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8559
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.0.11 / 2.1
Reporter: Dominic Letz
Assignee: Aleksey Yeschenko
  Labels: tombstone
 Fix For: 2.0.15, 2.1.5

 Attachments: 8559.txt, Selection_048.png, cassandra-2.0.11-8559.txt, 
 stacktrace.log


 When running with high amount of tombstones the error message generation from 
 CASSANDRA-6117 can lead to out of memory situation with the default setting.
 Attached a heapdump viewed in visualvm showing how this construct created two 
 777mb strings to print the error message for a read query and then crashed 
 OOM.
 {code}
 if (respectTombstoneThresholds()  columnCounter.ignored()  
 DatabaseDescriptor.getTombstoneWarnThreshold())
 {
 StringBuilder sb = new StringBuilder();
 CellNameType type = container.metadata().comparator;
 for (ColumnSlice sl : slices)
 {
 assert sl != null;
 sb.append('[');
 sb.append(type.getString(sl.start));
 sb.append('-');
 sb.append(type.getString(sl.finish));
 sb.append(']');
 }
 logger.warn(Read {} live and {} tombstoned cells in {}.{} (see 
 tombstone_warn_threshold). {} columns was requested, slices={}, delInfo={},
 columnCounter.live(), columnCounter.ignored(), 
 container.metadata().ksName, container.metadata().cfName, count, sb, 
 container.deletionInfo());
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8564) Harmless exception logged as ERROR


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535253#comment-14535253
 ] 

Tyler Hobbs commented on CASSANDRA-8564:


+1

 Harmless exception logged as ERROR
 --

 Key: CASSANDRA-8564
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8564
 Project: Cassandra
  Issue Type: Bug
Reporter: Philip Thompson
Assignee: T Jake Luciani
Priority: Minor
 Fix For: 2.1.x


 After CASSANDRA-8474, when running the dtest 
 counter_test.py:TestCounters.upgrade_test, we now see the following in the 
 log:
 {code}
 ERROR [CompactionExecutor:2] 2015-01-05 13:59:51,003 CassandraDaemon.java:170 
 - Exception in thread Thread[CompactionExecutor:2,1,main]
 java.util.concurrent.RejectedExecutionException: Task 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@5e8ea989 
 rejected from 
 org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor@7fc92f94[Terminated,
  pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 5]
 at 
 java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
  ~[na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821) 
 ~[na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325)
  ~[na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:530)
  ~[na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:619)
  ~[na:1.7.0_67]
 at 
 org.apache.cassandra.io.sstable.SSTableReader.scheduleTidy(SSTableReader.java:638)
  ~[main/:na]
 at 
 org.apache.cassandra.io.sstable.SSTableReader.tidy(SSTableReader.java:619) 
 ~[main/:na]
 at 
 org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1650)
  ~[main/:na]
 at 
 org.apache.cassandra.db.DataTracker.replaceReaders(DataTracker.java:409) 
 ~[main/:na]
 at 
 org.apache.cassandra.db.DataTracker.replaceWithNewInstances(DataTracker.java:303)
  ~[main/:na]
 at 
 org.apache.cassandra.io.sstable.SSTableRewriter.moveStarts(SSTableRewriter.java:254)
  ~[main/:na]
 at 
 org.apache.cassandra.io.sstable.SSTableRewriter.abort(SSTableRewriter.java:180)
  ~[main/:na]
 at 
 org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:205)
  ~[main/:na]
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[main/:na]
 at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
  ~[main/:na]
 at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
  ~[main/:na]
 at 
 org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
  ~[main/:na]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 ~[na:1.7.0_67]
 at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
 ~[na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  ~[na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_67]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
 Suppressed: java.util.concurrent.RejectedExecutionException: Task 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@681c91de 
 rejected from 
 org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor@7fc92f94[Terminated,
  pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 5]
 at 
 java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
  ~[na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821) 
 ~[na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325)
  ~[na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:530)
  ~[na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:619)
  ~[na:1.7.0_67]
 at 
 org.apache.cassandra.io.sstable.SSTableReader.scheduleTidy(SSTableReader.java:638)
  ~[main/:na]
 at 
 org.apache.cassandra.io.sstable.SSTableReader.tidy(SSTableReader.java:619) 
 ~[main/:na]
 at

[jira] [Commented] (CASSANDRA-9060) Anticompaction hangs on bloom filter bitset serialization

[
https://issues.apache.org/jira/browse/CASSANDRA-9060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535255#comment-14535255
]

Ariel Weisberg commented on CASSANDRA-9060:
---

[~krummas] Did this only happen because of the upgrade or is there a
workload/data model we should be testing anti-compaction with?

Anticompaction hangs on bloom filter bitset serialization
--

Key: CASSANDRA-9060
URL: https://issues.apache.org/jira/browse/CASSANDRA-9060
Project: Cassandra
Issue Type: Bug
Reporter: Gustav Munkby
Assignee: Gustav Munkby
Priority: Minor
Fix For: 2.1.5

Attachments: 0001-another-tweak-to-9060.patch, 2.1-9060-simple.patch,
trunk-9060.patch

I tried running an incremental repair against a 15-node vnode-cluster with
roughly 500GB data running on 2.1.3-SNAPSHOT, without performing the
suggested migration steps. I manually chose a small range for the repair
(using --start/end-token). The actual repair part took almost no time at all,
but the anticompactions took a lot of time (not surprisingly).
Obviously, this might not be the ideal way to run incremental repairs, but I
wanted to look into what made the whole process so slow. The results were
rather surprising. The majority of the time was spent serializing bloom
filters.
The reason seemed to be two-fold. First, the bloom-filters generated were
huge (probably because the original SSTables were large). With a proper
migration to incremental repairs, I'm guessing this would not happen.
Secondly, however, the bloom filters were being written to the output one
byte at a time (with quite a few type-conversions on the way) to transform
the little-endian in-memory representation to the big-endian on-disk
representation.
I have implemented a solution where big-endian is used in-memory as well as
on-disk, which obviously makes de-/serialization much, much faster. This
introduces some slight overhead when checking the bloom filter, but I can't
see how that would be problematic. An obvious alternative would be to still
perform the serialization/deserialization using a byte array, but perform the
byte-order swap there.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9321) Aggregate UDFs allow SFUNC return type to differ from STYPE if FFUNC specified


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535297#comment-14535297
 ] 

Tyler Hobbs commented on CASSANDRA-9321:


Tests look good (no regressions), +1.

 Aggregate UDFs allow SFUNC return type to differ from STYPE if FFUNC specified
 --

 Key: CASSANDRA-9321
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9321
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Zachary Kurey
Assignee: Robert Stupp
 Fix For: 3.0

 Attachments: 9321.txt


 When a final function is specified in an aggregate C* allows the return type 
 of the state function to not match the state type.  
 Allowing the mismatch if a final function is specified seems to be 
 intentional as if you don't provide a final function and you provide a state 
 function with a return type that doesn't match the state type, then C* gives 
 you an error that states that they must match unless a final function is 
 specified.  
 It seems incorrect regardless of whether or not a final function is present 
 to allow the state functions return type to vary from state type.  And indeed 
 if you do so it produces an error when you try to use the aggregate.
 Here is a simple example that shows the problem:
 {code}
 CREATE OR REPLACE FUNCTION state_func(state int, p2 int)
 RETURNS double LANGUAGE java AS 'return Double.valueOf(1.0);';
 CREATE OR REPLACE FUNCTION final_func(state int)
 RETURNS int
 LANGUAGE java
 AS 'return Integer.valueOf(1);';
 CREATE OR REPLACE AGGREGATE my_aggregate( int )
 SFUNC state_func
 STYPE int
 FINALFUNC final_func
 INITCOND 1;
 SELECT my_aggregate(column) FROM table;
 {code}
 The select produces the error:
 {noformat}
 Aggregate 'ks.my_aggregate : (int) - int' exists but hasn't been loaded 
 successfully for the following reason: Referenced state function 
 'ks.state_func [double, int]' for aggregate 'ks.my_aggregate' does not exist.
 {noformat}
 I was reproducing this with 3.0 trunk, though now I just grabbed the latest 
 and there is an NPE instead of the error above:
 {noformat}
 java.lang.NullPointerException: at index 1
   at 
 com.google.common.collect.ObjectArrays.checkElementNotNull(ObjectArrays.java:240)
  ~[guava-16.0.jar:na]
   at 
 com.google.common.collect.ImmutableSet.construct(ImmutableSet.java:195) 
 ~[guava-16.0.jar:na]
   at com.google.common.collect.ImmutableSet.of(ImmutableSet.java:116) 
 ~[guava-16.0.jar:na]
   at 
 org.apache.cassandra.cql3.functions.UDAggregate.getFunctions(UDAggregate.java:110)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.selection.AbstractFunctionSelector$1.getFunctions(AbstractFunctionSelector.java:78)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.selection.SelectorFactories.getFunctions(SelectorFactories.java:105)
  ~[main/:na]
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8954) risk analysis of patches based on past defects

2015-05-08 Thread Russ Hatch (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Russ Hatch updated CASSANDRA-8954:
--
Description:
Some changes to source are much more risky than others, and we can analyze data
from JIRA + git to make educated guesses about risk level. This is a backwards
looking technique with limitations but still may be useful (yes, the past does
not equal the future!).

(disclaimer: I did not come up with this technique).

The executive summary: 1) correlate changes with defects, by code unit such as
filename 2) quantify risk of new patches by combining correlation with a
measure of change size, as (correlation * change_size)

The basic idea is to build a tool which correlates past Defect tickets to the
files which were changed to fix them. If a Defect required changes to specific
files to fix, then in some sense past changes to those files (or their original
implementations) were problematic. Therefore, future changes to those files
carry some potential risk as well.

This requires getting an occasional dump of Defect type issues, and an
occasional dump of commit messages. Defects would have to be associated to
commits based on a text search of commit messages. From there we build a
weighted model of which source files get touched the most to fix defects (say
giving each file name a ranking of 1 to 10 where 10 carries the most risk).

To analyze specific patches going forward we look at the defect weight for that
source file, and factor in a metric for a patch's changes in that file (maybe
(lines changed/total lines), OR (change in cyclomatic complexity/total
complexity)). Out of this we get a number representing a theoretical risk.

was:
Some changes to source are much more risky than others, and we can analyze data
from JIRA + git to make educated guesses about risk level. This is a backwards
looking technique with limitations but still may be useful (yes, the past does
not equal the future!).

(disclaimer: I did not come up with this technique).

risk analysis of patches based on past defects
--

Key: CASSANDRA-8954
URL: https://issues.apache.org/jira/browse/CASSANDRA-8954
Project: Cassandra
Issue Type: Test
Reporter: Russ Hatch
Assignee: Russ Hatch

Some changes to source are much more risky than others, and we can analyze
data from JIRA + git to make educated guesses about risk level. This is a
backwards looking technique with limitations but still may be useful (yes,
the past does not equal the future!).
(disclaimer: I did not come up with this technique).
The executive summary: 1) correlate changes with defects, by code unit such
as filename 2) quantify risk of new patches by combining correlation with a
measure of change size, as (correlation * change_size)
The basic idea is to build a tool which correlates past Defect tickets to the
files which were changed to fix them. If a Defect required changes to
specific files to fix, then in some sense past changes to those files (or
their original implementations) were problematic. Therefore, future changes
to those files carry some potential risk as well.
This requires getting an occasional dump of Defect type issues, and an
occasional dump of commit messages. Defects would have to be associated to
commits based on a text search of commit messages. From there we build a
weighted model of which source files get touched the most to fix defects (say
giving each file name a ranking of 1 to 10 where 10 carries the most risk).
To analyze specific patches going forward we look at the defect weight for
that source file, and factor in a metric for a patch's changes in that file
(maybe (lines changed/total lines), OR (change in cyclomatic complexity/total

[jira] [Comment Edited] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535369#comment-14535369
]

Benedict edited comment on CASSANDRA-9318 at 5/8/15 7:52 PM:
-

bq. because our existing load shedding is fine at recovering from temporary
spikes in load

Are you certain? The recent testing Ariel did on CASSANDRA-8670 demonstrated
the MUTATION stage was what was bringing the cluster down, not the ExpiringMap;
and this was in a small cluster.

If anything, I suspect our ability to prune these messages is also
theoretically worse, on top of this practical datapoint, because it is done on
dequeue, whereas the ExpiringMap and MessagingService (whilst having a slightly
longer expiry) is done asynchronously (or on enqueue) and cannot be blocked by
e.g. flush. What I'm effectively suggesting is simply making all of the load
shedding all happen on enqueue, and be based on queue length as well as time,
so that our load shedding really is simply more robust.

The coordinator is also on the right side of the equation: as the cluster
grows, any single problems should spread out to the coordinators more slowly,
whereas the coordinator's ability to flood a processing node scales up at the
same (well, inverted) rate.

was (Author: benedict):
bq. because our existing load shedding is fine at recovering from temporary
spikes in load

Are you certain? The recent testing Ariel did on CASSANDRA-8670 demonstrated
the MUTATION stage was what was bringing the cluster down, not the ExpiringMap;
and this was in a small cluster.

Bound the number of in-flight requests at the coordinator
-

Key: CASSANDRA-9318
URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
Project: Cassandra
Issue Type: Improvement
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Fix For: 2.1.x

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9328) WriteTimeoutException thrown when LWT concurrency 1, despite the query duration taking MUCH less than cas_contention_timeout_in_ms

2015-05-08 Thread Sylvain Lebresne (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535151#comment-14535151
 ] 

Sylvain Lebresne commented on CASSANDRA-9328:
-

bq.  it seems that some updates that throw WTE really succeed

A WTE means that update may or may not be applied, so yes, the update may have 
succeeded if you get a WTE.

 WriteTimeoutException thrown when LWT concurrency  1, despite the query 
 duration taking MUCH less than cas_contention_timeout_in_ms
 

 Key: CASSANDRA-9328
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9328
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Aaron Whiteside
Assignee: Benjamin Lerer
Priority: Critical
 Fix For: 2.1.x

 Attachments: CassandraLWTTest.java, CassandraLWTTest2.java


 WriteTimeoutException thrown when LWT concurrency  1, despite the query 
 duration taking MUCH less than cas_contention_timeout_in_ms.
 Unit test attached, run against a 3 node cluster running 2.1.5.
 If you reduce the threadCount to 1, you never see a WriteTimeoutException. If 
 the WTE is due to not being able to communicate with other nodes, why does 
 the concurrency 1 cause inter-node communication to fail?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d8b79d5a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d8b79d5a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d8b79d5a

Branch: refs/heads/cassandra-2.1
Commit: d8b79d5aff93b75198bb60e5140ff3fea912d387
Parents: a431d84 15235ee
Author: Tyler Hobbs tylerho...@apache.org
Authored: Fri May 8 13:36:10 2015 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Fri May 8 13:36:10 2015 -0500

--
 CHANGES.txt|  2 ++
 .../cassandra/cql3/statements/SelectStatement.java |  2 +-
 .../apache/cassandra/db/marshal/CollectionType.java| 13 -
 3 files changed, 11 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d8b79d5a/CHANGES.txt
--
diff --cc CHANGES.txt
index 2b7a355,d3715c4..162b34f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,21 -1,6 +1,23 @@@
 -2.0.15:
 +2.1.6
 + * Delete processed sstables in sstablesplit/sstableupgrade (CASSANDRA-8606)
 + * Improve sstable exclusion from partition tombstones (CASSANDRA-9298)
 + * Validate the indexed column rather than the cell's contents for 2i 
(CASSANDRA-9057)
 + * Add support for top-k custom 2i queries (CASSANDRA-8717)
 + * Fix error when dropping table during compaction (CASSANDRA-9251)
 + * cassandra-stress supports validation operations over user profiles 
(CASSANDRA-8773)
 + * Add support for rate limiting log messages (CASSANDRA-9029)
 + * Log the partition key with tombstone warnings (CASSANDRA-8561)
 + * Reduce runWithCompactionsDisabled poll interval to 1ms (CASSANDRA-9271)
 + * Fix PITR commitlog replay (CASSANDRA-9195)
 + * GCInspector logs very different times (CASSANDRA-9124)
 + * Fix deleting from an empty list (CASSANDRA-9198)
 + * Update tuple and collection types that use a user-defined type when that 
UDT
 +   is modified (CASSANDRA-9148, CASSANDRA-9192)
 + * Use higher timeout for prepair and snapshot in repair (CASSANDRA-9261)
 + * Fix anticompaction blocking ANTI_ENTROPY stage (CASSANDRA-9151)
 +Merged from 2.0:
+  * Include keyspace and table name in error log for collections over the size
+limit (CASSANDRA-9286)
   * Avoid potential overlap in LCS with single-partition sstables 
(CASSANDRA-9322)
   * Log warning message when a table is queried before the schema has fully
 propagated (CASSANDRA-9136)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d8b79d5a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --cc src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 07e60d4,8a4deb6..51f4941
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@@ -1288,25 -1300,12 +1288,25 @@@ public class SelectStatement implement
  }
  }
  
 -private boolean hasValueForQuery(ColumnGroupMap staticGroup)
 +private static void addValue(Selection.ResultSetBuilder result, 
ColumnDefinition def, CQL3Row row, QueryOptions options)
  {
 -for (CFDefinition.Name name : 
Iterables.filter(selection.getColumns(), isStaticFilter))
 -if (staticGroup.hasValueFor(name.name.key))
 -return true;
 -return false;
 +if (row == null)
 +{
 +result.add((ByteBuffer)null);
 +return;
 +}
 +
 +if (def.type.isMultiCell())
 +{
 +ListCell cells = row.getMultiCellColumn(def.name);
 +ByteBuffer buffer = cells == null
 + ? null
-  : 
((CollectionType)def.type).serializeForNativeProtocol(cells, 
options.getProtocolVersion());
++ : 
((CollectionType)def.type).serializeForNativeProtocol(def, cells, 
options.getProtocolVersion());
 +result.add(buffer);
 +return;
 +}
 +
 +result.add(row.getColumn(def.name));
  }
  
  private boolean hasNoClusteringColumnsRestriction()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d8b79d5a/src/java/org/apache/cassandra/db/marshal/CollectionType.java
--
diff --cc src/java/org/apache/cassandra/db/marshal/CollectionType.java
index 24ad533,d1ae130..8662c1e
--- a/src/java/org/apache/cassandra/db/marshal/CollectionType.java
+++ b/src/java/org/apache/cassandra/db/marshal/CollectionType.java
@@@ -20,8 -20,6 +20,9 @@@ package org.apache.cassandra.db.marshal
  import java.nio.ByteBuffer;
  import

cassandra git commit: Add ks and table to err log about large collections

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 917068dcb - 15235ee63


Add ks and table to err log about large collections

Patch by Carl Yeksigian; reviewed by Tyler Hobbs for CASSANDRA-9286


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/15235ee6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/15235ee6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/15235ee6

Branch: refs/heads/cassandra-2.0
Commit: 15235ee63be2d8f39486ff97dbbdd8ed24bb9812
Parents: 917068d
Author: Tyler Hobbs tylerho...@apache.org
Authored: Fri May 8 13:28:33 2015 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Fri May 8 13:28:33 2015 -0500

--
 CHANGES.txt  |  2 ++
 .../cassandra/cql3/statements/SelectStatement.java   |  2 +-
 .../org/apache/cassandra/db/marshal/CollectionType.java  | 11 +++
 src/java/org/apache/cassandra/db/marshal/ListType.java   |  5 +++--
 src/java/org/apache/cassandra/db/marshal/MapType.java|  5 +++--
 src/java/org/apache/cassandra/db/marshal/SetType.java|  5 +++--
 6 files changed, 19 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/15235ee6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1540d28..d3715c4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 2.0.15:
+ * Include keyspace and table name in error log for collections over the size
+   limit (CASSANDRA-9286)
  * Avoid potential overlap in LCS with single-partition sstables 
(CASSANDRA-9322)
  * Log warning message when a table is queried before the schema has fully
propagated (CASSANDRA-9136)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/15235ee6/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 6b3c781..8a4deb6 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -1399,7 +1399,7 @@ public class SelectStatement implements CQLStatement, 
MeasurableForPreparedCache
 if (name.type.isCollection())
 {
 ListPairByteBuffer, Column collection = 
group.getCollection(name.name.key);
-result.add(collection == null ? null : 
((CollectionType)name.type).serialize(collection));
+result.add(collection == null ? null : 
((CollectionType)name.type).serialize(name, collection));
 }
 else
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/15235ee6/src/java/org/apache/cassandra/db/marshal/CollectionType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/CollectionType.java 
b/src/java/org/apache/cassandra/db/marshal/CollectionType.java
index 29b77a0..d1ae130 100644
--- a/src/java/org/apache/cassandra/db/marshal/CollectionType.java
+++ b/src/java/org/apache/cassandra/db/marshal/CollectionType.java
@@ -23,6 +23,7 @@ import java.util.List;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import org.apache.cassandra.cql3.CFDefinition;
 import org.apache.cassandra.cql3.CQL3Type;
 import org.apache.cassandra.db.Column;
 import org.apache.cassandra.serializers.MarshalException;
@@ -58,7 +59,7 @@ public abstract class CollectionTypeT extends 
AbstractTypeT
 
 protected abstract void appendToStringBuilder(StringBuilder sb);
 
-public abstract ByteBuffer serialize(ListPairByteBuffer, Column 
columns);
+public abstract ByteBuffer serialize(CFDefinition.Name name, 
ListPairByteBuffer, Column columns);
 
 @Override
 public String toString()
@@ -108,13 +109,15 @@ public abstract class CollectionTypeT extends 
AbstractTypeT
 return (ByteBuffer)result.flip();
 }
 
-protected ListPairByteBuffer, Column 
enforceLimit(ListPairByteBuffer, Column columns)
+protected ListPairByteBuffer, Column enforceLimit(CFDefinition.Name 
name, ListPairByteBuffer, Column columns)
 {
 if (columns.size() = MAX_ELEMENTS)
 return columns;
 
-logger.error(Detected collection with {} elements, more than the {} 
limit. Only the first {} elements will be returned to the client. 
-   + Please see 
http://cassandra.apache.org/doc/cql3/CQL.html#collections for more details., 
columns.size(), MAX_ELEMENTS, MAX_ELEMENTS);
+logger.error(Detected collection for table {}.{} with {} elements, 
more than the {} limit. Only the first {}
+

[jira] [Commented] (CASSANDRA-9300) token-generator - generated tokens too long


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535275#comment-14535275
 ] 

Tyler Hobbs commented on CASSANDRA-9300:


[~snazy] thanks for the patch, this tool doesn't get much love :)

Would you mind writing some very basic dtests?  They don't have to be 
comprehensive, but it would be good to catch at least obvious failures like 
this one.

 token-generator - generated tokens too long
 ---

 Key: CASSANDRA-9300
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9300
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Minor
 Fix For: 3.x, 2.1.x, 2.0.x

 Attachments: 9300.txt


 {{token-generator}} tool does not generate valid long values (2.0, 2.1, 
 trunk).
 E.g. 
 {code}
 tools/bin/token-generator 4
 DC #1:
   Node #1:0
   Node #2:   42535295865117307932921825928971026432
   Node #3:   85070591730234615865843651857942052864
   Node #4:  127605887595351923798765477786913079296
 {code}
 But should generate tokens like
 {code}
 -9223372036854775808
 -4611686018427387904
 0
 4611686018427387904
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9231) Support Routing Key as part of Partition Key


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535326#comment-14535326
 ] 

Benedict commented on CASSANDRA-9231:
-

bq. They wouldn't be providing arbitrary tokens, they would be providing 
arbitrary input to the hash function (for Random, MP3).

{code}
CREATE FUNCTION myOrderedTokenFct(a bigint) RETURNS bigint AS 'return a';
CREATE TABLE t (
   a int PRIMARY KEY,
   b text,
   c text
) with tokenizer=myOrderedTokenFct;
{code}
 
bq. Basically, this gets you very close to a per-table partitioner. The actual 
partitioner would just define the domain of the tokens and how they sort, but 
the actual computation would be per-table. And this for very, very little 
change to the syntax and barely more complexity code-wise than the routing 
key idea.



 Support Routing Key as part of Partition Key
 

 Key: CASSANDRA-9231
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9231
 Project: Cassandra
  Issue Type: Wish
  Components: Core
Reporter: Matthias Broecheler
 Fix For: 3.x


 Provide support for sub-dividing the partition key into a routing key and a 
 non-routing key component. Currently, all columns that make up the partition 
 key of the primary key are also routing keys, i.e. they determine which nodes 
 store the data. This proposal would give the data modeler the ability to 
 designate only a subset of the columns that comprise the partition key to be 
 routing keys. The non-routing key columns of the partition key identify the 
 partition but are not used to determine where to store the data.
 Consider the following example table definition:
 CREATE TABLE foo (
   a int,
   b int,
   c int,
   d int,
   PRIMARY KEY  (([a], b), c ) );
 (a,b) is the partition key, c is the clustering key, and d is just a column. 
 In addition, the square brackets identify the routing key as column a. This 
 means that only the value of column a is used to determine the node for data 
 placement (i.e. only the value of column a is murmur3 hashed to compute the 
 token). In addition, column b is needed to identify the partition but does 
 not influence the placement.
 This has the benefit that all rows with the same routing key (but potentially 
 different non-routing key columns of the partition key) are stored on the 
 same node and that knowledge of such co-locality can be exploited by 
 applications build on top of Cassandra.
 Currently, the only way to achieve co-locality is within a partition. 
 However, this approach has the limitations that: a) there are theoretical and 
 (more importantly) practical limitations on the size of a partition and b) 
 rows within a partition are ordered and an index is build to exploit such 
 ordering. For large partitions that overhead is significant if ordering isn't 
 needed.
 In other words, routing keys afford a simple means to achieve scalable 
 node-level co-locality without ordering while clustering keys afford 
 page-level co-locality with ordering. As such, they address different 
 co-locality needs giving the data modeler the flexibility to choose what is 
 needed for their application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9231) Support Routing Key as part of Partition Key


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535326#comment-14535326
 ] 

Benedict edited comment on CASSANDRA-9231 at 5/8/15 7:22 PM:
-

bq. They wouldn't be providing arbitrary tokens, they would be providing 
arbitrary input to the hash function (for Random, MP3).

{code}
CREATE FUNCTION myOrderedTokenFct(a bigint) RETURNS bigint AS 'return a';
CREATE TABLE t (
   a int PRIMARY KEY,
   b text,
   c text
) with tokenizer=myOrderedTokenFct;
{code}
 
bq. Basically, this gets you very close to a per-table partitioner. The actual 
partitioner would just define the domain of the tokens and how they sort, but 
the actual computation would be per-table. And this for very, very little 
change to the syntax and barely more complexity code-wise than the routing 
key idea.

It looks to me like these two statements disagree, but I may be mistaken.


was (Author: benedict):
bq. They wouldn't be providing arbitrary tokens, they would be providing 
arbitrary input to the hash function (for Random, MP3).

{code}
CREATE FUNCTION myOrderedTokenFct(a bigint) RETURNS bigint AS 'return a';
CREATE TABLE t (
   a int PRIMARY KEY,
   b text,
   c text
) with tokenizer=myOrderedTokenFct;
{code}
 
bq. Basically, this gets you very close to a per-table partitioner. The actual 
partitioner would just define the domain of the tokens and how they sort, but 
the actual computation would be per-table. And this for very, very little 
change to the syntax and barely more complexity code-wise than the routing 
key idea.



 Support Routing Key as part of Partition Key
 

 Key: CASSANDRA-9231
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9231
 Project: Cassandra
  Issue Type: Wish
  Components: Core
Reporter: Matthias Broecheler
 Fix For: 3.x


 Provide support for sub-dividing the partition key into a routing key and a 
 non-routing key component. Currently, all columns that make up the partition 
 key of the primary key are also routing keys, i.e. they determine which nodes 
 store the data. This proposal would give the data modeler the ability to 
 designate only a subset of the columns that comprise the partition key to be 
 routing keys. The non-routing key columns of the partition key identify the 
 partition but are not used to determine where to store the data.
 Consider the following example table definition:
 CREATE TABLE foo (
   a int,
   b int,
   c int,
   d int,
   PRIMARY KEY  (([a], b), c ) );
 (a,b) is the partition key, c is the clustering key, and d is just a column. 
 In addition, the square brackets identify the routing key as column a. This 
 means that only the value of column a is used to determine the node for data 
 placement (i.e. only the value of column a is murmur3 hashed to compute the 
 token). In addition, column b is needed to identify the partition but does 
 not influence the placement.
 This has the benefit that all rows with the same routing key (but potentially 
 different non-routing key columns of the partition key) are stored on the 
 same node and that knowledge of such co-locality can be exploited by 
 applications build on top of Cassandra.
 Currently, the only way to achieve co-locality is within a partition. 
 However, this approach has the limitations that: a) there are theoretical and 
 (more importantly) practical limitations on the size of a partition and b) 
 rows within a partition are ordered and an index is build to exploit such 
 ordering. For large partitions that overhead is significant if ordering isn't 
 needed.
 In other words, routing keys afford a simple means to achieve scalable 
 node-level co-locality without ordering while clustering keys afford 
 page-level co-locality with ordering. As such, they address different 
 co-locality needs giving the data modeler the flexibility to choose what is 
 needed for their application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535356#comment-14535356
]

Jonathan Ellis commented on CASSANDRA-9318:
---

bq. it sounds like Jonathan is suggesting we simply prune our ExpiringMap based
on bytes tracked as well as time?

No, I'm suggesting we abort requests more aggressively with OverloadedException
*before sending them to replicas*. One place this might make sense is
sendToHintedEndpoints, where we already throw OE.

Right now we only throw OE once we start writing hints for a node that is in
trouble. This doesn't seem to be aggressive enough. (Although, most of our
users are on 2.0 where we allowed 8x as many hints in flight before starting to
throttle.)

So, I am suggesting we also track requests outstanding (perhaps with the
ExpiringMap as you suggest) as well and stop accepting requests once we hit a
reasonable limit of you can't possibly process more requests than this in
parallel.

The ExpiringMap requests are already in-flight and cannot be cancelled, so
their effect on other nodes cannot be rescinded, and imposing a limit does
not stop us issuing more requests to the nodes in the cluster that are
failing to keep up and respond to us.

Right, and I'm fine with that. The goal is not to keep the replica completely
out of trouble. The goal is to keep the coordinator from falling over from
buffering EM and MessagingService entries that it can't drain fast enough.
Secondarily, this will help the replica too because our existing load shedding
is fine at recovering from temporary spikes in load. But our load shedding
isn't good enough to save it when the coordinators keep throwing more at it
when it's already overwhelmed.

Bound the number of in-flight requests at the coordinator
-

Key: CASSANDRA-9318
URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
Project: Cassandra
Issue Type: Improvement
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Fix For: 2.1.x

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8930) Add a warn notification for clients

2015-05-08 Thread Carl Yeksigian (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian updated CASSANDRA-8930:
--
Attachment: 8930-trunk-v2.txt

I've attached a patch which address the review comments except the last one. 
Since getting the remote warnings will take more time and wanted to try to get 
this into v4, the patch doesn't yet address the last point; v1 I'd be happy 
with coordinator only warnings, and then figure out remote warnings in a follow 
up ticket.

 Add a warn notification for clients
 ---

 Key: CASSANDRA-8930
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8930
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Carl Yeksigian
Assignee: Carl Yeksigian
  Labels: client-impacting, protocolv4
 Fix For: 3.x

 Attachments: 8930-trunk-v2.txt, 8930-trunk.txt


 Currently, if a query generates a warning, it is going to be logged server 
 side. If the person writing the query is not the admin, that warning isn't 
 going to have an impact on the query, and we're just going to fill up the 
 server logs.
 We should push these warnings back to the client so the driver users can make 
 necessary changes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9282) Warn on unlogged batches


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535373#comment-14535373
 ] 

Jonathan Ellis commented on CASSANDRA-9282:
---

Building an extra Set with string concatenation (maybe use Pair instead?) 
sounds expensive.  Which would be fine if we were only counting them for bad 
statements but the problem is that you don't get the are these mutations in 
the same partition division for free that you did in the last place.

 Warn on unlogged batches
 

 Key: CASSANDRA-9282
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9282
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.1.x


 At least until CASSANDRA-8303 is done and we can block them entirely, we 
 should log a warning when unlogged batches across multiple partition keys are 
 used.  This could either be done by backporting NoSpamLogger and blindly 
 logging every time, or we could add a threshold and warn when more than 10 
 keys are seen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9328) WriteTimeoutException thrown when LWT concurrency 1, despite the query duration taking MUCH less than cas_contention_timeout_in_ms

2015-05-08 Thread Aaron Whiteside (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron Whiteside updated CASSANDRA-9328:
---
Attachment: CassandraLWTTest2.java

 WriteTimeoutException thrown when LWT concurrency  1, despite the query 
 duration taking MUCH less than cas_contention_timeout_in_ms
 

 Key: CASSANDRA-9328
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9328
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Aaron Whiteside
Assignee: Benjamin Lerer
Priority: Critical
 Fix For: 2.1.x

 Attachments: CassandraLWTTest.java, CassandraLWTTest2.java


 WriteTimeoutException thrown when LWT concurrency  1, despite the query 
 duration taking MUCH less than cas_contention_timeout_in_ms.
 Unit test attached, run against a 3 node cluster running 2.1.5.
 If you reduce the threadCount to 1, you never see a WriteTimeoutException. If 
 the WTE is due to not being able to communicate with other nodes, why does 
 the concurrency 1 cause inter-node communication to fail?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9328) WriteTimeoutException thrown when LWT concurrency 1, despite the query duration taking MUCH less than cas_contention_timeout_in_ms

2015-05-08 Thread Aaron Whiteside (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535107#comment-14535107
 ] 

Aaron Whiteside commented on CASSANDRA-9328:


I've found if I retry the WriteTimeoutException's I get corrupted data, it 
seems that some updates that throw WTE really succeed..

See new attached test.

I'd say that LWT/CAS does not work at all, it's completely broken (not atomic). 
If it were just slow duration contention that would be fine, since the result 
would be correct (one would hope).

 WriteTimeoutException thrown when LWT concurrency  1, despite the query 
 duration taking MUCH less than cas_contention_timeout_in_ms
 

 Key: CASSANDRA-9328
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9328
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Aaron Whiteside
Assignee: Benjamin Lerer
Priority: Critical
 Fix For: 2.1.x

 Attachments: CassandraLWTTest.java, CassandraLWTTest2.java


 WriteTimeoutException thrown when LWT concurrency  1, despite the query 
 duration taking MUCH less than cas_contention_timeout_in_ms.
 Unit test attached, run against a 3 node cluster running 2.1.5.
 If you reduce the threadCount to 1, you never see a WriteTimeoutException. If 
 the WTE is due to not being able to communicate with other nodes, why does 
 the concurrency 1 cause inter-node communication to fail?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9306) Test coverage for cqlsh COPY

2015-05-08 Thread Jim Witschey (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535139#comment-14535139
 ] 

Jim Witschey commented on CASSANDRA-9306:
-

CQL {{COPY}} [only has dtests in the {{cqlsh}} 
tests|https://gist.github.com/mambocab/1263032317de6f2caf79]. I'll see if I can 
make my tests use both cqlsh and straight CQL. I can't, I'll open another 
ticket to test {{COPY}} in CQL. 

 Test coverage for cqlsh COPY
 

 Key: CASSANDRA-9306
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9306
 Project: Cassandra
  Issue Type: Test
  Components: Core
Reporter: Tyler Hobbs
Assignee: Jim Witschey
  Labels: cqlsh
 Fix For: 3.x, 2.1.x


 We need much more thorough test coverage for cqlsh's COPY TO/FROM commands.  
 There is one existing basic dtest ({{cqlsh_tests.py:TestCqlsh.test_copy_to}}) 
 that we can use as a starting point for new tests.
 The following things need to be tested:
 * Non-default delimiters
 * Null fields and non-default null markers
 * Skipping a header line
 * Explicit column ordering
 * Column names that need to be quoted
 * Every supported C* data type
 * Data that fails validation server-side
 * Wrong number of columns
 * Node going down during COPY operation
 In the non-failure cases, the tests should generally inserted data into 
 Cassandra, run COPY TO to dump the data to CSV, truncate, run COPY FROM to 
 reimport the data, and then verify that the reloaded data matches the 
 originally inserted data.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535339#comment-14535339
]

Benedict commented on CASSANDRA-9318:
-

I'm filling the blanks for CASSANDRA-5039, since I didn't write the ticket, but
that is the only sane implementation: for one, we no longer have a
BlockingQueue in most places that matter, but more importantly a bound that
blocked task submission could lead to problems with various other systems
seizing up, and e.g. write saturation stopping us serving reads, etc.

bq. At CL=1 and the way I hear people think about availability in C* I think
what you want is to get better at failing to hinting before the coordinator or
processing node overloads. Under overload conditions CL=1 is basically
synonymous with writing hints right?

Right.

bq. Maybe this is a congestion control problem? If we piggybacked information
in responses on congestion issues maybe we could make better decisions about
new requests such as rejecting a %age or going straight to hints before
resources have been committed across the cluster?

Yes, I completely agree (in broad strokes, anyway), but that's much more
complex than we seem to be discussing here, and I'm pretty sure not 2.1
material. Ideally we would be sending information back to coordinators
informing them that we have been load shedding, so that they can invalidate
their handlers for us in preference to those for anyone else. Another related
ticket is CASSANDRA-3852, which also IMO depends on a system like this.

Bound the number of in-flight requests at the coordinator
-

Key: CASSANDRA-9318
URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
Project: Cassandra
Issue Type: Improvement
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Fix For: 2.1.x

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535369#comment-14535369
 ] 

Benedict commented on CASSANDRA-9318:
-

bq. because our existing load shedding is fine at recovering from temporary 
spikes in load

Are you certain? The recent testing Ariel did on CASSANDRA-8670 demonstrated 
the MUTATION stage was what was bringing the cluster down, not the ExpiringMap; 
and this was in a small cluster.

If anything, I suspect our ability to prune these messages is also 
theoretically worse, on top of this practical datapoint, because it is done on 
dequeue, whereas the ExpiringMap (whilst having a slightly longer expiry) is 
done asynchronously and cannot be blocked by e.g. flush.

The coordinator is also on the right side of the equation: as the cluster 
grows, any single problems should spread out to the coordinators more slowly, 
whereas the coordinator's ability to flood a processing node scales up at the 
same (well, inverted) rate.

 Bound the number of in-flight requests at the coordinator
 -

 Key: CASSANDRA-9318
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
 Project: Cassandra
  Issue Type: Improvement
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 2.1.x


 It's possible to somewhat bound the amount of load accepted into the cluster 
 by bounding the number of in-flight requests and request bytes.
 An implementation might do something like track the number of outstanding 
 bytes and requests and if it reaches a high watermark disable read on client 
 connections until it goes back below some low watermark.
 Need to make sure that disabling read on the client connection won't 
 introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535382#comment-14535382
 ] 

Jonathan Ellis commented on CASSANDRA-9318:
---

We see lots of production deployments that log occasional messages about load 
shedding being triggered, so while it's not perfect I am comfortable that it 
does work as designed for temporary spikes.

Again -- and I apologize if this was already clear -- I'm proposing we *stop 
requests before adding them to the EM in the first place*, not more aggressive 
pruning of the EM.

 Bound the number of in-flight requests at the coordinator
 -

 Key: CASSANDRA-9318
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
 Project: Cassandra
  Issue Type: Improvement
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 2.1.x


 It's possible to somewhat bound the amount of load accepted into the cluster 
 by bounding the number of in-flight requests and request bytes.
 An implementation might do something like track the number of outstanding 
 bytes and requests and if it reaches a high watermark disable read on client 
 connections until it goes back below some low watermark.
 Need to make sure that disabling read on the client connection won't 
 introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8267) Only stream from unrepaired sstables during incremental repair

2015-05-08 Thread Yuki Morishita (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535385#comment-14535385
 ] 

Yuki Morishita commented on CASSANDRA-8267:
---

+1

 Only stream from unrepaired sstables during incremental repair
 --

 Key: CASSANDRA-8267
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8267
 Project: Cassandra
  Issue Type: Bug
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson
 Fix For: 3.x

 Attachments: 
 0001-Only-stream-from-unrepaired-sstables-during-incremen.patch, 
 8267-trunk.patch


 Seems we stream from all sstables even if we do incremental repair, we should 
 limit this to only stream from the unrepaired sstables if we do incremental 
 repair



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] cassandra git commit: Add ks and table to err log about large collections

Repository: cassandra
Updated Branches:
  refs/heads/trunk 28132de16 - 1f23bb4b3


Add ks and table to err log about large collections

Patch by Carl Yeksigian; reviewed by Tyler Hobbs for CASSANDRA-9286


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/15235ee6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/15235ee6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/15235ee6

Branch: refs/heads/trunk
Commit: 15235ee63be2d8f39486ff97dbbdd8ed24bb9812
Parents: 917068d
Author: Tyler Hobbs tylerho...@apache.org
Authored: Fri May 8 13:28:33 2015 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Fri May 8 13:28:33 2015 -0500

--
 CHANGES.txt  |  2 ++
 .../cassandra/cql3/statements/SelectStatement.java   |  2 +-
 .../org/apache/cassandra/db/marshal/CollectionType.java  | 11 +++
 src/java/org/apache/cassandra/db/marshal/ListType.java   |  5 +++--
 src/java/org/apache/cassandra/db/marshal/MapType.java|  5 +++--
 src/java/org/apache/cassandra/db/marshal/SetType.java|  5 +++--
 6 files changed, 19 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/15235ee6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1540d28..d3715c4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 2.0.15:
+ * Include keyspace and table name in error log for collections over the size
+   limit (CASSANDRA-9286)
  * Avoid potential overlap in LCS with single-partition sstables 
(CASSANDRA-9322)
  * Log warning message when a table is queried before the schema has fully
propagated (CASSANDRA-9136)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/15235ee6/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 6b3c781..8a4deb6 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -1399,7 +1399,7 @@ public class SelectStatement implements CQLStatement, 
MeasurableForPreparedCache
 if (name.type.isCollection())
 {
 ListPairByteBuffer, Column collection = 
group.getCollection(name.name.key);
-result.add(collection == null ? null : 
((CollectionType)name.type).serialize(collection));
+result.add(collection == null ? null : 
((CollectionType)name.type).serialize(name, collection));
 }
 else
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/15235ee6/src/java/org/apache/cassandra/db/marshal/CollectionType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/CollectionType.java 
b/src/java/org/apache/cassandra/db/marshal/CollectionType.java
index 29b77a0..d1ae130 100644
--- a/src/java/org/apache/cassandra/db/marshal/CollectionType.java
+++ b/src/java/org/apache/cassandra/db/marshal/CollectionType.java
@@ -23,6 +23,7 @@ import java.util.List;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import org.apache.cassandra.cql3.CFDefinition;
 import org.apache.cassandra.cql3.CQL3Type;
 import org.apache.cassandra.db.Column;
 import org.apache.cassandra.serializers.MarshalException;
@@ -58,7 +59,7 @@ public abstract class CollectionTypeT extends 
AbstractTypeT
 
 protected abstract void appendToStringBuilder(StringBuilder sb);
 
-public abstract ByteBuffer serialize(ListPairByteBuffer, Column 
columns);
+public abstract ByteBuffer serialize(CFDefinition.Name name, 
ListPairByteBuffer, Column columns);
 
 @Override
 public String toString()
@@ -108,13 +109,15 @@ public abstract class CollectionTypeT extends 
AbstractTypeT
 return (ByteBuffer)result.flip();
 }
 
-protected ListPairByteBuffer, Column 
enforceLimit(ListPairByteBuffer, Column columns)
+protected ListPairByteBuffer, Column enforceLimit(CFDefinition.Name 
name, ListPairByteBuffer, Column columns)
 {
 if (columns.size() = MAX_ELEMENTS)
 return columns;
 
-logger.error(Detected collection with {} elements, more than the {} 
limit. Only the first {} elements will be returned to the client. 
-   + Please see 
http://cassandra.apache.org/doc/cql3/CQL.html#collections for more details., 
columns.size(), MAX_ELEMENTS, MAX_ELEMENTS);
+logger.error(Detected collection for table {}.{} with {} elements, 
more than the {} limit. Only the first {}
+ +

[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d8b79d5a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d8b79d5a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d8b79d5a

Branch: refs/heads/trunk
Commit: d8b79d5aff93b75198bb60e5140ff3fea912d387
Parents: a431d84 15235ee
Author: Tyler Hobbs tylerho...@apache.org
Authored: Fri May 8 13:36:10 2015 -0500
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Fri May 8 13:36:10 2015 -0500

--
 CHANGES.txt|  2 ++
 .../cassandra/cql3/statements/SelectStatement.java |  2 +-
 .../apache/cassandra/db/marshal/CollectionType.java| 13 -
 3 files changed, 11 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d8b79d5a/CHANGES.txt
--
diff --cc CHANGES.txt
index 2b7a355,d3715c4..162b34f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,21 -1,6 +1,23 @@@
 -2.0.15:
 +2.1.6
 + * Delete processed sstables in sstablesplit/sstableupgrade (CASSANDRA-8606)
 + * Improve sstable exclusion from partition tombstones (CASSANDRA-9298)
 + * Validate the indexed column rather than the cell's contents for 2i 
(CASSANDRA-9057)
 + * Add support for top-k custom 2i queries (CASSANDRA-8717)
 + * Fix error when dropping table during compaction (CASSANDRA-9251)
 + * cassandra-stress supports validation operations over user profiles 
(CASSANDRA-8773)
 + * Add support for rate limiting log messages (CASSANDRA-9029)
 + * Log the partition key with tombstone warnings (CASSANDRA-8561)
 + * Reduce runWithCompactionsDisabled poll interval to 1ms (CASSANDRA-9271)
 + * Fix PITR commitlog replay (CASSANDRA-9195)
 + * GCInspector logs very different times (CASSANDRA-9124)
 + * Fix deleting from an empty list (CASSANDRA-9198)
 + * Update tuple and collection types that use a user-defined type when that 
UDT
 +   is modified (CASSANDRA-9148, CASSANDRA-9192)
 + * Use higher timeout for prepair and snapshot in repair (CASSANDRA-9261)
 + * Fix anticompaction blocking ANTI_ENTROPY stage (CASSANDRA-9151)
 +Merged from 2.0:
+  * Include keyspace and table name in error log for collections over the size
+limit (CASSANDRA-9286)
   * Avoid potential overlap in LCS with single-partition sstables 
(CASSANDRA-9322)
   * Log warning message when a table is queried before the schema has fully
 propagated (CASSANDRA-9136)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d8b79d5a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --cc src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 07e60d4,8a4deb6..51f4941
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@@ -1288,25 -1300,12 +1288,25 @@@ public class SelectStatement implement
  }
  }
  
 -private boolean hasValueForQuery(ColumnGroupMap staticGroup)
 +private static void addValue(Selection.ResultSetBuilder result, 
ColumnDefinition def, CQL3Row row, QueryOptions options)
  {
 -for (CFDefinition.Name name : 
Iterables.filter(selection.getColumns(), isStaticFilter))
 -if (staticGroup.hasValueFor(name.name.key))
 -return true;
 -return false;
 +if (row == null)
 +{
 +result.add((ByteBuffer)null);
 +return;
 +}
 +
 +if (def.type.isMultiCell())
 +{
 +ListCell cells = row.getMultiCellColumn(def.name);
 +ByteBuffer buffer = cells == null
 + ? null
-  : 
((CollectionType)def.type).serializeForNativeProtocol(cells, 
options.getProtocolVersion());
++ : 
((CollectionType)def.type).serializeForNativeProtocol(def, cells, 
options.getProtocolVersion());
 +result.add(buffer);
 +return;
 +}
 +
 +result.add(row.getColumn(def.name));
  }
  
  private boolean hasNoClusteringColumnsRestriction()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d8b79d5a/src/java/org/apache/cassandra/db/marshal/CollectionType.java
--
diff --cc src/java/org/apache/cassandra/db/marshal/CollectionType.java
index 24ad533,d1ae130..8662c1e
--- a/src/java/org/apache/cassandra/db/marshal/CollectionType.java
+++ b/src/java/org/apache/cassandra/db/marshal/CollectionType.java
@@@ -20,8 -20,6 +20,9 @@@ package org.apache.cassandra.db.marshal
  import java.nio.ByteBuffer;
  import java.util.List;

[jira] [Commented] (CASSANDRA-9231) Support Routing Key as part of Partition Key

[
https://issues.apache.org/jira/browse/CASSANDRA-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535319#comment-14535319
]

Tyler Hobbs commented on CASSANDRA-9231:

bq. However I would point out that letting the user provide an arbitrary token
lets them, for instance, break the order preserving assumptions of BOP, or
the fair distribution assumptions of the hash partitioner.

They wouldn't be providing arbitrary tokens, they would be providing arbitrary
input to the hash function (for Random, MP3). The distribution would be
approximately as fair as it would be without the transform step.

For BOP they would maintain the order of whatever the function returns, which
makes sense and seems like exactly what the user would want.

FWIW, I agree with Sylvain's preference for using functions rather than a
routing key, for the same reasons he lists.

Support Routing Key as part of Partition Key

Key: CASSANDRA-9231
URL: https://issues.apache.org/jira/browse/CASSANDRA-9231
Project: Cassandra
Issue Type: Wish
Components: Core
Reporter: Matthias Broecheler
Fix For: 3.x

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9300) token-generator - generated tokens too long

2015-05-08 Thread Robert Stupp (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535320#comment-14535320
 ] 

Robert Stupp commented on CASSANDRA-9300:
-

yup - will add some dtest for it

 token-generator - generated tokens too long
 ---

 Key: CASSANDRA-9300
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9300
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Minor
 Fix For: 3.x, 2.1.x, 2.0.x

 Attachments: 9300.txt


 {{token-generator}} tool does not generate valid long values (2.0, 2.1, 
 trunk).
 E.g. 
 {code}
 tools/bin/token-generator 4
 DC #1:
   Node #1:0
   Node #2:   42535295865117307932921825928971026432
   Node #3:   85070591730234615865843651857942052864
   Node #4:  127605887595351923798765477786913079296
 {code}
 But should generate tokens like
 {code}
 -9223372036854775808
 -4611686018427387904
 0
 4611686018427387904
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: explain CL compression options

Repository: cassandra
Updated Branches:
  refs/heads/trunk e9d1e7994 - 28132de16


explain CL compression options


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/28132de1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/28132de1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/28132de1

Branch: refs/heads/trunk
Commit: 28132de16c63e0273fd058f38a559fba6a54487b
Parents: e9d1e79
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri May 8 13:28:31 2015 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri May 8 13:28:34 2015 -0500

--
 conf/cassandra.yaml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/28132de1/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 35326f3..3bcef06 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -290,7 +290,8 @@ commitlog_sync_period_in_ms: 1
 commitlog_segment_size_in_mb: 32
 
 # Compression to apply to the commit log. If omitted, the commit log
-# will be written uncompressed.
+# will be written uncompressed.  LZ4, Snappy, and Deflate compressors
+# are supported.
 #commitlog_compression:
 #   - class_name: LZ4Compressor
 # parameters:

[jira] [Updated] (CASSANDRA-8954) risk analysis of patches based on past defects

2015-05-08 Thread Russ Hatch (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

(disclaimer: I did not come up with this technique).

To analyze specific patches going forward we look at the defect weight for that
source file, and factor in a metric for a patch's changes in that file (maybe
lines changed, or change in cyclomatic complexity). Out of this we get a number
representing a theoretical risk.

risk analysis of patches based on past defects
--

Key: CASSANDRA-8954
URL: https://issues.apache.org/jira/browse/CASSANDRA-8954
Project: Cassandra
Issue Type: Test
Reporter: Russ Hatch
Assignee: Russ Hatch

Some changes to source are much more risky than others, and we can analyze
data from JIRA + git to make educated guesses about risk level. This is a
backwards looking technique with limitations but still may be useful (yes,
the past does not equal the future!).
(disclaimer: I did not come up with this technique).
The basic idea is to build a tool which correlates past Defect tickets to the
files which were changed to fix them. If a Defect required changes to
specific files to fix, then in some sense past changes to those files (or
their original implementations) were problematic. Therefore, future changes
to those files carry some potential risk as well.
This requires getting an occasional dump of Defect type issues, and an
occasional dump of commit messages. Defects would have to be associated to
commits based on a text search of commit messages. From there we build a
weighted model of which source files get touched the most to fix defects (say
giving each file name a ranking of 1 to 10 where 10 carries the most risk).
To analyze specific patches going forward we look at the defect weight for
that source file, and factor in a metric for a patch's changes in that file
(maybe (lines changed/total lines), OR (change in cyclomatic complexity/total
complexity)). Out of this we get a number representing a theoretical risk.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

2015-05-08 Thread jake

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e2515790
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e2515790
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e2515790

Branch: refs/heads/trunk
Commit: e25157901bec34d26f251dbc15fa024a82f2bbb1
Parents: 1f23bb4 9e9846e
Author: T Jake Luciani j...@apache.org
Authored: Fri May 8 15:05:02 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Fri May 8 15:05:02 2015 -0400

--
 CHANGES.txt |  1 +
 .../DebuggableScheduledThreadPoolExecutor.java  | 27 
 .../cassandra/service/StorageService.java   |  7 +
 3 files changed, 35 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e2515790/CHANGES.txt
--
diff --cc CHANGES.txt
index dd3e503,4a9b34a..0dbc281
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,105 -1,5 +1,106 @@@
 +3.0
 + * Failure detector detects and ignores local pauses (CASSANDRA-9183)
 + * Remove Thrift dependencies in bundled tools (CASSANDRA-8358)
 + * Disable memory mapping of hsperfdata file for JVM statistics 
(CASSANDRA-9242)
 + * Add pre-startup checks to detect potential incompatibilities 
(CASSANDRA-8049)
 + * Distinguish between null and unset in protocol v4 (CASSANDRA-7304)
 + * Add user/role permissions for user-defined functions (CASSANDRA-7557)
 + * Allow cassandra config to be updated to restart daemon without unloading 
classes (CASSANDRA-9046)
 + * Don't initialize compaction writer before checking if iter is empty 
(CASSANDRA-9117)
 + * Don't execute any functions at prepare-time (CASSANDRA-9037)
 + * Share file handles between all instances of a SegmentedFile 
(CASSANDRA-8893)
 + * Make it possible to major compact LCS (CASSANDRA-7272)
 + * Make FunctionExecutionException extend RequestExecutionException
 +   (CASSANDRA-9055)
 + * Add support for SELECT JSON, INSERT JSON syntax and new toJson(), 
fromJson()
 +   functions (CASSANDRA-7970)
 + * Optimise max purgeable timestamp calculation in compaction (CASSANDRA-8920)
 + * Constrain internode message buffer sizes, and improve IO class hierarchy 
(CASSANDRA-8670) 
 + * New tool added to validate all sstables in a node (CASSANDRA-5791)
 + * Push notification when tracing completes for an operation (CASSANDRA-7807)
 + * Delay node up and node added notifications until native protocol 
server is started (CASSANDRA-8236)
 + * Compressed Commit Log (CASSANDRA-6809)
 + * Optimise IntervalTree (CASSANDRA-8988)
 + * Add a key-value payload for third party usage (CASSANDRA-8553, 9212)
 + * Bump metrics-reporter-config dependency for metrics 3.0 (CASSANDRA-8149)
 + * Partition intra-cluster message streams by size, not type (CASSANDRA-8789)
 + * Add WriteFailureException to native protocol, notify coordinator of
 +   write failures (CASSANDRA-8592)
 + * Convert SequentialWriter to nio (CASSANDRA-8709)
 + * Add role based access control (CASSANDRA-7653, 8650, 7216, 8760, 8849, 
8761, 8850)
 + * Record client ip address in tracing sessions (CASSANDRA-8162)
 + * Indicate partition key columns in response metadata for prepared
 +   statements (CASSANDRA-7660)
 + * Merge UUIDType and TimeUUIDType parse logic (CASSANDRA-8759)
 + * Avoid memory allocation when searching index summary (CASSANDRA-8793)
 + * Optimise (Time)?UUIDType Comparisons (CASSANDRA-8730)
 + * Make CRC32Ex into a separate maven dependency (CASSANDRA-8836)
 + * Use preloaded jemalloc w/ Unsafe (CASSANDRA-8714)
 + * Avoid accessing partitioner through StorageProxy (CASSANDRA-8244, 8268)
 + * Upgrade Metrics library and remove depricated metrics (CASSANDRA-5657)
 + * Serializing Row cache alternative, fully off heap (CASSANDRA-7438)
 + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707)
 + * Make CassandraException unchecked, extend RuntimeException (CASSANDRA-8560)
 + * Support direct buffer decompression for reads (CASSANDRA-8464)
 + * DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039)
 + * Group sstables for anticompaction correctly (CASSANDRA-8578)
 + * Add ReadFailureException to native protocol, respond
 +   immediately when replicas encounter errors while handling
 +   a read request (CASSANDRA-7886)
 + * Switch CommitLogSegment from RandomAccessFile to nio (CASSANDRA-8308)
 + * Allow mixing token and partition key restrictions (CASSANDRA-7016)
 + * Support index key/value entries on map collections (CASSANDRA-8473)
 + * Modernize schema tables (CASSANDRA-8261)
 + * Support for user-defined aggregation functions (CASSANDRA-8053)
 + * Fix NPE in SelectStatement with empty IN values (CASSANDRA-8419)
 + * Refactor SelectStatement, return IN results in natural order

[1/2] cassandra git commit: Fix for harmless exceptions being logged as ERROR

2015-05-08 Thread jake

Repository: cassandra
Updated Branches:
  refs/heads/trunk 1f23bb4b3 - e25157901


Fix for harmless exceptions being logged as ERROR

Patch by tjake; reviewed by thobbs for CASSANDRA-8564


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9e9846e7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9e9846e7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9e9846e7

Branch: refs/heads/trunk
Commit: 9e9846e73d2e9c702c8652d9e8744de31923ef6d
Parents: d8b79d5
Author: T Jake Luciani j...@apache.org
Authored: Thu May 7 16:22:56 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Fri May 8 15:03:22 2015 -0400

--
 CHANGES.txt |  1 +
 .../DebuggableScheduledThreadPoolExecutor.java  | 27 
 .../cassandra/service/StorageService.java   |  7 +
 3 files changed, 35 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e9846e7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 162b34f..4a9b34a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.6
+ * Fix for harmless exceptions logged as ERROR (CASSANDRA-8564)
  * Delete processed sstables in sstablesplit/sstableupgrade (CASSANDRA-8606)
  * Improve sstable exclusion from partition tombstones (CASSANDRA-9298)
  * Validate the indexed column rather than the cell's contents for 2i 
(CASSANDRA-9057)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e9846e7/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
--
diff --git 
a/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
 
b/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
index a301923..7226db8 100644
--- 
a/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
+++ 
b/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
@@ -19,6 +19,10 @@ package org.apache.cassandra.concurrent;
 
 import java.util.concurrent.*;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.cassandra.service.StorageService;
 import org.apache.cassandra.utils.JVMStabilityInspector;
 
 /**
@@ -30,19 +34,42 @@ import org.apache.cassandra.utils.JVMStabilityInspector;
  */
 public class DebuggableScheduledThreadPoolExecutor extends 
ScheduledThreadPoolExecutor
 {
+private static final Logger logger = 
LoggerFactory.getLogger(DebuggableScheduledThreadPoolExecutor.class);
+
+public static final RejectedExecutionHandler rejectedExecutionHandler = 
new RejectedExecutionHandler()
+{
+public void rejectedExecution(Runnable task, ThreadPoolExecutor 
executor)
+{
+if (executor.isShutdown())
+{
+if (!StorageService.instance.isInShutdownHook())
+throw new 
RejectedExecutionException(ScheduledThreadPoolExecutor has shut down.);
+
+logger.debug(ScheduledThreadPoolExecutor has shut down as 
part of C* shutdown);
+}
+else
+{
+throw new AssertionError(Unknown rejection of 
ScheduledThreadPoolExecutor task);
+}
+}
+};
+
 public DebuggableScheduledThreadPoolExecutor(int corePoolSize, String 
threadPoolName, int priority)
 {
 super(corePoolSize, new NamedThreadFactory(threadPoolName, priority));
+setRejectedExecutionHandler(rejectedExecutionHandler);
 }
 
 public DebuggableScheduledThreadPoolExecutor(int corePoolSize, 
ThreadFactory threadFactory)
 {
 super(corePoolSize, threadFactory);
+setRejectedExecutionHandler(rejectedExecutionHandler);
 }
 
 public DebuggableScheduledThreadPoolExecutor(String threadPoolName)
 {
 this(1, threadPoolName, Thread.NORM_PRIORITY);
+setRejectedExecutionHandler(rejectedExecutionHandler);
 }
 
 // We need this as well as the wrapper for the benefit of non-repeating 
tasks

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e9846e7/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 8521256..7f9259c 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -200,9 +200,15 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 public volatile VersionedValue.VersionedValueFactory

[jira] [Comment Edited] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535356#comment-14535356
]

Jonathan Ellis edited comment on CASSANDRA-9318 at 5/8/15 7:40 PM:
---

bq. it sounds like Jonathan is suggesting we simply prune our ExpiringMap based
on bytes tracked as well as time?

No, I'm suggesting we abort requests more aggressively with OverloadedException
*before sending them to replicas*. One place this might make sense is
sendToHintedEndpoints, where we already throw OE.

bq. The ExpiringMap requests are already in-flight and cannot be cancelled,
so their effect on other nodes cannot be rescinded, and imposing a limit does
not stop us issuing more requests to the nodes in the cluster that are failing
to keep up and respond to us.

was (Author: jbellis):
bq. it sounds like Jonathan is suggesting we simply prune our ExpiringMap based
on bytes tracked as well as time?

No, I'm suggesting we abort requests more aggressively with OverloadedException
*before sending them to replicas*. One place this might make sense is
sendToHintedEndpoints, where we already throw OE.

Bound the number of in-flight requests at the coordinator
-

Key: CASSANDRA-9318
URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
Project: Cassandra
Issue Type: Improvement
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Fix For: 2.1.x

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8954) risk analysis of patches based on past defects

2015-05-08 Thread Russ Hatch (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535367#comment-14535367
]

Russ Hatch commented on CASSANDRA-8954:
---

The proposed formula isn't necessarily rigid, and is not meant as a replacement
for common sense. We all know that a one line change can break an entire
system, and that a 5000 line patch can have zero impact too.

But it's a good starting point for quantifying change and evaluating risk. We
could adjust the metrics over time to become more (or less) sophisticated.

I think a couple of build tasks would be a good starting place for a prototype
of such a system. One to dump JIRA data, one to correlate JIRA with git commit
messages, and a final task to do code analysis and output the magic number.

risk analysis of patches based on past defects
--

Key: CASSANDRA-8954
URL: https://issues.apache.org/jira/browse/CASSANDRA-8954
Project: Cassandra
Issue Type: Test
Reporter: Russ Hatch
Assignee: Russ Hatch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: make sure lock gets unlocked

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 9e9846e73 - b66475c4e


make sure lock gets unlocked


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b66475c4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b66475c4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b66475c4

Branch: refs/heads/cassandra-2.1
Commit: b66475c4e0b7eb39e0225bbb69a67eff6e6405f8
Parents: 9e9846e
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Fri May 8 15:58:54 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Fri May 8 15:58:54 2015 -0400

--
 tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b66475c4/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java 
b/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
index e9d43a3..e2bdd4e 100644
--- a/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
+++ b/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
@@ -153,10 +153,10 @@ public class DynamicListE
 public void remove(NodeE node)
 {
 lock.writeLock().lock();
-assert node.value != null;
 node.value = null;
 try
 {
+assert node.value != null;
 size--;
 
 // go up through each level in the skip list, unlinking this node; 
this entails

[1/2] cassandra git commit: make sure lock gets unlocked

Repository: cassandra
Updated Branches:
  refs/heads/trunk e25157901 - 81c93084b


make sure lock gets unlocked


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b66475c4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b66475c4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b66475c4

Branch: refs/heads/trunk
Commit: b66475c4e0b7eb39e0225bbb69a67eff6e6405f8
Parents: 9e9846e
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Fri May 8 15:58:54 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Fri May 8 15:58:54 2015 -0400

--
 tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b66475c4/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java 
b/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
index e9d43a3..e2bdd4e 100644
--- a/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
+++ b/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
@@ -153,10 +153,10 @@ public class DynamicListE
 public void remove(NodeE node)
 {
 lock.writeLock().lock();
-assert node.value != null;
 node.value = null;
 try
 {
+assert node.value != null;
 size--;
 
 // go up through each level in the skip list, unlinking this node; 
this entails

[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/81c93084
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/81c93084
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/81c93084

Branch: refs/heads/trunk
Commit: 81c93084ba482508cb66c45bf7f19c07a35605a0
Parents: e251579 b66475c
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Fri May 8 15:59:37 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Fri May 8 15:59:37 2015 -0400

--
 tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535400#comment-14535400
 ] 

Ariel Weisberg commented on CASSANDRA-9318:
---

So it's news to me that we can throw OE. I thought if we were capable of 
writing a hint we had to do it. With token aware clients and CL=1 we will 
always be in a position to accept requests and meet the contract so I thought 
to have A we must accept it.



 Bound the number of in-flight requests at the coordinator
 -

 Key: CASSANDRA-9318
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
 Project: Cassandra
  Issue Type: Improvement
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 2.1.x


 It's possible to somewhat bound the amount of load accepted into the cluster 
 by bounding the number of in-flight requests and request bytes.
 An implementation might do something like track the number of outstanding 
 bytes and requests and if it reaches a high watermark disable read on client 
 connections until it goes back below some low watermark.
 Need to make sure that disabling read on the client connection won't 
 introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535402#comment-14535402
]

Benedict commented on CASSANDRA-9318:
-

bq. We see lots of production deployments that log occasional messages about
load shedding being triggered

Regrettably this log message doesn't indicate the load shedding was
sufficiently useful. There could be no load shedding for several minutes, or an
arbitrary interval, then for it to happen en masse, so seeing this message only
indicates load shedding was needed, and it fortunately happened in time to
prevent the node failing, but not that it generally is capable of preventing
the node failing.

bq. Again – and I apologize if this was already clear

Apologies; I did not fully digest this aspect of your most recent response
before responding myself. I'm not at all opposed to that, but I'm not sure why
it isn't already preventing these problems from occurring in our simple tests?
It currently bounds the number of in flight requests low enough we should be
seeing these overloaded exceptions during a lengthy flush on another node, but
we don't (again, referring to ariel's recent test as an example).

I don't really see them as in conflict with one another, but it seems like this
is something that could be lowered by operators? Especially with Jake's patch
providing histograms of request size. What changes are you proposing?

Bound the number of in-flight requests at the coordinator
-

Key: CASSANDRA-9318
URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
Project: Cassandra
Issue Type: Improvement
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Fix For: 2.1.x

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: fixup last fix for bad lock guard

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 b66475c4e - def6b5fa1


fixup last fix for bad lock guard


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/def6b5fa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/def6b5fa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/def6b5fa

Branch: refs/heads/cassandra-2.1
Commit: def6b5fa19c27d74bc69710b4e26d74dde9dd609
Parents: b66475c
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Fri May 8 16:08:36 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Fri May 8 16:08:36 2015 -0400

--
 .../stress/src/org/apache/cassandra/stress/util/DynamicList.java  | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/def6b5fa/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java 
b/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
index e2bdd4e..ee04063 100644
--- a/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
+++ b/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
@@ -153,10 +153,11 @@ public class DynamicListE
 public void remove(NodeE node)
 {
 lock.writeLock().lock();
-node.value = null;
 try
 {
 assert node.value != null;
+node.value = null;
+
 size--;
 
 // go up through each level in the skip list, unlinking this node; 
this entails

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535404#comment-14535404
 ] 

Benedict commented on CASSANDRA-9318:
-

bq. So it's news to me that we can throw OE

Must admit it's also news to me :)

 Bound the number of in-flight requests at the coordinator
 -

 Key: CASSANDRA-9318
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
 Project: Cassandra
  Issue Type: Improvement
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 2.1.x


 It's possible to somewhat bound the amount of load accepted into the cluster 
 by bounding the number of in-flight requests and request bytes.
 An implementation might do something like track the number of outstanding 
 bytes and requests and if it reaches a high watermark disable read on client 
 connections until it goes back below some low watermark.
 Need to make sure that disabling read on the client connection won't 
 introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535533#comment-14535533
 ] 

Jonathan Ellis commented on CASSANDRA-9318:
---

bq. With vnodes this will mean total unavalability.

Assuming we do this the way I am proposing -- after determining what replicas a 
mutation belongs on -- you get the same amount of unavailability with or 
without vnodes.  (More token ranges are affected, but each range is smaller.)

 Bound the number of in-flight requests at the coordinator
 -

 Key: CASSANDRA-9318
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
 Project: Cassandra
  Issue Type: Improvement
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 2.1.x


 It's possible to somewhat bound the amount of load accepted into the cluster 
 by bounding the number of in-flight requests and request bytes.
 An implementation might do something like track the number of outstanding 
 bytes and requests and if it reaches a high watermark disable read on client 
 connections until it goes back below some low watermark.
 Need to make sure that disabling read on the client connection won't 
 introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9282) Warn on unlogged batches


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535539#comment-14535539
 ] 

Jonathan Ellis commented on CASSANDRA-9282:
---

bq. Mutations for different keyspaces but the same key would still require a 
batchlog (current logic). 

This definitely isn't true at the replica level, do we not handle this 
correctly in SelectStatement and/or StorageProxy?

 Warn on unlogged batches
 

 Key: CASSANDRA-9282
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9282
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.1.x


 At least until CASSANDRA-8303 is done and we can block them entirely, we 
 should log a warning when unlogged batches across multiple partition keys are 
 used.  This could either be done by backporting NoSpamLogger and blindly 
 logging every time, or we could add a threshold and warn when more than 10 
 keys are seen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6702) Upgrading node uses the wrong port in gossiping


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535540#comment-14535540
 ] 

Ariel Weisberg commented on CASSANDRA-6702:
---

[~enigmacurry] I moved this one to testing just in case it's worth looking at 
how this might effect other scenarios. Blake did manual testing of the fix and 
there are some dtests so it's not completely untested. I am not sure how we 
feel about the dtest coverage and whether that is enough.

 Upgrading node uses the wrong port in gossiping
 ---

 Key: CASSANDRA-6702
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6702
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 1.1.7, AWS, Ec2MultiRegionSnitch
Reporter: Minh Do
Assignee: Blake Eggleston
Priority: Minor
 Fix For: 2.0.x

 Attachments: C6702-1.2.txt, C6702-2.0.txt


 When upgrading a node in 1.1.7 (or 1.1.11) cluster to 1.2.15 and inspecting 
 the gossip information on port/Ip, I could see that the upgrading node (1.2 
 version) communicates to one other node in the same region using Public IP 
 and non-encrypted port.
 For the rest, the upgrading node uses the correct ports and IPs to 
 communicate in this manner:
Same region: private IP and non-encrypted port 
and
Different region: public IP and encrypted port
 Because there is one node like this (or 2 out of 12 nodes cluster in which 
 nodes are split equally on 2 AWS regions), we have to modify Security Group 
 to allow the new traffics.
 Without modifying the SG, the 95th and 99th latencies for both reads and 
 writes in the cluster are very bad (due to RPC timeout).  Inspecting closer, 
 that upgraded node (1.2 node) is contributing to all of the high latencies 
 whenever it acts as a coordinator node. 
  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2da468d9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2da468d9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2da468d9

Branch: refs/heads/trunk
Commit: 2da468d9bbcf66a0635fb39d26da83d7349829a8
Parents: 81c9308 def6b5f
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Fri May 8 16:09:04 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Fri May 8 16:09:04 2015 -0400

--
 .../stress/src/org/apache/cassandra/stress/util/DynamicList.java  | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--

[1/2] cassandra git commit: fixup last fix for bad lock guard

Repository: cassandra
Updated Branches:
  refs/heads/trunk 81c93084b - 2da468d9b


fixup last fix for bad lock guard


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/def6b5fa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/def6b5fa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/def6b5fa

Branch: refs/heads/trunk
Commit: def6b5fa19c27d74bc69710b4e26d74dde9dd609
Parents: b66475c
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Fri May 8 16:08:36 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Fri May 8 16:08:36 2015 -0400

--
 .../stress/src/org/apache/cassandra/stress/util/DynamicList.java  | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/def6b5fa/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java 
b/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
index e2bdd4e..ee04063 100644
--- a/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
+++ b/tools/stress/src/org/apache/cassandra/stress/util/DynamicList.java
@@ -153,10 +153,11 @@ public class DynamicListE
 public void remove(NodeE node)
 {
 lock.writeLock().lock();
-node.value = null;
 try
 {
 assert node.value != null;
+node.value = null;
+
 size--;
 
 // go up through each level in the skip list, unlinking this node; 
this entails

[jira] [Comment Edited] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator