[jira] [Commented] (CASSANDRA-7281) SELECT on tuple relations are broken for mixed ASC/DESC clustering order
[
https://issues.apache.org/jira/browse/CASSANDRA-7281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550010#comment-14550010
]
Marcin Szymaniuk commented on CASSANDRA-7281:
-
I will provide another patch by the end of this week
SELECT on tuple relations are broken for mixed ASC/DESC clustering order
Key: CASSANDRA-7281
URL: https://issues.apache.org/jira/browse/CASSANDRA-7281
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Marcin Szymaniuk
Fix For: 2.1.x
Attachments:
0001-CASSANDRA-7281-SELECT-on-tuple-relations-are-broken-.patch,
0001-CASSANDRA-7281-SELECT-on-tuple-relations-are-broken-v2.patch,
0001-CASSANDRA-7281-SELECT-on-tuple-relations-are-broken-v3.patch
As noted on
[CASSANDRA-6875|https://issues.apache.org/jira/browse/CASSANDRA-6875?focusedCommentId=13992153page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13992153],
the tuple notation is broken when the clustering order mixes ASC and DESC
directives because the range of data they describe don't correspond to a
single continuous slice internally. To copy the example from CASSANDRA-6875:
{noformat}
cqlsh:ks create table foo (a int, b int, c int, PRIMARY KEY (a, b, c)) WITH
CLUSTERING ORDER BY (b DESC, c ASC);
cqlsh:ks INSERT INTO foo (a, b, c) VALUES (0, 2, 0);
cqlsh:ks INSERT INTO foo (a, b, c) VALUES (0, 1, 0);
cqlsh:ks INSERT INTO foo (a, b, c) VALUES (0, 1, 1);
cqlsh:ks INSERT INTO foo (a, b, c) VALUES (0, 0, 0);
cqlsh:ks SELECT * FROM foo WHERE a=0;
a | b | c
---+---+---
0 | 2 | 0
0 | 1 | 0
0 | 1 | 1
0 | 0 | 0
(4 rows)
cqlsh:ks SELECT * FROM foo WHERE a=0 AND (b, c) (1, 0);
a | b | c
---+---+---
0 | 2 | 0
(1 rows)
{noformat}
The last query should really return {{(0, 2, 0)}} and {{(0, 1, 1)}}.
For that specific example we should generate 2 internal slices, but I believe
that with more clustering columns we may have more slices.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9413) Add a default limit size (in bytes) for requests
[ https://issues.apache.org/jira/browse/CASSANDRA-9413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549953#comment-14549953 ] Antoine Blanchet commented on CASSANDRA-9413: - Got it. Thank you. Add a default limit size (in bytes) for requests Key: CASSANDRA-9413 URL: https://issues.apache.org/jira/browse/CASSANDRA-9413 Project: Cassandra Issue Type: Improvement Components: Core Environment: Cassandra 2.0.10, requested using Thrift Reporter: Antoine Blanchet We experienced a crash on our production cluster following a massive wide row read using Thrift A client tried to read a wide row (~4GB of raw data) without specifying any slice condition, which resulted in the crash of multiple nodes (as many as the replication factor) after long garbage collections. We know that wide rows should not be that big, but it is not the topic here. My question is the following: Is it possible to prevent Cassandra from OOM'ing when a client does this kind of requests? I'd rather have an error thrown to the client than a multi-server crash. The issue has already been discussed on the user mailing list, the thread is here : https://www.mail-archive.com/user@cassandra.apache.org/msg42340.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9186) AbstractType vs. CQL3Type loses frozen keyword
[
https://issues.apache.org/jira/browse/CASSANDRA-9186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549920#comment-14549920
]
Sylvain Lebresne commented on CASSANDRA-9186:
-
bq. should we really make a frozen/non-frozen distinction at UDF level at all?
Good question. I suppose we don't have too and not doing it would be more
flexible (typically it could be annoying to have to write twice the same
function just because it's sometimes applied to a frozen type and sometimes
not). So I like the idea of ignoring frozenness for functions but that
probably mean we want to complain if a function is defined with a frozen type
instead of just ignoring it silently as the later would probably be a bit
confusing.
AbstractType vs. CQL3Type loses frozen keyword
--
Key: CASSANDRA-9186
URL: https://issues.apache.org/jira/browse/CASSANDRA-9186
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Minor
Fix For: 3.x
{code}
create keyspace functionmetadatatest with replication =
{'class':'SimpleStrategy', 'replication_factor':1};
use functionmetadatatest ;
CREATE TYPE udtx (x int);
CREATE FUNCTION u_frozen(u frozenudtx)RETURNS int LANGUAGE java AS 'return
new Integer(0);';
SELECT function_name, signature, argument_types FROM system.schema_functions
WHERE keyspace_name='functionmetadatatest';
function_name | signature| argument_types
---+--+--
u_frozen | ['udtx'] |
['org.apache.cassandra.db.marshal.UserType(functionmetadatatest,75647478,78:org.apache.cassandra.db.marshal.Int32Type)']
{code}
Problem is that {{UserType}} and {{TupleType}} do not have a _frozen_ (or
_isMultiCell_) attribute. A conversion from type to cql3type therefore loses
the _frozen_ status.
EDIT: The issue has been discovered by [~aholmber]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9420) Table option for promising that you will never touch a column twice
[
https://issues.apache.org/jira/browse/CASSANDRA-9420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549993#comment-14549993
]
Sylvain Lebresne commented on CASSANDRA-9420:
-
bq. The problem stems from the fact that Cassandra plays safe when a TTL has
expired, and turns it into a tombstone, rather than getting rid of it on the
spot.
If {{gcGrace == 0}}, which you seem fine with, then we'll get rid of expired
column on the spot. The fact that expired column are turned into tombstones
is a technicality that only exists so we keep expired columns for gcGrace
seconds after expiration, but if the later is 0, it's removed right away. If
you've observed differently in practice, then that would sound like a bug and
if you provided reproduction steps we certainly can look into it. But, and
unless I've misunderstood what you're saying, we don't need yet another option
for this.
Table option for promising that you will never touch a column twice
---
Key: CASSANDRA-9420
URL: https://issues.apache.org/jira/browse/CASSANDRA-9420
Project: Cassandra
Issue Type: New Feature
Reporter: Björn Hegerfors
There are time series use cases where you write all values with various TTLs,
have GC grace = 0 and never ever update or delete a column after insertion.
In the case where all TTLs are the same, DTCS with recent patches works
great. But when there is lots of variations in TTLs, you are forced to choose
between splitting your table into multiple TTL tiers or having your SSTables
filled to the majority with tombstones. Or running frequent major compactions.
The problem stems from the fact that Cassandra plays safe when a TTL has
expired, and turns it into a tombstone, rather than getting rid of it on the
spot. The reason is that this TTL _may_ have been in a column which has had
an earlier write without (or with a higher) TTL. And then that one should now
be deleted too.
I propose that there should be table level setting to say I guarantee that
there will never be any updates to any columns. The effect of enabling that
option is that all tombstones and expired TTLs should always be immediately
removed during compaction. And the check for dropping entirely expired
SSTables can be very loosened for these tables.
This option should probably require gc_grace_seconds to be set to zero. It's
also questionable if writes without TTL should be allowed to such a table,
since those would become constants.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9399) cqlsh support for native protocol v4 features
[
https://issues.apache.org/jira/browse/CASSANDRA-9399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550022#comment-14550022
]
Stefania commented on CASSANDRA-9399:
-
tinyint and smallint have been added to cqlsh along with dtests to check:
- tracing
- date and time types
- tinyint and smallint.
Still todo in cqlsh:
- client warnings
- tests for CASSANDRA-8975
cqlsh support for native protocol v4 features
-
Key: CASSANDRA-9399
URL: https://issues.apache.org/jira/browse/CASSANDRA-9399
Project: Cassandra
Issue Type: Bug
Reporter: Aleksey Yeschenko
Assignee: Stefania
Labels: cqlsh
Fix For: 2.2 rc1
cqlsh/python-driver need to add support for all the new 2.2 CQL features:
- {{date}} and {{time}} types - CASSANDRA-7523 - not in cqlsh yet
- {{smallint}} and {{tinyint}} types - CASSANDRA-8951 - not in the driver yet
- client warnings - CASSANDRA-8930
- tracing improvements - CASSANDRA-7807
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: Log error when closing streaming connection at debug
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 7acd0e69d - 59b8e171e
Log error when closing streaming connection at debug
(this is good hygiene and remove a findBugs warning in particular)
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/59b8e171
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/59b8e171
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/59b8e171
Branch: refs/heads/cassandra-2.1
Commit: 59b8e171e91e4d714acac5ec195f40158e2e2971
Parents: 7acd0e6
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Tue May 19 10:31:48 2015 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Tue May 19 10:31:52 2015 +0200
--
.../org/apache/cassandra/streaming/ConnectionHandler.java | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/59b8e171/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --git a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index 6092046..aa3504a 100644
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@ -215,7 +215,12 @@ public class ConnectionHandler
{
socket.close();
}
-catch (IOException ignore) {}
+catch (IOException e)
+{
+// Erroring out while closing shouldn't happen but is not
really a big deal, so just log
+// it at DEBUG and ignore otherwise.
+logger.debug(Unexpected error while closing streaming
connection, e);
+}
}
}
[1/3] cassandra git commit: Log error when closing streaming connection at debug
Repository: cassandra
Updated Branches:
refs/heads/trunk 6f5542792 - c844bfeea
Log error when closing streaming connection at debug
(this is good hygiene and remove a findBugs warning in particular)
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/59b8e171
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/59b8e171
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/59b8e171
Branch: refs/heads/trunk
Commit: 59b8e171e91e4d714acac5ec195f40158e2e2971
Parents: 7acd0e6
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Tue May 19 10:31:48 2015 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Tue May 19 10:31:52 2015 +0200
--
.../org/apache/cassandra/streaming/ConnectionHandler.java | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/59b8e171/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --git a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index 6092046..aa3504a 100644
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@ -215,7 +215,12 @@ public class ConnectionHandler
{
socket.close();
}
-catch (IOException ignore) {}
+catch (IOException e)
+{
+// Erroring out while closing shouldn't happen but is not
really a big deal, so just log
+// it at DEBUG and ignore otherwise.
+logger.debug(Unexpected error while closing streaming
connection, e);
+}
}
}
[2/3] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/816c2b3f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/816c2b3f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/816c2b3f Branch: refs/heads/trunk Commit: 816c2b3f777843279295e67b0906f3df92ccc84a Parents: aa133b7 59b8e17 Author: Sylvain Lebresne sylv...@datastax.com Authored: Tue May 19 10:33:06 2015 +0200 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Tue May 19 10:33:06 2015 +0200 -- .../org/apache/cassandra/streaming/ConnectionHandler.java | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/816c2b3f/src/java/org/apache/cassandra/streaming/ConnectionHandler.java --
[3/3] cassandra git commit: Merge branch 'cassandra-2.2' into trunk
Merge branch 'cassandra-2.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c844bfee Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c844bfee Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c844bfee Branch: refs/heads/trunk Commit: c844bfeeabd252f1b9c871964bfcf3abfaf17d35 Parents: 6f55427 816c2b3 Author: Sylvain Lebresne sylv...@datastax.com Authored: Tue May 19 10:33:25 2015 +0200 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Tue May 19 10:33:25 2015 +0200 -- .../org/apache/cassandra/streaming/ConnectionHandler.java | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) --
[1/2] cassandra git commit: Log error when closing streaming connection at debug
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 aa133b70f - 816c2b3f7
Log error when closing streaming connection at debug
(this is good hygiene and remove a findBugs warning in particular)
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/59b8e171
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/59b8e171
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/59b8e171
Branch: refs/heads/cassandra-2.2
Commit: 59b8e171e91e4d714acac5ec195f40158e2e2971
Parents: 7acd0e6
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Tue May 19 10:31:48 2015 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Tue May 19 10:31:52 2015 +0200
--
.../org/apache/cassandra/streaming/ConnectionHandler.java | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/59b8e171/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --git a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index 6092046..aa3504a 100644
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@ -215,7 +215,12 @@ public class ConnectionHandler
{
socket.close();
}
-catch (IOException ignore) {}
+catch (IOException e)
+{
+// Erroring out while closing shouldn't happen but is not
really a big deal, so just log
+// it at DEBUG and ignore otherwise.
+logger.debug(Unexpected error while closing streaming
connection, e);
+}
}
}
[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/816c2b3f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/816c2b3f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/816c2b3f Branch: refs/heads/cassandra-2.2 Commit: 816c2b3f777843279295e67b0906f3df92ccc84a Parents: aa133b7 59b8e17 Author: Sylvain Lebresne sylv...@datastax.com Authored: Tue May 19 10:33:06 2015 +0200 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Tue May 19 10:33:06 2015 +0200 -- .../org/apache/cassandra/streaming/ConnectionHandler.java | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/816c2b3f/src/java/org/apache/cassandra/streaming/ConnectionHandler.java --
[jira] [Commented] (CASSANDRA-8696) nodetool repair on cassandra 2.1.2 keyspaces return java.lang.RuntimeException: Could not create snapshot
[
https://issues.apache.org/jira/browse/CASSANDRA-8696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550024#comment-14550024
]
Rafał Furmański commented on CASSANDRA-8696:
I see no errors after increasing this timeout to very high value. I hope repair
will end now..
nodetool repair on cassandra 2.1.2 keyspaces return
java.lang.RuntimeException: Could not create snapshot
-
Key: CASSANDRA-8696
URL: https://issues.apache.org/jira/browse/CASSANDRA-8696
Project: Cassandra
Issue Type: Bug
Reporter: Jeff Liu
Assignee: Yuki Morishita
Fix For: 2.1.x
When trying to run nodetool repair -pr on cassandra node ( 2.1.2), cassandra
throw java exceptions: cannot create snapshot.
the error log from system.log:
{noformat}
INFO [STREAM-IN-/10.97.9.110] 2015-01-28 02:07:28,815
StreamResultFuture.java:166 - [Stream #692c1450-a692-11e4-9973-070e938df227
ID#0] Prepare completed. Receiving 2 files(221187 bytes), sending 5
files(632105 bytes)
INFO [STREAM-IN-/10.97.9.110] 2015-01-28 02:07:29,046
StreamResultFuture.java:180 - [Stream #692c1450-a692-11e4-9973-070e938df227]
Session with /10.97.9.110 is complete
INFO [STREAM-IN-/10.97.9.110] 2015-01-28 02:07:29,046
StreamResultFuture.java:212 - [Stream #692c1450-a692-11e4-9973-070e938df227]
All sessions completed
INFO [STREAM-IN-/10.97.9.110] 2015-01-28 02:07:29,047
StreamingRepairTask.java:96 - [repair #685e3d00-a692-11e4-9973-070e938df227]
streaming task succeed, returning response to /10.98.194.68
INFO [RepairJobTask:1] 2015-01-28 02:07:29,065 StreamResultFuture.java:86 -
[Stream #692c6270-a692-11e4-9973-070e938df227] Executing streaming plan for
Repair
INFO [StreamConnectionEstablisher:4] 2015-01-28 02:07:29,065
StreamSession.java:213 - [Stream #692c6270-a692-11e4-9973-070e938df227]
Starting streaming to /10.66.187.201
INFO [StreamConnectionEstablisher:4] 2015-01-28 02:07:29,070
StreamCoordinator.java:209 - [Stream #692c6270-a692-11e4-9973-070e938df227,
ID#0] Beginning stream session with /10.66.187.201
INFO [STREAM-IN-/10.66.187.201] 2015-01-28 02:07:29,465
StreamResultFuture.java:166 - [Stream #692c6270-a692-11e4-9973-070e938df227
ID#0] Prepare completed. Receiving 5 files(627994 bytes), sending 5
files(632105 bytes)
INFO [StreamReceiveTask:22] 2015-01-28 02:07:31,971
StreamResultFuture.java:180 - [Stream #692c6270-a692-11e4-9973-070e938df227]
Session with /10.66.187.201 is complete
INFO [StreamReceiveTask:22] 2015-01-28 02:07:31,972
StreamResultFuture.java:212 - [Stream #692c6270-a692-11e4-9973-070e938df227]
All sessions completed
INFO [StreamReceiveTask:22] 2015-01-28 02:07:31,972
StreamingRepairTask.java:96 - [repair #685e3d00-a692-11e4-9973-070e938df227]
streaming task succeed, returning response to /10.98.194.68
ERROR [RepairJobTask:1] 2015-01-28 02:07:39,444 RepairJob.java:127 - Error
occurred during snapshot phase
java.lang.RuntimeException: Could not create snapshot at /10.97.9.110
at
org.apache.cassandra.repair.SnapshotTask$SnapshotCallback.onFailure(SnapshotTask.java:77)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.net.MessagingService$5$1.run(MessagingService.java:347)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
INFO [AntiEntropySessions:6] 2015-01-28 02:07:39,445 RepairSession.java:260
- [repair #6f85e740-a692-11e4-9973-070e938df227] new session: will sync
/10.98.194.68, /10.66.187.201, /10.226.218.135 on range
(12817179804668051873746972069086
2638799,12863540308359254031520865977436165] for events.[bigint0text,
bigint0boolean, bigint0int, dataset_catalog, column_categories,
bigint0double, bigint0bigint]
ERROR [AntiEntropySessions:5] 2015-01-28 02:07:39,445 RepairSession.java:303
- [repair #685e3d00-a692-11e4-9973-070e938df227] session completed with the
following error
java.io.IOException: Failed during snapshot creation.
at
org.apache.cassandra.repair.RepairSession.failedSnapshot(RepairSession.java:344)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.repair.RepairJob$2.onFailure(RepairJob.java:128)
~[apache-cassandra-2.1.2.jar:2.1.2]
at com.google.common.util.concurrent.Futures$4.run(Futures.java:1172)
[jira] [Commented] (CASSANDRA-7281) SELECT on tuple relations are broken for mixed ASC/DESC clustering order
[
https://issues.apache.org/jira/browse/CASSANDRA-7281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550027#comment-14550027
]
Benjamin Lerer commented on CASSANDRA-7281:
---
[~marcinszymaniuk] Great! Thanks
SELECT on tuple relations are broken for mixed ASC/DESC clustering order
Key: CASSANDRA-7281
URL: https://issues.apache.org/jira/browse/CASSANDRA-7281
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Marcin Szymaniuk
Fix For: 2.1.x
Attachments:
0001-CASSANDRA-7281-SELECT-on-tuple-relations-are-broken-.patch,
0001-CASSANDRA-7281-SELECT-on-tuple-relations-are-broken-v2.patch,
0001-CASSANDRA-7281-SELECT-on-tuple-relations-are-broken-v3.patch
As noted on
[CASSANDRA-6875|https://issues.apache.org/jira/browse/CASSANDRA-6875?focusedCommentId=13992153page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13992153],
the tuple notation is broken when the clustering order mixes ASC and DESC
directives because the range of data they describe don't correspond to a
single continuous slice internally. To copy the example from CASSANDRA-6875:
{noformat}
cqlsh:ks create table foo (a int, b int, c int, PRIMARY KEY (a, b, c)) WITH
CLUSTERING ORDER BY (b DESC, c ASC);
cqlsh:ks INSERT INTO foo (a, b, c) VALUES (0, 2, 0);
cqlsh:ks INSERT INTO foo (a, b, c) VALUES (0, 1, 0);
cqlsh:ks INSERT INTO foo (a, b, c) VALUES (0, 1, 1);
cqlsh:ks INSERT INTO foo (a, b, c) VALUES (0, 0, 0);
cqlsh:ks SELECT * FROM foo WHERE a=0;
a | b | c
---+---+---
0 | 2 | 0
0 | 1 | 0
0 | 1 | 1
0 | 0 | 0
(4 rows)
cqlsh:ks SELECT * FROM foo WHERE a=0 AND (b, c) (1, 0);
a | b | c
---+---+---
0 | 2 | 0
(1 rows)
{noformat}
The last query should really return {{(0, 2, 0)}} and {{(0, 1, 1)}}.
For that specific example we should generate 2 internal slices, but I believe
that with more clustering columns we may have more slices.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[3/9] cassandra git commit: Improve javadoc around trigger execution
Improve javadoc around trigger execution
Patch by Sam Tunnicliffe; reviewed by brandonwilliams for CASSANDRA-9334
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/116de2be
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/116de2be
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/116de2be
Branch: refs/heads/cassandra-2.2
Commit: 116de2bee5920bc6842300a3f686d88f443b281d
Parents: 59b8e17
Author: Sam Tunnicliffe s...@beobal.com
Authored: Tue May 12 11:49:03 2015 +0100
Committer: Sam Tunnicliffe s...@beobal.com
Committed: Tue May 19 11:59:44 2015 +0100
--
.../org/apache/cassandra/triggers/ITrigger.java | 4 +--
.../cassandra/triggers/TriggerExecutor.java | 32
2 files changed, 34 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/116de2be/src/java/org/apache/cassandra/triggers/ITrigger.java
--
diff --git a/src/java/org/apache/cassandra/triggers/ITrigger.java
b/src/java/org/apache/cassandra/triggers/ITrigger.java
index 4701b61..21aba05 100644
--- a/src/java/org/apache/cassandra/triggers/ITrigger.java
+++ b/src/java/org/apache/cassandra/triggers/ITrigger.java
@@ -36,7 +36,7 @@ import org.apache.cassandra.db.Mutation;
* 2) ITrigger implementation can be instantiated multiple times during the
server life time.
* (Depends on the number of times trigger folder is updated.)br
* 3) ITrigger implementation should be state-less (avoid dependency on
instance variables).br
- *
+ *
* brbThe API is still beta and can change./b
*/
public interface ITrigger
@@ -46,7 +46,7 @@ public interface ITrigger
*
* @param partitionKey - partition Key for the update.
* @param update - update received for the CF
- * @return modifications to be applied, null if no action to be performed.
+ * @return additional modifications to be applied along with the supplied
update
*/
public CollectionMutation augment(ByteBuffer partitionKey, ColumnFamily
update);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/116de2be/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
--
diff --git a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
index 677daad..bc6f46c 100644
--- a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
+++ b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
@@ -60,6 +60,23 @@ public class TriggerExecutor
cachedTriggers.clear();
}
+/**
+ * Augment a partition update by executing triggers to generate an
intermediate
+ * set of mutations, then merging the ColumnFamily from each mutation with
those
+ * supplied. This is called from @{link
org.apache.cassandra.service.StorageProxy#cas}
+ * which is scoped for a single partition. For that reason, any mutations
generated
+ * by triggers are checked to ensure that they are for the same table and
partition
+ * key as the primary update; if not, InvalidRequestException is thrown.
If no
+ * additional mutations are generated, the original updates are returned
unmodified.
+ *
+ * @param key partition key for the update
+ * @param updates partition update to be applied, contains the merge of
the original
+ *update and any generated mutations
+ * @return the final update to be applied, the original update merged with
any
+ * additional mutations generated by configured triggers
+ * @throws InvalidRequestException if any mutation generated by a trigger
does not
+ * apply to the exact same partition as the initial update
+ */
public ColumnFamily execute(ByteBuffer key, ColumnFamily updates) throws
InvalidRequestException
{
ListMutation intermediate = executeInternal(key, updates);
@@ -78,6 +95,21 @@ public class TriggerExecutor
return updates;
}
+/**
+ * Takes a collection of mutations and possibly augments it by adding
extra mutations
+ * generated by configured triggers. If no additional mutations are created
+ * this returns null, signalling to the caller that only the initial set of
+ * mutations should be applied. If additional mutations iare/i
generated,
+ * the total set (i.e. the original plus the additional mutations) are
applied
+ * together in a logged batch. Should this not be possible because the
initial
+ * mutations contain counter updates, InvalidRequestException is thrown.
+ *
+ * @param mutations initial collection of mutations
+ * @return augmented mutations. Either the union of the initial
[7/9] cassandra git commit: Add test for triggers which throw IRE
Add test for triggers which throw IRE
Patch by Sam Tunnicliffe; reviewed by brandonwilliams for CASSANDRA-9334
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/86c9c00e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/86c9c00e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/86c9c00e
Branch: refs/heads/cassandra-2.2
Commit: 86c9c00e98a1867deff3b8acc883d224b532197b
Parents: e2723a4
Author: Sam Tunnicliffe s...@beobal.com
Authored: Tue May 12 16:26:24 2015 +0100
Committer: Sam Tunnicliffe s...@beobal.com
Committed: Tue May 19 12:02:40 2015 +0100
--
.../apache/cassandra/triggers/TriggersTest.java | 49 +++-
1 file changed, 38 insertions(+), 11 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/86c9c00e/test/unit/org/apache/cassandra/triggers/TriggersTest.java
--
diff --git a/test/unit/org/apache/cassandra/triggers/TriggersTest.java
b/test/unit/org/apache/cassandra/triggers/TriggersTest.java
index 41d4bb8..b0a5aca 100644
--- a/test/unit/org/apache/cassandra/triggers/TriggersTest.java
+++ b/test/unit/org/apache/cassandra/triggers/TriggersTest.java
@@ -22,18 +22,13 @@ import java.nio.ByteBuffer;
import java.util.Collection;
import java.util.Collections;
-import org.junit.AfterClass;
-import org.junit.Before;
-import org.junit.BeforeClass;
-import org.junit.Test;
+import org.junit.*;
import org.apache.cassandra.SchemaLoader;
import org.apache.cassandra.config.Schema;
import org.apache.cassandra.cql3.QueryProcessor;
import org.apache.cassandra.cql3.UntypedResultSet;
-import org.apache.cassandra.db.ArrayBackedSortedColumns;
-import org.apache.cassandra.db.BufferCell;
-import org.apache.cassandra.db.ColumnFamily;
+import org.apache.cassandra.db.*;
import org.apache.cassandra.db.ConsistencyLevel;
import org.apache.cassandra.db.Mutation;
import org.apache.cassandra.exceptions.ConfigurationException;
@@ -42,11 +37,10 @@ import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.thrift.*;
import org.apache.thrift.protocol.TBinaryProtocol;
-import static org.junit.Assert.assertEquals;
-import static org.junit.Assert.assertTrue;
-
import static org.apache.cassandra.utils.ByteBufferUtil.bytes;
import static org.apache.cassandra.utils.ByteBufferUtil.toInt;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
public class TriggersTest
{
@@ -61,12 +55,12 @@ public class TriggersTest
public static void beforeTest() throws ConfigurationException
{
SchemaLoader.loadSchema();
+StorageService.instance.initServer(0);
}
@Before
public void setup() throws Exception
{
-StorageService.instance.initServer(0);
if (thriftServer == null || ! thriftServer.isRunning())
{
thriftServer = new ThriftServer(InetAddress.getLocalHost(), 9170,
50);
@@ -283,6 +277,29 @@ public class TriggersTest
}
}
+@Test(expected=RuntimeException.class)
+public void ifTriggerThrowsErrorNoMutationsAreApplied() throws Exception
+{
+String cf = cf + System.nanoTime();
+try
+{
+setupTableWithTrigger(cf, ErrorTrigger.class);
+String cql = String.format(INSERT INTO %s.%s (k, v1) VALUES (11,
11), ksName, cf);
+QueryProcessor.process(cql, ConsistencyLevel.ONE);
+}
+catch (Exception e)
+{
+Throwable cause = e.getCause();
+assertTrue((cause instanceof
org.apache.cassandra.exceptions.InvalidRequestException));
+assertTrue(cause.getMessage().equals(ErrorTrigger.MESSAGE));
+throw e;
+}
+finally
+{
+assertUpdateNotExecuted(cf, 11);
+}
+}
+
private void setupTableWithTrigger(String cf, Class? extends ITrigger
triggerImpl)
throws RequestExecutionException
{
@@ -350,4 +367,14 @@ public class TriggersTest
return Collections.singletonList(new Mutation(ksName, key,
extraUpdate));
}
}
+
+public static class ErrorTrigger implements ITrigger
+{
+public static final String MESSAGE = Thrown by ErrorTrigger;
+public CollectionMutation augment(ByteBuffer partitionKey,
ColumnFamily update)
+{
+throw new
org.apache.cassandra.exceptions.InvalidRequestException(MESSAGE);
+}
+}
+
}
[9/9] cassandra git commit: Merge branch 'cassandra-2.2' into trunk
Merge branch 'cassandra-2.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3a6e9c91 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3a6e9c91 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3a6e9c91 Branch: refs/heads/trunk Commit: 3a6e9c91b83ad0dc9b2423b51975277c790af66d Parents: c844bfe 86c9c00 Author: Sam Tunnicliffe s...@beobal.com Authored: Tue May 19 12:05:24 2015 +0100 Committer: Sam Tunnicliffe s...@beobal.com Committed: Tue May 19 12:05:24 2015 +0100 -- .../org/apache/cassandra/triggers/ITrigger.java | 4 +- .../cassandra/triggers/TriggerExecutor.java | 32 + .../apache/cassandra/triggers/TriggersTest.java | 49 +++- 3 files changed, 72 insertions(+), 13 deletions(-) --
[4/9] cassandra git commit: Improve javadoc around trigger execution
Improve javadoc around trigger execution
Patch by Sam Tunnicliffe; reviewed by brandonwilliams for CASSANDRA-9334
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/116de2be
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/116de2be
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/116de2be
Branch: refs/heads/trunk
Commit: 116de2bee5920bc6842300a3f686d88f443b281d
Parents: 59b8e17
Author: Sam Tunnicliffe s...@beobal.com
Authored: Tue May 12 11:49:03 2015 +0100
Committer: Sam Tunnicliffe s...@beobal.com
Committed: Tue May 19 11:59:44 2015 +0100
--
.../org/apache/cassandra/triggers/ITrigger.java | 4 +--
.../cassandra/triggers/TriggerExecutor.java | 32
2 files changed, 34 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/116de2be/src/java/org/apache/cassandra/triggers/ITrigger.java
--
diff --git a/src/java/org/apache/cassandra/triggers/ITrigger.java
b/src/java/org/apache/cassandra/triggers/ITrigger.java
index 4701b61..21aba05 100644
--- a/src/java/org/apache/cassandra/triggers/ITrigger.java
+++ b/src/java/org/apache/cassandra/triggers/ITrigger.java
@@ -36,7 +36,7 @@ import org.apache.cassandra.db.Mutation;
* 2) ITrigger implementation can be instantiated multiple times during the
server life time.
* (Depends on the number of times trigger folder is updated.)br
* 3) ITrigger implementation should be state-less (avoid dependency on
instance variables).br
- *
+ *
* brbThe API is still beta and can change./b
*/
public interface ITrigger
@@ -46,7 +46,7 @@ public interface ITrigger
*
* @param partitionKey - partition Key for the update.
* @param update - update received for the CF
- * @return modifications to be applied, null if no action to be performed.
+ * @return additional modifications to be applied along with the supplied
update
*/
public CollectionMutation augment(ByteBuffer partitionKey, ColumnFamily
update);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/116de2be/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
--
diff --git a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
index 677daad..bc6f46c 100644
--- a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
+++ b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
@@ -60,6 +60,23 @@ public class TriggerExecutor
cachedTriggers.clear();
}
+/**
+ * Augment a partition update by executing triggers to generate an
intermediate
+ * set of mutations, then merging the ColumnFamily from each mutation with
those
+ * supplied. This is called from @{link
org.apache.cassandra.service.StorageProxy#cas}
+ * which is scoped for a single partition. For that reason, any mutations
generated
+ * by triggers are checked to ensure that they are for the same table and
partition
+ * key as the primary update; if not, InvalidRequestException is thrown.
If no
+ * additional mutations are generated, the original updates are returned
unmodified.
+ *
+ * @param key partition key for the update
+ * @param updates partition update to be applied, contains the merge of
the original
+ *update and any generated mutations
+ * @return the final update to be applied, the original update merged with
any
+ * additional mutations generated by configured triggers
+ * @throws InvalidRequestException if any mutation generated by a trigger
does not
+ * apply to the exact same partition as the initial update
+ */
public ColumnFamily execute(ByteBuffer key, ColumnFamily updates) throws
InvalidRequestException
{
ListMutation intermediate = executeInternal(key, updates);
@@ -78,6 +95,21 @@ public class TriggerExecutor
return updates;
}
+/**
+ * Takes a collection of mutations and possibly augments it by adding
extra mutations
+ * generated by configured triggers. If no additional mutations are created
+ * this returns null, signalling to the caller that only the initial set of
+ * mutations should be applied. If additional mutations iare/i
generated,
+ * the total set (i.e. the original plus the additional mutations) are
applied
+ * together in a logged batch. Should this not be possible because the
initial
+ * mutations contain counter updates, InvalidRequestException is thrown.
+ *
+ * @param mutations initial collection of mutations
+ * @return augmented mutations. Either the union of the initial and
[1/9] cassandra git commit: Improve javadoc around trigger execution
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 0b92967d3 - 37d81b9dc
refs/heads/cassandra-2.1 59b8e171e - 116de2bee
refs/heads/cassandra-2.2 816c2b3f7 - 86c9c00e9
refs/heads/trunk c844bfeea - 3a6e9c91b
Improve javadoc around trigger execution
Patch by Sam Tunnicliffe; reviewed by brandonwilliams for CASSANDRA-9334
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/37d81b9d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/37d81b9d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/37d81b9d
Branch: refs/heads/cassandra-2.0
Commit: 37d81b9dcb08dd8c16a560591f46481b5d316841
Parents: 0b92967
Author: Sam Tunnicliffe s...@beobal.com
Authored: Tue May 12 11:49:03 2015 +0100
Committer: Sam Tunnicliffe s...@beobal.com
Committed: Tue May 19 09:39:56 2015 +0100
--
.../org/apache/cassandra/triggers/ITrigger.java | 4 +--
.../cassandra/triggers/TriggerExecutor.java | 32
2 files changed, 34 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/37d81b9d/src/java/org/apache/cassandra/triggers/ITrigger.java
--
diff --git a/src/java/org/apache/cassandra/triggers/ITrigger.java
b/src/java/org/apache/cassandra/triggers/ITrigger.java
index 15ed7ba..fe2a9e6 100644
--- a/src/java/org/apache/cassandra/triggers/ITrigger.java
+++ b/src/java/org/apache/cassandra/triggers/ITrigger.java
@@ -36,7 +36,7 @@ import org.apache.cassandra.db.RowMutation;
* 2) ITrigger implementation can be instantiated multiple times during the
server life time.
* (Depends on the number of times trigger folder is updated.)br
* 3) ITrigger implementation should be state-less (avoid dependency on
instance variables).br
- *
+ *
* brbThe API is still beta and can change./b
*/
public interface ITrigger
@@ -46,7 +46,7 @@ public interface ITrigger
*
* @param key - Row Key for the update.
* @param update - Update received for the CF
- * @return modifications to be applied, null if no action to be performed.
+ * @return additional modifications to be applied along with the supplied
update
*/
public CollectionRowMutation augment(ByteBuffer key, ColumnFamily
update);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/37d81b9d/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
--
diff --git a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
index 988c6a7..bae0da6 100644
--- a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
+++ b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
@@ -64,6 +64,23 @@ public class TriggerExecutor
cachedTriggers.clear();
}
+/**
+ * Augment a partition update by executing triggers to generate an
intermediate
+ * set of mutations, then merging the ColumnFamily from each mutation with
those
+ * supplied. This is called from @{link
org.apache.cassandra.service.StorageProxy#cas}
+ * which is scoped for a single partition. For that reason, any mutations
generated
+ * by triggers are checked to ensure that they are for the same table and
partition
+ * key as the primary update; if not, InvalidRequestException is thrown.
If no
+ * additional mutations are generated, the original updates are returned
unmodified.
+ *
+ * @param key partition key for the update
+ * @param updates partition update to be applied, contains the merge of
the original
+ *update and any generated mutations
+ * @return the final update to be applied, the original update merged with
any
+ * additional mutations generated by configured triggers
+ * @throws InvalidRequestException if any mutation generated by a trigger
does not
+ * apply to the exact same partition as the initial update
+ */
public ColumnFamily execute(ByteBuffer key, ColumnFamily updates) throws
InvalidRequestException
{
ListRowMutation intermediate = executeInternal(key, updates);
@@ -79,6 +96,21 @@ public class TriggerExecutor
return updates;
}
+/**
+ * Takes a collection of mutations and possibly augments it by adding
extra mutations
+ * generated by configured triggers. If no additional mutations are created
+ * this returns null, signalling to the caller that only the initial set of
+ * mutations should be applied. If additional mutations iare/i
generated,
+ * the total set (i.e. the original plus the additional mutations) are
applied
+ * together in a logged batch. Should this not be possible because the
[6/9] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e2723a40 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e2723a40 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e2723a40 Branch: refs/heads/trunk Commit: e2723a400d1fec823d853374d3a64c6d061c55a5 Parents: 816c2b3 116de2b Author: Sam Tunnicliffe s...@beobal.com Authored: Tue May 19 12:02:33 2015 +0100 Committer: Sam Tunnicliffe s...@beobal.com Committed: Tue May 19 12:02:33 2015 +0100 -- .../org/apache/cassandra/triggers/ITrigger.java | 4 +-- .../cassandra/triggers/TriggerExecutor.java | 32 2 files changed, 34 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/e2723a40/src/java/org/apache/cassandra/triggers/TriggerExecutor.java --
[5/9] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e2723a40 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e2723a40 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e2723a40 Branch: refs/heads/cassandra-2.2 Commit: e2723a400d1fec823d853374d3a64c6d061c55a5 Parents: 816c2b3 116de2b Author: Sam Tunnicliffe s...@beobal.com Authored: Tue May 19 12:02:33 2015 +0100 Committer: Sam Tunnicliffe s...@beobal.com Committed: Tue May 19 12:02:33 2015 +0100 -- .../org/apache/cassandra/triggers/ITrigger.java | 4 +-- .../cassandra/triggers/TriggerExecutor.java | 32 2 files changed, 34 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/e2723a40/src/java/org/apache/cassandra/triggers/TriggerExecutor.java --
[2/9] cassandra git commit: Improve javadoc around trigger execution
Improve javadoc around trigger execution
Patch by Sam Tunnicliffe; reviewed by brandonwilliams for CASSANDRA-9334
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/116de2be
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/116de2be
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/116de2be
Branch: refs/heads/cassandra-2.1
Commit: 116de2bee5920bc6842300a3f686d88f443b281d
Parents: 59b8e17
Author: Sam Tunnicliffe s...@beobal.com
Authored: Tue May 12 11:49:03 2015 +0100
Committer: Sam Tunnicliffe s...@beobal.com
Committed: Tue May 19 11:59:44 2015 +0100
--
.../org/apache/cassandra/triggers/ITrigger.java | 4 +--
.../cassandra/triggers/TriggerExecutor.java | 32
2 files changed, 34 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/116de2be/src/java/org/apache/cassandra/triggers/ITrigger.java
--
diff --git a/src/java/org/apache/cassandra/triggers/ITrigger.java
b/src/java/org/apache/cassandra/triggers/ITrigger.java
index 4701b61..21aba05 100644
--- a/src/java/org/apache/cassandra/triggers/ITrigger.java
+++ b/src/java/org/apache/cassandra/triggers/ITrigger.java
@@ -36,7 +36,7 @@ import org.apache.cassandra.db.Mutation;
* 2) ITrigger implementation can be instantiated multiple times during the
server life time.
* (Depends on the number of times trigger folder is updated.)br
* 3) ITrigger implementation should be state-less (avoid dependency on
instance variables).br
- *
+ *
* brbThe API is still beta and can change./b
*/
public interface ITrigger
@@ -46,7 +46,7 @@ public interface ITrigger
*
* @param partitionKey - partition Key for the update.
* @param update - update received for the CF
- * @return modifications to be applied, null if no action to be performed.
+ * @return additional modifications to be applied along with the supplied
update
*/
public CollectionMutation augment(ByteBuffer partitionKey, ColumnFamily
update);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/116de2be/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
--
diff --git a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
index 677daad..bc6f46c 100644
--- a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
+++ b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
@@ -60,6 +60,23 @@ public class TriggerExecutor
cachedTriggers.clear();
}
+/**
+ * Augment a partition update by executing triggers to generate an
intermediate
+ * set of mutations, then merging the ColumnFamily from each mutation with
those
+ * supplied. This is called from @{link
org.apache.cassandra.service.StorageProxy#cas}
+ * which is scoped for a single partition. For that reason, any mutations
generated
+ * by triggers are checked to ensure that they are for the same table and
partition
+ * key as the primary update; if not, InvalidRequestException is thrown.
If no
+ * additional mutations are generated, the original updates are returned
unmodified.
+ *
+ * @param key partition key for the update
+ * @param updates partition update to be applied, contains the merge of
the original
+ *update and any generated mutations
+ * @return the final update to be applied, the original update merged with
any
+ * additional mutations generated by configured triggers
+ * @throws InvalidRequestException if any mutation generated by a trigger
does not
+ * apply to the exact same partition as the initial update
+ */
public ColumnFamily execute(ByteBuffer key, ColumnFamily updates) throws
InvalidRequestException
{
ListMutation intermediate = executeInternal(key, updates);
@@ -78,6 +95,21 @@ public class TriggerExecutor
return updates;
}
+/**
+ * Takes a collection of mutations and possibly augments it by adding
extra mutations
+ * generated by configured triggers. If no additional mutations are created
+ * this returns null, signalling to the caller that only the initial set of
+ * mutations should be applied. If additional mutations iare/i
generated,
+ * the total set (i.e. the original plus the additional mutations) are
applied
+ * together in a logged batch. Should this not be possible because the
initial
+ * mutations contain counter updates, InvalidRequestException is thrown.
+ *
+ * @param mutations initial collection of mutations
+ * @return augmented mutations. Either the union of the initial
[8/9] cassandra git commit: Add test for triggers which throw IRE
Add test for triggers which throw IRE
Patch by Sam Tunnicliffe; reviewed by brandonwilliams for CASSANDRA-9334
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/86c9c00e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/86c9c00e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/86c9c00e
Branch: refs/heads/trunk
Commit: 86c9c00e98a1867deff3b8acc883d224b532197b
Parents: e2723a4
Author: Sam Tunnicliffe s...@beobal.com
Authored: Tue May 12 16:26:24 2015 +0100
Committer: Sam Tunnicliffe s...@beobal.com
Committed: Tue May 19 12:02:40 2015 +0100
--
.../apache/cassandra/triggers/TriggersTest.java | 49 +++-
1 file changed, 38 insertions(+), 11 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/86c9c00e/test/unit/org/apache/cassandra/triggers/TriggersTest.java
--
diff --git a/test/unit/org/apache/cassandra/triggers/TriggersTest.java
b/test/unit/org/apache/cassandra/triggers/TriggersTest.java
index 41d4bb8..b0a5aca 100644
--- a/test/unit/org/apache/cassandra/triggers/TriggersTest.java
+++ b/test/unit/org/apache/cassandra/triggers/TriggersTest.java
@@ -22,18 +22,13 @@ import java.nio.ByteBuffer;
import java.util.Collection;
import java.util.Collections;
-import org.junit.AfterClass;
-import org.junit.Before;
-import org.junit.BeforeClass;
-import org.junit.Test;
+import org.junit.*;
import org.apache.cassandra.SchemaLoader;
import org.apache.cassandra.config.Schema;
import org.apache.cassandra.cql3.QueryProcessor;
import org.apache.cassandra.cql3.UntypedResultSet;
-import org.apache.cassandra.db.ArrayBackedSortedColumns;
-import org.apache.cassandra.db.BufferCell;
-import org.apache.cassandra.db.ColumnFamily;
+import org.apache.cassandra.db.*;
import org.apache.cassandra.db.ConsistencyLevel;
import org.apache.cassandra.db.Mutation;
import org.apache.cassandra.exceptions.ConfigurationException;
@@ -42,11 +37,10 @@ import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.thrift.*;
import org.apache.thrift.protocol.TBinaryProtocol;
-import static org.junit.Assert.assertEquals;
-import static org.junit.Assert.assertTrue;
-
import static org.apache.cassandra.utils.ByteBufferUtil.bytes;
import static org.apache.cassandra.utils.ByteBufferUtil.toInt;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
public class TriggersTest
{
@@ -61,12 +55,12 @@ public class TriggersTest
public static void beforeTest() throws ConfigurationException
{
SchemaLoader.loadSchema();
+StorageService.instance.initServer(0);
}
@Before
public void setup() throws Exception
{
-StorageService.instance.initServer(0);
if (thriftServer == null || ! thriftServer.isRunning())
{
thriftServer = new ThriftServer(InetAddress.getLocalHost(), 9170,
50);
@@ -283,6 +277,29 @@ public class TriggersTest
}
}
+@Test(expected=RuntimeException.class)
+public void ifTriggerThrowsErrorNoMutationsAreApplied() throws Exception
+{
+String cf = cf + System.nanoTime();
+try
+{
+setupTableWithTrigger(cf, ErrorTrigger.class);
+String cql = String.format(INSERT INTO %s.%s (k, v1) VALUES (11,
11), ksName, cf);
+QueryProcessor.process(cql, ConsistencyLevel.ONE);
+}
+catch (Exception e)
+{
+Throwable cause = e.getCause();
+assertTrue((cause instanceof
org.apache.cassandra.exceptions.InvalidRequestException));
+assertTrue(cause.getMessage().equals(ErrorTrigger.MESSAGE));
+throw e;
+}
+finally
+{
+assertUpdateNotExecuted(cf, 11);
+}
+}
+
private void setupTableWithTrigger(String cf, Class? extends ITrigger
triggerImpl)
throws RequestExecutionException
{
@@ -350,4 +367,14 @@ public class TriggersTest
return Collections.singletonList(new Mutation(ksName, key,
extraUpdate));
}
}
+
+public static class ErrorTrigger implements ITrigger
+{
+public static final String MESSAGE = Thrown by ErrorTrigger;
+public CollectionMutation augment(ByteBuffer partitionKey,
ColumnFamily update)
+{
+throw new
org.apache.cassandra.exceptions.InvalidRequestException(MESSAGE);
+}
+}
+
}
[jira] [Updated] (CASSANDRA-9421) TriggerExecutor should not wrap all execution exceptions in a RuntimeException
[
https://issues.apache.org/jira/browse/CASSANDRA-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-9421:
---
Priority: Minor (was: Major)
Fix Version/s: 2.2.x
Issue Type: Improvement (was: Bug)
TriggerExecutor should not wrap all execution exceptions in a RuntimeException
--
Key: CASSANDRA-9421
URL: https://issues.apache.org/jira/browse/CASSANDRA-9421
Project: Cassandra
Issue Type: Improvement
Reporter: Sam Tunnicliffe
Priority: Minor
Fix For: 2.2.x
It may be desirable to have implementations of {{ITrigger#augment}} throw
{{RequestValidationException}} or {{RequestExecutionException}} when
conditions for applying the trigger are not met (see CASSANDRA-9334 for more
discussion). {{TriggerExecutor#executeInternal}} currently catches any
exception thrown by a trigger, wraps it in a {{RuntimeException}} and
rethrows. It would be better to allow {{CassandraExceptions}} to propagate
back to clients directly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9421) TriggerExecutor should not wrap all execution exceptions in a RuntimeException
Sam Tunnicliffe created CASSANDRA-9421:
--
Summary: TriggerExecutor should not wrap all execution exceptions
in a RuntimeException
Key: CASSANDRA-9421
URL: https://issues.apache.org/jira/browse/CASSANDRA-9421
Project: Cassandra
Issue Type: Bug
Reporter: Sam Tunnicliffe
It may be desirable to have implementations of {{ITrigger#augment}} throw
{{RequestValidationException}} or {{RequestExecutionException}} when conditions
for applying the trigger are not met (see CASSANDRA-9334 for more discussion).
{{TriggerExecutor#executeInternal}} currently catches any exception thrown by a
trigger, wraps it in a {{RuntimeException}} and rethrows. It would be better to
allow {{CassandraExceptions}} to propagate back to clients directly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9419) CqlRecordWriter does not adequately determine if driver failed to connect due to interrupt
[ https://issues.apache.org/jira/browse/CASSANDRA-9419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-9419: --- Reviewer: Sam Tunnicliffe (was: Aleksey Yeschenko) CqlRecordWriter does not adequately determine if driver failed to connect due to interrupt -- Key: CASSANDRA-9419 URL: https://issues.apache.org/jira/browse/CASSANDRA-9419 Project: Cassandra Issue Type: Bug Reporter: Philip Thompson Assignee: Philip Thompson Fix For: 2.2 rc1 Attachments: 9419.txt Until https://datastax-oss.atlassian.net/browse/JAVA-769 is implemented, I have had to implement a workaround. Java's thread interrupt status flag can be cleared for any number of reasons. After discussion, I've changed to parsing the driver's DriverException to tell if we are safe to retry the failed connection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9186) AbstractType vs. CQL3Type loses frozen keyword
[
https://issues.apache.org/jira/browse/CASSANDRA-9186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550285#comment-14550285
]
Aleksey Yeschenko commented on CASSANDRA-9186:
--
bq. but that probably mean we want to complain if a function is defined with a
frozen type instead of just ignoring it silently as the later would probably be
a bit confusing.
Right. +1
AbstractType vs. CQL3Type loses frozen keyword
--
Key: CASSANDRA-9186
URL: https://issues.apache.org/jira/browse/CASSANDRA-9186
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Minor
Fix For: 3.x
{code}
create keyspace functionmetadatatest with replication =
{'class':'SimpleStrategy', 'replication_factor':1};
use functionmetadatatest ;
CREATE TYPE udtx (x int);
CREATE FUNCTION u_frozen(u frozenudtx)RETURNS int LANGUAGE java AS 'return
new Integer(0);';
SELECT function_name, signature, argument_types FROM system.schema_functions
WHERE keyspace_name='functionmetadatatest';
function_name | signature| argument_types
---+--+--
u_frozen | ['udtx'] |
['org.apache.cassandra.db.marshal.UserType(functionmetadatatest,75647478,78:org.apache.cassandra.db.marshal.Int32Type)']
{code}
Problem is that {{UserType}} and {{TupleType}} do not have a _frozen_ (or
_isMultiCell_) attribute. A conversion from type to cql3type therefore loses
the _frozen_ status.
EDIT: The issue has been discovered by [~aholmber]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9422) SystemKeyspaceTest fails with non-release version string
Sam Tunnicliffe created CASSANDRA-9422:
--
Summary: SystemKeyspaceTest fails with non-release version string
Key: CASSANDRA-9422
URL: https://issues.apache.org/jira/browse/CASSANDRA-9422
Project: Cassandra
Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Fix For: 2.2 beta 1
{{SystemKeyspaceTest#snapshotSystemKeyspaceIfUpgrading}} is currently failing
on the 2.2 branch because the build version {{2.2.0-beta1-SNAPSHOT}} cannot be
parsed as a {{SemanticVersion}}. This is only a problem with the test, as the
code under test doesn't actually use {{SemanticVersion}}, it just checks
whether the old new version strings are different.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8163) Re-introduce DESCRIBE permission
[ https://issues.apache.org/jira/browse/CASSANDRA-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550333#comment-14550333 ] Aleksey Yeschenko commented on CASSANDRA-8163: -- Can we, for 2.2, just introduce the permission to the enum, and have it be a no-op? I don't want us to break the API again in 3.0. Re-introduce DESCRIBE permission Key: CASSANDRA-8163 URL: https://issues.apache.org/jira/browse/CASSANDRA-8163 Project: Cassandra Issue Type: Improvement Reporter: Vishy Kasar Priority: Minor Fix For: 3.x We have a cluster like this: project1_keyspace table101 table102 project2_keyspace table201 table202 We have set up following users and grants: project1_user has all access to project1_keyspace project2_user has all access to project2_keyspace However project1_user can still do a 'describe schema' and get the schema for project2_keyspace as well. We do not want project1_user to have any knowledge for project2 in any way (cqlsh/java-driver etc) . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9399) cqlsh support for native protocol v4 features
[
https://issues.apache.org/jira/browse/CASSANDRA-9399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551818#comment-14551818
]
Stefania commented on CASSANDRA-9399:
-
Implemented client warnings in cqlsh and fixed formatting issue for dates,
CASSANDRA-8975.
cqlsh support for native protocol v4 features
-
Key: CASSANDRA-9399
URL: https://issues.apache.org/jira/browse/CASSANDRA-9399
Project: Cassandra
Issue Type: Bug
Reporter: Aleksey Yeschenko
Assignee: Stefania
Labels: cqlsh
Fix For: 2.2.0 rc1
cqlsh/python-driver need to add support for all the new 2.2 CQL features:
- {{date}} and {{time}} types - CASSANDRA-7523 - not in cqlsh yet
- {{smallint}} and {{tinyint}} types - CASSANDRA-8951 - not in the driver yet
- client warnings - CASSANDRA-8930
- tracing improvements - CASSANDRA-7807
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-9403) Experiment with skipping file syncs during unit tests to reduce test time
[
https://issues.apache.org/jira/browse/CASSANDRA-9403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551589#comment-14551589
]
Stefania edited comment on CASSANDRA-9403 at 5/20/15 1:09 AM:
--
Regarding this utest failure,
http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-C-9403-testall/lastCompletedBuild/testReport/org.apache.cassandra.io.sstable/SSTableRewriterTest/testNumberOfFilesAndSizes/,
I would simply store {{s.bytesOnDisk()}} in a variable just like
{{startStorageMetricsLoad}}.
As far as I understand the code, once the sstable is compacted and replaced, at
line 285, the reference is released and there is nothing stopping the deleting
task from removing the files, see DescriptorTypeTidy in SSTableReader and
SSTableDeletingTask. SSTableDeletingTask runs asynchronously.
I can reliably fail that assertion by calling
SSTableDeletingTask.waitForDeletions(); just before it.
Alternatively you can try increasing the reference count for s,
{{s.selfRef().tryRef()}}, and this should keep the files but you also need to
release it at the end of the test, {{s.selfRef().release()}}.
was (Author: stefania):
Regarding this utest failure,
http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-C-9403-testall/lastCompletedBuild/testReport/org.apache.cassandra.io.sstable/SSTableRewriterTest/testNumberOfFilesAndSizes/,
I would simply store {{s.bytesOnDisk()}} in a variable just like
{{startStorageMetricsLoad}}.
As far as I understand the code, once the sstable is compacted and replaced, at
line 285, the reference is released and there is nothing stopping the deleting
task from removing the files, see DescriptorTypeTidy in SSTableReader and
SSTableDeletingTask. The SSTableDeletingTask runs asynchronously.
I can reliably fail that assertion by calling
SSTableDeletingTask.waitForDeletions(); just before it.
Alternatively you can try increasing the reference count for s,
{{s.selfRef().tryRef()}}, and this should keep the files but you also need to
release it at the end of the test, {{s.selfRef().release()}}.
Experiment with skipping file syncs during unit tests to reduce test time
-
Key: CASSANDRA-9403
URL: https://issues.apache.org/jira/browse/CASSANDRA-9403
Project: Cassandra
Issue Type: Test
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Some environments have ridiculous outliers for disk syncing. 20 seconds
ridiculous.
Unit tests aren't testing crash safety so it is a pointless exercise.
Instead we could intercept calls to sync files and check whether it looks
like the sync would succeed. Check that the things are not null, mapped,
closed etc. Outside of units tests it can go straight to the regular sync
call.
I would also like to have the disks for unit and dtests mounted with
barrier=0,noatime,nodiratime to further reduce susceptibility to outliers. We
aren't going to recover these nodes if they crash/restart.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9434) If a node loses schema_columns SSTables it could delete all secondary indexes from the schema
Richard Low created CASSANDRA-9434:
--
Summary: If a node loses schema_columns SSTables it could delete
all secondary indexes from the schema
Key: CASSANDRA-9434
URL: https://issues.apache.org/jira/browse/CASSANDRA-9434
Project: Cassandra
Issue Type: Bug
Reporter: Richard Low
It is possible that a single bad node can delete all secondary indexes if it
restarts and cannot read its schema_columns SSTables. Here's a reproduction:
* Create a 2 node cluster (we saw it on 2.0.11)
* Create the schema:
create keyspace myks with replication = {'class':'SimpleStrategy',
'replication_factor':1};
use myks;
create table mytable (a text, b text, c text, PRIMARY KEY (a, b) );
create index myindex on mytable(b);
NB index must be on clustering column to repro
* Kill one node
* Wipe its commitlog and system/schema_columns sstables.
* Start it again
* Run on this node
select index_name from system.schema_columns where keyspace_name = 'myks' and
columnfamily_name = 'mytable' and column_name = 'b';
and you'll see the index is null.
* Run 'describe schema' on the other node. Sometimes it will not show the
index, but you might need to bounce for it to disappear.
I think the culprit is SystemKeyspace.copyAllAliasesToColumnsProper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9206) Remove seed gossip probability
[
https://issues.apache.org/jira/browse/CASSANDRA-9206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551604#comment-14551604
]
sankalp kohli commented on CASSANDRA-9206:
--
Instead of removing this completely, why can't we make the default 1 and make
it tunable. We have seen Gossip backing up many times due to heavy load. With
this, the point at which things go south will come much earlier.
Remove seed gossip probability
--
Key: CASSANDRA-9206
URL: https://issues.apache.org/jira/browse/CASSANDRA-9206
Project: Cassandra
Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Brandon Williams
Fix For: 3.x
Attachments: 9206.txt
Currently, we use probability to determine whether a node will gossip with a
seed:
{noformat}
double probability = seeds.size() / (double)
(liveEndpoints.size() + unreachableEndpoints.size());
double randDbl = random.nextDouble();
if (randDbl = probability)
sendGossip(prod, seeds);
{noformat}
I propose that we remove this probability, and instead *always* gossip with a
seed. This of course means increased traffic and processing on the seed(s),
but even a 1000 node cluster with a single seed will only put ~1000 messages
per second on the seed, which is virtually nothing. Should it become a
problem, the solution is simple: add more seeds. Since seeds will also
always gossip with each other, this effectively gives us a poor man's
spanning tree, with the only cost being removing a few lines of code, and
should greatly improve our gossip convergence time, especially in large
clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9422) SystemKeyspaceTest fails with non-release version string
[
https://issues.apache.org/jira/browse/CASSANDRA-9422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551657#comment-14551657
]
Jeremiah Jordan commented on CASSANDRA-9422:
Better to fix the SemanticVersion code to parse the version string? We have
other code that sends the version string to SemanticVersion to gate messages
during rolling upgrades. Will the beta-snapshot version mess those up?
SystemKeyspaceTest fails with non-release version string
Key: CASSANDRA-9422
URL: https://issues.apache.org/jira/browse/CASSANDRA-9422
Project: Cassandra
Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Fix For: 2.2.0 rc1
{{SystemKeyspaceTest#snapshotSystemKeyspaceIfUpgrading}} is currently failing
on the 2.2 branch because the build version {{2.2.0-beta1-SNAPSHOT}} cannot
be parsed as a {{SemanticVersion}}. This is only a problem with the test, as
the code under test doesn't actually use {{SemanticVersion}}, it just checks
whether the old new version strings are different.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-9403) Experiment with skipping file syncs during unit tests to reduce test time
[
https://issues.apache.org/jira/browse/CASSANDRA-9403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551589#comment-14551589
]
Stefania edited comment on CASSANDRA-9403 at 5/20/15 1:10 AM:
--
Regarding this utest failure,
http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-C-9403-testall/lastCompletedBuild/testReport/org.apache.cassandra.io.sstable/SSTableRewriterTest/testNumberOfFilesAndSizes/,
I would simply store {{s.bytesOnDisk()}} in a variable just like
{{startStorageMetricsLoad}}.
As far as I understand the code, once the sstable is compacted and replaced, at
line 285, the reference is released and there is nothing stopping the deleting
task from removing the files, see DescriptorTypeTidy in SSTableReader and
SSTableDeletingTask. SSTableDeletingTask runs asynchronously.
I can reliably fail that assertion by calling
SSTableDeletingTask.waitForDeletions(); just before it.
Alternatively we can try increasing the reference count for s,
{{s.selfRef().tryRef()}}, and this should keep the files but we also need to
release it at the end of the test, {{s.selfRef().release()}}.
Odd that this test is only failing on this branch, else this would merit its
own ticket.
was (Author: stefania):
Regarding this utest failure,
http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-C-9403-testall/lastCompletedBuild/testReport/org.apache.cassandra.io.sstable/SSTableRewriterTest/testNumberOfFilesAndSizes/,
I would simply store {{s.bytesOnDisk()}} in a variable just like
{{startStorageMetricsLoad}}.
As far as I understand the code, once the sstable is compacted and replaced, at
line 285, the reference is released and there is nothing stopping the deleting
task from removing the files, see DescriptorTypeTidy in SSTableReader and
SSTableDeletingTask. SSTableDeletingTask runs asynchronously.
I can reliably fail that assertion by calling
SSTableDeletingTask.waitForDeletions(); just before it.
Alternatively you can try increasing the reference count for s,
{{s.selfRef().tryRef()}}, and this should keep the files but you also need to
release it at the end of the test, {{s.selfRef().release()}}.
Experiment with skipping file syncs during unit tests to reduce test time
-
Key: CASSANDRA-9403
URL: https://issues.apache.org/jira/browse/CASSANDRA-9403
Project: Cassandra
Issue Type: Test
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Some environments have ridiculous outliers for disk syncing. 20 seconds
ridiculous.
Unit tests aren't testing crash safety so it is a pointless exercise.
Instead we could intercept calls to sync files and check whether it looks
like the sync would succeed. Check that the things are not null, mapped,
closed etc. Outside of units tests it can go straight to the regular sync
call.
I would also like to have the disks for unit and dtests mounted with
barrier=0,noatime,nodiratime to further reduce susceptibility to outliers. We
aren't going to recover these nodes if they crash/restart.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9434) If a node loses schema_columns SSTables it could delete all secondary indexes from the schema
[
https://issues.apache.org/jira/browse/CASSANDRA-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551601#comment-14551601
]
Richard Low commented on CASSANDRA-9434:
cc [~iamaleksey]
If a node loses schema_columns SSTables it could delete all secondary indexes
from the schema
-
Key: CASSANDRA-9434
URL: https://issues.apache.org/jira/browse/CASSANDRA-9434
Project: Cassandra
Issue Type: Bug
Reporter: Richard Low
It is possible that a single bad node can delete all secondary indexes if it
restarts and cannot read its schema_columns SSTables. Here's a reproduction:
* Create a 2 node cluster (we saw it on 2.0.11)
* Create the schema:
create keyspace myks with replication = {'class':'SimpleStrategy',
'replication_factor':1};
use myks;
create table mytable (a text, b text, c text, PRIMARY KEY (a, b) );
create index myindex on mytable(b);
NB index must be on clustering column to repro
* Kill one node
* Wipe its commitlog and system/schema_columns sstables.
* Start it again
* Run on this node
select index_name from system.schema_columns where keyspace_name = 'myks' and
columnfamily_name = 'mytable' and column_name = 'b';
and you'll see the index is null.
* Run 'describe schema' on the other node. Sometimes it will not show the
index, but you might need to bounce for it to disappear.
I think the culprit is SystemKeyspace.copyAllAliasesToColumnsProper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9432) SliceQueryFilterWithTombstonesTest.testExpiredTombstones is failing infrequently
[ https://issues.apache.org/jira/browse/CASSANDRA-9432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551489#comment-14551489 ] Aleksey Yeschenko commented on CASSANDRA-9432: -- +1 SliceQueryFilterWithTombstonesTest.testExpiredTombstones is failing infrequently Key: CASSANDRA-9432 URL: https://issues.apache.org/jira/browse/CASSANDRA-9432 Project: Cassandra Issue Type: Test Reporter: Ariel Weisberg Assignee: Ariel Weisberg I have only seen one instance of this, but haven't looked hard. http://cassci.datastax.com/view/trunk/job/trunk_testall/98/testReport/junit/org.apache.cassandra.cql3/SliceQueryFilterWithTombstonesTest/testExpiredTombstones/ I have it running in a loop and it's not reproducing. For now I just want to log the exception that is causing the failure. Need the ticket to to tie the commit to. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9399) cqlsh support for native protocol v4 features
[
https://issues.apache.org/jira/browse/CASSANDRA-9399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551488#comment-14551488
]
Stefania commented on CASSANDRA-9399:
-
Noted. I will update the tracing tests accordingly, since there is no longer
any need to test multiple protocol versions.
I will also revert the part of my patch in cqlsh where I added support for
specifying the protocol version via a command line argument,
{{--protocolversion}} unless you think this might be useful for other reasons
or tests.
cqlsh support for native protocol v4 features
-
Key: CASSANDRA-9399
URL: https://issues.apache.org/jira/browse/CASSANDRA-9399
Project: Cassandra
Issue Type: Bug
Reporter: Aleksey Yeschenko
Assignee: Stefania
Labels: cqlsh
Fix For: 2.2.0 rc1
cqlsh/python-driver need to add support for all the new 2.2 CQL features:
- {{date}} and {{time}} types - CASSANDRA-7523 - not in cqlsh yet
- {{smallint}} and {{tinyint}} types - CASSANDRA-8951 - not in the driver yet
- client warnings - CASSANDRA-8930
- tracing improvements - CASSANDRA-7807
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9403) Experiment with skipping file syncs during unit tests to reduce test time
[
https://issues.apache.org/jira/browse/CASSANDRA-9403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551589#comment-14551589
]
Stefania commented on CASSANDRA-9403:
-
Regarding this utest failure,
http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-C-9403-testall/lastCompletedBuild/testReport/org.apache.cassandra.io.sstable/SSTableRewriterTest/testNumberOfFilesAndSizes/,
I would simply store {{s.bytesOnDisk()}} in a variable just like
{{startStorageMetricsLoad}}.
As far as I understand the code, once the sstable is compacted and replaced, at
line 285, the reference is released and there is nothing stopping the deleting
task from removing the files, see DescriptorTypeTidy in SSTableReader and
SSTableDeletingTask. The SSTableDeletingTask runs asynchronously.
I can reliably fail that assertion by calling
SSTableDeletingTask.waitForDeletions(); just before it.
Alternatively you can try increasing the reference count for s,
{{s.selfRef().tryRef()}}, and this should keep the files but you also need to
release it at the end of the test, {{s.selfRef().release()}}.
Experiment with skipping file syncs during unit tests to reduce test time
-
Key: CASSANDRA-9403
URL: https://issues.apache.org/jira/browse/CASSANDRA-9403
Project: Cassandra
Issue Type: Test
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Some environments have ridiculous outliers for disk syncing. 20 seconds
ridiculous.
Unit tests aren't testing crash safety so it is a pointless exercise.
Instead we could intercept calls to sync files and check whether it looks
like the sync would succeed. Check that the things are not null, mapped,
closed etc. Outside of units tests it can go straight to the regular sync
call.
I would also like to have the disks for unit and dtests mounted with
barrier=0,noatime,nodiratime to further reduce susceptibility to outliers. We
aren't going to recover these nodes if they crash/restart.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9434) If a node loses schema_columns SSTables it could delete all secondary indexes from the schema
[
https://issues.apache.org/jira/browse/CASSANDRA-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Richard Low updated CASSANDRA-9434:
---
Description:
It is possible that a single bad node can delete all secondary indexes if it
restarts and cannot read its schema_columns SSTables. Here's a reproduction:
* Create a 2 node cluster (we saw it on 2.0.11)
* Create the schema:
{code}
create keyspace myks with replication = {'class':'SimpleStrategy',
'replication_factor':1};
use myks;
create table mytable (a text, b text, c text, PRIMARY KEY (a, b) );
create index myindex on mytable(b);
{code}
NB index must be on clustering column to repro
* Kill one node
* Wipe its commitlog and system/schema_columns sstables.
* Start it again
* Run on this node
select index_name from system.schema_columns where keyspace_name = 'myks' and
columnfamily_name = 'mytable' and column_name = 'b';
and you'll see the index is null.
* Run 'describe schema' on the other node. Sometimes it will not show the
index, but you might need to bounce for it to disappear.
I think the culprit is SystemKeyspace.copyAllAliasesToColumnsProper.
was:
It is possible that a single bad node can delete all secondary indexes if it
restarts and cannot read its schema_columns SSTables. Here's a reproduction:
* Create a 2 node cluster (we saw it on 2.0.11)
* Create the schema:
create keyspace myks with replication = {'class':'SimpleStrategy',
'replication_factor':1};
use myks;
create table mytable (a text, b text, c text, PRIMARY KEY (a, b) );
create index myindex on mytable(b);
NB index must be on clustering column to repro
* Kill one node
* Wipe its commitlog and system/schema_columns sstables.
* Start it again
* Run on this node
select index_name from system.schema_columns where keyspace_name = 'myks' and
columnfamily_name = 'mytable' and column_name = 'b';
and you'll see the index is null.
* Run 'describe schema' on the other node. Sometimes it will not show the
index, but you might need to bounce for it to disappear.
I think the culprit is SystemKeyspace.copyAllAliasesToColumnsProper.
If a node loses schema_columns SSTables it could delete all secondary indexes
from the schema
-
Key: CASSANDRA-9434
URL: https://issues.apache.org/jira/browse/CASSANDRA-9434
Project: Cassandra
Issue Type: Bug
Reporter: Richard Low
It is possible that a single bad node can delete all secondary indexes if it
restarts and cannot read its schema_columns SSTables. Here's a reproduction:
* Create a 2 node cluster (we saw it on 2.0.11)
* Create the schema:
{code}
create keyspace myks with replication = {'class':'SimpleStrategy',
'replication_factor':1};
use myks;
create table mytable (a text, b text, c text, PRIMARY KEY (a, b) );
create index myindex on mytable(b);
{code}
NB index must be on clustering column to repro
* Kill one node
* Wipe its commitlog and system/schema_columns sstables.
* Start it again
* Run on this node
select index_name from system.schema_columns where keyspace_name = 'myks' and
columnfamily_name = 'mytable' and column_name = 'b';
and you'll see the index is null.
* Run 'describe schema' on the other node. Sometimes it will not show the
index, but you might need to bounce for it to disappear.
I think the culprit is SystemKeyspace.copyAllAliasesToColumnsProper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9432) SliceQueryFilterWithTombstonesTest.testExpiredTombstones is failing infrequently
[ https://issues.apache.org/jira/browse/CASSANDRA-9432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-9432: - Reviewer: Aleksey Yeschenko SliceQueryFilterWithTombstonesTest.testExpiredTombstones is failing infrequently Key: CASSANDRA-9432 URL: https://issues.apache.org/jira/browse/CASSANDRA-9432 Project: Cassandra Issue Type: Test Reporter: Ariel Weisberg Assignee: Ariel Weisberg I have only seen one instance of this, but haven't looked hard. http://cassci.datastax.com/view/trunk/job/trunk_testall/98/testReport/junit/org.apache.cassandra.cql3/SliceQueryFilterWithTombstonesTest/testExpiredTombstones/ I have it running in a loop and it's not reproducing. For now I just want to log the exception that is causing the failure. Need the ticket to to tie the commit to. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8917) Upgrading from 2.0.9 to 2.1.3 with 3 nodes, CL = quorum causes exceptions
[ https://issues.apache.org/jira/browse/CASSANDRA-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Ogden updated CASSANDRA-8917: -- Attachment: Screen Shot 2015-05-19 at 10.50.23 AM.png Upgrading from 2.0.9 to 2.1.3 with 3 nodes, CL = quorum causes exceptions - Key: CASSANDRA-8917 URL: https://issues.apache.org/jira/browse/CASSANDRA-8917 Project: Cassandra Issue Type: Bug Environment: C* 2.0.9, Centos 6.5, Java 1.7.0_72, spring data cassandra 1.1.1, cassandra java driver 2.0.9 Reporter: Gary Ogden Fix For: 2.1.x Attachments: Screen Shot 2015-05-19 at 10.50.23 AM.png, b_output.log, jersey_error.log, node1-cassandra.yaml, node1-system.log, node2-cassandra.yaml, node2-system.log, node3-cassandra.yaml, node3-system.log We have java apps running on glassfish that read/write to our 3 node cluster running on 2.0.9. we have the CL set to quorum for all reads and writes. When we started to upgrade the first node and did the sstable upgrade on that node, we started getting this error on reads and writes: com.datastax.driver.core.exceptions.UnavailableException: Not enough replica available for query at consistency QUORUM (2 required but only 1 alive) How is that possible when we have 3 nodes total, and there was 2 that were up and it's saying we can't get the required CL? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8917) Upgrading from 2.0.9 to 2.1.3 with 3 nodes, CL = quorum causes exceptions
[ https://issues.apache.org/jira/browse/CASSANDRA-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550468#comment-14550468 ] Gary Ogden commented on CASSANDRA-8917: --- I'm actually able to reproduce this same error under a different scenario in a test cluster using devcenter. One of our nodes got into a funny state but is still actually running. Here's the log output: ERROR 13:29:32 JVM state determined to be unstable. Exiting forcefully due to: java.lang.OutOfMemoryError: Java heap space at org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:69) ~[apache-cassandra-2.1.3.jar:2.1.3] at org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(CompressedRandomAccessReader.java:77) ~[apache-cassandra-2.1.3.jar:2.1.3] at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:56) ~[apache-cassandra-2.1.3.jar:2.1.3] at org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1832) ~[apache-cassandra-2.1.3.jar:2.1.3] at org.apache.cassandra.io.sstable.SSTableScanner.init(SSTableScanner.java:78) ~[apache-cassandra-2.1.3.jar:2.1.3] at org.apache.cassandra.io.sstable.SSTableScanner.getScanner(SSTableScanner.java:57) ~[apache-cassandra-2.1.3.jar:2.1.3] at org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1602) ~[apache-cassandra-2.1.3.jar:2.1.3] at org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:67) ~[apache-cassandra-2.1.3.jar:2.1.3] at org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1988) ~[apache-cassandra-2.1.3.jar:2.1.3] at org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:2105) ~[apache-cassandra-2.1.3.jar:2.1.3] at org.apache.cassandra.db.RangeSliceCommand.executeLocally(RangeSliceCommand.java:132) ~[apache-cassandra-2.1.3.jar:2.1.3] at org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:39) ~[apache-cassandra-2.1.3.jar:2.1.3] at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62) ~[apache-cassandra-2.1.3.jar:2.1.3] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_75] at org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164) ~[apache-cassandra-2 .1.3.jar:2.1.3] at org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$TraceSessionFutureTask.run(AbstractTracingAwareExecutorService.java:136) [apache- cassandra-2.1.3.jar:2.1.3] at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [apache-cassandra-2.1.3.jar:2.1.3] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75] CompilerOracle: inline org/apache/cassandra/db/AbstractNativeCell.compareTo (Lorg/apache/cassandra/db/composites/Composite;)I CompilerOracle: inline org/apache/cassandra/db/composites/AbstractSimpleCellNameType.compareUnsigned (Lorg/apache/cassandra/db/composites/Composite;Lorg/apache/ cassandra/db/composites/Composite;)I CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare (Ljava/nio/ByteBuffer;[B)I CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare ([BLjava/nio/ByteBuffer;)I CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/lang/Object;JI)I CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/nio/ByteBuffer;)I CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I CompilerOracle: inline org/apache/cassandra/db/AbstractNativeCell.compareTo (Lorg/apache/cassandra/db/composites/Composite;)I CompilerOracle: inline org/apache/cassandra/db/composites/AbstractSimpleCellNameType.compareUnsigned (Lorg/apache/cassandra/db/composites/Composite;Lorg/apache/ cassandra/db/composites/Composite;)I CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare (Ljava/nio/ByteBuffer;[B)I CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare ([BLjava/nio/ByteBuffer;)I CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/lang/Object;JI)I CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo
[12/16] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/src/java/org/apache/cassandra/tools/NodeTool.java -- diff --cc src/java/org/apache/cassandra/tools/NodeTool.java index 2c913e0,000..5156479 mode 100644,00..100644 --- a/src/java/org/apache/cassandra/tools/NodeTool.java +++ b/src/java/org/apache/cassandra/tools/NodeTool.java @@@ -1,2685 -1,0 +1,2690 @@@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * License); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an AS IS BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.tools; + +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkState; +import static com.google.common.base.Throwables.getStackTraceAsString; +import static com.google.common.collect.Iterables.toArray; +import static com.google.common.collect.Lists.newArrayList; +import static java.lang.Integer.parseInt; +import static java.lang.String.format; +import static org.apache.commons.lang3.ArrayUtils.EMPTY_STRING_ARRAY; +import static org.apache.commons.lang3.StringUtils.EMPTY; +import static org.apache.commons.lang3.StringUtils.isEmpty; +import static org.apache.commons.lang3.StringUtils.isNotEmpty; +import static org.apache.commons.lang3.StringUtils.join; +import io.airlift.command.Arguments; +import io.airlift.command.Cli; +import io.airlift.command.Command; +import io.airlift.command.Help; +import io.airlift.command.Option; +import io.airlift.command.OptionType; +import io.airlift.command.ParseArgumentsMissingException; +import io.airlift.command.ParseArgumentsUnexpectedException; +import io.airlift.command.ParseCommandMissingException; +import io.airlift.command.ParseCommandUnrecognizedException; +import io.airlift.command.ParseOptionConversionException; +import io.airlift.command.ParseOptionMissingException; +import io.airlift.command.ParseOptionMissingValueException; + +import java.io.Console; +import java.io.File; +import java.io.FileNotFoundException; +import java.io.FileWriter; +import java.io.IOError; +import java.io.IOException; +import java.lang.management.MemoryUsage; +import java.net.InetAddress; +import java.net.UnknownHostException; +import java.text.DecimalFormat; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.Comparator; +import java.util.Date; +import java.util.HashMap; +import java.util.Iterator; +import java.util.List; +import java.util.Map; +import java.util.Map.Entry; +import java.util.Scanner; +import java.util.Set; +import java.util.concurrent.ExecutionException; + +import javax.management.InstanceNotFoundException; +import javax.management.openmbean.CompositeData; +import javax.management.openmbean.OpenDataException; +import javax.management.openmbean.TabularData; +import javax.management.openmbean.TabularDataSupport; + +import org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutorMBean; +import org.apache.cassandra.db.ColumnFamilyStoreMBean; +import org.apache.cassandra.db.Keyspace; +import org.apache.cassandra.db.compaction.CompactionManagerMBean; +import org.apache.cassandra.db.compaction.OperationType; +import org.apache.cassandra.io.util.FileUtils; +import org.apache.cassandra.locator.EndpointSnitchInfoMBean; +import org.apache.cassandra.metrics.ColumnFamilyMetrics.Sampler; +import org.apache.cassandra.net.MessagingServiceMBean; +import org.apache.cassandra.repair.RepairParallelism; +import org.apache.cassandra.service.CacheServiceMBean; +import org.apache.cassandra.service.StorageProxyMBean; +import org.apache.cassandra.streaming.ProgressInfo; +import org.apache.cassandra.streaming.SessionInfo; +import org.apache.cassandra.streaming.StreamState; +import org.apache.cassandra.utils.FBUtilities; +import org.apache.cassandra.utils.JVMStabilityInspector; +import org.apache.commons.lang3.ArrayUtils; + +import com.google.common.base.Joiner; +import com.google.common.base.Throwables; +import com.google.common.collect.ArrayListMultimap; +import com.google.common.collect.LinkedHashMultimap; +import com.google.common.collect.Lists; +import
[04/16] cassandra git commit: Add option to not validate atoms during scrub
Add option to not validate atoms during scrub
patch by Jordan West; reviewed by yukim for CASSANDRA-9406
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/be9eff57
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/be9eff57
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/be9eff57
Branch: refs/heads/trunk
Commit: be9eff573342cd4993eb4fc8ee694dc885e8a8e4
Parents: 37d81b9
Author: Jordan West jorda...@gmail.com
Authored: Tue May 19 08:26:27 2015 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Tue May 19 08:26:27 2015 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 4 +--
.../db/compaction/CompactionManager.java| 12 +++
.../cassandra/db/compaction/Scrubber.java | 12 ---
.../cassandra/service/StorageService.java | 7 +++-
.../cassandra/service/StorageServiceMBean.java | 2 ++
.../org/apache/cassandra/tools/NodeCmd.java | 5 ++-
.../org/apache/cassandra/tools/NodeProbe.java | 4 +--
.../cassandra/tools/StandaloneScrubber.java | 6 +++-
.../unit/org/apache/cassandra/db/ScrubTest.java | 37 +---
10 files changed, 60 insertions(+), 30 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/be9eff57/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a9d04d6..cf124b4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,7 @@
* Clone SliceQueryFilter in AbstractReadCommand implementations
(CASSANDRA-8940)
* Push correct protocol notification for DROP INDEX (CASSANDRA-9310)
* token-generator - generated tokens too long (CASSANDRA-9300)
+ * Add option not to validate atoms during scrub (CASSANDRA-9406)
2.0.15:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/be9eff57/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 5ea1287..eec4044 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -1122,12 +1122,12 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
CompactionManager.instance.performCleanup(ColumnFamilyStore.this,
renewer);
}
-public void scrub(boolean disableSnapshot, boolean skipCorrupted) throws
ExecutionException, InterruptedException
+public void scrub(boolean disableSnapshot, boolean skipCorrupted, boolean
checkData) throws ExecutionException, InterruptedException
{
// skip snapshot creation during scrub, SEE JIRA 5891
if(!disableSnapshot)
snapshotWithoutFlush(pre-scrub- + System.currentTimeMillis());
-CompactionManager.instance.performScrub(ColumnFamilyStore.this,
skipCorrupted);
+CompactionManager.instance.performScrub(ColumnFamilyStore.this,
skipCorrupted, checkData);
}
public void sstablesRewrite(boolean excludeCurrentVersion) throws
ExecutionException, InterruptedException
http://git-wip-us.apache.org/repos/asf/cassandra/blob/be9eff57/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 0978ae6..207b90d 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -234,13 +234,13 @@ public class CompactionManager implements
CompactionManagerMBean
executor.submit(runnable).get();
}
-public void performScrub(ColumnFamilyStore cfStore, final boolean
skipCorrupted) throws InterruptedException, ExecutionException
+public void performScrub(ColumnFamilyStore cfStore, final boolean
skipCorrupted, final boolean checkData) throws InterruptedException,
ExecutionException
{
performAllSSTableOperation(cfStore, new AllSSTablesOperation()
{
public void perform(ColumnFamilyStore store,
IterableSSTableReader sstables) throws IOException
{
-doScrub(store, sstables, skipCorrupted);
+doScrub(store, sstables, skipCorrupted, checkData);
}
});
}
@@ -432,16 +432,16 @@ public class CompactionManager implements
CompactionManagerMBean
*
* @throws IOException
*/
-private void doScrub(ColumnFamilyStore cfs, IterableSSTableReader
sstables, boolean skipCorrupted) throws
[15/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ff1a1562
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ff1a1562
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ff1a1562
Branch: refs/heads/trunk
Commit: ff1a156290440934826cf2e6a7f080d160683a16
Parents: 678291a d693ca1
Author: Yuki Morishita yu...@apache.org
Authored: Tue May 19 09:00:08 2015 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Tue May 19 09:00:08 2015 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 8 ++--
.../db/compaction/CompactionManager.java| 8 ++--
.../cassandra/db/compaction/Scrubber.java | 8 ++--
.../cassandra/service/StorageService.java | 7 +++-
.../cassandra/service/StorageServiceMBean.java | 2 +
.../org/apache/cassandra/tools/NodeProbe.java | 8 ++--
.../cassandra/tools/StandaloneScrubber.java | 6 ++-
.../apache/cassandra/tools/nodetool/Scrub.java | 9 -
.../unit/org/apache/cassandra/db/ScrubTest.java | 42 ++--
10 files changed, 66 insertions(+), 33 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff1a1562/CHANGES.txt
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff1a1562/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 062efe8,0951c01..738e9eb
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -1387,64 -1399,12 +1387,64 @@@ public class ColumnFamilyStore implemen
return
CompactionManager.instance.performCleanup(ColumnFamilyStore.this);
}
- public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted) throws ExecutionException,
InterruptedException
+ public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted, boolean checkData) throws
ExecutionException, InterruptedException
{
- return scrub(disableSnapshot, skipCorrupted, false);
++return scrub(disableSnapshot, skipCorrupted, false, checkData);
+}
+
+@VisibleForTesting
- public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted, boolean alwaysFail) throws
ExecutionException, InterruptedException
++public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted, boolean alwaysFail, boolean checkData)
throws ExecutionException, InterruptedException
+{
// skip snapshot creation during scrub, SEE JIRA 5891
if(!disableSnapshot)
snapshotWithoutFlush(pre-scrub- + System.currentTimeMillis());
-return
CompactionManager.instance.performScrub(ColumnFamilyStore.this, skipCorrupted,
checkData);
+
+try
+{
- return
CompactionManager.instance.performScrub(ColumnFamilyStore.this, skipCorrupted);
++return
CompactionManager.instance.performScrub(ColumnFamilyStore.this, skipCorrupted,
checkData);
+}
+catch(Throwable t)
+{
+if (!rebuildOnFailedScrub(t))
+throw t;
+
+return alwaysFail ? CompactionManager.AllSSTableOpStatus.ABORTED
: CompactionManager.AllSSTableOpStatus.SUCCESSFUL;
+}
+}
+
+/**
+ * CASSANDRA-5174 : For an index cfs we may be able to discard everything
and just rebuild
+ * the index when a scrub fails.
+ *
+ * @return true if we are an index cfs and we successfully rebuilt the
index
+ */
+public boolean rebuildOnFailedScrub(Throwable failure)
+{
+if (!isIndex())
+return false;
+
+SecondaryIndex index = null;
+if
(metadata.cfName.contains(Directories.SECONDARY_INDEX_NAME_SEPARATOR))
+{
+String[] parts = metadata.cfName.split(\\ +
Directories.SECONDARY_INDEX_NAME_SEPARATOR, 2);
+ColumnFamilyStore parentCfs =
keyspace.getColumnFamilyStore(parts[0]);
+index = parentCfs.indexManager.getIndexByName(metadata.cfName);
+assert index != null;
+}
+
+if (index == null)
+return false;
+
+truncateBlocking();
+
+logger.warn(Rebuilding index for {} because of {}, name,
failure.getMessage());
+index.getBaseCfs().rebuildSecondaryIndex(index.getIndexName());
+return true;
+}
+
+public CompactionManager.AllSSTableOpStatus verify(boolean
[14/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ff1a1562
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ff1a1562
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ff1a1562
Branch: refs/heads/cassandra-2.2
Commit: ff1a156290440934826cf2e6a7f080d160683a16
Parents: 678291a d693ca1
Author: Yuki Morishita yu...@apache.org
Authored: Tue May 19 09:00:08 2015 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Tue May 19 09:00:08 2015 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 8 ++--
.../db/compaction/CompactionManager.java| 8 ++--
.../cassandra/db/compaction/Scrubber.java | 8 ++--
.../cassandra/service/StorageService.java | 7 +++-
.../cassandra/service/StorageServiceMBean.java | 2 +
.../org/apache/cassandra/tools/NodeProbe.java | 8 ++--
.../cassandra/tools/StandaloneScrubber.java | 6 ++-
.../apache/cassandra/tools/nodetool/Scrub.java | 9 -
.../unit/org/apache/cassandra/db/ScrubTest.java | 42 ++--
10 files changed, 66 insertions(+), 33 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff1a1562/CHANGES.txt
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff1a1562/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 062efe8,0951c01..738e9eb
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -1387,64 -1399,12 +1387,64 @@@ public class ColumnFamilyStore implemen
return
CompactionManager.instance.performCleanup(ColumnFamilyStore.this);
}
- public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted) throws ExecutionException,
InterruptedException
+ public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted, boolean checkData) throws
ExecutionException, InterruptedException
{
- return scrub(disableSnapshot, skipCorrupted, false);
++return scrub(disableSnapshot, skipCorrupted, false, checkData);
+}
+
+@VisibleForTesting
- public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted, boolean alwaysFail) throws
ExecutionException, InterruptedException
++public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted, boolean alwaysFail, boolean checkData)
throws ExecutionException, InterruptedException
+{
// skip snapshot creation during scrub, SEE JIRA 5891
if(!disableSnapshot)
snapshotWithoutFlush(pre-scrub- + System.currentTimeMillis());
-return
CompactionManager.instance.performScrub(ColumnFamilyStore.this, skipCorrupted,
checkData);
+
+try
+{
- return
CompactionManager.instance.performScrub(ColumnFamilyStore.this, skipCorrupted);
++return
CompactionManager.instance.performScrub(ColumnFamilyStore.this, skipCorrupted,
checkData);
+}
+catch(Throwable t)
+{
+if (!rebuildOnFailedScrub(t))
+throw t;
+
+return alwaysFail ? CompactionManager.AllSSTableOpStatus.ABORTED
: CompactionManager.AllSSTableOpStatus.SUCCESSFUL;
+}
+}
+
+/**
+ * CASSANDRA-5174 : For an index cfs we may be able to discard everything
and just rebuild
+ * the index when a scrub fails.
+ *
+ * @return true if we are an index cfs and we successfully rebuilt the
index
+ */
+public boolean rebuildOnFailedScrub(Throwable failure)
+{
+if (!isIndex())
+return false;
+
+SecondaryIndex index = null;
+if
(metadata.cfName.contains(Directories.SECONDARY_INDEX_NAME_SEPARATOR))
+{
+String[] parts = metadata.cfName.split(\\ +
Directories.SECONDARY_INDEX_NAME_SEPARATOR, 2);
+ColumnFamilyStore parentCfs =
keyspace.getColumnFamilyStore(parts[0]);
+index = parentCfs.indexManager.getIndexByName(metadata.cfName);
+assert index != null;
+}
+
+if (index == null)
+return false;
+
+truncateBlocking();
+
+logger.warn(Rebuilding index for {} because of {}, name,
failure.getMessage());
+index.getBaseCfs().rebuildSecondaryIndex(index.getIndexName());
+return true;
+}
+
+public CompactionManager.AllSSTableOpStatus
[08/16] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
--
diff --cc src/java/org/apache/cassandra/tools/StandaloneScrubber.java
index d420218,26768f3..459df98
--- a/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
+++ b/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
@@@ -109,7 -121,7 +110,7 @@@ public class StandaloneScrubbe
{
try
{
- Scrubber scrubber = new Scrubber(cfs, sstable,
options.skipCorrupted, handler, true);
-Scrubber scrubber = new Scrubber(cfs, sstable,
options.skipCorrupted, !options.noValidate, handler, true);
++Scrubber scrubber = new Scrubber(cfs, sstable,
options.skipCorrupted, handler, true, !options.noValidate);
try
{
scrubber.scrub();
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/test/unit/org/apache/cassandra/db/ScrubTest.java
--
diff --cc test/unit/org/apache/cassandra/db/ScrubTest.java
index a19c76d,94c7e34..028cf6c
--- a/test/unit/org/apache/cassandra/db/ScrubTest.java
+++ b/test/unit/org/apache/cassandra/db/ScrubTest.java
@@@ -30,7 -30,10 +30,11 @@@ import java.util.Set
import java.util.concurrent.ExecutionException;
import org.apache.cassandra.cql3.QueryProcessor;
-import org.apache.cassandra.db.compaction.OperationType;
++import org.apache.cassandra.db.composites.CellNameType;
+import org.apache.cassandra.exceptions.ConfigurationException;
+ import org.apache.cassandra.db.marshal.CompositeType;
+ import org.apache.cassandra.db.marshal.LongType;
+ import org.apache.cassandra.db.marshal.UTF8Type;
import org.apache.cassandra.exceptions.RequestExecutionException;
import org.apache.cassandra.io.compress.CompressionMetadata;
import org.apache.cassandra.utils.UUIDGen;
@@@ -124,7 -123,7 +128,7 @@@ public class ScrubTest extends SchemaLo
overrideWithGarbage(sstable, ByteBufferUtil.bytes(0),
ByteBufferUtil.bytes(1));
// with skipCorrupted == false, the scrub is expected to fail
- Scrubber scrubber = new Scrubber(cfs, sstable, false, false);
-Scrubber scrubber = new Scrubber(cfs, sstable, false, true);
++Scrubber scrubber = new Scrubber(cfs, sstable, false, false, true);
try
{
scrubber.scrub();
@@@ -134,9 -133,10 +138,9 @@@
// with skipCorrupted == true, the corrupt row will be skipped
Scrubber.ScrubResult scrubResult;
- scrubber = new Scrubber(cfs, sstable, true, false);
-scrubber = new Scrubber(cfs, sstable, true, true);
++scrubber = new Scrubber(cfs, sstable, true, false, true);
scrubResult = scrubber.scrubWithResult();
scrubber.close();
-cfs.replaceCompactedSSTables(Collections.singletonList(sstable),
Collections.singletonList(scrubber.getNewSSTable()), OperationType.SCRUB);
assertNotNull(scrubResult);
@@@ -182,7 -181,7 +186,7 @@@
overrideWithGarbage(sstable, ByteBufferUtil.bytes(0),
ByteBufferUtil.bytes(1));
// with skipCorrupted == false, the scrub is expected to fail
- Scrubber scrubber = new Scrubber(cfs, sstable, false, false);
-Scrubber scrubber = new Scrubber(cfs, sstable, false, true);
++Scrubber scrubber = new Scrubber(cfs, sstable, false, false, true);
try
{
scrubber.scrub();
@@@ -191,9 -190,10 +195,9 @@@
catch (IOError err) {}
// with skipCorrupted == true, the corrupt row will be skipped
- scrubber = new Scrubber(cfs, sstable, true, false);
-scrubber = new Scrubber(cfs, sstable, true, true);
++scrubber = new Scrubber(cfs, sstable, true, false, true);
scrubber.scrub();
scrubber.close();
-cfs.replaceCompactedSSTables(Collections.singletonList(sstable),
Collections.singletonList(scrubber.getNewSSTable()), OperationType.SCRUB);
assertEquals(1, cfs.getSSTables().size());
// verify that we can read all of the rows, and there is now one less
row
@@@ -324,7 -324,7 +328,7 @@@
components.add(Component.TOC);
SSTableReader sstable = SSTableReader.openNoValidation(desc,
components, metadata);
--Scrubber scrubber = new Scrubber(cfs, sstable, false, true);
++Scrubber scrubber = new Scrubber(cfs, sstable, false, true, true);
scrubber.scrub();
cfs.loadNewSSTables();
@@@ -420,9 -420,19 +424,19 @@@
Keyspace keyspace = Keyspace.open(Keyspace1);
ColumnFamilyStore cfs =
keyspace.getColumnFamilyStore(test_compact_static_columns);
-QueryProcessor.processInternal(INSERT INTO
[16/16] cassandra git commit: Merge branch 'cassandra-2.2' into trunk
Merge branch 'cassandra-2.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5c413ef7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5c413ef7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5c413ef7 Branch: refs/heads/trunk Commit: 5c413ef76062da188ff661064ef45a02b5e5beda Parents: bade0fb ff1a156 Author: Yuki Morishita yu...@apache.org Authored: Tue May 19 09:00:17 2015 -0500 Committer: Yuki Morishita yu...@apache.org Committed: Tue May 19 09:00:17 2015 -0500 -- CHANGES.txt | 1 + .../apache/cassandra/db/ColumnFamilyStore.java | 8 ++-- .../db/compaction/CompactionManager.java| 8 ++-- .../cassandra/db/compaction/Scrubber.java | 8 ++-- .../cassandra/service/StorageService.java | 7 +++- .../cassandra/service/StorageServiceMBean.java | 2 + .../org/apache/cassandra/tools/NodeProbe.java | 8 ++-- .../cassandra/tools/StandaloneScrubber.java | 6 ++- .../apache/cassandra/tools/nodetool/Scrub.java | 9 - .../unit/org/apache/cassandra/db/ScrubTest.java | 42 ++-- 10 files changed, 66 insertions(+), 33 deletions(-) --
[02/16] cassandra git commit: Add option to not validate atoms during scrub
Add option to not validate atoms during scrub
patch by Jordan West; reviewed by yukim for CASSANDRA-9406
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/be9eff57
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/be9eff57
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/be9eff57
Branch: refs/heads/cassandra-2.1
Commit: be9eff573342cd4993eb4fc8ee694dc885e8a8e4
Parents: 37d81b9
Author: Jordan West jorda...@gmail.com
Authored: Tue May 19 08:26:27 2015 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Tue May 19 08:26:27 2015 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 4 +--
.../db/compaction/CompactionManager.java| 12 +++
.../cassandra/db/compaction/Scrubber.java | 12 ---
.../cassandra/service/StorageService.java | 7 +++-
.../cassandra/service/StorageServiceMBean.java | 2 ++
.../org/apache/cassandra/tools/NodeCmd.java | 5 ++-
.../org/apache/cassandra/tools/NodeProbe.java | 4 +--
.../cassandra/tools/StandaloneScrubber.java | 6 +++-
.../unit/org/apache/cassandra/db/ScrubTest.java | 37 +---
10 files changed, 60 insertions(+), 30 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/be9eff57/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a9d04d6..cf124b4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,7 @@
* Clone SliceQueryFilter in AbstractReadCommand implementations
(CASSANDRA-8940)
* Push correct protocol notification for DROP INDEX (CASSANDRA-9310)
* token-generator - generated tokens too long (CASSANDRA-9300)
+ * Add option not to validate atoms during scrub (CASSANDRA-9406)
2.0.15:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/be9eff57/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 5ea1287..eec4044 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -1122,12 +1122,12 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
CompactionManager.instance.performCleanup(ColumnFamilyStore.this,
renewer);
}
-public void scrub(boolean disableSnapshot, boolean skipCorrupted) throws
ExecutionException, InterruptedException
+public void scrub(boolean disableSnapshot, boolean skipCorrupted, boolean
checkData) throws ExecutionException, InterruptedException
{
// skip snapshot creation during scrub, SEE JIRA 5891
if(!disableSnapshot)
snapshotWithoutFlush(pre-scrub- + System.currentTimeMillis());
-CompactionManager.instance.performScrub(ColumnFamilyStore.this,
skipCorrupted);
+CompactionManager.instance.performScrub(ColumnFamilyStore.this,
skipCorrupted, checkData);
}
public void sstablesRewrite(boolean excludeCurrentVersion) throws
ExecutionException, InterruptedException
http://git-wip-us.apache.org/repos/asf/cassandra/blob/be9eff57/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 0978ae6..207b90d 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -234,13 +234,13 @@ public class CompactionManager implements
CompactionManagerMBean
executor.submit(runnable).get();
}
-public void performScrub(ColumnFamilyStore cfStore, final boolean
skipCorrupted) throws InterruptedException, ExecutionException
+public void performScrub(ColumnFamilyStore cfStore, final boolean
skipCorrupted, final boolean checkData) throws InterruptedException,
ExecutionException
{
performAllSSTableOperation(cfStore, new AllSSTablesOperation()
{
public void perform(ColumnFamilyStore store,
IterableSSTableReader sstables) throws IOException
{
-doScrub(store, sstables, skipCorrupted);
+doScrub(store, sstables, skipCorrupted, checkData);
}
});
}
@@ -432,16 +432,16 @@ public class CompactionManager implements
CompactionManagerMBean
*
* @throws IOException
*/
-private void doScrub(ColumnFamilyStore cfs, IterableSSTableReader
sstables, boolean
[13/16] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d693ca12
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d693ca12
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d693ca12
Branch: refs/heads/trunk
Commit: d693ca12c76c2651df1769e137a94b954174e061
Parents: adb8831 be9eff5
Author: Yuki Morishita yu...@apache.org
Authored: Tue May 19 08:50:28 2015 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Tue May 19 08:50:28 2015 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 4 +--
.../db/compaction/CompactionManager.java| 8 ++---
.../cassandra/db/compaction/Scrubber.java | 12 ---
.../cassandra/service/StorageService.java | 7 +++-
.../cassandra/service/StorageServiceMBean.java | 2 ++
.../org/apache/cassandra/tools/NodeProbe.java | 8 ++---
.../org/apache/cassandra/tools/NodeTool.java| 7 +++-
.../cassandra/tools/StandaloneScrubber.java | 6 +++-
.../unit/org/apache/cassandra/db/ScrubTest.java | 38 +---
10 files changed, 63 insertions(+), 30 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/CHANGES.txt
--
diff --cc CHANGES.txt
index 198935b,cf124b4..6fc1c9c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -30,6 -3,10 +30,7 @@@ Merged from 2.0
* Clone SliceQueryFilter in AbstractReadCommand implementations
(CASSANDRA-8940)
* Push correct protocol notification for DROP INDEX (CASSANDRA-9310)
* token-generator - generated tokens too long (CASSANDRA-9300)
+ * Add option not to validate atoms during scrub (CASSANDRA-9406)
-
-
-2.0.15:
* Fix counting of tombstones for TombstoneOverwhelmingException
(CASSANDRA-9299)
* Fix ReconnectableSnitch reconnecting to peers during upgrade
(CASSANDRA-6702)
* Include keyspace and table name in error log for collections over the size
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index bdc2d8b,eec4044..0951c01
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -1394,22 -1117,22 +1394,22 @@@ public class ColumnFamilyStore implemen
return maxFile;
}
-public void forceCleanup(CounterId.OneShotRenewer renewer) throws
ExecutionException, InterruptedException
+public CompactionManager.AllSSTableOpStatus forceCleanup() throws
ExecutionException, InterruptedException
{
-CompactionManager.instance.performCleanup(ColumnFamilyStore.this,
renewer);
+return
CompactionManager.instance.performCleanup(ColumnFamilyStore.this);
}
- public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted) throws ExecutionException,
InterruptedException
-public void scrub(boolean disableSnapshot, boolean skipCorrupted, boolean
checkData) throws ExecutionException, InterruptedException
++public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted, boolean checkData) throws
ExecutionException, InterruptedException
{
// skip snapshot creation during scrub, SEE JIRA 5891
if(!disableSnapshot)
snapshotWithoutFlush(pre-scrub- + System.currentTimeMillis());
- return
CompactionManager.instance.performScrub(ColumnFamilyStore.this, skipCorrupted);
-CompactionManager.instance.performScrub(ColumnFamilyStore.this,
skipCorrupted, checkData);
++return
CompactionManager.instance.performScrub(ColumnFamilyStore.this, skipCorrupted,
checkData);
}
-public void sstablesRewrite(boolean excludeCurrentVersion) throws
ExecutionException, InterruptedException
+public CompactionManager.AllSSTableOpStatus sstablesRewrite(boolean
excludeCurrentVersion) throws ExecutionException, InterruptedException
{
-
CompactionManager.instance.performSSTableRewrite(ColumnFamilyStore.this,
excludeCurrentVersion);
+return
CompactionManager.instance.performSSTableRewrite(ColumnFamilyStore.this,
excludeCurrentVersion);
}
public void markObsolete(CollectionSSTableReader sstables,
OperationType compactionType)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --cc
[jira] [Commented] (CASSANDRA-9407) SSTableRewriterTest is hard failing after Fix canonical view returning early opened SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-9407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550494#comment-14550494 ] Benedict commented on CASSANDRA-9407: - bq. Separate tickets kind of maybe aid in searchability and tracking. I am fine with reopening instead. I think the opposite is true, but I'm not wed to any approach. JIRA is messy, and slow. The less mess we need to filter through the better in my book. Historically we have generally reopened unless the problem made it into a release. Hopefully before long that will in itself make this point a bit moot, so probably not worth agonizing over. SSTableRewriterTest is hard failing after Fix canonical view returning early opened SSTables -- Key: CASSANDRA-9407 URL: https://issues.apache.org/jira/browse/CASSANDRA-9407 Project: Cassandra Issue Type: Test Reporter: Ariel Weisberg Assignee: Benedict Fix For: 2.2.0 rc1 When I bisected it seems to start failing after https://github.com/apache/cassandra/commit/7d6a60ddd0e156f6028c05d97fa9179f86fe02ee -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/4] cassandra git commit: Improve javadoc around trigger execution
Repository: cassandra
Updated Branches:
refs/heads/trunk 3a6e9c91b - f7d15d6af
Improve javadoc around trigger execution
Patch by Sam Tunnicliffe; reviewed by brandonwilliams for CASSANDRA-9334
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/37d81b9d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/37d81b9d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/37d81b9d
Branch: refs/heads/trunk
Commit: 37d81b9dcb08dd8c16a560591f46481b5d316841
Parents: 0b92967
Author: Sam Tunnicliffe s...@beobal.com
Authored: Tue May 12 11:49:03 2015 +0100
Committer: Sam Tunnicliffe s...@beobal.com
Committed: Tue May 19 09:39:56 2015 +0100
--
.../org/apache/cassandra/triggers/ITrigger.java | 4 +--
.../cassandra/triggers/TriggerExecutor.java | 32
2 files changed, 34 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/37d81b9d/src/java/org/apache/cassandra/triggers/ITrigger.java
--
diff --git a/src/java/org/apache/cassandra/triggers/ITrigger.java
b/src/java/org/apache/cassandra/triggers/ITrigger.java
index 15ed7ba..fe2a9e6 100644
--- a/src/java/org/apache/cassandra/triggers/ITrigger.java
+++ b/src/java/org/apache/cassandra/triggers/ITrigger.java
@@ -36,7 +36,7 @@ import org.apache.cassandra.db.RowMutation;
* 2) ITrigger implementation can be instantiated multiple times during the
server life time.
* (Depends on the number of times trigger folder is updated.)br
* 3) ITrigger implementation should be state-less (avoid dependency on
instance variables).br
- *
+ *
* brbThe API is still beta and can change./b
*/
public interface ITrigger
@@ -46,7 +46,7 @@ public interface ITrigger
*
* @param key - Row Key for the update.
* @param update - Update received for the CF
- * @return modifications to be applied, null if no action to be performed.
+ * @return additional modifications to be applied along with the supplied
update
*/
public CollectionRowMutation augment(ByteBuffer key, ColumnFamily
update);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/37d81b9d/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
--
diff --git a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
index 988c6a7..bae0da6 100644
--- a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
+++ b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
@@ -64,6 +64,23 @@ public class TriggerExecutor
cachedTriggers.clear();
}
+/**
+ * Augment a partition update by executing triggers to generate an
intermediate
+ * set of mutations, then merging the ColumnFamily from each mutation with
those
+ * supplied. This is called from @{link
org.apache.cassandra.service.StorageProxy#cas}
+ * which is scoped for a single partition. For that reason, any mutations
generated
+ * by triggers are checked to ensure that they are for the same table and
partition
+ * key as the primary update; if not, InvalidRequestException is thrown.
If no
+ * additional mutations are generated, the original updates are returned
unmodified.
+ *
+ * @param key partition key for the update
+ * @param updates partition update to be applied, contains the merge of
the original
+ *update and any generated mutations
+ * @return the final update to be applied, the original update merged with
any
+ * additional mutations generated by configured triggers
+ * @throws InvalidRequestException if any mutation generated by a trigger
does not
+ * apply to the exact same partition as the initial update
+ */
public ColumnFamily execute(ByteBuffer key, ColumnFamily updates) throws
InvalidRequestException
{
ListRowMutation intermediate = executeInternal(key, updates);
@@ -79,6 +96,21 @@ public class TriggerExecutor
return updates;
}
+/**
+ * Takes a collection of mutations and possibly augments it by adding
extra mutations
+ * generated by configured triggers. If no additional mutations are created
+ * this returns null, signalling to the caller that only the initial set of
+ * mutations should be applied. If additional mutations iare/i
generated,
+ * the total set (i.e. the original plus the additional mutations) are
applied
+ * together in a logged batch. Should this not be possible because the
initial
+ * mutations contain counter updates, InvalidRequestException is thrown.
+ *
+ * @param mutations initial collection of mutations
+ *
cassandra git commit: Fix 2.2 CHANGES.txt
Repository: cassandra Updated Branches: refs/heads/cassandra-2.2 889c1872e - 678291a75 Fix 2.2 CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/678291a7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/678291a7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/678291a7 Branch: refs/heads/cassandra-2.2 Commit: 678291a750e9a0c412957fbb87ce9aeb98e7e673 Parents: 889c187 Author: Aleksey Yeschenko alek...@apache.org Authored: Tue May 19 16:48:56 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Tue May 19 16:48:56 2015 +0300 -- CHANGES.txt | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/678291a7/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 7885e0e..2ffd6ed 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,3 +1,8 @@ +2.2 +Merged from 2.1: + * Use configured gcgs in anticompaction (CASSANDRA-9397) + + 2.2.0-beta1 * Introduce Transactional API for internal state changes (CASSANDRA-8984) * Add a flag in cassandra.yaml to enable UDFs (CASSANDRA-9404) @@ -114,7 +119,6 @@ 2.1.6 - * Use configured gcgs in anticompaction (CASSANDRA-9397) * Warn on misuse of unlogged batches (CASSANDRA-9282) * Failure detector detects and ignores local pauses (CASSANDRA-9183) * Add utility class to support for rate limiting a given log statement (CASSANDRA-9029)
[jira] [Updated] (CASSANDRA-9397) Wrong gc_grace_seconds used in anticompaction
[ https://issues.apache.org/jira/browse/CASSANDRA-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-9397: - Fix Version/s: (was: 2.2.0 beta 1) 2.2.0 rc1 Wrong gc_grace_seconds used in anticompaction - Key: CASSANDRA-9397 URL: https://issues.apache.org/jira/browse/CASSANDRA-9397 Project: Cassandra Issue Type: Improvement Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.1.6, 3.0 beta 1, 2.2.0 rc1 Attachments: 0001-fix.patch looks like we use CFMetaData.DEFAULT_GC_GRACE_SECONDS instead of the configured one during anticompaction attached patch fixes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9407) SSTableRewriterTest is hard failing after Fix canonical view returning early opened SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-9407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550515#comment-14550515 ] Ariel Weisberg commented on CASSANDRA-9407: --- Thinking on it more. If you use JIRA and you are all in on the agile features you don't really open tickets from previous iterations and that is what the habit stems from. The history I am thinking of is the history of what was completed in previous iterations. The way we use JIRA I think the policy of reopening when something hasn't shipped makes more sense. SSTableRewriterTest is hard failing after Fix canonical view returning early opened SSTables -- Key: CASSANDRA-9407 URL: https://issues.apache.org/jira/browse/CASSANDRA-9407 Project: Cassandra Issue Type: Test Reporter: Ariel Weisberg Assignee: Benedict Fix For: 2.2.0 rc1 When I bisected it seems to start failing after https://github.com/apache/cassandra/commit/7d6a60ddd0e156f6028c05d97fa9179f86fe02ee -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9422) SystemKeyspaceTest fails with non-release version string
[
https://issues.apache.org/jira/browse/CASSANDRA-9422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
T Jake Luciani updated CASSANDRA-9422:
--
Fix Version/s: (was: 2.2.0 beta 1)
2.2.0 rc1
SystemKeyspaceTest fails with non-release version string
Key: CASSANDRA-9422
URL: https://issues.apache.org/jira/browse/CASSANDRA-9422
Project: Cassandra
Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Fix For: 2.2.0 rc1
{{SystemKeyspaceTest#snapshotSystemKeyspaceIfUpgrading}} is currently failing
on the 2.2 branch because the build version {{2.2.0-beta1-SNAPSHOT}} cannot
be parsed as a {{SemanticVersion}}. This is only a problem with the test, as
the code under test doesn't actually use {{SemanticVersion}}, it just checks
whether the old new version strings are different.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9369) HSHA dtest for closing connections is failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-9369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550458#comment-14550458
]
Michael Shuler commented on CASSANDRA-9369:
---
Why are we disabling tests that happen to fail? Disabling failing tests from
the dtest suite only hides the failure from the people that want to see *all*
failures, disrupts viewing of test history in CI, and does not seem to have a
clear positive advantage.
In my opinion, there are only 2 reasons to disable a test:
1) The test completely hangs the test server.
2) A feature test is committed before the feature itself has been committed to
Cassandra.
HSHA dtest for closing connections is failing on trunk
--
Key: CASSANDRA-9369
URL: https://issues.apache.org/jira/browse/CASSANDRA-9369
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Tyler Hobbs
Assignee: Ariel Weisberg
Fix For: 2.2.x
The {{thrift_hsha_test.ThriftHSHATest.test_closing_connections}} dtest is
failing against trunk (but not against 2.1). Here's an example failure:
{noformat}
Error Message
There are non-closed connections: COMMAND PID USER FD TYPE DEVICE
SIZE/OFF NODE NAME
java28202 automaton7u IPv4 60767 0t0 TCP
localhost:58436-localhost:9160 (CLOSE_WAIT)
begin captured logging
dtest: DEBUG: cluster ccm directory: /tmp/dtest-U0pP7H
dtest: DEBUG: Creating connection pools..
dtest: DEBUG: Disabling/Enabling thrift iteration #0
dtest: DEBUG: Closing connections from the client side..
pycassa.pool: INFO: Pool 13972199784 was disposed
pycassa.pool: INFO: Pool 139721997714512 was disposed
pycassa.pool: INFO: Pool 139721997749904 was disposed
- end captured logging -
Stacktrace
File /usr/lib/python2.7/unittest/case.py, line 331, in run
testMethod()
File /home/automaton/cassandra-dtest/thrift_hsha_test.py, line 59, in
test_closing_connections
self.assertEqual(len(lines), 0, There are non-closed connections: %s %
stdout)
File /usr/lib/python2.7/unittest/case.py, line 515, in assertEqual
assertion_func(first, second, msg=msg)
File /usr/lib/python2.7/unittest/case.py, line 508, in _baseAssertEqual
raise self.failureException(msg)
'There are non-closed connections: COMMAND PID USER FD TYPE DEVICE
SIZE/OFF NODE NAME\njava28202 automaton7u IPv4 60767 0t0 TCP
localhost:58436-localhost:9160 (CLOSE_WAIT)\n\n begin
captured logging \ndtest: DEBUG: cluster ccm
directory: /tmp/dtest-U0pP7H\ndtest: DEBUG: Creating connection
pools..\ndtest: DEBUG: Disabling/Enabling thrift iteration #0\ndtest: DEBUG:
Closing connections from the client side..\npycassa.pool: INFO: Pool
13972199784 was disposed\npycassa.pool: INFO: Pool 139721997714512 was
disposed\npycassa.pool: INFO: Pool 139721997749904 was
disposed\n- end captured logging
-'
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8163) Re-introduce DESCRIBE permission
[ https://issues.apache.org/jira/browse/CASSANDRA-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550333#comment-14550333 ] Aleksey Yeschenko edited comment on CASSANDRA-8163 at 5/19/15 1:58 PM: --- -Can we, for 2.2, just introduce the permission to the enum, and have it be a no-op?- -I don't want us to break the API again in 3.0.- Scratch that, we already have it in the enum, for roles. Sorry for the confusion. was (Author: iamaleksey): Can we, for 2.2, just introduce the permission to the enum, and have it be a no-op? I don't want us to break the API again in 3.0. Re-introduce DESCRIBE permission Key: CASSANDRA-8163 URL: https://issues.apache.org/jira/browse/CASSANDRA-8163 Project: Cassandra Issue Type: Improvement Reporter: Vishy Kasar Priority: Minor Fix For: 3.x We have a cluster like this: project1_keyspace table101 table102 project2_keyspace table201 table202 We have set up following users and grants: project1_user has all access to project1_keyspace project2_user has all access to project2_keyspace However project1_user can still do a 'describe schema' and get the schema for project2_keyspace as well. We do not want project1_user to have any knowledge for project2 in any way (cqlsh/java-driver etc) . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Git Push Summary
Repository: cassandra Updated Tags: refs/tags/cassandra-2.2.0-beta1 [created] f7e42e94e
[2/3] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e8766e53 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e8766e53 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e8766e53 Branch: refs/heads/trunk Commit: e8766e53d11645b41e25d904d5bdbdb2b1c0c9a9 Parents: ff1a156 7af9c6a Author: T Jake Luciani j...@apache.org Authored: Tue May 19 10:13:14 2015 -0400 Committer: T Jake Luciani j...@apache.org Committed: Tue May 19 10:13:14 2015 -0400 -- .../DebuggableScheduledThreadPoolExecutor.java | 9 ++ ...buggableScheduledThreadPoolExecutorTest.java | 115 +++ 2 files changed, 124 insertions(+) --
[1/3] cassandra git commit: cancel any rejected tasks on shutdown so callers who care can progress
Repository: cassandra
Updated Branches:
refs/heads/trunk 5c413ef76 - 77f0d99fa
cancel any rejected tasks on shutdown so callers who care can progress
patch by tjake; reviewed by Sergio Bossa for CASSANDRA-9417
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7af9c6ac
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7af9c6ac
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7af9c6ac
Branch: refs/heads/trunk
Commit: 7af9c6ac88278d34202fea922606905b326f7811
Parents: d693ca1
Author: T Jake Luciani j...@apache.org
Authored: Mon May 18 15:18:24 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Tue May 19 10:12:07 2015 -0400
--
.../DebuggableScheduledThreadPoolExecutor.java | 9 ++
...buggableScheduledThreadPoolExecutorTest.java | 115 +++
2 files changed, 124 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7af9c6ac/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
--
diff --git
a/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
b/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
index 7226db8..ea0715c 100644
---
a/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
+++
b/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
@@ -31,6 +31,11 @@ import org.apache.cassandra.utils.JVMStabilityInspector;
*
* DebuggableScheduledThreadPoolExecutor also catches exceptions during Task
execution
* so that they don't supress subsequent invocations of the task.
+ *
+ * Finally, there is a special rejected execution handler for tasks rejected
during the shutdown hook.
+ *
+ * For fire and forget tasks (like ref tidy) we can safely ignore the
exceptions.
+ * For any callers that care to know their task was rejected we cancel passed
task.
*/
public class DebuggableScheduledThreadPoolExecutor extends
ScheduledThreadPoolExecutor
{
@@ -45,6 +50,10 @@ public class DebuggableScheduledThreadPoolExecutor extends
ScheduledThreadPoolEx
if (!StorageService.instance.isInShutdownHook())
throw new
RejectedExecutionException(ScheduledThreadPoolExecutor has shut down.);
+//Give some notification to the caller the task isn't going to
run
+if (task instanceof Future)
+((Future) task).cancel(false);
+
logger.debug(ScheduledThreadPoolExecutor has shut down as
part of C* shutdown);
}
else
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7af9c6ac/test/unit/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutorTest.java
--
diff --git
a/test/unit/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutorTest.java
b/test/unit/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutorTest.java
new file mode 100644
index 000..46b2764
--- /dev/null
+++
b/test/unit/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutorTest.java
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.concurrent;
+
+import java.io.IOException;
+import java.util.concurrent.CancellationException;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import junit.framework.Assert;
+import org.apache.cassandra.service.EmbeddedCassandraService;
+import org.apache.cassandra.service.StorageService;
+
+public class DebuggableScheduledThreadPoolExecutorTest
+{
+
+static
[3/3] cassandra git commit: Merge branch 'cassandra-2.2' into trunk
Merge branch 'cassandra-2.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/77f0d99f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/77f0d99f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/77f0d99f Branch: refs/heads/trunk Commit: 77f0d99facbfdd744fe289ec5c1edae3abf7f4d9 Parents: 5c413ef e8766e5 Author: T Jake Luciani j...@apache.org Authored: Tue May 19 10:13:54 2015 -0400 Committer: T Jake Luciani j...@apache.org Committed: Tue May 19 10:13:54 2015 -0400 -- .../DebuggableScheduledThreadPoolExecutor.java | 9 ++ ...buggableScheduledThreadPoolExecutorTest.java | 115 +++ 2 files changed, 124 insertions(+) --
[jira] [Commented] (CASSANDRA-9369) HSHA dtest for closing connections is failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-9369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550496#comment-14550496
]
Ariel Weisberg commented on CASSANDRA-9369:
---
If we were caught up and all tests were passing then yes disabling failing
tests would be a no-no. Right now when we are far behind and trying to catch up
it makes sense to disable tests we know aren't passing.
See CASSANDRA-8997 and CASSANDRA-9002 for why (and feel free to comment and
disagree).
Discussing this might be good retrospective fodder. I want to bring up the fact
that it's been a few months and utests and dtests still aren't passing, and
people are still doing feature work. What's the point of doing feature work for
a system that you can't ship anyways because the tests don't pass? I wouldn't
be surprised if the tests don't pass by the time we want to ship 2.2. By
branching 2.2 we increased test load an complexity because now we have two
places to merge to and two sets of tests to monitor.
I don't agree with the thinking that addressing test issues delays feature
work. It's already delayed you are just in denial.
HSHA dtest for closing connections is failing on trunk
--
Key: CASSANDRA-9369
URL: https://issues.apache.org/jira/browse/CASSANDRA-9369
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Tyler Hobbs
Assignee: Ariel Weisberg
Fix For: 2.2.x
The {{thrift_hsha_test.ThriftHSHATest.test_closing_connections}} dtest is
failing against trunk (but not against 2.1). Here's an example failure:
{noformat}
Error Message
There are non-closed connections: COMMAND PID USER FD TYPE DEVICE
SIZE/OFF NODE NAME
java28202 automaton7u IPv4 60767 0t0 TCP
localhost:58436-localhost:9160 (CLOSE_WAIT)
begin captured logging
dtest: DEBUG: cluster ccm directory: /tmp/dtest-U0pP7H
dtest: DEBUG: Creating connection pools..
dtest: DEBUG: Disabling/Enabling thrift iteration #0
dtest: DEBUG: Closing connections from the client side..
pycassa.pool: INFO: Pool 13972199784 was disposed
pycassa.pool: INFO: Pool 139721997714512 was disposed
pycassa.pool: INFO: Pool 139721997749904 was disposed
- end captured logging -
Stacktrace
File /usr/lib/python2.7/unittest/case.py, line 331, in run
testMethod()
File /home/automaton/cassandra-dtest/thrift_hsha_test.py, line 59, in
test_closing_connections
self.assertEqual(len(lines), 0, There are non-closed connections: %s %
stdout)
File /usr/lib/python2.7/unittest/case.py, line 515, in assertEqual
assertion_func(first, second, msg=msg)
File /usr/lib/python2.7/unittest/case.py, line 508, in _baseAssertEqual
raise self.failureException(msg)
'There are non-closed connections: COMMAND PID USER FD TYPE DEVICE
SIZE/OFF NODE NAME\njava28202 automaton7u IPv4 60767 0t0 TCP
localhost:58436-localhost:9160 (CLOSE_WAIT)\n\n begin
captured logging \ndtest: DEBUG: cluster ccm
directory: /tmp/dtest-U0pP7H\ndtest: DEBUG: Creating connection
pools..\ndtest: DEBUG: Disabling/Enabling thrift iteration #0\ndtest: DEBUG:
Closing connections from the client side..\npycassa.pool: INFO: Pool
13972199784 was disposed\npycassa.pool: INFO: Pool 139721997714512 was
disposed\npycassa.pool: INFO: Pool 139721997749904 was
disposed\n- end captured logging
-'
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9422) SystemKeyspaceTest fails with non-release version string
[
https://issues.apache.org/jira/browse/CASSANDRA-9422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550346#comment-14550346
]
Sam Tunnicliffe commented on CASSANDRA-9422:
Test fix pushed [here|https://github.com/beobal/cassandra/tree/9422]
SystemKeyspaceTest fails with non-release version string
Key: CASSANDRA-9422
URL: https://issues.apache.org/jira/browse/CASSANDRA-9422
Project: Cassandra
Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Fix For: 2.2 beta 1
{{SystemKeyspaceTest#snapshotSystemKeyspaceIfUpgrading}} is currently failing
on the 2.2 branch because the build version {{2.2.0-beta1-SNAPSHOT}} cannot
be parsed as a {{SemanticVersion}}. This is only a problem with the test, as
the code under test doesn't actually use {{SemanticVersion}}, it just checks
whether the old new version strings are different.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Git Push Summary
Repository: cassandra Updated Tags: refs/tags/2.2.0-beta1-tentative [deleted] 1735249eb
[3/4] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/889c1872 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/889c1872 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/889c1872 Branch: refs/heads/trunk Commit: 889c1872e036362636ef0be5419006f19c35a796 Parents: 86c9c00 adb8831 Author: Aleksey Yeschenko alek...@apache.org Authored: Tue May 19 16:43:11 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Tue May 19 16:43:11 2015 +0300 -- --
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/889c1872 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/889c1872 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/889c1872 Branch: refs/heads/cassandra-2.2 Commit: 889c1872e036362636ef0be5419006f19c35a796 Parents: 86c9c00 adb8831 Author: Aleksey Yeschenko alek...@apache.org Authored: Tue May 19 16:43:11 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Tue May 19 16:43:11 2015 +0300 -- --
[1/3] cassandra git commit: Improve javadoc around trigger execution
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 86c9c00e9 - 889c1872e
Improve javadoc around trigger execution
Patch by Sam Tunnicliffe; reviewed by brandonwilliams for CASSANDRA-9334
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/37d81b9d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/37d81b9d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/37d81b9d
Branch: refs/heads/cassandra-2.2
Commit: 37d81b9dcb08dd8c16a560591f46481b5d316841
Parents: 0b92967
Author: Sam Tunnicliffe s...@beobal.com
Authored: Tue May 12 11:49:03 2015 +0100
Committer: Sam Tunnicliffe s...@beobal.com
Committed: Tue May 19 09:39:56 2015 +0100
--
.../org/apache/cassandra/triggers/ITrigger.java | 4 +--
.../cassandra/triggers/TriggerExecutor.java | 32
2 files changed, 34 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/37d81b9d/src/java/org/apache/cassandra/triggers/ITrigger.java
--
diff --git a/src/java/org/apache/cassandra/triggers/ITrigger.java
b/src/java/org/apache/cassandra/triggers/ITrigger.java
index 15ed7ba..fe2a9e6 100644
--- a/src/java/org/apache/cassandra/triggers/ITrigger.java
+++ b/src/java/org/apache/cassandra/triggers/ITrigger.java
@@ -36,7 +36,7 @@ import org.apache.cassandra.db.RowMutation;
* 2) ITrigger implementation can be instantiated multiple times during the
server life time.
* (Depends on the number of times trigger folder is updated.)br
* 3) ITrigger implementation should be state-less (avoid dependency on
instance variables).br
- *
+ *
* brbThe API is still beta and can change./b
*/
public interface ITrigger
@@ -46,7 +46,7 @@ public interface ITrigger
*
* @param key - Row Key for the update.
* @param update - Update received for the CF
- * @return modifications to be applied, null if no action to be performed.
+ * @return additional modifications to be applied along with the supplied
update
*/
public CollectionRowMutation augment(ByteBuffer key, ColumnFamily
update);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/37d81b9d/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
--
diff --git a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
index 988c6a7..bae0da6 100644
--- a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
+++ b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
@@ -64,6 +64,23 @@ public class TriggerExecutor
cachedTriggers.clear();
}
+/**
+ * Augment a partition update by executing triggers to generate an
intermediate
+ * set of mutations, then merging the ColumnFamily from each mutation with
those
+ * supplied. This is called from @{link
org.apache.cassandra.service.StorageProxy#cas}
+ * which is scoped for a single partition. For that reason, any mutations
generated
+ * by triggers are checked to ensure that they are for the same table and
partition
+ * key as the primary update; if not, InvalidRequestException is thrown.
If no
+ * additional mutations are generated, the original updates are returned
unmodified.
+ *
+ * @param key partition key for the update
+ * @param updates partition update to be applied, contains the merge of
the original
+ *update and any generated mutations
+ * @return the final update to be applied, the original update merged with
any
+ * additional mutations generated by configured triggers
+ * @throws InvalidRequestException if any mutation generated by a trigger
does not
+ * apply to the exact same partition as the initial update
+ */
public ColumnFamily execute(ByteBuffer key, ColumnFamily updates) throws
InvalidRequestException
{
ListRowMutation intermediate = executeInternal(key, updates);
@@ -79,6 +96,21 @@ public class TriggerExecutor
return updates;
}
+/**
+ * Takes a collection of mutations and possibly augments it by adding
extra mutations
+ * generated by configured triggers. If no additional mutations are created
+ * this returns null, signalling to the caller that only the initial set of
+ * mutations should be applied. If additional mutations iare/i
generated,
+ * the total set (i.e. the original plus the additional mutations) are
applied
+ * together in a logged batch. Should this not be possible because the
initial
+ * mutations contain counter updates, InvalidRequestException is thrown.
+ *
+ * @param mutations initial collection of
[2/4] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/adb88316 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/adb88316 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/adb88316 Branch: refs/heads/trunk Commit: adb88316ac7ec26be96751062615c3ad0c22aaff Parents: 116de2b 37d81b9 Author: Aleksey Yeschenko alek...@apache.org Authored: Tue May 19 16:41:41 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Tue May 19 16:41:41 2015 +0300 -- --
[06/16] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/src/java/org/apache/cassandra/tools/NodeTool.java -- diff --cc src/java/org/apache/cassandra/tools/NodeTool.java index 2c913e0,000..5156479 mode 100644,00..100644 --- a/src/java/org/apache/cassandra/tools/NodeTool.java +++ b/src/java/org/apache/cassandra/tools/NodeTool.java @@@ -1,2685 -1,0 +1,2690 @@@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * License); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an AS IS BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.tools; + +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkState; +import static com.google.common.base.Throwables.getStackTraceAsString; +import static com.google.common.collect.Iterables.toArray; +import static com.google.common.collect.Lists.newArrayList; +import static java.lang.Integer.parseInt; +import static java.lang.String.format; +import static org.apache.commons.lang3.ArrayUtils.EMPTY_STRING_ARRAY; +import static org.apache.commons.lang3.StringUtils.EMPTY; +import static org.apache.commons.lang3.StringUtils.isEmpty; +import static org.apache.commons.lang3.StringUtils.isNotEmpty; +import static org.apache.commons.lang3.StringUtils.join; +import io.airlift.command.Arguments; +import io.airlift.command.Cli; +import io.airlift.command.Command; +import io.airlift.command.Help; +import io.airlift.command.Option; +import io.airlift.command.OptionType; +import io.airlift.command.ParseArgumentsMissingException; +import io.airlift.command.ParseArgumentsUnexpectedException; +import io.airlift.command.ParseCommandMissingException; +import io.airlift.command.ParseCommandUnrecognizedException; +import io.airlift.command.ParseOptionConversionException; +import io.airlift.command.ParseOptionMissingException; +import io.airlift.command.ParseOptionMissingValueException; + +import java.io.Console; +import java.io.File; +import java.io.FileNotFoundException; +import java.io.FileWriter; +import java.io.IOError; +import java.io.IOException; +import java.lang.management.MemoryUsage; +import java.net.InetAddress; +import java.net.UnknownHostException; +import java.text.DecimalFormat; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.Comparator; +import java.util.Date; +import java.util.HashMap; +import java.util.Iterator; +import java.util.List; +import java.util.Map; +import java.util.Map.Entry; +import java.util.Scanner; +import java.util.Set; +import java.util.concurrent.ExecutionException; + +import javax.management.InstanceNotFoundException; +import javax.management.openmbean.CompositeData; +import javax.management.openmbean.OpenDataException; +import javax.management.openmbean.TabularData; +import javax.management.openmbean.TabularDataSupport; + +import org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutorMBean; +import org.apache.cassandra.db.ColumnFamilyStoreMBean; +import org.apache.cassandra.db.Keyspace; +import org.apache.cassandra.db.compaction.CompactionManagerMBean; +import org.apache.cassandra.db.compaction.OperationType; +import org.apache.cassandra.io.util.FileUtils; +import org.apache.cassandra.locator.EndpointSnitchInfoMBean; +import org.apache.cassandra.metrics.ColumnFamilyMetrics.Sampler; +import org.apache.cassandra.net.MessagingServiceMBean; +import org.apache.cassandra.repair.RepairParallelism; +import org.apache.cassandra.service.CacheServiceMBean; +import org.apache.cassandra.service.StorageProxyMBean; +import org.apache.cassandra.streaming.ProgressInfo; +import org.apache.cassandra.streaming.SessionInfo; +import org.apache.cassandra.streaming.StreamState; +import org.apache.cassandra.utils.FBUtilities; +import org.apache.cassandra.utils.JVMStabilityInspector; +import org.apache.commons.lang3.ArrayUtils; + +import com.google.common.base.Joiner; +import com.google.common.base.Throwables; +import com.google.common.collect.ArrayListMultimap; +import com.google.common.collect.LinkedHashMultimap; +import com.google.common.collect.Lists; +import
[07/16] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d693ca12
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d693ca12
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d693ca12
Branch: refs/heads/cassandra-2.2
Commit: d693ca12c76c2651df1769e137a94b954174e061
Parents: adb8831 be9eff5
Author: Yuki Morishita yu...@apache.org
Authored: Tue May 19 08:50:28 2015 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Tue May 19 08:50:28 2015 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 4 +--
.../db/compaction/CompactionManager.java| 8 ++---
.../cassandra/db/compaction/Scrubber.java | 12 ---
.../cassandra/service/StorageService.java | 7 +++-
.../cassandra/service/StorageServiceMBean.java | 2 ++
.../org/apache/cassandra/tools/NodeProbe.java | 8 ++---
.../org/apache/cassandra/tools/NodeTool.java| 7 +++-
.../cassandra/tools/StandaloneScrubber.java | 6 +++-
.../unit/org/apache/cassandra/db/ScrubTest.java | 38 +---
10 files changed, 63 insertions(+), 30 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/CHANGES.txt
--
diff --cc CHANGES.txt
index 198935b,cf124b4..6fc1c9c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -30,6 -3,10 +30,7 @@@ Merged from 2.0
* Clone SliceQueryFilter in AbstractReadCommand implementations
(CASSANDRA-8940)
* Push correct protocol notification for DROP INDEX (CASSANDRA-9310)
* token-generator - generated tokens too long (CASSANDRA-9300)
+ * Add option not to validate atoms during scrub (CASSANDRA-9406)
-
-
-2.0.15:
* Fix counting of tombstones for TombstoneOverwhelmingException
(CASSANDRA-9299)
* Fix ReconnectableSnitch reconnecting to peers during upgrade
(CASSANDRA-6702)
* Include keyspace and table name in error log for collections over the size
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index bdc2d8b,eec4044..0951c01
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -1394,22 -1117,22 +1394,22 @@@ public class ColumnFamilyStore implemen
return maxFile;
}
-public void forceCleanup(CounterId.OneShotRenewer renewer) throws
ExecutionException, InterruptedException
+public CompactionManager.AllSSTableOpStatus forceCleanup() throws
ExecutionException, InterruptedException
{
-CompactionManager.instance.performCleanup(ColumnFamilyStore.this,
renewer);
+return
CompactionManager.instance.performCleanup(ColumnFamilyStore.this);
}
- public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted) throws ExecutionException,
InterruptedException
-public void scrub(boolean disableSnapshot, boolean skipCorrupted, boolean
checkData) throws ExecutionException, InterruptedException
++public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted, boolean checkData) throws
ExecutionException, InterruptedException
{
// skip snapshot creation during scrub, SEE JIRA 5891
if(!disableSnapshot)
snapshotWithoutFlush(pre-scrub- + System.currentTimeMillis());
- return
CompactionManager.instance.performScrub(ColumnFamilyStore.this, skipCorrupted);
-CompactionManager.instance.performScrub(ColumnFamilyStore.this,
skipCorrupted, checkData);
++return
CompactionManager.instance.performScrub(ColumnFamilyStore.this, skipCorrupted,
checkData);
}
-public void sstablesRewrite(boolean excludeCurrentVersion) throws
ExecutionException, InterruptedException
+public CompactionManager.AllSSTableOpStatus sstablesRewrite(boolean
excludeCurrentVersion) throws ExecutionException, InterruptedException
{
-
CompactionManager.instance.performSSTableRewrite(ColumnFamilyStore.this,
excludeCurrentVersion);
+return
CompactionManager.instance.performSSTableRewrite(ColumnFamilyStore.this,
excludeCurrentVersion);
}
public void markObsolete(CollectionSSTableReader sstables,
OperationType compactionType)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --cc
[09/16] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/src/java/org/apache/cassandra/tools/NodeTool.java -- diff --cc src/java/org/apache/cassandra/tools/NodeTool.java index 2c913e0,000..5156479 mode 100644,00..100644 --- a/src/java/org/apache/cassandra/tools/NodeTool.java +++ b/src/java/org/apache/cassandra/tools/NodeTool.java @@@ -1,2685 -1,0 +1,2690 @@@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * License); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an AS IS BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.tools; + +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkState; +import static com.google.common.base.Throwables.getStackTraceAsString; +import static com.google.common.collect.Iterables.toArray; +import static com.google.common.collect.Lists.newArrayList; +import static java.lang.Integer.parseInt; +import static java.lang.String.format; +import static org.apache.commons.lang3.ArrayUtils.EMPTY_STRING_ARRAY; +import static org.apache.commons.lang3.StringUtils.EMPTY; +import static org.apache.commons.lang3.StringUtils.isEmpty; +import static org.apache.commons.lang3.StringUtils.isNotEmpty; +import static org.apache.commons.lang3.StringUtils.join; +import io.airlift.command.Arguments; +import io.airlift.command.Cli; +import io.airlift.command.Command; +import io.airlift.command.Help; +import io.airlift.command.Option; +import io.airlift.command.OptionType; +import io.airlift.command.ParseArgumentsMissingException; +import io.airlift.command.ParseArgumentsUnexpectedException; +import io.airlift.command.ParseCommandMissingException; +import io.airlift.command.ParseCommandUnrecognizedException; +import io.airlift.command.ParseOptionConversionException; +import io.airlift.command.ParseOptionMissingException; +import io.airlift.command.ParseOptionMissingValueException; + +import java.io.Console; +import java.io.File; +import java.io.FileNotFoundException; +import java.io.FileWriter; +import java.io.IOError; +import java.io.IOException; +import java.lang.management.MemoryUsage; +import java.net.InetAddress; +import java.net.UnknownHostException; +import java.text.DecimalFormat; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.Comparator; +import java.util.Date; +import java.util.HashMap; +import java.util.Iterator; +import java.util.List; +import java.util.Map; +import java.util.Map.Entry; +import java.util.Scanner; +import java.util.Set; +import java.util.concurrent.ExecutionException; + +import javax.management.InstanceNotFoundException; +import javax.management.openmbean.CompositeData; +import javax.management.openmbean.OpenDataException; +import javax.management.openmbean.TabularData; +import javax.management.openmbean.TabularDataSupport; + +import org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutorMBean; +import org.apache.cassandra.db.ColumnFamilyStoreMBean; +import org.apache.cassandra.db.Keyspace; +import org.apache.cassandra.db.compaction.CompactionManagerMBean; +import org.apache.cassandra.db.compaction.OperationType; +import org.apache.cassandra.io.util.FileUtils; +import org.apache.cassandra.locator.EndpointSnitchInfoMBean; +import org.apache.cassandra.metrics.ColumnFamilyMetrics.Sampler; +import org.apache.cassandra.net.MessagingServiceMBean; +import org.apache.cassandra.repair.RepairParallelism; +import org.apache.cassandra.service.CacheServiceMBean; +import org.apache.cassandra.service.StorageProxyMBean; +import org.apache.cassandra.streaming.ProgressInfo; +import org.apache.cassandra.streaming.SessionInfo; +import org.apache.cassandra.streaming.StreamState; +import org.apache.cassandra.utils.FBUtilities; +import org.apache.cassandra.utils.JVMStabilityInspector; +import org.apache.commons.lang3.ArrayUtils; + +import com.google.common.base.Joiner; +import com.google.common.base.Throwables; +import com.google.common.collect.ArrayListMultimap; +import com.google.common.collect.LinkedHashMultimap; +import com.google.common.collect.Lists; +import
[11/16] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
--
diff --cc src/java/org/apache/cassandra/tools/StandaloneScrubber.java
index d420218,26768f3..459df98
--- a/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
+++ b/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
@@@ -109,7 -121,7 +110,7 @@@ public class StandaloneScrubbe
{
try
{
- Scrubber scrubber = new Scrubber(cfs, sstable,
options.skipCorrupted, handler, true);
-Scrubber scrubber = new Scrubber(cfs, sstable,
options.skipCorrupted, !options.noValidate, handler, true);
++Scrubber scrubber = new Scrubber(cfs, sstable,
options.skipCorrupted, handler, true, !options.noValidate);
try
{
scrubber.scrub();
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/test/unit/org/apache/cassandra/db/ScrubTest.java
--
diff --cc test/unit/org/apache/cassandra/db/ScrubTest.java
index a19c76d,94c7e34..028cf6c
--- a/test/unit/org/apache/cassandra/db/ScrubTest.java
+++ b/test/unit/org/apache/cassandra/db/ScrubTest.java
@@@ -30,7 -30,10 +30,11 @@@ import java.util.Set
import java.util.concurrent.ExecutionException;
import org.apache.cassandra.cql3.QueryProcessor;
-import org.apache.cassandra.db.compaction.OperationType;
++import org.apache.cassandra.db.composites.CellNameType;
+import org.apache.cassandra.exceptions.ConfigurationException;
+ import org.apache.cassandra.db.marshal.CompositeType;
+ import org.apache.cassandra.db.marshal.LongType;
+ import org.apache.cassandra.db.marshal.UTF8Type;
import org.apache.cassandra.exceptions.RequestExecutionException;
import org.apache.cassandra.io.compress.CompressionMetadata;
import org.apache.cassandra.utils.UUIDGen;
@@@ -124,7 -123,7 +128,7 @@@ public class ScrubTest extends SchemaLo
overrideWithGarbage(sstable, ByteBufferUtil.bytes(0),
ByteBufferUtil.bytes(1));
// with skipCorrupted == false, the scrub is expected to fail
- Scrubber scrubber = new Scrubber(cfs, sstable, false, false);
-Scrubber scrubber = new Scrubber(cfs, sstable, false, true);
++Scrubber scrubber = new Scrubber(cfs, sstable, false, false, true);
try
{
scrubber.scrub();
@@@ -134,9 -133,10 +138,9 @@@
// with skipCorrupted == true, the corrupt row will be skipped
Scrubber.ScrubResult scrubResult;
- scrubber = new Scrubber(cfs, sstable, true, false);
-scrubber = new Scrubber(cfs, sstable, true, true);
++scrubber = new Scrubber(cfs, sstable, true, false, true);
scrubResult = scrubber.scrubWithResult();
scrubber.close();
-cfs.replaceCompactedSSTables(Collections.singletonList(sstable),
Collections.singletonList(scrubber.getNewSSTable()), OperationType.SCRUB);
assertNotNull(scrubResult);
@@@ -182,7 -181,7 +186,7 @@@
overrideWithGarbage(sstable, ByteBufferUtil.bytes(0),
ByteBufferUtil.bytes(1));
// with skipCorrupted == false, the scrub is expected to fail
- Scrubber scrubber = new Scrubber(cfs, sstable, false, false);
-Scrubber scrubber = new Scrubber(cfs, sstable, false, true);
++Scrubber scrubber = new Scrubber(cfs, sstable, false, false, true);
try
{
scrubber.scrub();
@@@ -191,9 -190,10 +195,9 @@@
catch (IOError err) {}
// with skipCorrupted == true, the corrupt row will be skipped
- scrubber = new Scrubber(cfs, sstable, true, false);
-scrubber = new Scrubber(cfs, sstable, true, true);
++scrubber = new Scrubber(cfs, sstable, true, false, true);
scrubber.scrub();
scrubber.close();
-cfs.replaceCompactedSSTables(Collections.singletonList(sstable),
Collections.singletonList(scrubber.getNewSSTable()), OperationType.SCRUB);
assertEquals(1, cfs.getSSTables().size());
// verify that we can read all of the rows, and there is now one less
row
@@@ -324,7 -324,7 +328,7 @@@
components.add(Component.TOC);
SSTableReader sstable = SSTableReader.openNoValidation(desc,
components, metadata);
--Scrubber scrubber = new Scrubber(cfs, sstable, false, true);
++Scrubber scrubber = new Scrubber(cfs, sstable, false, true, true);
scrubber.scrub();
cfs.loadNewSSTables();
@@@ -420,9 -420,19 +424,19 @@@
Keyspace keyspace = Keyspace.open(Keyspace1);
ColumnFamilyStore cfs =
keyspace.getColumnFamilyStore(test_compact_static_columns);
-QueryProcessor.processInternal(INSERT INTO
[10/16] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d693ca12
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d693ca12
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d693ca12
Branch: refs/heads/cassandra-2.1
Commit: d693ca12c76c2651df1769e137a94b954174e061
Parents: adb8831 be9eff5
Author: Yuki Morishita yu...@apache.org
Authored: Tue May 19 08:50:28 2015 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Tue May 19 08:50:28 2015 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 4 +--
.../db/compaction/CompactionManager.java| 8 ++---
.../cassandra/db/compaction/Scrubber.java | 12 ---
.../cassandra/service/StorageService.java | 7 +++-
.../cassandra/service/StorageServiceMBean.java | 2 ++
.../org/apache/cassandra/tools/NodeProbe.java | 8 ++---
.../org/apache/cassandra/tools/NodeTool.java| 7 +++-
.../cassandra/tools/StandaloneScrubber.java | 6 +++-
.../unit/org/apache/cassandra/db/ScrubTest.java | 38 +---
10 files changed, 63 insertions(+), 30 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/CHANGES.txt
--
diff --cc CHANGES.txt
index 198935b,cf124b4..6fc1c9c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -30,6 -3,10 +30,7 @@@ Merged from 2.0
* Clone SliceQueryFilter in AbstractReadCommand implementations
(CASSANDRA-8940)
* Push correct protocol notification for DROP INDEX (CASSANDRA-9310)
* token-generator - generated tokens too long (CASSANDRA-9300)
+ * Add option not to validate atoms during scrub (CASSANDRA-9406)
-
-
-2.0.15:
* Fix counting of tombstones for TombstoneOverwhelmingException
(CASSANDRA-9299)
* Fix ReconnectableSnitch reconnecting to peers during upgrade
(CASSANDRA-6702)
* Include keyspace and table name in error log for collections over the size
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index bdc2d8b,eec4044..0951c01
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -1394,22 -1117,22 +1394,22 @@@ public class ColumnFamilyStore implemen
return maxFile;
}
-public void forceCleanup(CounterId.OneShotRenewer renewer) throws
ExecutionException, InterruptedException
+public CompactionManager.AllSSTableOpStatus forceCleanup() throws
ExecutionException, InterruptedException
{
-CompactionManager.instance.performCleanup(ColumnFamilyStore.this,
renewer);
+return
CompactionManager.instance.performCleanup(ColumnFamilyStore.this);
}
- public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted) throws ExecutionException,
InterruptedException
-public void scrub(boolean disableSnapshot, boolean skipCorrupted, boolean
checkData) throws ExecutionException, InterruptedException
++public CompactionManager.AllSSTableOpStatus scrub(boolean
disableSnapshot, boolean skipCorrupted, boolean checkData) throws
ExecutionException, InterruptedException
{
// skip snapshot creation during scrub, SEE JIRA 5891
if(!disableSnapshot)
snapshotWithoutFlush(pre-scrub- + System.currentTimeMillis());
- return
CompactionManager.instance.performScrub(ColumnFamilyStore.this, skipCorrupted);
-CompactionManager.instance.performScrub(ColumnFamilyStore.this,
skipCorrupted, checkData);
++return
CompactionManager.instance.performScrub(ColumnFamilyStore.this, skipCorrupted,
checkData);
}
-public void sstablesRewrite(boolean excludeCurrentVersion) throws
ExecutionException, InterruptedException
+public CompactionManager.AllSSTableOpStatus sstablesRewrite(boolean
excludeCurrentVersion) throws ExecutionException, InterruptedException
{
-
CompactionManager.instance.performSSTableRewrite(ColumnFamilyStore.this,
excludeCurrentVersion);
+return
CompactionManager.instance.performSSTableRewrite(ColumnFamilyStore.this,
excludeCurrentVersion);
}
public void markObsolete(CollectionSSTableReader sstables,
OperationType compactionType)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --cc
[03/16] cassandra git commit: Add option to not validate atoms during scrub
Add option to not validate atoms during scrub
patch by Jordan West; reviewed by yukim for CASSANDRA-9406
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/be9eff57
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/be9eff57
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/be9eff57
Branch: refs/heads/cassandra-2.2
Commit: be9eff573342cd4993eb4fc8ee694dc885e8a8e4
Parents: 37d81b9
Author: Jordan West jorda...@gmail.com
Authored: Tue May 19 08:26:27 2015 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Tue May 19 08:26:27 2015 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 4 +--
.../db/compaction/CompactionManager.java| 12 +++
.../cassandra/db/compaction/Scrubber.java | 12 ---
.../cassandra/service/StorageService.java | 7 +++-
.../cassandra/service/StorageServiceMBean.java | 2 ++
.../org/apache/cassandra/tools/NodeCmd.java | 5 ++-
.../org/apache/cassandra/tools/NodeProbe.java | 4 +--
.../cassandra/tools/StandaloneScrubber.java | 6 +++-
.../unit/org/apache/cassandra/db/ScrubTest.java | 37 +---
10 files changed, 60 insertions(+), 30 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/be9eff57/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a9d04d6..cf124b4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,7 @@
* Clone SliceQueryFilter in AbstractReadCommand implementations
(CASSANDRA-8940)
* Push correct protocol notification for DROP INDEX (CASSANDRA-9310)
* token-generator - generated tokens too long (CASSANDRA-9300)
+ * Add option not to validate atoms during scrub (CASSANDRA-9406)
2.0.15:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/be9eff57/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 5ea1287..eec4044 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -1122,12 +1122,12 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
CompactionManager.instance.performCleanup(ColumnFamilyStore.this,
renewer);
}
-public void scrub(boolean disableSnapshot, boolean skipCorrupted) throws
ExecutionException, InterruptedException
+public void scrub(boolean disableSnapshot, boolean skipCorrupted, boolean
checkData) throws ExecutionException, InterruptedException
{
// skip snapshot creation during scrub, SEE JIRA 5891
if(!disableSnapshot)
snapshotWithoutFlush(pre-scrub- + System.currentTimeMillis());
-CompactionManager.instance.performScrub(ColumnFamilyStore.this,
skipCorrupted);
+CompactionManager.instance.performScrub(ColumnFamilyStore.this,
skipCorrupted, checkData);
}
public void sstablesRewrite(boolean excludeCurrentVersion) throws
ExecutionException, InterruptedException
http://git-wip-us.apache.org/repos/asf/cassandra/blob/be9eff57/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 0978ae6..207b90d 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -234,13 +234,13 @@ public class CompactionManager implements
CompactionManagerMBean
executor.submit(runnable).get();
}
-public void performScrub(ColumnFamilyStore cfStore, final boolean
skipCorrupted) throws InterruptedException, ExecutionException
+public void performScrub(ColumnFamilyStore cfStore, final boolean
skipCorrupted, final boolean checkData) throws InterruptedException,
ExecutionException
{
performAllSSTableOperation(cfStore, new AllSSTablesOperation()
{
public void perform(ColumnFamilyStore store,
IterableSSTableReader sstables) throws IOException
{
-doScrub(store, sstables, skipCorrupted);
+doScrub(store, sstables, skipCorrupted, checkData);
}
});
}
@@ -432,16 +432,16 @@ public class CompactionManager implements
CompactionManagerMBean
*
* @throws IOException
*/
-private void doScrub(ColumnFamilyStore cfs, IterableSSTableReader
sstables, boolean
[01/16] cassandra git commit: Add option to not validate atoms during scrub
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 37d81b9dc - be9eff573
refs/heads/cassandra-2.1 adb88316a - d693ca12c
refs/heads/cassandra-2.2 678291a75 - ff1a15629
refs/heads/trunk bade0fb1a - 5c413ef76
Add option to not validate atoms during scrub
patch by Jordan West; reviewed by yukim for CASSANDRA-9406
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/be9eff57
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/be9eff57
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/be9eff57
Branch: refs/heads/cassandra-2.0
Commit: be9eff573342cd4993eb4fc8ee694dc885e8a8e4
Parents: 37d81b9
Author: Jordan West jorda...@gmail.com
Authored: Tue May 19 08:26:27 2015 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Tue May 19 08:26:27 2015 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 4 +--
.../db/compaction/CompactionManager.java| 12 +++
.../cassandra/db/compaction/Scrubber.java | 12 ---
.../cassandra/service/StorageService.java | 7 +++-
.../cassandra/service/StorageServiceMBean.java | 2 ++
.../org/apache/cassandra/tools/NodeCmd.java | 5 ++-
.../org/apache/cassandra/tools/NodeProbe.java | 4 +--
.../cassandra/tools/StandaloneScrubber.java | 6 +++-
.../unit/org/apache/cassandra/db/ScrubTest.java | 37 +---
10 files changed, 60 insertions(+), 30 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/be9eff57/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a9d04d6..cf124b4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,7 @@
* Clone SliceQueryFilter in AbstractReadCommand implementations
(CASSANDRA-8940)
* Push correct protocol notification for DROP INDEX (CASSANDRA-9310)
* token-generator - generated tokens too long (CASSANDRA-9300)
+ * Add option not to validate atoms during scrub (CASSANDRA-9406)
2.0.15:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/be9eff57/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 5ea1287..eec4044 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -1122,12 +1122,12 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
CompactionManager.instance.performCleanup(ColumnFamilyStore.this,
renewer);
}
-public void scrub(boolean disableSnapshot, boolean skipCorrupted) throws
ExecutionException, InterruptedException
+public void scrub(boolean disableSnapshot, boolean skipCorrupted, boolean
checkData) throws ExecutionException, InterruptedException
{
// skip snapshot creation during scrub, SEE JIRA 5891
if(!disableSnapshot)
snapshotWithoutFlush(pre-scrub- + System.currentTimeMillis());
-CompactionManager.instance.performScrub(ColumnFamilyStore.this,
skipCorrupted);
+CompactionManager.instance.performScrub(ColumnFamilyStore.this,
skipCorrupted, checkData);
}
public void sstablesRewrite(boolean excludeCurrentVersion) throws
ExecutionException, InterruptedException
http://git-wip-us.apache.org/repos/asf/cassandra/blob/be9eff57/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 0978ae6..207b90d 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -234,13 +234,13 @@ public class CompactionManager implements
CompactionManagerMBean
executor.submit(runnable).get();
}
-public void performScrub(ColumnFamilyStore cfStore, final boolean
skipCorrupted) throws InterruptedException, ExecutionException
+public void performScrub(ColumnFamilyStore cfStore, final boolean
skipCorrupted, final boolean checkData) throws InterruptedException,
ExecutionException
{
performAllSSTableOperation(cfStore, new AllSSTablesOperation()
{
public void perform(ColumnFamilyStore store,
IterableSSTableReader sstables) throws IOException
{
-doScrub(store, sstables, skipCorrupted);
+doScrub(store, sstables, skipCorrupted, checkData);
}
});
}
@@
[05/16] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
--
diff --cc src/java/org/apache/cassandra/tools/StandaloneScrubber.java
index d420218,26768f3..459df98
--- a/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
+++ b/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
@@@ -109,7 -121,7 +110,7 @@@ public class StandaloneScrubbe
{
try
{
- Scrubber scrubber = new Scrubber(cfs, sstable,
options.skipCorrupted, handler, true);
-Scrubber scrubber = new Scrubber(cfs, sstable,
options.skipCorrupted, !options.noValidate, handler, true);
++Scrubber scrubber = new Scrubber(cfs, sstable,
options.skipCorrupted, handler, true, !options.noValidate);
try
{
scrubber.scrub();
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d693ca12/test/unit/org/apache/cassandra/db/ScrubTest.java
--
diff --cc test/unit/org/apache/cassandra/db/ScrubTest.java
index a19c76d,94c7e34..028cf6c
--- a/test/unit/org/apache/cassandra/db/ScrubTest.java
+++ b/test/unit/org/apache/cassandra/db/ScrubTest.java
@@@ -30,7 -30,10 +30,11 @@@ import java.util.Set
import java.util.concurrent.ExecutionException;
import org.apache.cassandra.cql3.QueryProcessor;
-import org.apache.cassandra.db.compaction.OperationType;
++import org.apache.cassandra.db.composites.CellNameType;
+import org.apache.cassandra.exceptions.ConfigurationException;
+ import org.apache.cassandra.db.marshal.CompositeType;
+ import org.apache.cassandra.db.marshal.LongType;
+ import org.apache.cassandra.db.marshal.UTF8Type;
import org.apache.cassandra.exceptions.RequestExecutionException;
import org.apache.cassandra.io.compress.CompressionMetadata;
import org.apache.cassandra.utils.UUIDGen;
@@@ -124,7 -123,7 +128,7 @@@ public class ScrubTest extends SchemaLo
overrideWithGarbage(sstable, ByteBufferUtil.bytes(0),
ByteBufferUtil.bytes(1));
// with skipCorrupted == false, the scrub is expected to fail
- Scrubber scrubber = new Scrubber(cfs, sstable, false, false);
-Scrubber scrubber = new Scrubber(cfs, sstable, false, true);
++Scrubber scrubber = new Scrubber(cfs, sstable, false, false, true);
try
{
scrubber.scrub();
@@@ -134,9 -133,10 +138,9 @@@
// with skipCorrupted == true, the corrupt row will be skipped
Scrubber.ScrubResult scrubResult;
- scrubber = new Scrubber(cfs, sstable, true, false);
-scrubber = new Scrubber(cfs, sstable, true, true);
++scrubber = new Scrubber(cfs, sstable, true, false, true);
scrubResult = scrubber.scrubWithResult();
scrubber.close();
-cfs.replaceCompactedSSTables(Collections.singletonList(sstable),
Collections.singletonList(scrubber.getNewSSTable()), OperationType.SCRUB);
assertNotNull(scrubResult);
@@@ -182,7 -181,7 +186,7 @@@
overrideWithGarbage(sstable, ByteBufferUtil.bytes(0),
ByteBufferUtil.bytes(1));
// with skipCorrupted == false, the scrub is expected to fail
- Scrubber scrubber = new Scrubber(cfs, sstable, false, false);
-Scrubber scrubber = new Scrubber(cfs, sstable, false, true);
++Scrubber scrubber = new Scrubber(cfs, sstable, false, false, true);
try
{
scrubber.scrub();
@@@ -191,9 -190,10 +195,9 @@@
catch (IOError err) {}
// with skipCorrupted == true, the corrupt row will be skipped
- scrubber = new Scrubber(cfs, sstable, true, false);
-scrubber = new Scrubber(cfs, sstable, true, true);
++scrubber = new Scrubber(cfs, sstable, true, false, true);
scrubber.scrub();
scrubber.close();
-cfs.replaceCompactedSSTables(Collections.singletonList(sstable),
Collections.singletonList(scrubber.getNewSSTable()), OperationType.SCRUB);
assertEquals(1, cfs.getSSTables().size());
// verify that we can read all of the rows, and there is now one less
row
@@@ -324,7 -324,7 +328,7 @@@
components.add(Component.TOC);
SSTableReader sstable = SSTableReader.openNoValidation(desc,
components, metadata);
--Scrubber scrubber = new Scrubber(cfs, sstable, false, true);
++Scrubber scrubber = new Scrubber(cfs, sstable, false, true, true);
scrubber.scrub();
cfs.loadNewSSTables();
@@@ -420,9 -420,19 +424,19 @@@
Keyspace keyspace = Keyspace.open(Keyspace1);
ColumnFamilyStore cfs =
keyspace.getColumnFamilyStore(test_compact_static_columns);
-QueryProcessor.processInternal(INSERT INTO
cassandra git commit: cancel any rejected tasks on shutdown so callers who care can progress
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 d693ca12c - 7af9c6ac8
cancel any rejected tasks on shutdown so callers who care can progress
patch by tjake; reviewed by Sergio Bossa for CASSANDRA-9417
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7af9c6ac
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7af9c6ac
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7af9c6ac
Branch: refs/heads/cassandra-2.1
Commit: 7af9c6ac88278d34202fea922606905b326f7811
Parents: d693ca1
Author: T Jake Luciani j...@apache.org
Authored: Mon May 18 15:18:24 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Tue May 19 10:12:07 2015 -0400
--
.../DebuggableScheduledThreadPoolExecutor.java | 9 ++
...buggableScheduledThreadPoolExecutorTest.java | 115 +++
2 files changed, 124 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7af9c6ac/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
--
diff --git
a/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
b/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
index 7226db8..ea0715c 100644
---
a/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
+++
b/src/java/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutor.java
@@ -31,6 +31,11 @@ import org.apache.cassandra.utils.JVMStabilityInspector;
*
* DebuggableScheduledThreadPoolExecutor also catches exceptions during Task
execution
* so that they don't supress subsequent invocations of the task.
+ *
+ * Finally, there is a special rejected execution handler for tasks rejected
during the shutdown hook.
+ *
+ * For fire and forget tasks (like ref tidy) we can safely ignore the
exceptions.
+ * For any callers that care to know their task was rejected we cancel passed
task.
*/
public class DebuggableScheduledThreadPoolExecutor extends
ScheduledThreadPoolExecutor
{
@@ -45,6 +50,10 @@ public class DebuggableScheduledThreadPoolExecutor extends
ScheduledThreadPoolEx
if (!StorageService.instance.isInShutdownHook())
throw new
RejectedExecutionException(ScheduledThreadPoolExecutor has shut down.);
+//Give some notification to the caller the task isn't going to
run
+if (task instanceof Future)
+((Future) task).cancel(false);
+
logger.debug(ScheduledThreadPoolExecutor has shut down as
part of C* shutdown);
}
else
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7af9c6ac/test/unit/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutorTest.java
--
diff --git
a/test/unit/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutorTest.java
b/test/unit/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutorTest.java
new file mode 100644
index 000..46b2764
--- /dev/null
+++
b/test/unit/org/apache/cassandra/concurrent/DebuggableScheduledThreadPoolExecutorTest.java
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.concurrent;
+
+import java.io.IOException;
+import java.util.concurrent.CancellationException;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import junit.framework.Assert;
+import org.apache.cassandra.service.EmbeddedCassandraService;
+import org.apache.cassandra.service.StorageService;
+
+public class DebuggableScheduledThreadPoolExecutorTest
+{
+
+static
[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e8766e53 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e8766e53 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e8766e53 Branch: refs/heads/cassandra-2.2 Commit: e8766e53d11645b41e25d904d5bdbdb2b1c0c9a9 Parents: ff1a156 7af9c6a Author: T Jake Luciani j...@apache.org Authored: Tue May 19 10:13:14 2015 -0400 Committer: T Jake Luciani j...@apache.org Committed: Tue May 19 10:13:14 2015 -0400 -- .../DebuggableScheduledThreadPoolExecutor.java | 9 ++ ...buggableScheduledThreadPoolExecutorTest.java | 115 +++ 2 files changed, 124 insertions(+) --
[jira] [Commented] (CASSANDRA-9407) SSTableRewriterTest is hard failing after Fix canonical view returning early opened SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-9407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550490#comment-14550490 ] Ariel Weisberg commented on CASSANDRA-9407: --- I am in the habit of being pretty generous with opening tickets for commits because JIRA has a really nice feature we don't have hooked up where all the commits for a ticket are shown in JIRA under one of the tabs at the bottom if they mention the ticket in the commit message. That is why I always include the ticket name in every commit. Separate tickets kind of maybe aid in searchability and tracking. I am fine with reopening instead. SSTableRewriterTest is hard failing after Fix canonical view returning early opened SSTables -- Key: CASSANDRA-9407 URL: https://issues.apache.org/jira/browse/CASSANDRA-9407 Project: Cassandra Issue Type: Test Reporter: Ariel Weisberg Assignee: Benedict Fix For: 2.2.0 rc1 When I bisected it seems to start failing after https://github.com/apache/cassandra/commit/7d6a60ddd0e156f6028c05d97fa9179f86fe02ee -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9419) CqlRecordWriter does not adequately determine if driver failed to connect due to interrupt
[
https://issues.apache.org/jira/browse/CASSANDRA-9419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550493#comment-14550493
]
Sam Tunnicliffe commented on CASSANDRA-9419:
I'm still seeing failures locally, where the log output looks like this (with
some of the WARNs elided):
{noformat}
[junit] Testsuite: org.apache.cassandra.pig.CqlTableTest
[junit] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
47.007 sec
[junit]
[junit] - Standard Output ---
...
[junit] WARN 13:27:58 job_local_0003
[junit] java.io.IOException:
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried
for query failed (tried: localhost/127.0.0.1:9042
(com.datastax.driver.core.exceptions.DriverException: Connection thread
interrupted))
[junit] at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter$RangeClient.run(CqlRecordWriter.java:329)
~[main/:na]
[junit] Caused by:
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried
for query failed (tried: localhost/127.0.0.1:9042
(com.datastax.driver.core.exceptions.DriverException: Connection thread
interrupted))
[junit] at
com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:223)
~[cassandra-driver-core-2.1.5-shaded.jar:na]
[junit] at
com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:78)
~[cassandra-driver-core-2.1.5-shaded.jar:na]
[junit] at
com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1230)
~[cassandra-driver-core-2.1.5-shaded.jar:na]
[junit] at com.datastax.driver.core.Cluster.init(Cluster.java:157)
~[cassandra-driver-core-2.1.5-shaded.jar:na]
[junit] at com.datastax.driver.core.Cluster.connect(Cluster.java:245)
~[cassandra-driver-core-2.1.5-shaded.jar:na]
[junit] at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter$RangeClient.run(CqlRecordWriter.java:320)
~[main/:na]
[junit] WARN 13:27:58 job_local_0003
[junit] java.io.IOException:
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried
for query failed (tried: localhost/127.0.0.1:9042
(com.datastax.driver.core.exceptions.DriverException: Connection thread
interrupted))
[junit] at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter$RangeClient.run(CqlRecordWriter.java:329)
~[main/:na]
[junit] Caused by:
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried
for query failed (tried: localhost/127.0.0.1:9042
(com.datastax.driver.core.exceptions.DriverException: Connection thread
interrupted))
[junit] at
com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:223)
~[cassandra-driver-core-2.1.5-shaded.jar:na]
[junit] at
com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:78)
~[cassandra-driver-core-2.1.5-shaded.jar:na]
[junit] at
com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1230)
~[cassandra-driver-core-2.1.5-shaded.jar:na]
[junit] at com.datastax.driver.core.Cluster.init(Cluster.java:157)
~[cassandra-driver-core-2.1.5-shaded.jar:na]
[junit] at com.datastax.driver.core.Cluster.connect(Cluster.java:245)
~[cassandra-driver-core-2.1.5-shaded.jar:na]
[junit] at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter$RangeClient.run(CqlRecordWriter.java:320)
~[main/:na]
[junit] WARN 13:27:58 Ooops! Some job has failed! Specify -stop_on_failure
if you want Pig to stop immediately on failure.
[junit] WARN 13:27:58 Ooops! Some job has failed! Specify -stop_on_failure
if you want Pig to stop immediately on failure.
[junit] ERROR 13:27:58 1 map reduce job(s) failed!
[junit] ERROR 13:27:58 1 map reduce job(s) failed!
[junit] WARN 13:27:59 Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
[junit] WARN 13:27:59 Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
[junit] WARN 13:27:59 No job jar file set. User classes may not be found.
See JobConf(Class) or JobConf#setJar(String).
[junit] WARN 13:27:59 No job jar file set. User classes may not be found.
See JobConf(Class) or JobConf#setJar(String).
[junit] WARN 13:28:02 Failed to get RunningJob for job job_local_0004
[junit] WARN 13:28:02 Failed to get RunningJob for job job_local_0004
[junit] WARN 13:28:02 SchemaTupleBackend has already been initialized
[junit] WARN 13:28:02 SchemaTupleBackend has already been initialized
[junit] WARN 13:28:03 Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
[junit] WARN 13:28:03 Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
[junit] WARN 13:28:03
[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-8603: -- Attachment: 8603-2.1-V3.txt Patch for 2.1 that perform the check during deserialization Cut tombstone memory footprint in half for cql deletes -- Key: CASSANDRA-8603 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Dominic Letz Assignee: Dominic Letz Labels: tombstone Attachments: 8603-2.1-V3.txt, cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt, cassandra-2.1-8603_v2.txt, system.log As CQL does not yet support range deletes every delete from CQL results in a Semi-RangeTombstone which actually has the same start and end values - but until today they are copies. Effectively doubling the required heap memory to store the RangeTombstone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8163) Re-introduce DESCRIBE permission
[ https://issues.apache.org/jira/browse/CASSANDRA-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550527#comment-14550527 ] Jonathan Ellis commented on CASSANDRA-8163: --- Since this is a breaking API change, we'll have to do it in 4.0 if it doesn't make 3.0. (But I'm not tagging it for 3.0 b1 since I'd prefer to reserve that for must-have tickets.) Re-introduce DESCRIBE permission Key: CASSANDRA-8163 URL: https://issues.apache.org/jira/browse/CASSANDRA-8163 Project: Cassandra Issue Type: Improvement Reporter: Vishy Kasar Priority: Minor Fix For: 3.x We have a cluster like this: project1_keyspace table101 table102 project2_keyspace table201 table202 We have set up following users and grants: project1_user has all access to project1_keyspace project2_user has all access to project2_keyspace However project1_user can still do a 'describe schema' and get the schema for project2_keyspace as well. We do not want project1_user to have any knowledge for project2 in any way (cqlsh/java-driver etc) . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8163) Re-introduce DESCRIBE permission
[ https://issues.apache.org/jira/browse/CASSANDRA-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550527#comment-14550527 ] Jonathan Ellis edited comment on CASSANDRA-8163 at 5/19/15 2:44 PM: If this is a breaking API change, we'll have to do it in 4.0 if it doesn't make 3.0. (But I'm not tagging it for 3.0 b1 since I'd prefer to reserve that for must-have tickets.) was (Author: jbellis): Since this is a breaking API change, we'll have to do it in 4.0 if it doesn't make 3.0. (But I'm not tagging it for 3.0 b1 since I'd prefer to reserve that for must-have tickets.) Re-introduce DESCRIBE permission Key: CASSANDRA-8163 URL: https://issues.apache.org/jira/browse/CASSANDRA-8163 Project: Cassandra Issue Type: Improvement Reporter: Vishy Kasar Priority: Minor Fix For: 3.x We have a cluster like this: project1_keyspace table101 table102 project2_keyspace table201 table202 We have set up following users and grants: project1_user has all access to project1_keyspace project2_user has all access to project2_keyspace However project1_user can still do a 'describe schema' and get the schema for project2_keyspace as well. We do not want project1_user to have any knowledge for project2 in any way (cqlsh/java-driver etc) . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-9414) CommitLogTest failing intermittently on Windows
[
https://issues.apache.org/jira/browse/CASSANDRA-9414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie resolved CASSANDRA-9414.
Resolution: Not A Problem
No longer occurring either locally or on CI box on 15 runs (was about 90%
failure rate when I opened this ticket).
Closing as Not a Problem as there's enough other things to fix (see
CASSANDRA-9418) that it's not worth chasing a transient utest failure right now.
CommitLogTest failing intermittently on Windows
---
Key: CASSANDRA-9414
URL: https://issues.apache.org/jira/browse/CASSANDRA-9414
Project: Cassandra
Issue Type: Bug
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
Priority: Minor
Labels: Windows
Fix For: 2.2.0 rc1
Failure is intermittent enough that bisect is proving to be more hassle than
it's worth. Seems pretty consistent in CI.
{noformat}
[junit] Testcase:
testDeleteIfNotDirty(org.apache.cassandra.db.CommitLogTest): Caused an
ERROR
[junit] java.nio.file.AccessDeniedException:
build\test\cassandra\commitlog;0\CommitLog-5-1431965988394.log
[junit] FSWriteError in
build\test\cassandra\commitlog;0\CommitLog-5-1431965988394.log
[junit] at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:131)
[junit] at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:148)
[junit] at
org.apache.cassandra.db.commitlog.CommitLogSegmentManager.recycleSegment(CommitLogSegmentManager.java:360)
[junit] at
org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:166)
[junit] at
org.apache.cassandra.db.commitlog.CommitLog.startUnsafe(CommitLog.java:416)
[junit] at
org.apache.cassandra.db.commitlog.CommitLog.resetUnsafe(CommitLog.java:389)
[junit] at
org.apache.cassandra.db.CommitLogTest.testDeleteIfNotDirty(CommitLogTest.java:178)
[junit] Caused by: java.nio.file.AccessDeniedException:
build\test\cassandra\commitlog;0\CommitLog-5-1431965988394.log
[junit] at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
[junit] at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
[junit] at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
[junit] at
sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
[junit] at
sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
[junit] at java.nio.file.Files.delete(Files.java:1126)
[junit] at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:125)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550559#comment-14550559 ] Aleksey Yeschenko commented on CASSANDRA-8603: -- v3 LGTM. We don't need new tests here, but I'd like to see cassci results before committing. Cut tombstone memory footprint in half for cql deletes -- Key: CASSANDRA-8603 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Dominic Letz Assignee: Benjamin Lerer Labels: tombstone Attachments: 8603-2.1-V3.txt, cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt, cassandra-2.1-8603_v2.txt, system.log As CQL does not yet support range deletes every delete from CQL results in a Semi-RangeTombstone which actually has the same start and end values - but until today they are copies. Effectively doubling the required heap memory to store the RangeTombstone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9369) HSHA dtest for closing connections is failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-9369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550567#comment-14550567
]
Michael Shuler commented on CASSANDRA-9369:
---
Speaking for myself and my own workflow, the only way to see what tests are
failing is to run them continually in CI. CI is the tool I am using to track
the tests that are failing repeatedly, intermittently failing, or blowing up
everything. If they are hidden from my view and I don't see them fail in CI,
then they don't get brought to my attention. I volunteered to track the testing
to try to get to the goal of 100% passing, and I found this particular ticket
from a git commit message that caught my attention - not from the tool I am
using to track the state of testing, which is where I would like it.
As an example of the problem I have with disabling tests with {{@require}}, I
just now removed this disabling for CASSANDRA-8049 CASSANDRA-9300 and
CASSANDRA-9294 - the committers of those may or may not have known these dtests
were disabled, I don't know.
So I understand your point, but I would kindly request that you leave failing
tests to fail in CI. It is much easier for me to help with tracking the state
of tests.
HSHA dtest for closing connections is failing on trunk
--
Key: CASSANDRA-9369
URL: https://issues.apache.org/jira/browse/CASSANDRA-9369
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Tyler Hobbs
Assignee: Ariel Weisberg
Fix For: 2.2.x
The {{thrift_hsha_test.ThriftHSHATest.test_closing_connections}} dtest is
failing against trunk (but not against 2.1). Here's an example failure:
{noformat}
Error Message
There are non-closed connections: COMMAND PID USER FD TYPE DEVICE
SIZE/OFF NODE NAME
java28202 automaton7u IPv4 60767 0t0 TCP
localhost:58436-localhost:9160 (CLOSE_WAIT)
begin captured logging
dtest: DEBUG: cluster ccm directory: /tmp/dtest-U0pP7H
dtest: DEBUG: Creating connection pools..
dtest: DEBUG: Disabling/Enabling thrift iteration #0
dtest: DEBUG: Closing connections from the client side..
pycassa.pool: INFO: Pool 13972199784 was disposed
pycassa.pool: INFO: Pool 139721997714512 was disposed
pycassa.pool: INFO: Pool 139721997749904 was disposed
- end captured logging -
Stacktrace
File /usr/lib/python2.7/unittest/case.py, line 331, in run
testMethod()
File /home/automaton/cassandra-dtest/thrift_hsha_test.py, line 59, in
test_closing_connections
self.assertEqual(len(lines), 0, There are non-closed connections: %s %
stdout)
File /usr/lib/python2.7/unittest/case.py, line 515, in assertEqual
assertion_func(first, second, msg=msg)
File /usr/lib/python2.7/unittest/case.py, line 508, in _baseAssertEqual
raise self.failureException(msg)
'There are non-closed connections: COMMAND PID USER FD TYPE DEVICE
SIZE/OFF NODE NAME\njava28202 automaton7u IPv4 60767 0t0 TCP
localhost:58436-localhost:9160 (CLOSE_WAIT)\n\n begin
captured logging \ndtest: DEBUG: cluster ccm
directory: /tmp/dtest-U0pP7H\ndtest: DEBUG: Creating connection
pools..\ndtest: DEBUG: Disabling/Enabling thrift iteration #0\ndtest: DEBUG:
Closing connections from the client side..\npycassa.pool: INFO: Pool
13972199784 was disposed\npycassa.pool: INFO: Pool 139721997714512 was
disposed\npycassa.pool: INFO: Pool 139721997749904 was
disposed\n- end captured logging
-'
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-9369) HSHA dtest for closing connections is failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-9369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550567#comment-14550567
]
Michael Shuler edited comment on CASSANDRA-9369 at 5/19/15 2:56 PM:
Speaking for myself and my own workflow, the only way to see what tests are
failing is to run them continually in CI. CI is the tool I am using to track
the tests that are failing repeatedly, intermittently failing, or blowing up
everything. If they are hidden from my view and I don't see them fail in CI,
then they don't get brought to my attention. I volunteered to track the testing
to try to get to the goal of 100% passing, and I found this particular ticket
from a git commit message that caught my attention - not from the tool I am
using to track the state of testing, which is where I would like it.
As an example of the problem I have with disabling tests with {{@require}}, I
just now removed this disabling for CASSANDRA-8049 CASSANDRA-9300 and
CASSANDRA-9194 - the committers of those may or may not have known these dtests
were disabled, I don't know.
So I understand your point, but I would kindly request that you leave failing
tests to fail in CI. It is much easier for me to help with tracking the state
of tests.
was (Author: mshuler):
Speaking for myself and my own workflow, the only way to see what tests are
failing is to run them continually in CI. CI is the tool I am using to track
the tests that are failing repeatedly, intermittently failing, or blowing up
everything. If they are hidden from my view and I don't see them fail in CI,
then they don't get brought to my attention. I volunteered to track the testing
to try to get to the goal of 100% passing, and I found this particular ticket
from a git commit message that caught my attention - not from the tool I am
using to track the state of testing, which is where I would like it.
As an example of the problem I have with disabling tests with {{@require}}, I
just now removed this disabling for CASSANDRA-8049 CASSANDRA-9300 and
CASSANDRA-9294 - the committers of those may or may not have known these dtests
were disabled, I don't know.
So I understand your point, but I would kindly request that you leave failing
tests to fail in CI. It is much easier for me to help with tracking the state
of tests.
HSHA dtest for closing connections is failing on trunk
--
Key: CASSANDRA-9369
URL: https://issues.apache.org/jira/browse/CASSANDRA-9369
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Tyler Hobbs
Assignee: Ariel Weisberg
Fix For: 2.2.x
The {{thrift_hsha_test.ThriftHSHATest.test_closing_connections}} dtest is
failing against trunk (but not against 2.1). Here's an example failure:
{noformat}
Error Message
There are non-closed connections: COMMAND PID USER FD TYPE DEVICE
SIZE/OFF NODE NAME
java28202 automaton7u IPv4 60767 0t0 TCP
localhost:58436-localhost:9160 (CLOSE_WAIT)
begin captured logging
dtest: DEBUG: cluster ccm directory: /tmp/dtest-U0pP7H
dtest: DEBUG: Creating connection pools..
dtest: DEBUG: Disabling/Enabling thrift iteration #0
dtest: DEBUG: Closing connections from the client side..
pycassa.pool: INFO: Pool 13972199784 was disposed
pycassa.pool: INFO: Pool 139721997714512 was disposed
pycassa.pool: INFO: Pool 139721997749904 was disposed
- end captured logging -
Stacktrace
File /usr/lib/python2.7/unittest/case.py, line 331, in run
testMethod()
File /home/automaton/cassandra-dtest/thrift_hsha_test.py, line 59, in
test_closing_connections
self.assertEqual(len(lines), 0, There are non-closed connections: %s %
stdout)
File /usr/lib/python2.7/unittest/case.py, line 515, in assertEqual
assertion_func(first, second, msg=msg)
File /usr/lib/python2.7/unittest/case.py, line 508, in _baseAssertEqual
raise self.failureException(msg)
'There are non-closed connections: COMMAND PID USER FD TYPE DEVICE
SIZE/OFF NODE NAME\njava28202 automaton7u IPv4 60767 0t0 TCP
localhost:58436-localhost:9160 (CLOSE_WAIT)\n\n begin
captured logging \ndtest: DEBUG: cluster ccm
directory: /tmp/dtest-U0pP7H\ndtest: DEBUG: Creating connection
pools..\ndtest: DEBUG: Disabling/Enabling thrift iteration #0\ndtest: DEBUG:
Closing connections from the client side..\npycassa.pool: INFO: Pool
13972199784 was disposed\npycassa.pool: INFO: Pool 139721997714512 was
disposed\npycassa.pool: INFO: Pool 139721997749904 was
disposed\n- end captured logging
-'
{noformat}
[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8603: - Assignee: Benjamin Lerer (was: Dominic Letz) Cut tombstone memory footprint in half for cql deletes -- Key: CASSANDRA-8603 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Dominic Letz Assignee: Benjamin Lerer Labels: tombstone Attachments: 8603-2.1-V3.txt, cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt, cassandra-2.1-8603_v2.txt, system.log As CQL does not yet support range deletes every delete from CQL results in a Semi-RangeTombstone which actually has the same start and end values - but until today they are copies. Effectively doubling the required heap memory to store the RangeTombstone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[
https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-7304:
-
Labels: client-impacting cql protocolv4 (was: cql protocolv4)
Ability to distinguish between NULL and UNSET values in Prepared Statements
---
Key: CASSANDRA-7304
URL: https://issues.apache.org/jira/browse/CASSANDRA-7304
Project: Cassandra
Issue Type: Sub-task
Reporter: Drew Kutcharian
Assignee: Oded Peer
Labels: client-impacting, cql, protocolv4
Fix For: 2.2.0 beta 1
Attachments: 7304-03.patch, 7304-04.patch, 7304-05.patch,
7304-06.patch, 7304-07.patch, 7304-2.patch, 7304-V8.txt, 7304.patch
Currently Cassandra inserts tombstones when a value of a column is bound to
NULL in a prepared statement. At higher insert rates managing all these
tombstones becomes an unnecessary overhead. This limits the usefulness of the
prepared statements since developers have to either create multiple prepared
statements (each with a different combination of column names, which at times
is just unfeasible because of the sheer number of possible combinations) or
fall back to using regular (non-prepared) statements.
This JIRA is here to explore the possibility of either:
A. Have a flag on prepared statements that once set, tells Cassandra to
ignore null columns
or
B. Have an UNSET value which makes Cassandra skip the null columns and not
tombstone them
Basically, in the context of a prepared statement, a null value means delete,
but we don’t have anything that means ignore (besides creating a new
prepared statement without the ignored column).
Please refer to the original conversation on DataStax Java Driver mailing
list for more background:
https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion
*EDIT 18/12/14 - [~odpeer] Implementation Notes:*
The motivation hasn't changed.
Protocol version 4 specifies that bind variables do not require having a
value when executing a statement. Bind variables without a value are called
'unset'. The 'unset' bind variable is serialized as the int value '-2'
without following bytes.
\\
\\
* An unset bind variable in an EXECUTE or BATCH request
** On a {{value}} does not modify the value and does not create a tombstone
** On the {{ttl}} clause is treated as 'unlimited'
** On the {{timestamp}} clause is treated as 'now'
** On a map key or a list index throws {{InvalidRequestException}}
** On a {{counter}} increment or decrement operation does not change the
counter value, e.g. {{UPDATE my_tab SET c = c - ? WHERE k = 1}} does change
the value of counter {{c}}
** On a tuple field or UDT field throws {{InvalidRequestException}}
* An unset bind variable in a QUERY request
** On a partition column, clustering column or index column in the {{WHERE}}
clause throws {{InvalidRequestException}}
** On the {{limit}} clause is treated as 'unlimited'
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8603: - Reviewer: Aleksey Yeschenko (was: Benjamin Lerer) Cut tombstone memory footprint in half for cql deletes -- Key: CASSANDRA-8603 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Dominic Letz Assignee: Benjamin Lerer Labels: tombstone Attachments: 8603-2.1-V3.txt, cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt, cassandra-2.1-8603_v2.txt, system.log As CQL does not yet support range deletes every delete from CQL results in a Semi-RangeTombstone which actually has the same start and end values - but until today they are copies. Effectively doubling the required heap memory to store the RangeTombstone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
svn commit: r1680337 - /cassandra/site/publish/doc/cql3/CQL-2.2.html
Author: tylerhobbs Date: Tue May 19 16:37:26 2015 New Revision: 1680337 URL: http://svn.apache.org/r1680337 Log: Add CQL3 docs for Cassandra 2.2 Added: cassandra/site/publish/doc/cql3/CQL-2.2.html Added: cassandra/site/publish/doc/cql3/CQL-2.2.html URL: http://svn.apache.org/viewvc/cassandra/site/publish/doc/cql3/CQL-2.2.html?rev=1680337view=auto == --- cassandra/site/publish/doc/cql3/CQL-2.2.html (added) +++ cassandra/site/publish/doc/cql3/CQL-2.2.html Tue May 19 16:37:26 2015 @@ -0,0 +1,516 @@ +?xml version='1.0' encoding='utf-8' ?!DOCTYPE html PUBLIC -//W3C//DTD XHTML 1.0 Transitional//EN http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;html xmlns=http://www.w3.org/1999/xhtml;headmeta http-equiv=Content-Type content=text/html; charset=utf-8/titleCQL/title/headbodyplink rel=StyleSheet href=CQL.css type=text/css media=screen/ph1 id=CassandraQueryLanguageCQLv3.3.0Cassandra Query Language (CQL) v3.3.0/h1span id=tableOfContentsol style=list-style: none;lia href=CQL.html#CassandraQueryLanguageCQLv3.3.0Cassandra Query Language (CQL) v3.3.0/aol style=list-style: none;lia href=CQL.html#CQLSyntaxCQL Syntax/aol style=list-style: none;lia href=CQL.html#PreamblePreamble/a/lilia href=CQL.html#ConventionsConventions/a/lilia href=CQL.html#identifiersIdentifiers and keywords/a/lilia href=CQL.html#constantsConstants/a/lilia href=CQL.html#CommentsComments/a/l ilia href=CQL.html#statementsStatements/a/lilia href=CQL.html#preparedStatementPrepared Statement/a/li/ol/lilia href=CQL.html#dataDefinitionData Definition/aol style=list-style: none;lia href=CQL.html#createKeyspaceStmtCREATE KEYSPACE/a/lilia href=CQL.html#useStmtUSE/a/lilia href=CQL.html#alterKeyspaceStmtALTER KEYSPACE/a/lilia href=CQL.html#dropKeyspaceStmtDROP KEYSPACE/a/lilia href=CQL.html#createTableStmtCREATE TABLE/a/lilia href=CQL.html#alterTableStmtALTER TABLE/a/lilia href=CQL.html#dropTableStmtDROP TABLE/a/lilia href=CQL.html#truncateStmtTRUNCATE/a/lilia href=CQL.html#createIndexStmtCREATE INDEX/a/lilia href=CQL.html#dropIndexStmtDROP INDEX/a/lilia href=CQL.html#createTypeStmtCREATE TYPE/a/lilia href=CQL.html#alterTypeStmtALTER TYPE/a/lilia href=CQL.html#dropTypeStmtDROP TYPE/a/lilia href=CQL.html#createTri ggerStmtCREATE TRIGGER/a/lilia href=CQL.html#dropTriggerStmtDROP TRIGGER/a/lilia href=CQL.html#createFunctionStmtCREATE FUNCTION/a/lilia href=CQL.html#dropFunctionStmtDROP FUNCTION/a/lilia href=CQL.html#createAggregateStmtCREATE AGGREGATE/a/lilia href=CQL.html#dropAggregateStmtDROP AGGREGATE/a/li/ol/lilia href=CQL.html#dataManipulationData Manipulation/aol style=list-style: none;lia href=CQL.html#insertStmtINSERT/a/lilia href=CQL.html#updateStmtUPDATE/a/lilia href=CQL.html#deleteStmtDELETE/a/lilia href=CQL.html#batchStmtBATCH/a/li/ol/lilia href=CQL.html#queriesQueries/aol style=list-style: none;lia href=CQL.html#selectStmtSELECT/a/li/ol/lilia href=CQL.html#typesData Types/aol style=list-style: none;lia href=CQL.html#usingtimestampsWorking with timestamps/a/lilia href=CQL.html#usingdatesWorking with dates/a/lili a href=CQL.html#usingtimeWorking with time/a/lilia href=CQL.html#countersCounters/a/lilia href=CQL.html#collectionsWorking with collections/a/li/ol/lilia href=CQL.html#functionsFunctions/aol style=list-style: none;lia href=CQL.html#tokenFunToken/a/lilia href=CQL.html#uuidFunUuid/a/lilia href=CQL.html#timeuuidFunTimeuuid functions/a/lilia href=CQL.html#blobFunBlob conversion functions/a/li/ol/lilia href=CQL.html#udfsUser-Defined Functions/a/lilia href=CQL.html#udasUser-Defined Aggregates/a/lilia href=CQL.html#jsonJSON Support/aol style=list-style: none;lia href=CQL.html#selectJsonSELECT JSON/a/lilia href=CQL.html#insertJsonINSERT JSON/a/lilia href=CQL.html#jsonEncodingJSON Encoding of Cassandra Data Types/a/lilia href=CQL.html#fromJsonThe fromJson() Function/a/lilia href=CQL.html#toJsonThe toJson() Function/a/li/ol /lilia href=CQL.html#appendixAAppendix A: CQL Keywords/a/lilia href=CQL.html#appendixBAppendix B: CQL Reserved Types/a/lilia href=CQL.html#changesChanges/aol style=list-style: none;lia href=CQL.html#a3.3.03.3.0/a/lilia href=CQL.html#a3.2.03.2.0/a/lilia href=CQL.html#a3.1.73.1.7/a/lilia href=CQL.html#a3.1.63.1.6/a/lilia href=CQL.html#a3.1.53.1.5/a/lilia href=CQL.html#a3.1.43.1.4/a/lilia href=CQL.html#a3.1.33.1.3/a/lilia href=CQL.html#a3.1.23.1.2/a/lilia href=CQL.html#a3.1.13.1.1/a/lilia href=CQL.html#a3.1.03.1.0/a/lilia href=CQL.html#a3.0.53.0.5/a/lilia href=CQL.html#a3.0.43.0.4/a/lilia href=CQL.html#a3.0.33.0.3/a/lilia href=CQL.html#a3.0.23.0.2/a/lilia href=CQL.html#a3.0.13.0.1/a/li/ol/lilia href=CQL.html#VersioningVersioning/a/li/ol/li/ol/spanh2 id=CQLSyntaxCQL Syntax/h2h 3 id=PreamblePreamble/h3pThis document describes the Cassandra Query Language (CQL) version 3. CQL v3 is not backward compatible with CQL v2 and differs from
[jira] [Updated] (CASSANDRA-9425) Make node-local schema fully immutable
[
https://issues.apache.org/jira/browse/CASSANDRA-9425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-9425:
-
Issue Type: Sub-task (was: Improvement)
Parent: CASSANDRA-9424
Make node-local schema fully immutable
--
Key: CASSANDRA-9425
URL: https://issues.apache.org/jira/browse/CASSANDRA-9425
Project: Cassandra
Issue Type: Sub-task
Reporter: Aleksey Yeschenko
Assignee: Aleksey Yeschenko
Fix For: 3.x
The way we handle schema changes currently is inherently racy.
All of our {{SchemaAlteringStatement}} s perform validation on a schema state
that's won't necessarily be there when the statement gets executed and
mutates schema.
We should make all the *Metadata classes ({{KeyspaceMetadata,
TableMetadata}}, {{ColumnMetadata}}, immutable, and local schema persistently
snapshottable, with a single top-level {{AtomicReference}} to the current
snapshot. Have DDL statements perform validation and transformation on the
same state.
In pseudo-code, think
{code}
public interface DDLStatement
{
/**
* Validates that the DDL statement can be applied to the provided schema
snapshot.
*
* @param schema snapshot of schema before executing CREATE KEYSPACE
*/
void validate(SchemaSnapshot schema);
/**
* Applies the DDL statement to the provided schema snapshot.
* Implies that validate() has already been called on the provided
snapshot.
*
* @param schema snapshot of schema before executing the statement
* @return snapshot of schema as it would be after executing the statement
*/
SchemaSnapshot transform(SchemaSnapshot schema);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
svn commit: r1680340 - /cassandra/site/publish/doc/cql3/CQL-2.2.html
Author: tylerhobbs Date: Tue May 19 16:42:12 2015 New Revision: 1680340 URL: http://svn.apache.org/r1680340 Log: Fix links in CQL-2.2.html Modified: cassandra/site/publish/doc/cql3/CQL-2.2.html Modified: cassandra/site/publish/doc/cql3/CQL-2.2.html URL: http://svn.apache.org/viewvc/cassandra/site/publish/doc/cql3/CQL-2.2.html?rev=1680340r1=1680339r2=1680340view=diff == --- cassandra/site/publish/doc/cql3/CQL-2.2.html (original) +++ cassandra/site/publish/doc/cql3/CQL-2.2.html Tue May 19 16:42:12 2015 @@ -1,4 +1,4 @@ -?xml version='1.0' encoding='utf-8' ?!DOCTYPE html PUBLIC -//W3C//DTD XHTML 1.0 Transitional//EN http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;html xmlns=http://www.w3.org/1999/xhtml;headmeta http-equiv=Content-Type content=text/html; charset=utf-8/titleCQL/title/headbodyplink rel=StyleSheet href=CQL.css type=text/css media=screen/ph1 id=CassandraQueryLanguageCQLv3.3.0Cassandra Query Language (CQL) v3.3.0/h1span id=tableOfContentsol style=list-style: none;lia href=CQL.html#CassandraQueryLanguageCQLv3.3.0Cassandra Query Language (CQL) v3.3.0/aol style=list-style: none;lia href=CQL.html#CQLSyntaxCQL Syntax/aol style=list-style: none;lia href=CQL.html#PreamblePreamble/a/lilia href=CQL.html#ConventionsConventions/a/lilia href=CQL.html#identifiersIdentifiers and keywords/a/lilia href=CQL.html#constantsConstants/a/lilia href=CQL.html#CommentsComments/a/l ilia href=CQL.html#statementsStatements/a/lilia href=CQL.html#preparedStatementPrepared Statement/a/li/ol/lilia href=CQL.html#dataDefinitionData Definition/aol style=list-style: none;lia href=CQL.html#createKeyspaceStmtCREATE KEYSPACE/a/lilia href=CQL.html#useStmtUSE/a/lilia href=CQL.html#alterKeyspaceStmtALTER KEYSPACE/a/lilia href=CQL.html#dropKeyspaceStmtDROP KEYSPACE/a/lilia href=CQL.html#createTableStmtCREATE TABLE/a/lilia href=CQL.html#alterTableStmtALTER TABLE/a/lilia href=CQL.html#dropTableStmtDROP TABLE/a/lilia href=CQL.html#truncateStmtTRUNCATE/a/lilia href=CQL.html#createIndexStmtCREATE INDEX/a/lilia href=CQL.html#dropIndexStmtDROP INDEX/a/lilia href=CQL.html#createTypeStmtCREATE TYPE/a/lilia href=CQL.html#alterTypeStmtALTER TYPE/a/lilia href=CQL.html#dropTypeStmtDROP TYPE/a/lilia href=CQL.html#createTri ggerStmtCREATE TRIGGER/a/lilia href=CQL.html#dropTriggerStmtDROP TRIGGER/a/lilia href=CQL.html#createFunctionStmtCREATE FUNCTION/a/lilia href=CQL.html#dropFunctionStmtDROP FUNCTION/a/lilia href=CQL.html#createAggregateStmtCREATE AGGREGATE/a/lilia href=CQL.html#dropAggregateStmtDROP AGGREGATE/a/li/ol/lilia href=CQL.html#dataManipulationData Manipulation/aol style=list-style: none;lia href=CQL.html#insertStmtINSERT/a/lilia href=CQL.html#updateStmtUPDATE/a/lilia href=CQL.html#deleteStmtDELETE/a/lilia href=CQL.html#batchStmtBATCH/a/li/ol/lilia href=CQL.html#queriesQueries/aol style=list-style: none;lia href=CQL.html#selectStmtSELECT/a/li/ol/lilia href=CQL.html#typesData Types/aol style=list-style: none;lia href=CQL.html#usingtimestampsWorking with timestamps/a/lilia href=CQL.html#usingdatesWorking with dates/a/lili a href=CQL.html#usingtimeWorking with time/a/lilia href=CQL.html#countersCounters/a/lilia href=CQL.html#collectionsWorking with collections/a/li/ol/lilia href=CQL.html#functionsFunctions/aol style=list-style: none;lia href=CQL.html#tokenFunToken/a/lilia href=CQL.html#uuidFunUuid/a/lilia href=CQL.html#timeuuidFunTimeuuid functions/a/lilia href=CQL.html#blobFunBlob conversion functions/a/li/ol/lilia href=CQL.html#udfsUser-Defined Functions/a/lilia href=CQL.html#udasUser-Defined Aggregates/a/lilia href=CQL.html#jsonJSON Support/aol style=list-style: none;lia href=CQL.html#selectJsonSELECT JSON/a/lilia href=CQL.html#insertJsonINSERT JSON/a/lilia href=CQL.html#jsonEncodingJSON Encoding of Cassandra Data Types/a/lilia href=CQL.html#fromJsonThe fromJson() Function/a/lilia href=CQL.html#toJsonThe toJson() Function/a/li/ol /lilia href=CQL.html#appendixAAppendix A: CQL Keywords/a/lilia href=CQL.html#appendixBAppendix B: CQL Reserved Types/a/lilia href=CQL.html#changesChanges/aol style=list-style: none;lia href=CQL.html#a3.3.03.3.0/a/lilia href=CQL.html#a3.2.03.2.0/a/lilia href=CQL.html#a3.1.73.1.7/a/lilia href=CQL.html#a3.1.63.1.6/a/lilia href=CQL.html#a3.1.53.1.5/a/lilia href=CQL.html#a3.1.43.1.4/a/lilia href=CQL.html#a3.1.33.1.3/a/lilia href=CQL.html#a3.1.23.1.2/a/lilia href=CQL.html#a3.1.13.1.1/a/lilia href=CQL.html#a3.1.03.1.0/a/lilia href=CQL.html#a3.0.53.0.5/a/lilia href=CQL.html#a3.0.43.0.4/a/lilia href=CQL.html#a3.0.33.0.3/a/lilia href=CQL.html#a3.0.23.0.2/a/lilia href=CQL.html#a3.0.13.0.1/a/li/ol/lilia href=CQL.html#VersioningVersioning/a/li/ol/li/ol/spanh2 id=CQLSyntaxCQL Syntax/h2h 3 id=PreamblePreamble/h3pThis document describes the Cassandra Query Language (CQL) version 3. CQL v3 is not backward compatible with CQL
[jira] [Commented] (CASSANDRA-9425) Make node-local schema fully immutable
[
https://issues.apache.org/jira/browse/CASSANDRA-9425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550746#comment-14550746
]
Aleksey Yeschenko commented on CASSANDRA-9425:
--
FWIW I have most of the work done, originally as part of CASSANDRA-6717. That
said, it's not correct to bundle the two, given that they aren't, strictly
speaking, related, and one of them is blocking 3.0beta1.
Make node-local schema fully immutable
--
Key: CASSANDRA-9425
URL: https://issues.apache.org/jira/browse/CASSANDRA-9425
Project: Cassandra
Issue Type: Sub-task
Reporter: Aleksey Yeschenko
Assignee: Aleksey Yeschenko
Fix For: 3.x
The way we handle schema changes currently is inherently racy.
All of our {{SchemaAlteringStatement}} s perform validation on a schema state
that's won't necessarily be there when the statement gets executed and
mutates schema.
We should make all the *Metadata classes ({{KeyspaceMetadata,
TableMetadata}}, {{ColumnMetadata}}, immutable, and local schema persistently
snapshottable, with a single top-level {{AtomicReference}} to the current
snapshot. Have DDL statements perform validation and transformation on the
same state.
In pseudo-code, think
{code}
public interface DDLStatement
{
/**
* Validates that the DDL statement can be applied to the provided schema
snapshot.
*
* @param schema snapshot of schema before executing CREATE KEYSPACE
*/
void validate(SchemaSnapshot schema);
/**
* Applies the DDL statement to the provided schema snapshot.
* Implies that validate() has already been called on the provided
snapshot.
*
* @param schema snapshot of schema before executing the statement
* @return snapshot of schema as it would be after executing the statement
*/
SchemaSnapshot transform(SchemaSnapshot schema);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6038) Support schema changes in mixed version clusters
[ https://issues.apache.org/jira/browse/CASSANDRA-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-6038: - Assignee: (was: Aleksey Yeschenko) Support schema changes in mixed version clusters Key: CASSANDRA-6038 URL: https://issues.apache.org/jira/browse/CASSANDRA-6038 Project: Cassandra Issue Type: Sub-task Reporter: Aleksey Yeschenko Labels: client-impacting Fix For: 3.x CASSANDRA-5845 made it so that schema changes in a major-mixed cluster are not propagated to the minorer-major nodes. This lets us perform backwards-incompatible schema changes in major releases safely - like adding the schema_triggers table, moving all the aliases to schema_columns, getting rid of the deprecated schema columns, etc. Even this limitation might be too strict, however, and should be avoided if possible (starting with at least versioning schema separately from messaging service versioning, and resorting to major-minor schema block only when truly necessary and not for every x-y pair). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (CASSANDRA-9367) Builtin CQL functions should be added to the schema_functions table
[
https://issues.apache.org/jira/browse/CASSANDRA-9367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko reopened CASSANDRA-9367:
--
Builtin CQL functions should be added to the schema_functions table
---
Key: CASSANDRA-9367
URL: https://issues.apache.org/jira/browse/CASSANDRA-9367
Project: Cassandra
Issue Type: Improvement
Reporter: Michaël Figuière
Priority: Minor
As CASSANDRA-6717 will not be part of Cassandra 2.2, my comment there got to
be considered for the current {{system.schema_functions}} table, to avoid
creating a special case just for 2.2:
{quote}
Builtin CQL functions are not described in the {{system}} keyspace in its
current representation, this new schema should include them next to the user
defined ones as:
* The {{system}} keyspace and its tables are described in the {{system}}
keyspace, therefore it would be consistent.
* Having builtin CQL functions described there would allow external tools to
manipulate all the functions in a similar way.
* This would document the available builtin function for users that don't
remember the ones available in their current version of Cassandra.
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9367) Expose builtin-in CQL functions via schema data dictionary
[
https://issues.apache.org/jira/browse/CASSANDRA-9367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-9367:
-
Summary: Expose builtin-in CQL functions via schema data dictionary (was:
Builtin CQL functions should be added to the schema_functions table)
Expose builtin-in CQL functions via schema data dictionary
--
Key: CASSANDRA-9367
URL: https://issues.apache.org/jira/browse/CASSANDRA-9367
Project: Cassandra
Issue Type: Improvement
Reporter: Michaël Figuière
Priority: Minor
As CASSANDRA-6717 will not be part of Cassandra 2.2, my comment there got to
be considered for the current {{system.schema_functions}} table, to avoid
creating a special case just for 2.2:
{quote}
Builtin CQL functions are not described in the {{system}} keyspace in its
current representation, this new schema should include them next to the user
defined ones as:
* The {{system}} keyspace and its tables are described in the {{system}}
keyspace, therefore it would be consistent.
* Having builtin CQL functions described there would allow external tools to
manipulate all the functions in a similar way.
* This would document the available builtin function for users that don't
remember the ones available in their current version of Cassandra.
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
svn commit: r1680344 - /cassandra/site/publish/doc/cql3/CQL-2.2.html
Author: tylerhobbs
Date: Tue May 19 16:51:55 2015
New Revision: 1680344
URL: http://svn.apache.org/r1680344
Log:
Replace Cassandra 3.0 reference with 2.2
Modified:
cassandra/site/publish/doc/cql3/CQL-2.2.html
Modified: cassandra/site/publish/doc/cql3/CQL-2.2.html
URL:
http://svn.apache.org/viewvc/cassandra/site/publish/doc/cql3/CQL-2.2.html?rev=1680344r1=1680343r2=1680344view=diff
==
--- cassandra/site/publish/doc/cql3/CQL-2.2.html (original)
+++ cassandra/site/publish/doc/cql3/CQL-2.2.html Tue May 19 16:51:55 2015
@@ -512,5 +512,5 @@ INSERT INTO atable (pk, val) VALUES (2,2
INSERT INTO atable (pk, val) VALUES (3,3);
INSERT INTO atable (pk, val) VALUES (4,4);
SELECT average(val) FROM atable;
-/pre/prep/ppSee a href=#createAggregateStmtcodeCREATE
AGGREGATE/code/a and a href=#dropAggregateStmtcodeDROP
AGGREGATE/code/a./ph2 id=jsonJSON Support/h2pCassandra 3.0
introduces JSON support to a href=#selectStmtcodeSELECT/code/a and a
href=#insertStmtcodeINSERT/code/a statements. This support does not
fundamentally alter the CQL API (for example, the schema is still enforced), it
simply provides a convenient way to work with JSON documents./ph3
id=selectJsonSELECT JSON/h3pWith codeSELECT/code statements, the new
codeJSON/code keyword can be used to return each row as a single
codeJSON/code encoded map. The remainder of the codeSELECT/code
statment behavior is the same./ppThe result map keys are the same as the
column names in a normal result set. For example, a statement like
codeSELECT JSON a, ttl(b) FROM .../code would result in a map with keys
codea/code and codettl(b)/
code. However, this is one notable exception: for symmetry with codeINSERT
JSON/code behavior, case-sensitive column names with upper-case letters will
be surrounded with double quotes. For example, codeSELECT JSON myColumn
FROM .../code would result in a map key code\myColumn\/code (note
the escaped quotes)./ppThe map values will codeJSON/code-encoded
representations (as described below) of the result set values./ph3
id=insertJsonINSERT JSON/h3pWith codeINSERT/code statements, the new
codeJSON/code keyword can be used to enable inserting a codeJSON/code
encoded map as a single row. The format of the codeJSON/code map should
generally match that returned by a codeSELECT JSON/code statement on the
same table. In particular, case-sensitive column names should be surrounded
with double quotes. For example, to insert into a table with two columns named
#8220;myKey#8221; and #8220;value#8221;, you would do the following:/
ppre class=samplepreINSERT INTO mytable JSON '{\myKey\: 0, value:
0}'
-/pre/prepAny columns which are ommitted from the codeJSON/code map
will be defaulted to a codeNULL/code value (which will result in a
tombstone being created)./ph3 id=jsonEncodingJSON Encoding of Cassandra
Data Types/h3pWhere possible, Cassandra will represent and accept data
types in their native codeJSON/code representation. Cassandra will also
accept string representations matching the CQL literal format for all data
types. The following table describes the encodings that Cassandra will accept
in codeINSERT JSON/code values (and codefromJson()/code arguments) as
well as the format Cassandra will use when returning data for codeSELECT
JSON/code statements (and codefromJson()/code):/ptabletrthtype
/ththformats accepted /ththreturn format
/ththnotes/th/trtrtdcodeascii/code/tdtdstring
/tdtdstring /tdtdUses JSON#8217;s code\u/code
character escape/td/tr
trtdcodebigint/code /tdtdinteger, string /tdtdinteger
/tdtdString must be valid 64 bit
integer/td/trtrtdcodeblob/code /tdtdstring
/tdtdstring /tdtdString should be 0x followed by an even
number of hex digits/td/trtrtdcodeboolean/code /tdtdboolean,
string /tdtdboolean /tdtdString must be
#8220;true#8221; or false/td/trtrtdcodedate/code
/tdtdstring/tdtdstring /tdtdDate in
format code-MM-DD/code, timezone
UTC/td/trtrtdcodedecimal/code /tdtdinteger, float,
string/tdtdfloat/tdtdMay exceed 32 or 64-bit IEEE-754
floating point precision in client-side
decoder/td/trtrtdcodedouble/code /tdtdinteger, float,
string/tdtdfloat/tdtdString must be valid integer or
float/td/trtrtdcodefloat/code/tdtdinteger, float,
string/tdtdfloat/tdtdString must be valid integer or
float/td/trtrtdcodeinet/code /tdtdstring
/tdtdstring /tdtdIPv4 or IPv6
address/td/trtrtdcodeint/code /tdtdinteger, string
/tdtdinteger /tdtdString must be valid 32 bit
integer/td/trtrtdcodetext/code /tdtdstring
/tdtdstring /tdtdUses JSON#8217;s code\u/code character
escape/td/trtrtdcodetime/code /tdtdstring
/tdtdstring /tdtdTime of day
[jira] [Updated] (CASSANDRA-9367) Expose builtin-in CQL functions via schema data dictionary
[
https://issues.apache.org/jira/browse/CASSANDRA-9367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-9367:
-
Issue Type: Sub-task (was: Improvement)
Parent: CASSANDRA-9424
Expose builtin-in CQL functions via schema data dictionary
--
Key: CASSANDRA-9367
URL: https://issues.apache.org/jira/browse/CASSANDRA-9367
Project: Cassandra
Issue Type: Sub-task
Reporter: Michaël Figuière
Priority: Minor
Fix For: 3.x
As CASSANDRA-6717 will not be part of Cassandra 2.2, my comment there got to
be considered for the current {{system.schema_functions}} table, to avoid
creating a special case just for 2.2:
{quote}
Builtin CQL functions are not described in the {{system}} keyspace in its
current representation, this new schema should include them next to the user
defined ones as:
* The {{system}} keyspace and its tables are described in the {{system}}
keyspace, therefore it would be consistent.
* Having builtin CQL functions described there would allow external tools to
manipulate all the functions in a similar way.
* This would document the available builtin function for users that don't
remember the ones available in their current version of Cassandra.
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
