date:20140611


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-6477:
--

Summary: Global indexes  (was: Partitioned indexes)

(Renaming this to Global Indexes, borrowing DynamoDB's term.)

 Global indexes
 --

 Key: CASSANDRA-6477
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
 Fix For: 3.0


 Local indexes are suitable for low-cardinality data, where spreading the 
 index across the cluster is a Good Thing.  However, for high-cardinality 
 data, local indexes require querying most nodes in the cluster even if only a 
 handful of rows is returned.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7371) DELETEs get lost

2014-06-11 Thread T Jake Luciani (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-7371:
--

Reviewer: Benedict

 DELETEs get lost
 

 Key: CASSANDRA-7371
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7371
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.1 git branch since merge commit 
 4722fe70aa9ae1b62772cfa1a1de58ef289445d5 (RefCount native frames from netty 
 to avoid corruption bugs)
Reporter: Robert Stupp
Assignee: T Jake Luciani
Priority: Blocker
 Fix For: 2.1.0

 Attachments: 7371.txt, Cassandra7371.java


 The mentioned commit introduced a bug which is not easy to reproduce:
 Workload description:
 - One INSERT into a table
 - multiple concurrent SELECTs against different tables (one select returns a 
 result)
 - One UPDATE against the same table as the INSERT
 - (same) multiple concurrent SELECTs against different tables (one select 
 returns a result)
 - One DELETE against the same table as the INSERT
 - (same) multiple concurrent SELECTs against different tables
 Expected is that the last bunch of SELECTs returns no result. But since 
 commit SHA  the DELETE gets not processed.
 To clarify - the DELETE is not delayed - it is not executed at all.
 Checked against a single node C* cluster.
 Does only affect unreleased 2.1 - not 2.0 nor 1.2.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7371) DELETEs get lost

2014-06-11 Thread T Jake Luciani (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-7371:
--

Attachment: 7371.txt

Fix attached. Issue was range tombstones didn't bring the start and end 
composites on heap.  

I also fixed a issue where the BatchStatements weren't tracking the source frame

 DELETEs get lost
 

 Key: CASSANDRA-7371
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7371
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.1 git branch since merge commit 
 4722fe70aa9ae1b62772cfa1a1de58ef289445d5 (RefCount native frames from netty 
 to avoid corruption bugs)
Reporter: Robert Stupp
Assignee: T Jake Luciani
Priority: Blocker
 Fix For: 2.1.0

 Attachments: 7371.txt, Cassandra7371.java


 The mentioned commit introduced a bug which is not easy to reproduce:
 Workload description:
 - One INSERT into a table
 - multiple concurrent SELECTs against different tables (one select returns a 
 result)
 - One UPDATE against the same table as the INSERT
 - (same) multiple concurrent SELECTs against different tables (one select 
 returns a result)
 - One DELETE against the same table as the INSERT
 - (same) multiple concurrent SELECTs against different tables
 Expected is that the last bunch of SELECTs returns no result. But since 
 commit SHA  the DELETE gets not processed.
 To clarify - the DELETE is not delayed - it is not executed at all.
 Checked against a single node C* cluster.
 Does only affect unreleased 2.1 - not 2.0 nor 1.2.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Explicitly use Long.MAX_VALUE timestamp for counter deletions

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 deaf5ba15 - 5fe755762


Explicitly use Long.MAX_VALUE timestamp for counter deletions

patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-7346


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5fe75576
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5fe75576
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5fe75576

Branch: refs/heads/cassandra-2.1
Commit: 5fe7557627fac6ace2554a4f8ef552c9d9512490
Parents: deaf5ba
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Jun 11 09:47:22 2014 -0500
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Jun 11 09:47:22 2014 -0500

--
 CHANGES.txt |  2 ++
 .../cassandra/cql/AbstractModification.java |  9 --
 .../apache/cassandra/cql/DeleteStatement.java   | 27 ++
 .../apache/cassandra/cql/UpdateStatement.java   |  7 -
 .../apache/cassandra/cql3/UpdateParameters.java | 19 ++---
 .../cql3/statements/DeleteStatement.java|  2 +-
 .../apache/cassandra/db/CounterMutation.java|  3 ++
 .../cassandra/thrift/CassandraServer.java   | 29 
 8 files changed, 59 insertions(+), 39 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5fe75576/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b70782b..a8a84d8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 2.1.0
+ * Explicitly use Long.MAX_VALUE timestamp for counter deletions
+   (CASSANDRA-7346)
  * Fix native protocol CAS batches (CASSANDRA-7337)
  * Reduce likelihood of contention on local paxos locking (CASSANDRA-7359)
  * Upgrade to Pig 0.12.1 (CASSANDRA-6556)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5fe75576/src/java/org/apache/cassandra/cql/AbstractModification.java
--
diff --git a/src/java/org/apache/cassandra/cql/AbstractModification.java 
b/src/java/org/apache/cassandra/cql/AbstractModification.java
index 8da2611..9b88b5e 100644
--- a/src/java/org/apache/cassandra/cql/AbstractModification.java
+++ b/src/java/org/apache/cassandra/cql/AbstractModification.java
@@ -107,8 +107,11 @@ public abstract class AbstractModification
  *
  * @throws InvalidRequestException on the wrong request
  */
-public abstract ListIMutation prepareRowMutations(String keyspace, 
ThriftClientState clientState, ListByteBuffer variables)
-throws InvalidRequestException, UnauthorizedException;
+public ListIMutation prepareRowMutations(String keyspace, 
ThriftClientState clientState, ListByteBuffer variables)
+throws InvalidRequestException, UnauthorizedException
+{
+return prepareRowMutations(keyspace, clientState, null, variables);
+}
 
 /**
  * Convert statement into a list of mutations to apply on the server
@@ -121,6 +124,6 @@ public abstract class AbstractModification
  *
  * @throws InvalidRequestException on the wrong request
  */
-public abstract ListIMutation prepareRowMutations(String keyspace, 
ThriftClientState clientState, Long timestamp, ListByteBuffer variables)
+public abstract ListIMutation prepareRowMutations(String keyspace, 
ThriftClientState clientState, Long batchTimestamp, ListByteBuffer variables)
 throws InvalidRequestException, UnauthorizedException;
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5fe75576/src/java/org/apache/cassandra/cql/DeleteStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql/DeleteStatement.java 
b/src/java/org/apache/cassandra/cql/DeleteStatement.java
index 71942e4..e00ffc7 100644
--- a/src/java/org/apache/cassandra/cql/DeleteStatement.java
+++ b/src/java/org/apache/cassandra/cql/DeleteStatement.java
@@ -24,6 +24,7 @@ import java.util.List;
 import org.apache.cassandra.auth.Permission;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.Schema;
+import org.apache.cassandra.db.CounterMutation;
 import org.apache.cassandra.db.Mutation;
 import org.apache.cassandra.db.composites.CellName;
 import org.apache.cassandra.db.IMutation;
@@ -62,13 +63,7 @@ public class DeleteStatement extends AbstractModification
 return keys;
 }
 
-public ListIMutation prepareRowMutations(String keyspace, 
ThriftClientState clientState, ListByteBuffer variables)
-throws InvalidRequestException, UnauthorizedException
-{
-return prepareRowMutations(keyspace, clientState, null, variables);
-}
-
-public ListIMutation prepareRowMutations(String keyspace, 
ThriftClientState

[1/2] git commit: Explicitly use Long.MAX_VALUE timestamp for counter deletions

Repository: cassandra
Updated Branches:
  refs/heads/trunk 204442452 - 9aace4836


Explicitly use Long.MAX_VALUE timestamp for counter deletions

patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-7346


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5fe75576
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5fe75576
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5fe75576

Branch: refs/heads/trunk
Commit: 5fe7557627fac6ace2554a4f8ef552c9d9512490
Parents: deaf5ba
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Jun 11 09:47:22 2014 -0500
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Jun 11 09:47:22 2014 -0500

--
 CHANGES.txt |  2 ++
 .../cassandra/cql/AbstractModification.java |  9 --
 .../apache/cassandra/cql/DeleteStatement.java   | 27 ++
 .../apache/cassandra/cql/UpdateStatement.java   |  7 -
 .../apache/cassandra/cql3/UpdateParameters.java | 19 ++---
 .../cql3/statements/DeleteStatement.java|  2 +-
 .../apache/cassandra/db/CounterMutation.java|  3 ++
 .../cassandra/thrift/CassandraServer.java   | 29 
 8 files changed, 59 insertions(+), 39 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5fe75576/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b70782b..a8a84d8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 2.1.0
+ * Explicitly use Long.MAX_VALUE timestamp for counter deletions
+   (CASSANDRA-7346)
  * Fix native protocol CAS batches (CASSANDRA-7337)
  * Reduce likelihood of contention on local paxos locking (CASSANDRA-7359)
  * Upgrade to Pig 0.12.1 (CASSANDRA-6556)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5fe75576/src/java/org/apache/cassandra/cql/AbstractModification.java
--
diff --git a/src/java/org/apache/cassandra/cql/AbstractModification.java 
b/src/java/org/apache/cassandra/cql/AbstractModification.java
index 8da2611..9b88b5e 100644
--- a/src/java/org/apache/cassandra/cql/AbstractModification.java
+++ b/src/java/org/apache/cassandra/cql/AbstractModification.java
@@ -107,8 +107,11 @@ public abstract class AbstractModification
  *
  * @throws InvalidRequestException on the wrong request
  */
-public abstract ListIMutation prepareRowMutations(String keyspace, 
ThriftClientState clientState, ListByteBuffer variables)
-throws InvalidRequestException, UnauthorizedException;
+public ListIMutation prepareRowMutations(String keyspace, 
ThriftClientState clientState, ListByteBuffer variables)
+throws InvalidRequestException, UnauthorizedException
+{
+return prepareRowMutations(keyspace, clientState, null, variables);
+}
 
 /**
  * Convert statement into a list of mutations to apply on the server
@@ -121,6 +124,6 @@ public abstract class AbstractModification
  *
  * @throws InvalidRequestException on the wrong request
  */
-public abstract ListIMutation prepareRowMutations(String keyspace, 
ThriftClientState clientState, Long timestamp, ListByteBuffer variables)
+public abstract ListIMutation prepareRowMutations(String keyspace, 
ThriftClientState clientState, Long batchTimestamp, ListByteBuffer variables)
 throws InvalidRequestException, UnauthorizedException;
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5fe75576/src/java/org/apache/cassandra/cql/DeleteStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql/DeleteStatement.java 
b/src/java/org/apache/cassandra/cql/DeleteStatement.java
index 71942e4..e00ffc7 100644
--- a/src/java/org/apache/cassandra/cql/DeleteStatement.java
+++ b/src/java/org/apache/cassandra/cql/DeleteStatement.java
@@ -24,6 +24,7 @@ import java.util.List;
 import org.apache.cassandra.auth.Permission;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.Schema;
+import org.apache.cassandra.db.CounterMutation;
 import org.apache.cassandra.db.Mutation;
 import org.apache.cassandra.db.composites.CellName;
 import org.apache.cassandra.db.IMutation;
@@ -62,13 +63,7 @@ public class DeleteStatement extends AbstractModification
 return keys;
 }
 
-public ListIMutation prepareRowMutations(String keyspace, 
ThriftClientState clientState, ListByteBuffer variables)
-throws InvalidRequestException, UnauthorizedException
-{
-return prepareRowMutations(keyspace, clientState, null, variables);
-}
-
-public ListIMutation prepareRowMutations(String keyspace, 
ThriftClientState clientState,

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk

Conflicts:
src/java/org/apache/cassandra/cql/AbstractModification.java
src/java/org/apache/cassandra/cql/DeleteStatement.java
src/java/org/apache/cassandra/cql/UpdateStatement.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9aace483
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9aace483
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9aace483

Branch: refs/heads/trunk
Commit: 9aace4836ebbb1ddd93c5c9e27e0008a70f734ea
Parents: 2044424 5fe7557
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Jun 11 09:52:07 2014 -0500
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Jun 11 09:52:07 2014 -0500

--
 CHANGES.txt |  2 ++
 .../apache/cassandra/cql3/UpdateParameters.java | 19 ++---
 .../cql3/statements/DeleteStatement.java|  2 +-
 .../apache/cassandra/db/CounterMutation.java|  3 ++
 .../cassandra/thrift/CassandraServer.java   | 29 
 5 files changed, 38 insertions(+), 17 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9aace483/CHANGES.txt
--
diff --cc CHANGES.txt
index cded133,a8a84d8..aa2f96c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,14 -1,6 +1,16 @@@
 +3.0
 + * Move sstable RandomAccessReader to nio2, which allows using the
 +   FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050)
 + * Remove CQL2 (CASSANDRA-5918)
 + * Add Thrift get_multi_slice call (CASSANDRA-6757)
 + * Optimize fetching multiple cells by name (CASSANDRA-6933)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Make incremental repair default (CASSANDRA-7250)
 +
 +
  2.1.0
+  * Explicitly use Long.MAX_VALUE timestamp for counter deletions
+(CASSANDRA-7346)
   * Fix native protocol CAS batches (CASSANDRA-7337)
   * Reduce likelihood of contention on local paxos locking (CASSANDRA-7359)
   * Upgrade to Pig 0.12.1 (CASSANDRA-6556)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9aace483/src/java/org/apache/cassandra/thrift/CassandraServer.java
--

buildbot success in ASF Buildbot on cassandra-trunk

2014-06-11 Thread buildbot

The Buildbot has detected a restored build on builder cassandra-trunk while 
building cassandra.
Full details are available at:
 http://ci.apache.org/builders/cassandra-trunk/builds/314

Buildbot URL: http://ci.apache.org/

Buildslave for this Build: portunus_ubuntu

Build Reason: scheduler
Build Source Stamp: [branch trunk] 9aace4836ebbb1ddd93c5c9e27e0008a70f734ea
Blamelist: Aleksey Yeschenko alek...@apache.org

Build succeeded!

sincerely,
 -The Buildbot

[jira] [Updated] (CASSANDRA-7379) updating a row with composite key with a null value removes the entire row


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashot Golovenko updated CASSANDRA-7379:
---

Description: 
create a table

CREATE TABLE relation (
u1 bigint,
u2 bigint,
mf int,
PRIMARY KEY (u1, u2));

insert value:
UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;

SELECT * from relation ;

 u1 | u2 | mf
++
  1 |  2 |  1

insert null value:
UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;

SELECT * from relation ;

(0 rows) --- WRONG!

The INSERT statement however works:
INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 

SELECT * from relation ;

 u1 | u2 | mf
++--
  1 |  2 | null

(1 rows)
 






  was:
create a table

CREATE TABLE relation (
u1 bigint,
u2 bigint,
mf int,
PRIMARY KEY (u1, u2));

insert value:
UPDATE facebook_user_relation SET mf = 1 WHERE u1 = 1 and u2 = 2;

SELECT * from relation ;

 u1 | u2 | mf
++
  1 |  2 |  1

insert null value:
UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;

SELECT * from relation ;

(0 rows) --- WRONG!

The INSERT statement however works:
INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 

SELECT * from relation ;

 u1 | u2 | mf
++--
  1 |  2 | null

(1 rows)
 







 updating a row with composite key with a null value removes the entire row
 --

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-7379) updating a row with composite key with a null value removes the entire row

Ashot Golovenko created CASSANDRA-7379:
--

 Summary: updating a row with composite key with a null value 
removes the entire row
 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Priority: Critical


create a table

CREATE TABLE relation (
u1 bigint,
u2 bigint,
mf int,
PRIMARY KEY (u1, u2));

insert value:
UPDATE facebook_user_relation SET mf = 1 WHERE u1 = 1 and u2 = 2;

SELECT * from relation ;

 u1 | u2 | mf
++
  1 |  2 |  1

insert null value:
UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;

SELECT * from relation ;

(0 rows) --- WRONG!

The INSERT statement however works:
INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 

SELECT * from relation ;

 u1 | u2 | mf
++--
  1 |  2 | null

(1 rows)
 








--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7366) Use node's hostId instead of generating counterId-s

2014-06-11 Thread Aleksey Yeschenko (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027899#comment-14027899
]

Aleksey Yeschenko commented on CASSANDRA-7366:
--

Well, to me the point of the patch is not to get rid of some code, but to
further simplify the implementation conceptually and kill one more concept
before we hit 2.1 (counter ids in this case). (kill off remote/local shards
before 3.0, kill off contexts, logical timestamps (in favor of micros) and
CounterCell entirely in 3.0 - CASSANDRA-6506).

In fact, we could kill more code (CounterId the class, etc.), but I explicitly
went with the minimal amount of change now, b/c 2.1 is so close. Just enough to
remove the concept itself (we'll get rid of the rest in 3.0).

bq. The one downside I see might be that, unless I'm wrong, this will roughly
double the size of the counters for all existing ones

Not really. Only doubling, and only the value (the context) for existing
counters in clusters that have never renewed a counter id or made a topology
change before. One benefit is that it would allows us to go the
collectTimeOrderedData() path for looking up the value of the local shard,
which would help us with the read-before-write path, which should be a
reasonable win. Saving it for another ticket though.

Use node's hostId instead of generating counterId-s
---

Key: CASSANDRA-7366
URL: https://issues.apache.org/jira/browse/CASSANDRA-7366
Project: Cassandra
Issue Type: Improvement
Reporter: Aleksey Yeschenko
Assignee: Aleksey Yeschenko
Priority: Minor
Fix For: 2.1.0

Now that we no longer renew, or have to renew, counter ids - we can/should
simply reuse the node's hostId.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-5483) Repair tracing

2014-06-11 Thread Ben Chan (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027905#comment-14027905
]

Ben Chan commented on CASSANDRA-5483:
-

I'm sorry I can't think this through very deeply at the moment, so please allow
a little slack if I say something incorrect. I'm writing this during a small
window of time (where I can't do anything else) in a crisis I'm dealing with.

bq. Why is command part of events instead of sessions?

It was a somewhat arbitrary design decision. I can move it over to sessions.
The only real reason was historical (see the Mar 3 comment); it was a
proof-of-concept that never got followed up upon until just now.

bq. Also: should use an enum internally. Logging as string representation is
fine.

Just to be clear, you mean Java code should work with an enum, and the actual
cql table column is fine as a string?

The code actually does use an enum (of sorts; not an Enum proper), the
traceType. The traceType is passed to Tracing#getCommandName to look up the
String for command name.

bq. It makes people grumpy when we break JMX signatures. Can we add a new
overload instead, preserving the old? This should cut down on some of the code
churn in StorageService as well.

I will admit that I didn't really consider the entire ecosystem of tools that
use JMX to control Cassandra (though note that I did mention the JMX api change
in a Mar 8 comment ... the server isn't going to work with old versions of
nodetool. And a newer nodetool still won't work with older server versions.).

bq. It's a minor thing to get hung up on, but I'm not wild about all the work
needed to propagate TTLs around. Is it really super important to persist repair
traces much longer than queries? If so, what if we used a separate table and
just allowed users to modify the default ttl? (The original trace code predates
default TTLs, I think, or we would have made use of it there.)

I guess the question is how many different types of things (bootstrap, repair,
query, decommission, ... anything else?) might eventually end up being traced.
If n is small, then having n tables may be fine.

The logic was this (see Mar 1-3 discussion): Repair can take a long time, so 24
hours may be too short of a ttl.

I recall reading about problematic repairs taking several days, which wouldn't
mix well with a 24 hour ttl.

bq. Also have a nagging feeling that the notify/wait logic in StorageService +
TraceState is more complex than necessary.

If there is guaranteed to only be one consumer of notifications at a time, then
the updated v15 logic seems fine. But if there are ever two traces going on
(either of different or the same type; are you allowed to have two simultaneous
repairs of different keyspaces?) which require update notifications, then there
could be dropped notifications. The problem (I believe) is that all consumers
may not be in a wait state at the moment when notifyAll is signalled. This
means a notification could be missed, right? I'm not experienced in Java
concurrency, and this isn't the best time for me to slowly think things
through, so it's quite possible I'm wrong here.

However, if you can be completely sure there will never be concurrent repair
traces happening on the same node, or any other trace types (whatever types are
added in the future) that require update notifications in order to implement
on-the-fly reporting, then that issue is moot, and v15 should work fine, as far
as my cursory inspection goes.

bq. I should note I've made no attempt to corroborate this behaviour is
sensible; I've only simplified it.

Any feedback would be welcome. As I've said before, heuristics are messy. I
talked about the reasoning behind my design decisions, and a possibility for an
alternate implementation (with attendant tradeoffs) in a Mar 17 comment. I
honestly thought I'd get more comments on it at the time, but it's possible the
patch had already gotten into TL; DR territory even then.

---

Okay my short break from reality is over. Time to panic.

Repair tracing
--

Key: CASSANDRA-5483
URL: https://issues.apache.org/jira/browse/CASSANDRA-5483
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Yuki Morishita
Assignee: Ben Chan
Priority: Minor
Labels: repair
Attachments: 5483-full-trunk.txt,
5483-v06-04-Allow-tracing-ttl-to-be-configured.patch,
5483-v06-05-Add-a-command-column-to-system_traces.events.patch,
5483-v06-06-Fix-interruption-in-tracestate-propagation.patch,
5483-v07-07-Better-constructor-parameters-for-DebuggableThreadPoolExecutor.patch,
5483-v07-08-Fix-brace-style.patch,
5483-v07-09-Add-trace-option-to-a-more-complete-set-of-repair-functions.patch,

[jira] [Created] (CASSANDRA-7380) Native protocol needs keepalive, we should add it

2014-06-11 Thread Jose Martinez Poblete (JIRA)

Jose Martinez Poblete created CASSANDRA-7380:


 Summary: Native protocol needs keepalive, we should add it
 Key: CASSANDRA-7380
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7380
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 1.2, 2.0
Reporter: Jose Martinez Poblete


On clients connecting to C* 1.2.15 using native protocol. We see that when the 
client is bounced, the old connection is not going away

On Thrift, there is the rpc_timeout but there is no similar feature for the 
native protocol



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7380) Native protocol needs keepalive, we should add it

2014-06-11 Thread Jeremiah Jordan (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Jordan updated CASSANDRA-7380:
---

Since Version: 1.2.0 beta 1

 Native protocol needs keepalive, we should add it
 -

 Key: CASSANDRA-7380
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7380
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 1.2, 2.0
Reporter: Jose Martinez Poblete

 On clients connecting to C* 1.2.15 using native protocol. We see that when 
 the client is bounced, the old connection is not going away
 On Thrift, there is the rpc_keepalive but there is no similar feature for the 
 native protocol



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7380) Native protocol needs keepalive, we should add it

2014-06-11 Thread Jose Martinez Poblete (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jose Martinez Poblete updated CASSANDRA-7380:
-

Description: 
On clients connecting to C* 1.2.15 using native protocol. We see that when the 
client is bounced, the old connection is not going away

On Thrift, there is the rpc_keepalive but there is no similar feature for the 
native protocol

  was:
On clients connecting to C* 1.2.15 using native protocol. We see that when the 
client is bounced, the old connection is not going away

On Thrift, there is the rpc_timeout but there is no similar feature for the 
native protocol


 Native protocol needs keepalive, we should add it
 -

 Key: CASSANDRA-7380
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7380
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 1.2, 2.0
Reporter: Jose Martinez Poblete

 On clients connecting to C* 1.2.15 using native protocol. We see that when 
 the client is bounced, the old connection is not going away
 On Thrift, there is the rpc_keepalive but there is no similar feature for the 
 native protocol



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7366) Use node's hostId instead of generating counterId-s

2014-06-11 Thread Sylvain Lebresne (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027973#comment-14027973
]

Sylvain Lebresne commented on CASSANDRA-7366:
-

bq. Only doubling, and only the value (the context) for existing counters in
clusters that have never renewed a counter id or made a topology change before.

Do you mean by that that if you've already renewed counter ids then you don't
double the size of the counter context but only add some percentage of it? If
so, I agree, but I was just saying, if you've never renewed any counter id,
then each counter context will double. If you have renewed, we won't double
per-se, but we'll still grow those context the same way.

Overall, I'm not saying we shouldn't do that, I'm just listing the downside so
we're aware of what we do. And on the plus side, while we can indeed get rid of
the full CounterId code/concept, it's really just a UUID, so conceptually, I'm
not sure that we'd win that much by removing it. But don't get me wrong,
removing this would please my OCD, but since this has a small downside for
upgrader, I'm trying to convince myself that the pros outweight the cons, and
I'm not 100% convinced one way or another. But if you feel strongly, happy to
go ahead.

bq. One benefit is that it would allows us to go the collectTimeOrderedData()
path for looking up the value of the local shard

How does using the hostId makes that easier?

Use node's hostId instead of generating counterId-s
---

Now that we no longer renew, or have to renew, counter ids - we can/should
simply reuse the node's hostId.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7366) Use node's hostId instead of generating counterId-s

2014-06-11 Thread Aleksey Yeschenko (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028047#comment-14028047
 ] 

Aleksey Yeschenko commented on CASSANDRA-7366:
--

bq. How does using the hostId makes that easier?

b/c then we can safely assume that there are no local/remote shards with this 
counterId (hostId) - because we know we only started using them in 2.1, and we 
only write global shards in 2.1. So we don't need to accumulate the value over 
all the existing tables (can stop at the most recent one).

 Use node's hostId instead of generating counterId-s
 ---

 Key: CASSANDRA-7366
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7366
 Project: Cassandra
  Issue Type: Improvement
Reporter: Aleksey Yeschenko
Assignee: Aleksey Yeschenko
Priority: Minor
 Fix For: 2.1.0


 Now that we no longer renew, or have to renew, counter ids - we can/should 
 simply reuse the node's hostId.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (CASSANDRA-5483) Repair tracing

2014-06-11 Thread Ben Chan (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027905#comment-14027905
]

Ben Chan edited comment on CASSANDRA-5483 at 6/11/14 5:37 PM:
--

bq. Why is command part of events instead of sessions?

bq. Also: should use an enum internally. Logging as string representation is
fine.

Just to be clear, you mean Java code should work with an enum, and the actual
cql table column is fine as a string?

The code actually does use an enum (of sorts; not an Enum proper), the
traceType. The traceType is passed to Tracing#getCommandName to look up the
String for command name.

bq. It makes people grumpy when we break JMX signatures. Can we add a new
overload instead, preserving the old? This should cut down on some of the code
churn in StorageService as well.

The logic was this (see Mar 1-3 discussion): Repair can take a long time, so 24
hours may be too short of a ttl.

I recall reading about problematic repairs taking several days, which wouldn't
mix well with a 24 hour ttl.

bq. Also have a nagging feeling that the notify/wait logic in StorageService +
TraceState is more complex than necessary.

If there is guaranteed to only ever be one consumer of notifications at a time,
then the updated v15 logic seems fine. But if there are ever two threads
polling the same TraceState, then there could be dropped notifications. The
problem (I believe) is that all consumers may not be in a wait state at the
moment when notifyAll is signalled. This means a notification could be missed,
right? I'm not experienced in Java concurrency, and this isn't the best time
for me to slowly think things through, so it's quite possible I'm wrong here.

But it does seem reasonable there will only ever be one polling thread for any
given tracing session, so the v15 code should work fine in that respect, as far
as my cursory inspection goes.

Note, however, that the polling in this case is a heuristic only. Meaning that
it's *likely* that an external trace happened somewhere around this time plus
or minus (as far as I know, there is no way in Cassandra to be notified of cf
updates). Which means that the actual trace may only arrive *after* the
notification, meaning that for two notifications ~maxWait seconds apart, your
logging of events might be maxWait seconds late:

{noformat}
time actionresult
----
0 receive notification no events found
10 event A
1000 receive notification sendNotification(event A)
{noformat}

This is why I had an exponential backoff. Because I wanted to poll with high
frequency for a likely event, polling less and less often as the notification
recedes into the past. There are, of course, endless tweaks possible to this
basic algorithm. It depends upon what you can assume about the likely time
distribution of the events hinted at by the notification.

{noformat}
time actionresult
----
0 receive notification no events found
2 poll no events found
4 poll no events found
8 poll no events found
10 event A
16 poll sendNotification(event A)
32 poll no events found
1000 receive notification no events found
{noformat}

bq. I should note I've made no attempt to corroborate this behaviour is
sensible; I've only simplified it.

Any feedback would be welcome. As I've said before, heuristics are messy.

[jira] [Commented] (CASSANDRA-7380) Native protocol needs keepalive, we should add it


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028089#comment-14028089
 ] 

sankalp kohli commented on CASSANDRA-7380:
--

We should definitely add this. 

 Native protocol needs keepalive, we should add it
 -

 Key: CASSANDRA-7380
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7380
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 1.2, 2.0
Reporter: Jose Martinez Poblete

 On clients connecting to C* 1.2.15 using native protocol. We see that when 
 the client is bounced, the old connection is not going away
 On Thrift, there is the rpc_keepalive but there is no similar feature for the 
 native protocol



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-5483) Repair tracing

2014-06-11 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028104#comment-14028104
 ] 

Benedict commented on CASSANDRA-5483:
-

Like I said, I only replicated functionality - the exponential backoff is 
maintained, it's only defined more simply (as far as I could read you doubled 
the wait time every second timeout, which is what happens now also). As the 
code is defined now we explicitly create a repair session and spawn a specific 
thread to process it, so we have a guarantee there's only one thread. It 
doesn't matter if the consumer is in its method when notifyAll is called; if it 
isn't it will receive the notification as soon as it next enters the method.

When I have some time I'll take a look and see if there's an alternative 
approach.

 Repair tracing
 --

 Key: CASSANDRA-5483
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5483
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Yuki Morishita
Assignee: Ben Chan
Priority: Minor
  Labels: repair
 Attachments: 5483-full-trunk.txt, 
 5483-v06-04-Allow-tracing-ttl-to-be-configured.patch, 
 5483-v06-05-Add-a-command-column-to-system_traces.events.patch, 
 5483-v06-06-Fix-interruption-in-tracestate-propagation.patch, 
 5483-v07-07-Better-constructor-parameters-for-DebuggableThreadPoolExecutor.patch,
  5483-v07-08-Fix-brace-style.patch, 
 5483-v07-09-Add-trace-option-to-a-more-complete-set-of-repair-functions.patch,
  5483-v07-10-Correct-name-of-boolean-repairedAt-to-fullRepair.patch, 
 5483-v08-11-Shorten-trace-messages.-Use-Tracing-begin.patch, 
 5483-v08-12-Trace-streaming-in-Differencer-StreamingRepairTask.patch, 
 5483-v08-13-sendNotification-of-local-traces-back-to-nodetool.patch, 
 5483-v08-14-Poll-system_traces.events.patch, 
 5483-v08-15-Limit-trace-notifications.-Add-exponential-backoff.patch, 
 5483-v09-16-Fix-hang-caused-by-incorrect-exit-code.patch, 
 5483-v10-17-minor-bugfixes-and-changes.patch, 
 5483-v10-rebased-and-squashed-471f5cc.patch, 5483-v11-01-squashed.patch, 
 5483-v11-squashed-nits.patch, 5483-v12-02-cassandra-yaml-ttl-doc.patch, 
 5483-v13-608fb03-May-14-trace-formatting-changes.patch, 
 5483-v14-01-squashed.patch, 5483-v15.patch, ccm-repair-test, 
 cqlsh-left-justify-text-columns.patch, prerepair-vs-postbuggedrepair.diff, 
 test-5483-system_traces-events.txt, 
 trunk@4620823-5483-v02-0001-Trace-filtering-and-tracestate-propagation.patch, 
 trunk@4620823-5483-v02-0002-Put-a-few-traces-parallel-to-the-repair-logging.patch,
  tr...@8ebeee1-5483-v01-001-trace-filtering-and-tracestate-propagation.txt, 
 tr...@8ebeee1-5483-v01-002-simple-repair-tracing.txt, 
 v02p02-5483-v03-0003-Make-repair-tracing-controllable-via-nodetool.patch, 
 v02p02-5483-v04-0003-This-time-use-an-EnumSet-to-pass-boolean-repair-options.patch,
  v02p02-5483-v05-0003-Use-long-instead-of-EnumSet-to-work-with-JMX.patch


 I think it would be nice to log repair stats and results like query tracing 
 stores traces to system keyspace. With it, you don't have to lookup each log 
 file to see what was the status and how it performed the repair you invoked. 
 Instead, you can query the repair log with session ID to see the state and 
 stats of all nodes involved in that repair session.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7371) DELETEs get lost

2014-06-11 Thread Robert Stupp (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028127#comment-14028127
 ] 

Robert Stupp commented on CASSANDRA-7371:
-

patch works
from my side: +1

thanks for the quick fix :)

off topic: i still do not understand why this only happened on OSX - but it's 
an argument to stay on Mac for development
(flame off)

 DELETEs get lost
 

 Key: CASSANDRA-7371
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7371
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.1 git branch since merge commit 
 4722fe70aa9ae1b62772cfa1a1de58ef289445d5 (RefCount native frames from netty 
 to avoid corruption bugs)
Reporter: Robert Stupp
Assignee: T Jake Luciani
Priority: Blocker
 Fix For: 2.1.0

 Attachments: 7371.txt, Cassandra7371.java


 The mentioned commit introduced a bug which is not easy to reproduce:
 Workload description:
 - One INSERT into a table
 - multiple concurrent SELECTs against different tables (one select returns a 
 result)
 - One UPDATE against the same table as the INSERT
 - (same) multiple concurrent SELECTs against different tables (one select 
 returns a result)
 - One DELETE against the same table as the INSERT
 - (same) multiple concurrent SELECTs against different tables
 Expected is that the last bunch of SELECTs returns no result. But since 
 commit SHA  the DELETE gets not processed.
 To clarify - the DELETE is not delayed - it is not executed at all.
 Checked against a single node C* cluster.
 Does only affect unreleased 2.1 - not 2.0 nor 1.2.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Assigned] (CASSANDRA-7379) updating a row with composite key with a null value removes the entire row


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-7379:
-

Assignee: Michael Shuler

 updating a row with composite key with a null value removes the entire row
 --

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Assignee: Michael Shuler
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Assigned] (CASSANDRA-7380) Native protocol needs keepalive, we should add it


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-7380:
-

Assignee: Tyler Hobbs

 Native protocol needs keepalive, we should add it
 -

 Key: CASSANDRA-7380
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7380
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 1.2, 2.0
Reporter: Jose Martinez Poblete
Assignee: Tyler Hobbs

 On clients connecting to C* 1.2.15 using native protocol. We see that when 
 the client is bounced, the old connection is not going away
 On Thrift, there is the rpc_keepalive but there is no similar feature for the 
 native protocol



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7379) updating a row with composite key with a null value removes the entire row


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028157#comment-14028157
 ] 

Mikhail Stepura commented on CASSANDRA-7379:


duplicate of CASSANDRA-6951?

 updating a row with composite key with a null value removes the entire row
 --

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Assignee: Michael Shuler
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028187#comment-14028187
 ] 

Jeff Griffith commented on CASSANDRA-7373:
--

Adding a bit more detail to this... A warning comes on the COMMIT-LOG-WRITER 
thread saying it is skipping the append for a large row:

 INFO [FlushWriter:3571] 2014-06-11 10:19:59,696 Memtable.java (line 436) 
Completed flushing 
/home/y/var/cassandra/data/SyncCore/CommEvents/SyncCore-CommEvents-ic-84518-Data.db
 (3647831 bytes) for commitlog position ReplayPosition(segmentId=1402438439770, 
position=61789)
 INFO [FlushWriter:3566] 2014-06-11 10:19:59,727 Memtable.java (line 436) 
Completed flushing 
/home/y/var/cassandra/data/SyncCore/RecentComms/SyncCore-RecentComms-ic-83750-Data.db
 (5619016 bytes) for commitlog position ReplayPosition(segmentId=1402438439770, 
position=61789)
 INFO [GossipStage:1] 2014-06-11 10:20:07,302 Gossiper.java (line 823) 
InetAddress /10.198.26.179 is now DOWN
 INFO [OptionalTasks:1] 2014-06-11 10:20:09,754 ColumnFamilyStore.java (line 
633) Enqueuing flush of 
Memtable-SingleEndpointNetworks@1949516166(19390313/37748736 serialized/live 
bytes, 37981 ops)
 INFO [FlushWriter:3568] 2014-06-11 10:20:09,755 Memtable.java (line 398) 
Writing Memtable-SingleEndpointNetworks@1949516166(19390313/37748736 
serialized/live bytes, 37981 ops)
 INFO [OptionalTasks:1] 2014-06-11 10:20:09,755 ColumnFamilyStore.java (line 
633) Enqueuing flush of 
Memtable-EmailNetworkDeltas@1578090988(14498572/14502911 serialized/live bytes, 
4587 ops)
 INFO [FlushWriter:3567] 2014-06-11 10:20:09,756 Memtable.java (line 398) 
Writing Memtable-EmailNetworkDeltas@1578090988(14498572/14502911 
serialized/live bytes, 4587 ops)
 INFO [FlushWriter:3567] 2014-06-11 10:20:09,789 Memtable.java (line 436) 
Completed flushing 
/home/y/var/cassandra/data/SyncCore/EmailNetworkDeltas/SyncCore-EmailNetworkDeltas-ic-90590-Data.db
 (95318 bytes) for commitlog position ReplayPosition(segmentId=1402438439771, 
position=43352)
 INFO [FlushWriter:3568] 2014-06-11 10:20:09,985 Memtable.java (line 436) 
Completed flushing 
/home/y/var/cassandra/data/SyncCore/SingleEndpointNetworks/SyncCore-SingleEndpointNetworks-ic-88034-Data.db
 (9499480 bytes) for commitlog position ReplayPosition(segmentId=1402438439771, 
position=43352)
 INFO [OptionalTasks:1] 2014-06-11 10:20:32,805 MeteredFlusher.java (line 64) 
flushing high-traffic column family CFS(Keyspace='SyncCore', 
ColumnFamily='EndpointPrefixIndexMinimized') (estimated 650749167 bytes)
 INFO [OptionalTasks:1] 2014-06-11 10:20:32,806 ColumnFamilyStore.java (line 
633) Enqueuing flush of 
Memtable-EndpointPrefixIndexMinimized@1634472324(74779192/650749167 
serialized/live bytes, 3059166 ops)
 INFO [FlushWriter:3573] 2014-06-11 10:20:32,807 Memtable.java (line 398) 
Writing Memtable-EndpointPrefixIndexMinimized@1634472324(74779192/650749167 
serialized/live bytes, 3059166 ops)
 INFO [FlushWriter:3573] 2014-06-11 10:20:36,459 Memtable.java (line 436) 
Completed flushing 
/home/y/var/cassandra/data/SyncCore/EndpointPrefixIndexMinimized/SyncCore-EndpointPrefixIndexMinimized-ic-127891-Data.db
 (34290509 bytes) for commitlog position 
ReplayPosition(segmentId=1402438439773, position=7326447)
 WARN [COMMIT-LOG-WRITER] 2014-06-11 10:20:52,605 CommitLog.java (line 349) 
Skipping commitlog append of extremely large mutation (1886300377 bytes)
 INFO [OptionalTasks:1] 2014-06-11 10:20:52,902 MeteredFlusher.java (line 64) 
flushing high-traffic column family CFS(Keyspace='SyncCore', 
ColumnFamily='EmailNetworkDeltas') (estimated 1886275564 bytes)
 INFO [OptionalTasks:1] 2014-06-11 10:20:53,452 ColumnFamilyStore.java (line 
633) Enqueuing flush of 
Memtable-EmailNetworkDeltas@745275182(1886662001/1887226753 serialized/live 
bytes, 474 ops)
 INFO [FlushWriter:3577] 2014-06-11 10:20:53,452 Memtable.java (line 398) 
Writing Memtable-EmailNetworkDeltas@745275182(1886662001/1887226753 
serialized/live bytes, 474 ops)
ERROR [FlushWriter:3577] 2014-06-11 10:20:53,463 CassandraDaemon.java (line 
191) Exception in thread Thread[FlushWriter:3577,5,main]
java.lang.NegativeArraySizeException
at 
org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
at 
org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:220)
at java.io.DataOutputStream.write(DataOutputStream.java:107)
at 
org.apache.cassandra.io.util.DataOutputBuffer.write(DataOutputBuffer.java:60)
at 
org.apache.cassandra.utils.ByteBufferUtil.write(ByteBufferUtil.java:328)
at 
org.apache.cassandra.utils.ByteBufferUtil.writeWithLength(ByteBufferUtil.java:315)
at 
org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:55)
at 
org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:30)
at

[jira] [Commented] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028198#comment-14028198
 ] 

Jeff Griffith commented on CASSANDRA-7373:
--

The NegativeIndexException seems to come from here. Seems like the 1.8G size 
was getting doubled resulting an an integer overflow:

private void expand(int i) {
/* Can the buffer handle @i more bytes, if not expand it */
if (count + i = buf.length) {
return;
}

byte[] newbuf = new byte[(count + i) * 2];
System.arraycopy(buf, 0, newbuf, 0, count);
buf = newbuf;
}

 Commit logs no longer deleting and MemtablePostFlusher pending growing
 --

 Key: CASSANDRA-7373
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7373
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.5
 Cassandra 1.12.16
 Replication factor of 3
Reporter: Francois Richard

 We have this issue where once in a while, we get into a situation where the 
 MemtablePostFlusher is not executing and the space used by the commit logs on 
 disks keeps on increasing and increasing.
 We can observe the problem by invoking nodetool tpstats:
 {code}
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 6 6   46650213 0
  0
 RequestResponseStage  0 0  130547421 0
  0
 MutationStage 2 2  116813206 0
  0
 ReadRepairStage   0 02322201 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0 120780 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0  0 0
  0
 MemoryMeter   0 0456 0
  0
 MemtablePostFlusher   1   447   6344 0
  0
 FlushWriter   0 0   6132 0
 62
 MiscStage 0 0  0 0
  0
 PendingRangeCalculator0 0  6 0
  0
 commitlog_archiver0 0  0 0
  0
 InternalResponseStage 0 0  0 0
  0
 HintedHandoff 2 2  4 0
  0
 Message type   Dropped
 RANGE_SLICE  0
 READ_REPAIR  0
 BINARY   0
 READ 0
 MUTATION 0
 _TRACE   0
 REQUEST_RESPONSE 0
 COUNTER_MUTATION 0
 {code}
 Here is a potential error in the logs that can explain this:
 {code}
 ERROR [FlushWriter:2693] 2014-06-09 22:05:38,452 CassandraDaemon.java (line 
 191) Exception in thread Thread[FlushWriter:2693,5,main]
 java.lang.NegativeArraySizeException
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:220)
   at java.io.DataOutputStream.write(DataOutputStream.java:107)
   at 
 org.apache.cassandra.io.util.DataOutputBuffer.write(DataOutputBuffer.java:60)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.write(ByteBufferUtil.java:328)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithLength(ByteBufferUtil.java:315)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:55)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:30)
   at 
 org.apache.cassandra.db.OnDiskAtom$Serializer.serializeForSSTable(OnDiskAtom.java:62)
   at org.apache.cassandra.db.ColumnIndex$Builder.add(ColumnIndex.java:181)
   at 
 org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:133)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:185)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:430)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:385)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028198#comment-14028198
 ] 

Jeff Griffith edited comment on CASSANDRA-7373 at 6/11/14 6:46 PM:
---

The NegativeArraySizeException seems to come from here. Seems like the 1.8G 
size was getting doubled resulting an an integer overflow:

private void expand(int i) {
/* Can the buffer handle @i more bytes, if not expand it */
if (count + i = buf.length) {
return;
}

byte[] newbuf = new byte[(count + i) * 2];
System.arraycopy(buf, 0, newbuf, 0, count);
buf = newbuf;
}


was (Author: jeffery.griffith):
The NegativeArraySize seems to come from here. Seems like the 1.8G size was 
getting doubled resulting an an integer overflow:

private void expand(int i) {
/* Can the buffer handle @i more bytes, if not expand it */
if (count + i = buf.length) {
return;
}

byte[] newbuf = new byte[(count + i) * 2];
System.arraycopy(buf, 0, newbuf, 0, count);
buf = newbuf;
}

 Commit logs no longer deleting and MemtablePostFlusher pending growing
 --

 Key: CASSANDRA-7373
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7373
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.5
 Cassandra 1.12.16
 Replication factor of 3
Reporter: Francois Richard

 We have this issue where once in a while, we get into a situation where the 
 MemtablePostFlusher is not executing and the space used by the commit logs on 
 disks keeps on increasing and increasing.
 We can observe the problem by invoking nodetool tpstats:
 {code}
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 6 6   46650213 0
  0
 RequestResponseStage  0 0  130547421 0
  0
 MutationStage 2 2  116813206 0
  0
 ReadRepairStage   0 02322201 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0 120780 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0  0 0
  0
 MemoryMeter   0 0456 0
  0
 MemtablePostFlusher   1   447   6344 0
  0
 FlushWriter   0 0   6132 0
 62
 MiscStage 0 0  0 0
  0
 PendingRangeCalculator0 0  6 0
  0
 commitlog_archiver0 0  0 0
  0
 InternalResponseStage 0 0  0 0
  0
 HintedHandoff 2 2  4 0
  0
 Message type   Dropped
 RANGE_SLICE  0
 READ_REPAIR  0
 BINARY   0
 READ 0
 MUTATION 0
 _TRACE   0
 REQUEST_RESPONSE 0
 COUNTER_MUTATION 0
 {code}
 Here is a potential error in the logs that can explain this:
 {code}
 ERROR [FlushWriter:2693] 2014-06-09 22:05:38,452 CassandraDaemon.java (line 
 191) Exception in thread Thread[FlushWriter:2693,5,main]
 java.lang.NegativeArraySizeException
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:220)
   at java.io.DataOutputStream.write(DataOutputStream.java:107)
   at 
 org.apache.cassandra.io.util.DataOutputBuffer.write(DataOutputBuffer.java:60)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.write(ByteBufferUtil.java:328)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithLength(ByteBufferUtil.java:315)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:55)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:30)
   at 
 org.apache.cassandra.db.OnDiskAtom$Serializer.serializeForSSTable(OnDiskAtom.java:62)
   at org.apache.cassandra.db.ColumnIndex$Builder.add(ColumnIndex.java:181)
   at

[jira] [Comment Edited] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028198#comment-14028198
 ] 

Jeff Griffith edited comment on CASSANDRA-7373 at 6/11/14 6:46 PM:
---

The NegativeArraySize seems to come from here. Seems like the 1.8G size was 
getting doubled resulting an an integer overflow:

private void expand(int i) {
/* Can the buffer handle @i more bytes, if not expand it */
if (count + i = buf.length) {
return;
}

byte[] newbuf = new byte[(count + i) * 2];
System.arraycopy(buf, 0, newbuf, 0, count);
buf = newbuf;
}


was (Author: jeffery.griffith):
The NegativeIndexException seems to come from here. Seems like the 1.8G size 
was getting doubled resulting an an integer overflow:

private void expand(int i) {
/* Can the buffer handle @i more bytes, if not expand it */
if (count + i = buf.length) {
return;
}

byte[] newbuf = new byte[(count + i) * 2];
System.arraycopy(buf, 0, newbuf, 0, count);
buf = newbuf;
}

 Commit logs no longer deleting and MemtablePostFlusher pending growing
 --

 Key: CASSANDRA-7373
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7373
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.5
 Cassandra 1.12.16
 Replication factor of 3
Reporter: Francois Richard

 We have this issue where once in a while, we get into a situation where the 
 MemtablePostFlusher is not executing and the space used by the commit logs on 
 disks keeps on increasing and increasing.
 We can observe the problem by invoking nodetool tpstats:
 {code}
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 6 6   46650213 0
  0
 RequestResponseStage  0 0  130547421 0
  0
 MutationStage 2 2  116813206 0
  0
 ReadRepairStage   0 02322201 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0 120780 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0  0 0
  0
 MemoryMeter   0 0456 0
  0
 MemtablePostFlusher   1   447   6344 0
  0
 FlushWriter   0 0   6132 0
 62
 MiscStage 0 0  0 0
  0
 PendingRangeCalculator0 0  6 0
  0
 commitlog_archiver0 0  0 0
  0
 InternalResponseStage 0 0  0 0
  0
 HintedHandoff 2 2  4 0
  0
 Message type   Dropped
 RANGE_SLICE  0
 READ_REPAIR  0
 BINARY   0
 READ 0
 MUTATION 0
 _TRACE   0
 REQUEST_RESPONSE 0
 COUNTER_MUTATION 0
 {code}
 Here is a potential error in the logs that can explain this:
 {code}
 ERROR [FlushWriter:2693] 2014-06-09 22:05:38,452 CassandraDaemon.java (line 
 191) Exception in thread Thread[FlushWriter:2693,5,main]
 java.lang.NegativeArraySizeException
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:220)
   at java.io.DataOutputStream.write(DataOutputStream.java:107)
   at 
 org.apache.cassandra.io.util.DataOutputBuffer.write(DataOutputBuffer.java:60)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.write(ByteBufferUtil.java:328)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithLength(ByteBufferUtil.java:315)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:55)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:30)
   at 
 org.apache.cassandra.db.OnDiskAtom$Serializer.serializeForSSTable(OnDiskAtom.java:62)
   at org.apache.cassandra.db.ColumnIndex$Builder.add(ColumnIndex.java:181)
   at

[jira] [Comment Edited] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028187#comment-14028187
 ] 

Jeff Griffith edited comment on CASSANDRA-7373 at 6/11/14 6:48 PM:
---

Adding a bit more detail to this... A warning comes on the COMMIT-LOG-WRITER 
thread saying it is skipping the append for a large row. There is also a lot of 
GCing and Gossip thinks two nodes are down.

 INFO [FlushWriter:3571] 2014-06-11 10:19:59,696 Memtable.java (line 436) 
Completed flushing 
/home/y/var/cassandra/data/SyncCore/CommEvents/SyncCore-CommEvents-ic-84518-Data.db
 (3647831 bytes) for commitlog position ReplayPosition(segmentId=1402438439770, 
position=61789)
 INFO [FlushWriter:3566] 2014-06-11 10:19:59,727 Memtable.java (line 436) 
Completed flushing 
/home/y/var/cassandra/data/SyncCore/RecentComms/SyncCore-RecentComms-ic-83750-Data.db
 (5619016 bytes) for commitlog position ReplayPosition(segmentId=1402438439770, 
position=61789)
 INFO [GossipStage:1] 2014-06-11 10:20:07,302 Gossiper.java (line 823) 
InetAddress /10.198.26.179 is now DOWN
 INFO [OptionalTasks:1] 2014-06-11 10:20:09,754 ColumnFamilyStore.java (line 
633) Enqueuing flush of 
Memtable-SingleEndpointNetworks@1949516166(19390313/37748736 serialized/live 
bytes, 37981 ops)
 INFO [FlushWriter:3568] 2014-06-11 10:20:09,755 Memtable.java (line 398) 
Writing Memtable-SingleEndpointNetworks@1949516166(19390313/37748736 
serialized/live bytes, 37981 ops)
 INFO [OptionalTasks:1] 2014-06-11 10:20:09,755 ColumnFamilyStore.java (line 
633) Enqueuing flush of 
Memtable-EmailNetworkDeltas@1578090988(14498572/14502911 serialized/live bytes, 
4587 ops)
 INFO [FlushWriter:3567] 2014-06-11 10:20:09,756 Memtable.java (line 398) 
Writing Memtable-EmailNetworkDeltas@1578090988(14498572/14502911 
serialized/live bytes, 4587 ops)
 INFO [FlushWriter:3567] 2014-06-11 10:20:09,789 Memtable.java (line 436) 
Completed flushing 
/home/y/var/cassandra/data/SyncCore/EmailNetworkDeltas/SyncCore-EmailNetworkDeltas-ic-90590-Data.db
 (95318 bytes) for commitlog position ReplayPosition(segmentId=1402438439771, 
position=43352)
 INFO [FlushWriter:3568] 2014-06-11 10:20:09,985 Memtable.java (line 436) 
Completed flushing 
/home/y/var/cassandra/data/SyncCore/SingleEndpointNetworks/SyncCore-SingleEndpointNetworks-ic-88034-Data.db
 (9499480 bytes) for commitlog position ReplayPosition(segmentId=1402438439771, 
position=43352)
 INFO [OptionalTasks:1] 2014-06-11 10:20:32,805 MeteredFlusher.java (line 64) 
flushing high-traffic column family CFS(Keyspace='SyncCore', 
ColumnFamily='EndpointPrefixIndexMinimized') (estimated 650749167 bytes)
 INFO [OptionalTasks:1] 2014-06-11 10:20:32,806 ColumnFamilyStore.java (line 
633) Enqueuing flush of 
Memtable-EndpointPrefixIndexMinimized@1634472324(74779192/650749167 
serialized/live bytes, 3059166 ops)
 INFO [FlushWriter:3573] 2014-06-11 10:20:32,807 Memtable.java (line 398) 
Writing Memtable-EndpointPrefixIndexMinimized@1634472324(74779192/650749167 
serialized/live bytes, 3059166 ops)
 INFO [FlushWriter:3573] 2014-06-11 10:20:36,459 Memtable.java (line 436) 
Completed flushing 
/home/y/var/cassandra/data/SyncCore/EndpointPrefixIndexMinimized/SyncCore-EndpointPrefixIndexMinimized-ic-127891-Data.db
 (34290509 bytes) for commitlog position 
ReplayPosition(segmentId=1402438439773, position=7326447)
 WARN [COMMIT-LOG-WRITER] 2014-06-11 10:20:52,605 CommitLog.java (line 349) 
Skipping commitlog append of extremely large mutation (1886300377 bytes)
 INFO [OptionalTasks:1] 2014-06-11 10:20:52,902 MeteredFlusher.java (line 64) 
flushing high-traffic column family CFS(Keyspace='SyncCore', 
ColumnFamily='EmailNetworkDeltas') (estimated 1886275564 bytes)
 INFO [OptionalTasks:1] 2014-06-11 10:20:53,452 ColumnFamilyStore.java (line 
633) Enqueuing flush of 
Memtable-EmailNetworkDeltas@745275182(1886662001/1887226753 serialized/live 
bytes, 474 ops)
 INFO [FlushWriter:3577] 2014-06-11 10:20:53,452 Memtable.java (line 398) 
Writing Memtable-EmailNetworkDeltas@745275182(1886662001/1887226753 
serialized/live bytes, 474 ops)
ERROR [FlushWriter:3577] 2014-06-11 10:20:53,463 CassandraDaemon.java (line 
191) Exception in thread Thread[FlushWriter:3577,5,main]
java.lang.NegativeArraySizeException
at 
org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
at 
org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:220)
at java.io.DataOutputStream.write(DataOutputStream.java:107)
at 
org.apache.cassandra.io.util.DataOutputBuffer.write(DataOutputBuffer.java:60)
at 
org.apache.cassandra.utils.ByteBufferUtil.write(ByteBufferUtil.java:328)
at 
org.apache.cassandra.utils.ByteBufferUtil.writeWithLength(ByteBufferUtil.java:315)
at 
org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:55)

[jira] [Commented] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028207#comment-14028207
 ] 

Jeff Griffith commented on CASSANDRA-7373:
--

That NegativeArraySizeException seems to bubble up to the 
UncaughtExceptionHandler so we are wondering if the commit log handling is 
simply stopping because of this.

 Commit logs no longer deleting and MemtablePostFlusher pending growing
 --

 Key: CASSANDRA-7373
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7373
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.5
 Cassandra 1.12.16
 Replication factor of 3
Reporter: Francois Richard

 We have this issue where once in a while, we get into a situation where the 
 MemtablePostFlusher is not executing and the space used by the commit logs on 
 disks keeps on increasing and increasing.
 We can observe the problem by invoking nodetool tpstats:
 {code}
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 6 6   46650213 0
  0
 RequestResponseStage  0 0  130547421 0
  0
 MutationStage 2 2  116813206 0
  0
 ReadRepairStage   0 02322201 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0 120780 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0  0 0
  0
 MemoryMeter   0 0456 0
  0
 MemtablePostFlusher   1   447   6344 0
  0
 FlushWriter   0 0   6132 0
 62
 MiscStage 0 0  0 0
  0
 PendingRangeCalculator0 0  6 0
  0
 commitlog_archiver0 0  0 0
  0
 InternalResponseStage 0 0  0 0
  0
 HintedHandoff 2 2  4 0
  0
 Message type   Dropped
 RANGE_SLICE  0
 READ_REPAIR  0
 BINARY   0
 READ 0
 MUTATION 0
 _TRACE   0
 REQUEST_RESPONSE 0
 COUNTER_MUTATION 0
 {code}
 Here is a potential error in the logs that can explain this:
 {code}
 ERROR [FlushWriter:2693] 2014-06-09 22:05:38,452 CassandraDaemon.java (line 
 191) Exception in thread Thread[FlushWriter:2693,5,main]
 java.lang.NegativeArraySizeException
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:220)
   at java.io.DataOutputStream.write(DataOutputStream.java:107)
   at 
 org.apache.cassandra.io.util.DataOutputBuffer.write(DataOutputBuffer.java:60)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.write(ByteBufferUtil.java:328)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithLength(ByteBufferUtil.java:315)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:55)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:30)
   at 
 org.apache.cassandra.db.OnDiskAtom$Serializer.serializeForSSTable(OnDiskAtom.java:62)
   at org.apache.cassandra.db.ColumnIndex$Builder.add(ColumnIndex.java:181)
   at 
 org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:133)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:185)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:430)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:385)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7380) Native protocol needs keepalive, we should add it


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-7380:


Fix Version/s: 2.0.9
   1.2.17

 Native protocol needs keepalive, we should add it
 -

 Key: CASSANDRA-7380
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7380
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 1.2, 2.0
Reporter: Jose Martinez Poblete
Assignee: Tyler Hobbs
 Fix For: 1.2.17, 2.0.9


 On clients connecting to C* 1.2.15 using native protocol. We see that when 
 the client is bounced, the old connection is not going away
 On Thrift, there is the rpc_keepalive but there is no similar feature for the 
 native protocol



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7380) Native protocol needs keepalive, we should add it


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028257#comment-14028257
 ] 

Brandon Williams commented on CASSANDRA-7380:
-

Reusing the rpc_keepalive option should be fine, since I can't imagine we need 
thrift/native granularity here.

 Native protocol needs keepalive, we should add it
 -

 Key: CASSANDRA-7380
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7380
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 1.2, 2.0
Reporter: Jose Martinez Poblete
Assignee: Tyler Hobbs
 Fix For: 1.2.17, 2.0.9


 On clients connecting to C* 1.2.15 using native protocol. We see that when 
 the client is bounced, the old connection is not going away
 On Thrift, there is the rpc_keepalive but there is no similar feature for the 
 native protocol



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-5263) Increase merkle tree depth as needed


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5263:
--

  Component/s: (was: Config)
Fix Version/s: 2.1.1
 Assignee: Yuki Morishita  (was: Minh Do)
  Summary: Increase merkle tree depth as needed  (was: Allow Merkle 
tree maximum depth to be configurable)

 Increase merkle tree depth as needed
 

 Key: CASSANDRA-5263
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5263
 Project: Cassandra
  Issue Type: Improvement
Affects Versions: 1.1.9
Reporter: Ahmed Bashir
Assignee: Yuki Morishita
 Fix For: 2.1.1


 Currently, the maximum depth allowed for Merkle trees is hardcoded as 15.  
 This value should be configurable, just like phi_convict_treshold and other 
 properties.
 Given a cluster with nodes responsible for a large number of row keys, Merkle 
 tree comparisons can result in a large amount of unnecessary row keys being 
 streamed.
 Empirical testing indicates that reasonable changes to this depth (18, 20, 
 etc) don't affect the Merkle tree generation and differencing timings all 
 that much, and they can significantly reduce the amount of data being 
 streamed during repair. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (CASSANDRA-7379) updating a row with composite key with a null value removes the entire row


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028271#comment-14028271
 ] 

Jonathan Ellis edited comment on CASSANDRA-7379 at 6/11/14 7:27 PM:


6951 is marked duplicate of CASSANDRA-6782 which was fixed in 2.0.6, so...  
superficially I would say no.


was (Author: jbellis):
6951 is marked duplicate of CASSANDRA-6782 which was fixed in 2.0.6, so...  
superficially I would say no.a

 updating a row with composite key with a null value removes the entire row
 --

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Assignee: Michael Shuler
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7379) updating a row with composite key with a null value removes the entire row


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028271#comment-14028271
 ] 

Jonathan Ellis commented on CASSANDRA-7379:
---

6951 is marked duplicate of CASSANDRA-6782 which was fixed in 2.0.6, so...  
superficially I would say no.a

 updating a row with composite key with a null value removes the entire row
 --

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Assignee: Michael Shuler
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7379) updating a row with composite key with a null value removes the entire row


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028278#comment-14028278
 ] 

Brandon Williams commented on CASSANDRA-7379:
-

Failed for me on 2.0 head

 updating a row with composite key with a null value removes the entire row
 --

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Assignee: Michael Shuler
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7379) updating a row with composite key with a null value removes the entire row


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028329#comment-14028329
 ] 

Mikhail Stepura commented on CASSANDRA-7379:


I mean as per CASSANDRA-6951 it's kinda expected behavior

 updating a row with composite key with a null value removes the entire row
 --

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Assignee: Michael Shuler
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7379) updating a row with composite key with a null value removes the entire row

2014-06-11 Thread Jack Krupansky (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028349#comment-14028349
 ] 

Jack Krupansky commented on CASSANDRA-7379:
---

(Nit: Description says composite key, but the table definition has a 
compound key.)

 updating a row with composite key with a null value removes the entire row
 --

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Assignee: Michael Shuler
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028356#comment-14028356
 ] 

Mikhail Stepura commented on CASSANDRA-7373:


{{FlushRunnable.runWith}} doesn't count down its latch if an exception happens, 
so the task in MemtablePostFlusher will wait forever

 Commit logs no longer deleting and MemtablePostFlusher pending growing
 --

 Key: CASSANDRA-7373
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7373
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.5
 Cassandra 1.12.16
 Replication factor of 3
Reporter: Francois Richard

 We have this issue where once in a while, we get into a situation where the 
 MemtablePostFlusher is not executing and the space used by the commit logs on 
 disks keeps on increasing and increasing.
 We can observe the problem by invoking nodetool tpstats:
 {code}
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 6 6   46650213 0
  0
 RequestResponseStage  0 0  130547421 0
  0
 MutationStage 2 2  116813206 0
  0
 ReadRepairStage   0 02322201 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0 120780 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0  0 0
  0
 MemoryMeter   0 0456 0
  0
 MemtablePostFlusher   1   447   6344 0
  0
 FlushWriter   0 0   6132 0
 62
 MiscStage 0 0  0 0
  0
 PendingRangeCalculator0 0  6 0
  0
 commitlog_archiver0 0  0 0
  0
 InternalResponseStage 0 0  0 0
  0
 HintedHandoff 2 2  4 0
  0
 Message type   Dropped
 RANGE_SLICE  0
 READ_REPAIR  0
 BINARY   0
 READ 0
 MUTATION 0
 _TRACE   0
 REQUEST_RESPONSE 0
 COUNTER_MUTATION 0
 {code}
 Here is a potential error in the logs that can explain this:
 {code}
 ERROR [FlushWriter:2693] 2014-06-09 22:05:38,452 CassandraDaemon.java (line 
 191) Exception in thread Thread[FlushWriter:2693,5,main]
 java.lang.NegativeArraySizeException
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:220)
   at java.io.DataOutputStream.write(DataOutputStream.java:107)
   at 
 org.apache.cassandra.io.util.DataOutputBuffer.write(DataOutputBuffer.java:60)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.write(ByteBufferUtil.java:328)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithLength(ByteBufferUtil.java:315)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:55)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:30)
   at 
 org.apache.cassandra.db.OnDiskAtom$Serializer.serializeForSSTable(OnDiskAtom.java:62)
   at org.apache.cassandra.db.ColumnIndex$Builder.add(ColumnIndex.java:181)
   at 
 org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:133)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:185)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:430)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:385)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-7381) Snappy Compression does not work with PowerPC 64-bit, Little Endian

2014-06-11 Thread Iheanyi Ekechukwu (JIRA)

Iheanyi Ekechukwu created CASSANDRA-7381:


 Summary: Snappy Compression does not work with PowerPC 64-bit, 
Little Endian
 Key: CASSANDRA-7381
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7381
 Project: Cassandra
  Issue Type: Bug
Reporter: Iheanyi Ekechukwu


In PowerPC 64-bit, Little Endian, CompressedRandomAccessReaderTest, 
CompressedInputStreamTest, and CFMetaDataTest fail due to the included 
snappy-java JAR missing the ppc64le native library.

Testing on Ubuntu 14.04, ppc64le. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7381) Snappy Compression does not work with PowerPC 64-bit, Little Endian

2014-06-11 Thread Iheanyi Ekechukwu (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Iheanyi Ekechukwu updated CASSANDRA-7381:
-

Attachment: trunk-7381.txt

 Snappy Compression does not work with PowerPC 64-bit, Little Endian
 ---

 Key: CASSANDRA-7381
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7381
 Project: Cassandra
  Issue Type: Bug
Reporter: Iheanyi Ekechukwu
 Attachments: trunk-7381.txt


 In PowerPC 64-bit, Little Endian, CompressedRandomAccessReaderTest, 
 CompressedInputStreamTest, and CFMetaDataTest fail due to the included 
 snappy-java JAR missing the ppc64le native library.
 Testing on Ubuntu 14.04, ppc64le. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7381) Snappy Compression does not work with PowerPC 64-bit, Little Endian

2014-06-11 Thread Iheanyi Ekechukwu (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Iheanyi Ekechukwu updated CASSANDRA-7381:
-

Description: 
In PowerPC 64-bit, Little Endian, CompressedRandomAccessReaderTest, 
CompressedInputStreamTest, and CFMetaDataTest fail due to the included 
snappy-java JAR missing the ppc64le native library.

Testing on Ubuntu 14.04, ppc64le.

The specific fix for Snappy-Java and adding the native library can be found at 
https://github.com/xerial/snappy-java/pull/67.

  was:
In PowerPC 64-bit, Little Endian, CompressedRandomAccessReaderTest, 
CompressedInputStreamTest, and CFMetaDataTest fail due to the included 
snappy-java JAR missing the ppc64le native library.

Testing on Ubuntu 14.04, ppc64le. 


 Snappy Compression does not work with PowerPC 64-bit, Little Endian
 ---

 Key: CASSANDRA-7381
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7381
 Project: Cassandra
  Issue Type: Bug
Reporter: Iheanyi Ekechukwu
 Attachments: trunk-7381.txt


 In PowerPC 64-bit, Little Endian, CompressedRandomAccessReaderTest, 
 CompressedInputStreamTest, and CFMetaDataTest fail due to the included 
 snappy-java JAR missing the ppc64le native library.
 Testing on Ubuntu 14.04, ppc64le.
 The specific fix for Snappy-Java and adding the native library can be found 
 at https://github.com/xerial/snappy-java/pull/67.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028391#comment-14028391
 ] 

Jeff Griffith edited comment on CASSANDRA-7373 at 6/11/14 9:00 PM:
---

Thanks Mikhail. If my explanation for the exception above makes sense, it would 
seem the only mystery is how the high-traffic cf got to be over half of 
MAX-INT. In the logs below, is this amount about some crazy-large single 
mutation or about all values in memory for this column family?

 WARN [COMMIT-LOG-WRITER] 2014-06-11 20:25:29,505 CommitLog.java (line 349) 
Skipping commitlog append of extremely large mutation (1833599403 bytes)
 INFO [OptionalTasks:1] 2014-06-11 20:25:29,832 MeteredFlusher.java (line 64) 
flushing high-traffic column family CFS(Keyspace='SyncCore', 
ColumnFamily='EmailNetworkDeltas') (estimated 1836720026 bytes)
 INFO [OptionalTasks:1] 2014-06-11 20:25:30,325 ColumnFamilyStore.java (line 
633) Enqueuing flush of 
Memtable-EmailNetworkDeltas@79278257(1837683252/1864225045 serialized/live 
bytes, 1098 ops)
 INFO [FlushWriter:1520] 2014-06-11 20:25:30,326 Memtable.java (line 398) 
Writing Memtable-EmailNetworkDeltas@79278257(1837683252/1864225045 
serialized/live bytes, 1098 ops)
ERROR [FlushWriter:1520] 2014-06-11 20:25:30,344 CassandraDaemon.java (line 
191) Exception in thread Thread[FlushWriter:1520,5,main]
java.lang.NegativeArraySizeException
at 
org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)



was (Author: jeffery.griffith):
Thanks Mikhail. If my explanation for the exception above makes sense, it would 
seem the only mystery is how the high-traffic table got to be over half of 
MAX-INT. In the logs below, is this amount about some crazy-large single 
mutation or about all values in memory for this column family?

 WARN [COMMIT-LOG-WRITER] 2014-06-11 20:25:29,505 CommitLog.java (line 349) 
Skipping commitlog append of extremely large mutation (1833599403 bytes)
 INFO [OptionalTasks:1] 2014-06-11 20:25:29,832 MeteredFlusher.java (line 64) 
flushing high-traffic column family CFS(Keyspace='SyncCore', 
ColumnFamily='EmailNetworkDeltas') (estimated 1836720026 bytes)
 INFO [OptionalTasks:1] 2014-06-11 20:25:30,325 ColumnFamilyStore.java (line 
633) Enqueuing flush of 
Memtable-EmailNetworkDeltas@79278257(1837683252/1864225045 serialized/live 
bytes, 1098 ops)
 INFO [FlushWriter:1520] 2014-06-11 20:25:30,326 Memtable.java (line 398) 
Writing Memtable-EmailNetworkDeltas@79278257(1837683252/1864225045 
serialized/live bytes, 1098 ops)
ERROR [FlushWriter:1520] 2014-06-11 20:25:30,344 CassandraDaemon.java (line 
191) Exception in thread Thread[FlushWriter:1520,5,main]
java.lang.NegativeArraySizeException
at 
org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)


 Commit logs no longer deleting and MemtablePostFlusher pending growing
 --

 Key: CASSANDRA-7373
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7373
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.5
 Cassandra 1.12.16
 Replication factor of 3
Reporter: Francois Richard

 We have this issue where once in a while, we get into a situation where the 
 MemtablePostFlusher is not executing and the space used by the commit logs on 
 disks keeps on increasing and increasing.
 We can observe the problem by invoking nodetool tpstats:
 {code}
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 6 6   46650213 0
  0
 RequestResponseStage  0 0  130547421 0
  0
 MutationStage 2 2  116813206 0
  0
 ReadRepairStage   0 02322201 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0 120780 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0  0 0
  0
 MemoryMeter   0 0456 0
  0
 MemtablePostFlusher   1   447   6344 0
  0
 FlushWriter   0 0   6132 0
 62
 MiscStage 0 0  0 0
  0
 PendingRangeCalculator0 0  6 0
  0
 commitlog_archiver

[jira] [Commented] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028391#comment-14028391
 ] 

Jeff Griffith commented on CASSANDRA-7373:
--

Thanks Mikhail. If my explanation for the exception above makes sense, it would 
seem the only mystery is how the high-traffic table got to be over half of 
MAX-INT. In the logs below, is this amount about some crazy-large single 
mutation or about all values in memory for this column family?

 WARN [COMMIT-LOG-WRITER] 2014-06-11 20:25:29,505 CommitLog.java (line 349) 
Skipping commitlog append of extremely large mutation (1833599403 bytes)
 INFO [OptionalTasks:1] 2014-06-11 20:25:29,832 MeteredFlusher.java (line 64) 
flushing high-traffic column family CFS(Keyspace='SyncCore', 
ColumnFamily='EmailNetworkDeltas') (estimated 1836720026 bytes)
 INFO [OptionalTasks:1] 2014-06-11 20:25:30,325 ColumnFamilyStore.java (line 
633) Enqueuing flush of 
Memtable-EmailNetworkDeltas@79278257(1837683252/1864225045 serialized/live 
bytes, 1098 ops)
 INFO [FlushWriter:1520] 2014-06-11 20:25:30,326 Memtable.java (line 398) 
Writing Memtable-EmailNetworkDeltas@79278257(1837683252/1864225045 
serialized/live bytes, 1098 ops)
ERROR [FlushWriter:1520] 2014-06-11 20:25:30,344 CassandraDaemon.java (line 
191) Exception in thread Thread[FlushWriter:1520,5,main]
java.lang.NegativeArraySizeException
at 
org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)


 Commit logs no longer deleting and MemtablePostFlusher pending growing
 --

 Key: CASSANDRA-7373
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7373
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.5
 Cassandra 1.12.16
 Replication factor of 3
Reporter: Francois Richard

 We have this issue where once in a while, we get into a situation where the 
 MemtablePostFlusher is not executing and the space used by the commit logs on 
 disks keeps on increasing and increasing.
 We can observe the problem by invoking nodetool tpstats:
 {code}
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 6 6   46650213 0
  0
 RequestResponseStage  0 0  130547421 0
  0
 MutationStage 2 2  116813206 0
  0
 ReadRepairStage   0 02322201 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0 120780 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0  0 0
  0
 MemoryMeter   0 0456 0
  0
 MemtablePostFlusher   1   447   6344 0
  0
 FlushWriter   0 0   6132 0
 62
 MiscStage 0 0  0 0
  0
 PendingRangeCalculator0 0  6 0
  0
 commitlog_archiver0 0  0 0
  0
 InternalResponseStage 0 0  0 0
  0
 HintedHandoff 2 2  4 0
  0
 Message type   Dropped
 RANGE_SLICE  0
 READ_REPAIR  0
 BINARY   0
 READ 0
 MUTATION 0
 _TRACE   0
 REQUEST_RESPONSE 0
 COUNTER_MUTATION 0
 {code}
 Here is a potential error in the logs that can explain this:
 {code}
 ERROR [FlushWriter:2693] 2014-06-09 22:05:38,452 CassandraDaemon.java (line 
 191) Exception in thread Thread[FlushWriter:2693,5,main]
 java.lang.NegativeArraySizeException
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:220)
   at java.io.DataOutputStream.write(DataOutputStream.java:107)
   at 
 org.apache.cassandra.io.util.DataOutputBuffer.write(DataOutputBuffer.java:60)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.write(ByteBufferUtil.java:328)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithLength(ByteBufferUtil.java:315)
   at

[jira] [Comment Edited] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028391#comment-14028391
 ] 

Jeff Griffith edited comment on CASSANDRA-7373 at 6/11/14 9:01 PM:
---

Thanks Mikhail. If my explanation for the exception above makes sense, it would 
seem the only mystery is how the high-traffic cf got to be over half of 
MAX-INT. In the logs below, is this amount about some crazy-large or corrupted 
single mutation or about all values in memory for this column family?

 WARN [COMMIT-LOG-WRITER] 2014-06-11 20:25:29,505 CommitLog.java (line 349) 
Skipping commitlog append of extremely large mutation (1833599403 bytes)
 INFO [OptionalTasks:1] 2014-06-11 20:25:29,832 MeteredFlusher.java (line 64) 
flushing high-traffic column family CFS(Keyspace='SyncCore', 
ColumnFamily='EmailNetworkDeltas') (estimated 1836720026 bytes)
 INFO [OptionalTasks:1] 2014-06-11 20:25:30,325 ColumnFamilyStore.java (line 
633) Enqueuing flush of 
Memtable-EmailNetworkDeltas@79278257(1837683252/1864225045 serialized/live 
bytes, 1098 ops)
 INFO [FlushWriter:1520] 2014-06-11 20:25:30,326 Memtable.java (line 398) 
Writing Memtable-EmailNetworkDeltas@79278257(1837683252/1864225045 
serialized/live bytes, 1098 ops)
ERROR [FlushWriter:1520] 2014-06-11 20:25:30,344 CassandraDaemon.java (line 
191) Exception in thread Thread[FlushWriter:1520,5,main]
java.lang.NegativeArraySizeException
at 
org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)



was (Author: jeffery.griffith):
Thanks Mikhail. If my explanation for the exception above makes sense, it would 
seem the only mystery is how the high-traffic cf got to be over half of 
MAX-INT. In the logs below, is this amount about some crazy-large single 
mutation or about all values in memory for this column family?

 WARN [COMMIT-LOG-WRITER] 2014-06-11 20:25:29,505 CommitLog.java (line 349) 
Skipping commitlog append of extremely large mutation (1833599403 bytes)
 INFO [OptionalTasks:1] 2014-06-11 20:25:29,832 MeteredFlusher.java (line 64) 
flushing high-traffic column family CFS(Keyspace='SyncCore', 
ColumnFamily='EmailNetworkDeltas') (estimated 1836720026 bytes)
 INFO [OptionalTasks:1] 2014-06-11 20:25:30,325 ColumnFamilyStore.java (line 
633) Enqueuing flush of 
Memtable-EmailNetworkDeltas@79278257(1837683252/1864225045 serialized/live 
bytes, 1098 ops)
 INFO [FlushWriter:1520] 2014-06-11 20:25:30,326 Memtable.java (line 398) 
Writing Memtable-EmailNetworkDeltas@79278257(1837683252/1864225045 
serialized/live bytes, 1098 ops)
ERROR [FlushWriter:1520] 2014-06-11 20:25:30,344 CassandraDaemon.java (line 
191) Exception in thread Thread[FlushWriter:1520,5,main]
java.lang.NegativeArraySizeException
at 
org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)


 Commit logs no longer deleting and MemtablePostFlusher pending growing
 --

 Key: CASSANDRA-7373
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7373
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.5
 Cassandra 1.12.16
 Replication factor of 3
Reporter: Francois Richard

 We have this issue where once in a while, we get into a situation where the 
 MemtablePostFlusher is not executing and the space used by the commit logs on 
 disks keeps on increasing and increasing.
 We can observe the problem by invoking nodetool tpstats:
 {code}
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 6 6   46650213 0
  0
 RequestResponseStage  0 0  130547421 0
  0
 MutationStage 2 2  116813206 0
  0
 ReadRepairStage   0 02322201 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0 120780 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0  0 0
  0
 MemoryMeter   0 0456 0
  0
 MemtablePostFlusher   1   447   6344 0
  0
 FlushWriter   0 0   6132 0
 62
 MiscStage 0 0  0 0
  0
 PendingRangeCalculator0 0  6 0
  0
 commitlog_archiver

[jira] [Commented] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028406#comment-14028406
 ] 

Mikhail Stepura commented on CASSANDRA-7373:


[~jeffery.griffith] you have a pretty big row in that column family, and 
{{FastByteArrayOutputStream.expand}} doesn't validate if it can be safely 
expanded.

So, to summarize
* Unsafe {{FastByteArrayOutputStream.expand}} leads to an exception
* {{FlushRunnable.runWith}} fails to count down a latch, blocking tasks in 
MemtablePostFlusher

 Commit logs no longer deleting and MemtablePostFlusher pending growing
 --

 Key: CASSANDRA-7373
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7373
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.5
 Cassandra 1.12.16
 Replication factor of 3
Reporter: Francois Richard

 We have this issue where once in a while, we get into a situation where the 
 MemtablePostFlusher is not executing and the space used by the commit logs on 
 disks keeps on increasing and increasing.
 We can observe the problem by invoking nodetool tpstats:
 {code}
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 6 6   46650213 0
  0
 RequestResponseStage  0 0  130547421 0
  0
 MutationStage 2 2  116813206 0
  0
 ReadRepairStage   0 02322201 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0 120780 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0  0 0
  0
 MemoryMeter   0 0456 0
  0
 MemtablePostFlusher   1   447   6344 0
  0
 FlushWriter   0 0   6132 0
 62
 MiscStage 0 0  0 0
  0
 PendingRangeCalculator0 0  6 0
  0
 commitlog_archiver0 0  0 0
  0
 InternalResponseStage 0 0  0 0
  0
 HintedHandoff 2 2  4 0
  0
 Message type   Dropped
 RANGE_SLICE  0
 READ_REPAIR  0
 BINARY   0
 READ 0
 MUTATION 0
 _TRACE   0
 REQUEST_RESPONSE 0
 COUNTER_MUTATION 0
 {code}
 Here is a potential error in the logs that can explain this:
 {code}
 ERROR [FlushWriter:2693] 2014-06-09 22:05:38,452 CassandraDaemon.java (line 
 191) Exception in thread Thread[FlushWriter:2693,5,main]
 java.lang.NegativeArraySizeException
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:220)
   at java.io.DataOutputStream.write(DataOutputStream.java:107)
   at 
 org.apache.cassandra.io.util.DataOutputBuffer.write(DataOutputBuffer.java:60)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.write(ByteBufferUtil.java:328)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithLength(ByteBufferUtil.java:315)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:55)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:30)
   at 
 org.apache.cassandra.db.OnDiskAtom$Serializer.serializeForSSTable(OnDiskAtom.java:62)
   at org.apache.cassandra.db.ColumnIndex$Builder.add(ColumnIndex.java:181)
   at 
 org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:133)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:185)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:430)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:385)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7342) CAS writes does not have hint functionality.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028420#comment-14028420
 ] 

sankalp kohli commented on CASSANDRA-7342:
--

I was thinking of ways to do it. Can we use the same hints CF and add a new 
column called cas_commit and save commits there. This way we can distinguish 
while replaying the logs whether it is a commit or a normal mutation. 

CREATE TABLE hints (
target_id uuid,
hint_id timeuuid,
message_version int,
mutation blob,
cas_commit blob,
PRIMARY KEY (target_id, hint_id, message_version)
 * ) WITH COMPACT STORAGE;

 CAS writes does not have hint functionality. 
 -

 Key: CASSANDRA-7342
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7342
 Project: Cassandra
  Issue Type: Improvement
Reporter: sankalp kohli

 When a dead node comes up, it gets the last commit but not anything which it 
 has missed. 
 This reduces the durability of those writes compared to other writes. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (CASSANDRA-7342) CAS writes does not have hint functionality.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028420#comment-14028420
 ] 

sankalp kohli edited comment on CASSANDRA-7342 at 6/11/14 9:30 PM:
---

I was thinking of ways to do it. Can we use the same hints CF and add a new 
column called cas_commit and save commits there. This way we can distinguish 
while replaying the logs whether it is a commit or a normal mutation. 

CREATE TABLE hints (
target_id uuid,
hint_id timeuuid,
message_version int,
mutation blob,
cas_commit blob,  //New Column 
PRIMARY KEY (target_id, hint_id, message_version);


was (Author: kohlisankalp):
I was thinking of ways to do it. Can we use the same hints CF and add a new 
column called cas_commit and save commits there. This way we can distinguish 
while replaying the logs whether it is a commit or a normal mutation. 

CREATE TABLE hints (
target_id uuid,
hint_id timeuuid,
message_version int,
mutation blob,
cas_commit blob,
PRIMARY KEY (target_id, hint_id, message_version)
 * ) WITH COMPACT STORAGE;

 CAS writes does not have hint functionality. 
 -

 Key: CASSANDRA-7342
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7342
 Project: Cassandra
  Issue Type: Improvement
Reporter: sankalp kohli

 When a dead node comes up, it gets the last commit but not anything which it 
 has missed. 
 This reduces the durability of those writes compared to other writes. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7379) updating a row with composite key with a null value removes the entire row


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashot Golovenko updated CASSANDRA-7379:
---

Reproduced In: 2.0.8, 2.0.7  (was: 2.0.7, 2.0.8)

 updating a row with composite key with a null value removes the entire row
 --

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Assignee: Michael Shuler
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7379) updating a row with composite key with a null value removes the entire row


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028505#comment-14028505
 ] 

Ashot Golovenko commented on CASSANDRA-7379:


[~jkrupan], you're right, updated the title

 updating a row with composite key with a null value removes the entire row
 --

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Assignee: Michael Shuler
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7379) updating a row with compound key with a null value removes the entire row


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashot Golovenko updated CASSANDRA-7379:
---

Reproduced In: 2.0.8, 2.0.7  (was: 2.0.7, 2.0.8)
  Summary: updating a row with compound key with a null value removes 
the entire row  (was: updating a row with composite key with a null value 
removes the entire row)

 updating a row with compound key with a null value removes the entire row
 -

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Assignee: Michael Shuler
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Use node's host id in place of counter ids

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 5fe755762 - 99594cd68


Use node's host id in place of counter ids

patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-7366


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/99594cd6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/99594cd6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/99594cd6

Branch: refs/heads/cassandra-2.1
Commit: 99594cd6879c73da78d05a56232427936d2ee5d7
Parents: 5fe7557
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Jun 11 17:21:35 2014 -0500
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Jun 11 17:21:35 2014 -0500

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java |  6 ---
 .../org/apache/cassandra/config/KSMetaData.java |  1 -
 .../org/apache/cassandra/db/SystemKeyspace.java | 49 +---
 .../cassandra/service/StorageService.java   | 17 +++
 .../org/apache/cassandra/utils/CounterId.java   | 38 +--
 .../apache/cassandra/utils/CounterIdTest.java   | 49 
 7 files changed, 9 insertions(+), 152 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/99594cd6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a8a84d8..9dd54f9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.0
+ * Use node's host id in place of counter ids (CASSANDRA-7366)
  * Explicitly use Long.MAX_VALUE timestamp for counter deletions
(CASSANDRA-7346)
  * Fix native protocol CAS batches (CASSANDRA-7337)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/99594cd6/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index f6935e5..de2466c 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -144,12 +144,6 @@ public final class CFMetaData
  + PRIMARY KEY 
(table_name, index_name)
  + ) WITH COMPACT STORAGE 
AND COMMENT='indexes that have been completed');
 
-public static final CFMetaData CounterIdCf = compile(CREATE TABLE \ + 
SystemKeyspace.COUNTER_ID_CF + \ (
- + key text,
- + id timeuuid,
- + PRIMARY KEY (key, 
id)
- + ) WITH COMPACT 
STORAGE AND COMMENT='counter node IDs');
-
 public static final CFMetaData SchemaKeyspacesCf = compile(CREATE TABLE  
+ SystemKeyspace.SCHEMA_KEYSPACES_CF +  (
+ 
keyspace_name text PRIMARY KEY,
+ 
durable_writes boolean,

http://git-wip-us.apache.org/repos/asf/cassandra/blob/99594cd6/src/java/org/apache/cassandra/config/KSMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/KSMetaData.java 
b/src/java/org/apache/cassandra/config/KSMetaData.java
index d0cb613..7700394 100644
--- a/src/java/org/apache/cassandra/config/KSMetaData.java
+++ b/src/java/org/apache/cassandra/config/KSMetaData.java
@@ -96,7 +96,6 @@ public final class KSMetaData
 CFMetaData.PeerEventsCf,
 CFMetaData.HintsCf,
 CFMetaData.IndexCf,
-CFMetaData.CounterIdCf,
 CFMetaData.SchemaKeyspacesCf,
 
CFMetaData.SchemaColumnFamiliesCf,
 CFMetaData.SchemaColumnsCf,

http://git-wip-us.apache.org/repos/asf/cassandra/blob/99594cd6/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java 
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index 9cb6e94..659bc69 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -29,7 +29,6 @@ import com.google.common.collect.HashMultimap;
 import com.google.common.collect.Iterables;
 import

[jira] [Commented] (CASSANDRA-7379) updating a row with compound key with a null value removes the entire row


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028521#comment-14028521
 ] 

Ashot Golovenko commented on CASSANDRA-7379:


[~mishail], is it also expected from UPDATE and INSERT statements to produce 
different results?

 updating a row with compound key with a null value removes the entire row
 -

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Assignee: Michael Shuler
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk

Conflicts:
test/unit/org/apache/cassandra/utils/CounterIdTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a9b1b4c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a9b1b4c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a9b1b4c

Branch: refs/heads/trunk
Commit: 5a9b1b4c1b2b5f97195f20bf8b4c917c54552f21
Parents: 9aace48 99594cd
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Jun 11 17:43:29 2014 -0500
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Jun 11 17:43:29 2014 -0500

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java |  6 ---
 .../org/apache/cassandra/config/KSMetaData.java |  1 -
 .../org/apache/cassandra/db/SystemKeyspace.java | 48 +---
 .../cassandra/service/StorageService.java   | 17 +++
 .../org/apache/cassandra/utils/CounterId.java   | 38 +---
 .../apache/cassandra/utils/CounterIdTest.java   | 48 
 7 files changed, 9 insertions(+), 150 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a9b1b4c/CHANGES.txt
--
diff --cc CHANGES.txt
index aa2f96c,9dd54f9..598c793
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,14 -1,5 +1,15 @@@
 +3.0
 + * Move sstable RandomAccessReader to nio2, which allows using the
 +   FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050)
 + * Remove CQL2 (CASSANDRA-5918)
 + * Add Thrift get_multi_slice call (CASSANDRA-6757)
 + * Optimize fetching multiple cells by name (CASSANDRA-6933)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Make incremental repair default (CASSANDRA-7250)
 +
 +
  2.1.0
+  * Use node's host id in place of counter ids (CASSANDRA-7366)
   * Explicitly use Long.MAX_VALUE timestamp for counter deletions
 (CASSANDRA-7346)
   * Fix native protocol CAS batches (CASSANDRA-7337)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a9b1b4c/src/java/org/apache/cassandra/config/CFMetaData.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a9b1b4c/src/java/org/apache/cassandra/service/StorageService.java
--

[1/2] git commit: Use node's host id in place of counter ids

Repository: cassandra
Updated Branches:
  refs/heads/trunk 9aace4836 - 5a9b1b4c1


Use node's host id in place of counter ids

patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-7366


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/99594cd6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/99594cd6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/99594cd6

Branch: refs/heads/trunk
Commit: 99594cd6879c73da78d05a56232427936d2ee5d7
Parents: 5fe7557
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Jun 11 17:21:35 2014 -0500
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Jun 11 17:21:35 2014 -0500

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/CFMetaData.java |  6 ---
 .../org/apache/cassandra/config/KSMetaData.java |  1 -
 .../org/apache/cassandra/db/SystemKeyspace.java | 49 +---
 .../cassandra/service/StorageService.java   | 17 +++
 .../org/apache/cassandra/utils/CounterId.java   | 38 +--
 .../apache/cassandra/utils/CounterIdTest.java   | 49 
 7 files changed, 9 insertions(+), 152 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/99594cd6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a8a84d8..9dd54f9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.0
+ * Use node's host id in place of counter ids (CASSANDRA-7366)
  * Explicitly use Long.MAX_VALUE timestamp for counter deletions
(CASSANDRA-7346)
  * Fix native protocol CAS batches (CASSANDRA-7337)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/99594cd6/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index f6935e5..de2466c 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -144,12 +144,6 @@ public final class CFMetaData
  + PRIMARY KEY 
(table_name, index_name)
  + ) WITH COMPACT STORAGE 
AND COMMENT='indexes that have been completed');
 
-public static final CFMetaData CounterIdCf = compile(CREATE TABLE \ + 
SystemKeyspace.COUNTER_ID_CF + \ (
- + key text,
- + id timeuuid,
- + PRIMARY KEY (key, 
id)
- + ) WITH COMPACT 
STORAGE AND COMMENT='counter node IDs');
-
 public static final CFMetaData SchemaKeyspacesCf = compile(CREATE TABLE  
+ SystemKeyspace.SCHEMA_KEYSPACES_CF +  (
+ 
keyspace_name text PRIMARY KEY,
+ 
durable_writes boolean,

http://git-wip-us.apache.org/repos/asf/cassandra/blob/99594cd6/src/java/org/apache/cassandra/config/KSMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/KSMetaData.java 
b/src/java/org/apache/cassandra/config/KSMetaData.java
index d0cb613..7700394 100644
--- a/src/java/org/apache/cassandra/config/KSMetaData.java
+++ b/src/java/org/apache/cassandra/config/KSMetaData.java
@@ -96,7 +96,6 @@ public final class KSMetaData
 CFMetaData.PeerEventsCf,
 CFMetaData.HintsCf,
 CFMetaData.IndexCf,
-CFMetaData.CounterIdCf,
 CFMetaData.SchemaKeyspacesCf,
 
CFMetaData.SchemaColumnFamiliesCf,
 CFMetaData.SchemaColumnsCf,

http://git-wip-us.apache.org/repos/asf/cassandra/blob/99594cd6/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java 
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index 9cb6e94..659bc69 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -29,7 +29,6 @@ import com.google.common.collect.HashMultimap;
 import com.google.common.collect.Iterables;
 import

[jira] [Updated] (CASSANDRA-7079) allow filtering within wide row


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashot Golovenko updated CASSANDRA-7079:
---

Reproduced In: 2.0.8

 allow filtering within wide row
 ---

 Key: CASSANDRA-7079
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7079
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko

 Let's say I have a table with wide rows.
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 f boolean,
 PRIMARY KEY (u1, u2));
 Usually I need to retrieve the whole row: 
 select * from relation where u1 = ?;
 But sometimes I just need the relations within u1 with f = true.
 By now I can't perform the following without creating an index which will 
 degrade write performance:
 select * from relation where u1 = ? and f=true allow filtering;
 So now I filter rows on server side which means more network traffic and I 
 don't know how much more server resources. Filtering rows in this case on a 
 server side looks like nothing hard.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Stepura updated CASSANDRA-7373:
---

Attachment: 0002-Handle-possible-integer-overflow.patch
0001-Move-latch.countDown-into-finally-block.patch

I'm attaching two patches for 1.2 to address those two issues.
Though I'm not fully convinced that  2nd issue (with unsafe expand for very 
large buffers) is worth fixing 

 Commit logs no longer deleting and MemtablePostFlusher pending growing
 --

 Key: CASSANDRA-7373
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7373
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.5
 Cassandra 1.12.16
 Replication factor of 3
Reporter: Francois Richard
 Attachments: 0001-Move-latch.countDown-into-finally-block.patch, 
 0002-Handle-possible-integer-overflow.patch


 We have this issue where once in a while, we get into a situation where the 
 MemtablePostFlusher is not executing and the space used by the commit logs on 
 disks keeps on increasing and increasing.
 We can observe the problem by invoking nodetool tpstats:
 {code}
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 6 6   46650213 0
  0
 RequestResponseStage  0 0  130547421 0
  0
 MutationStage 2 2  116813206 0
  0
 ReadRepairStage   0 02322201 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0 120780 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0  0 0
  0
 MemoryMeter   0 0456 0
  0
 MemtablePostFlusher   1   447   6344 0
  0
 FlushWriter   0 0   6132 0
 62
 MiscStage 0 0  0 0
  0
 PendingRangeCalculator0 0  6 0
  0
 commitlog_archiver0 0  0 0
  0
 InternalResponseStage 0 0  0 0
  0
 HintedHandoff 2 2  4 0
  0
 Message type   Dropped
 RANGE_SLICE  0
 READ_REPAIR  0
 BINARY   0
 READ 0
 MUTATION 0
 _TRACE   0
 REQUEST_RESPONSE 0
 COUNTER_MUTATION 0
 {code}
 Here is a potential error in the logs that can explain this:
 {code}
 ERROR [FlushWriter:2693] 2014-06-09 22:05:38,452 CassandraDaemon.java (line 
 191) Exception in thread Thread[FlushWriter:2693,5,main]
 java.lang.NegativeArraySizeException
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:220)
   at java.io.DataOutputStream.write(DataOutputStream.java:107)
   at 
 org.apache.cassandra.io.util.DataOutputBuffer.write(DataOutputBuffer.java:60)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.write(ByteBufferUtil.java:328)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithLength(ByteBufferUtil.java:315)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:55)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:30)
   at 
 org.apache.cassandra.db.OnDiskAtom$Serializer.serializeForSSTable(OnDiskAtom.java:62)
   at org.apache.cassandra.db.ColumnIndex$Builder.add(ColumnIndex.java:181)
   at 
 org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:133)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:185)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:430)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:385)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7379) updating a row with compound key with a null value removes the entire row


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028543#comment-14028543
 ] 

Mikhail Stepura commented on CASSANDRA-7379:


[~asho...@gmail.com] no idea. I just pointed to a very similar ticket. /cc 
[~iamaleksey] and [~slebresne] to comment

 updating a row with compound key with a null value removes the entire row
 -

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Assignee: Michael Shuler
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (CASSANDRA-7379) updating a row with compound key with a null value removes the entire row

2014-06-11 Thread Aleksey Yeschenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-7379.
--

   Resolution: Duplicate
Reproduced In: 2.0.8, 2.0.7  (was: 2.0.7, 2.0.8)

This is the expected behavior. See CASSANDRA-6951.

 updating a row with compound key with a null value removes the entire row
 -

 Key: CASSANDRA-7379
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7379
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ashot Golovenko
Assignee: Michael Shuler
Priority: Critical

 create a table
 CREATE TABLE relation (
 u1 bigint,
 u2 bigint,
 mf int,
 PRIMARY KEY (u1, u2));
 insert value:
 UPDATE relation SET mf = 1 WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
  u1 | u2 | mf
 ++
   1 |  2 |  1
 insert null value:
 UPDATE relation SET mf = null WHERE u1 = 1 and u2 = 2;
 SELECT * from relation ;
 (0 rows) --- WRONG!
 The INSERT statement however works:
 INSERT INTO relation (u1, u2, mf) VALUES (1, 2, null); 
 SELECT * from relation ;
  u1 | u2 | mf
 ++--
   1 |  2 | null
 (1 rows)
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7367) add cql function to create timeuuid from date

2014-06-11 Thread Sylvain Lebresne (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028554#comment-14028554
 ] 

Sylvain Lebresne commented on CASSANDRA-7367:
-

How is that different from the existing {{minTimeuuid()}} and {{maxTimeuuid()}} 
functions (expect that it creates a clock and sequence number that are 
basically random which is likely a bad idea)?

 add cql function to create timeuuid from date
 -

 Key: CASSANDRA-7367
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7367
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Dave Brosius
Assignee: Dave Brosius
Priority: Trivial
 Fix For: 3.0

 Attachments: 7367.txt


 creating timeuuids in cql is unwieldy. Adding a toTimeuuid() function to 
 convert a date/time to a time uuid.
 ie, 
 create table foo (key timeuuid primary key, value text);
 insert into foo (key, value) values (toTimeuuid('2009-12-16 19:26:29-0500'), 
 'hello');



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7367) add cql function to create timeuuid from date

2014-06-11 Thread Sylvain Lebresne (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028561#comment-14028561
 ] 

Sylvain Lebresne commented on CASSANDRA-7367:
-

My bad, I realize that the goal is inserts here. In that case, I'm not entirely 
fan of such functions because it makes it very easy to create 2 timeuuid that 
conflicts while the goal of timeuuid is to avoid that. It feels to me that 
creating a timeuuid from a date for something else than a query is 
intrinsically wrong, hence my reluctance to add a function to make it easy.

 add cql function to create timeuuid from date
 -

 Key: CASSANDRA-7367
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7367
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Dave Brosius
Assignee: Dave Brosius
Priority: Trivial
 Fix For: 3.0

 Attachments: 7367.txt


 creating timeuuids in cql is unwieldy. Adding a toTimeuuid() function to 
 convert a date/time to a time uuid.
 ie, 
 create table foo (key timeuuid primary key, value text);
 insert into foo (key, value) values (toTimeuuid('2009-12-16 19:26:29-0500'), 
 'hello');



--
This message was sent by Atlassian JIRA
(v6.2#6252)

buildbot failure in ASF Buildbot on cassandra-2.1

2014-06-11 Thread buildbot

The Buildbot has detected a new failure on builder cassandra-2.1 while building 
cassandra.
Full details are available at:
 http://ci.apache.org/builders/cassandra-2.1/builds/134

Buildbot URL: http://ci.apache.org/

Buildslave for this Build: portunus_ubuntu

Build Reason: scheduler
Build Source Stamp: [branch cassandra-2.1] 
99594cd6879c73da78d05a56232427936d2ee5d7
Blamelist: Aleksey Yeschenko alek...@apache.org

BUILD FAILED: failed shell

sincerely,
 -The Buildbot

[jira] [Commented] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028590#comment-14028590
 ] 

Jeff Griffith commented on CASSANDRA-7373:
--

Excellent, thanks [~mishail]. We'll try too root out the large row which we 
were in the process of doing anyway. 

 Commit logs no longer deleting and MemtablePostFlusher pending growing
 --

 Key: CASSANDRA-7373
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7373
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.5
 Cassandra 1.12.16
 Replication factor of 3
Reporter: Francois Richard
 Fix For: 1.2.17, 2.0.9, 2.1.0

 Attachments: 0001-Move-latch.countDown-into-finally-block.patch, 
 0002-Handle-possible-integer-overflow.patch


 We have this issue where once in a while, we get into a situation where the 
 MemtablePostFlusher is not executing and the space used by the commit logs on 
 disks keeps on increasing and increasing.
 We can observe the problem by invoking nodetool tpstats:
 {code}
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 6 6   46650213 0
  0
 RequestResponseStage  0 0  130547421 0
  0
 MutationStage 2 2  116813206 0
  0
 ReadRepairStage   0 02322201 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0 120780 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0  0 0
  0
 MemoryMeter   0 0456 0
  0
 MemtablePostFlusher   1   447   6344 0
  0
 FlushWriter   0 0   6132 0
 62
 MiscStage 0 0  0 0
  0
 PendingRangeCalculator0 0  6 0
  0
 commitlog_archiver0 0  0 0
  0
 InternalResponseStage 0 0  0 0
  0
 HintedHandoff 2 2  4 0
  0
 Message type   Dropped
 RANGE_SLICE  0
 READ_REPAIR  0
 BINARY   0
 READ 0
 MUTATION 0
 _TRACE   0
 REQUEST_RESPONSE 0
 COUNTER_MUTATION 0
 {code}
 Here is a potential error in the logs that can explain this:
 {code}
 ERROR [FlushWriter:2693] 2014-06-09 22:05:38,452 CassandraDaemon.java (line 
 191) Exception in thread Thread[FlushWriter:2693,5,main]
 java.lang.NegativeArraySizeException
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:220)
   at java.io.DataOutputStream.write(DataOutputStream.java:107)
   at 
 org.apache.cassandra.io.util.DataOutputBuffer.write(DataOutputBuffer.java:60)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.write(ByteBufferUtil.java:328)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithLength(ByteBufferUtil.java:315)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:55)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:30)
   at 
 org.apache.cassandra.db.OnDiskAtom$Serializer.serializeForSSTable(OnDiskAtom.java:62)
   at org.apache.cassandra.db.ColumnIndex$Builder.add(ColumnIndex.java:181)
   at 
 org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:133)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:185)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:430)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:385)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7380) Native protocol needs keepalive, we should add it


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-7380:


Attachment: 7380-2.0.txt
7380-1.2.txt

Patches against 1.2 and 2.0 to reuse rpc_keepalive for the native proto.

 Native protocol needs keepalive, we should add it
 -

 Key: CASSANDRA-7380
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7380
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 1.2, 2.0
Reporter: Jose Martinez Poblete
Assignee: Tyler Hobbs
 Fix For: 1.2.17, 2.0.9

 Attachments: 7380-1.2.txt, 7380-2.0.txt


 On clients connecting to C* 1.2.15 using native protocol. We see that when 
 the client is bounced, the old connection is not going away
 On Thrift, there is the rpc_keepalive but there is no similar feature for the 
 native protocol



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6539) Track metrics at a keyspace level as well as column family level


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6539:


Attachment: (was: 6539-1.2.txt)

 Track metrics at a keyspace level as well as column family level
 

 Key: CASSANDRA-6539
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6539
 Project: Cassandra
  Issue Type: Improvement
Reporter: Nick Bailey
Assignee: Brandon Williams
Priority: Minor
  Labels: lhf
 Fix For: 1.2.17, 2.0.9

 Attachments: 6539-1.2.txt


 It would be useful to be able to see aggregated metrics (write/read 
 count/latency) at a keyspace level as well as at the individual column family 
 level.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6539) Track metrics at a keyspace level as well as column family level


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6539:


Attachment: (was: 6539-2.0.txt)

 Track metrics at a keyspace level as well as column family level
 

 Key: CASSANDRA-6539
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6539
 Project: Cassandra
  Issue Type: Improvement
Reporter: Nick Bailey
Assignee: Brandon Williams
Priority: Minor
  Labels: lhf
 Fix For: 1.2.17, 2.0.9

 Attachments: 6539-1.2.txt


 It would be useful to be able to see aggregated metrics (write/read 
 count/latency) at a keyspace level as well as at the individual column family 
 level.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6539) Track metrics at a keyspace level as well as column family level


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6539:


Attachment: 6539-1.2.txt

I think you're right.  I pared it down to anything that wasn't a simple sum or 
didn't make sense in the updated patch.

 Track metrics at a keyspace level as well as column family level
 

 Key: CASSANDRA-6539
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6539
 Project: Cassandra
  Issue Type: Improvement
Reporter: Nick Bailey
Assignee: Brandon Williams
Priority: Minor
  Labels: lhf
 Fix For: 1.2.17, 2.0.9

 Attachments: 6539-1.2.txt


 It would be useful to be able to see aggregated metrics (write/read 
 count/latency) at a keyspace level as well as at the individual column family 
 level.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7373) Commit logs no longer deleting and MemtablePostFlusher pending growing

2014-06-11 Thread Tyler Hobbs (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028652#comment-14028652
 ] 

Tyler Hobbs commented on CASSANDRA-7373:


The first patch is a dupe of CASSANDRA-7275, so feel free to either move the 
patch there or resolve it as a duplicate.

 Commit logs no longer deleting and MemtablePostFlusher pending growing
 --

 Key: CASSANDRA-7373
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7373
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.5
 Cassandra 1.12.16
 Replication factor of 3
Reporter: Francois Richard
 Fix For: 1.2.17, 2.0.9, 2.1.0

 Attachments: 0001-Move-latch.countDown-into-finally-block.patch, 
 0002-Handle-possible-integer-overflow.patch


 We have this issue where once in a while, we get into a situation where the 
 MemtablePostFlusher is not executing and the space used by the commit logs on 
 disks keeps on increasing and increasing.
 We can observe the problem by invoking nodetool tpstats:
 {code}
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 6 6   46650213 0
  0
 RequestResponseStage  0 0  130547421 0
  0
 MutationStage 2 2  116813206 0
  0
 ReadRepairStage   0 02322201 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0 120780 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0  0 0
  0
 MemoryMeter   0 0456 0
  0
 MemtablePostFlusher   1   447   6344 0
  0
 FlushWriter   0 0   6132 0
 62
 MiscStage 0 0  0 0
  0
 PendingRangeCalculator0 0  6 0
  0
 commitlog_archiver0 0  0 0
  0
 InternalResponseStage 0 0  0 0
  0
 HintedHandoff 2 2  4 0
  0
 Message type   Dropped
 RANGE_SLICE  0
 READ_REPAIR  0
 BINARY   0
 READ 0
 MUTATION 0
 _TRACE   0
 REQUEST_RESPONSE 0
 COUNTER_MUTATION 0
 {code}
 Here is a potential error in the logs that can explain this:
 {code}
 ERROR [FlushWriter:2693] 2014-06-09 22:05:38,452 CassandraDaemon.java (line 
 191) Exception in thread Thread[FlushWriter:2693,5,main]
 java.lang.NegativeArraySizeException
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
   at 
 org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:220)
   at java.io.DataOutputStream.write(DataOutputStream.java:107)
   at 
 org.apache.cassandra.io.util.DataOutputBuffer.write(DataOutputBuffer.java:60)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.write(ByteBufferUtil.java:328)
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithLength(ByteBufferUtil.java:315)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:55)
   at 
 org.apache.cassandra.db.ColumnSerializer.serialize(ColumnSerializer.java:30)
   at 
 org.apache.cassandra.db.OnDiskAtom$Serializer.serializeForSSTable(OnDiskAtom.java:62)
   at org.apache.cassandra.db.ColumnIndex$Builder.add(ColumnIndex.java:181)
   at 
 org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:133)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:185)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:430)
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:385)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7380) Native protocol needs keepalive, we should add it


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028654#comment-14028654
 ] 

sankalp kohli commented on CASSANDRA-7380:
--

looks good. There is SimpleClient.java which also sets some options like this. 
Don't know what it is used for. 
Also the patch will be different for trunk as options are set in Server.java

 Native protocol needs keepalive, we should add it
 -

 Key: CASSANDRA-7380
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7380
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 1.2, 2.0
Reporter: Jose Martinez Poblete
Assignee: Brandon Williams
 Fix For: 1.2.17, 2.0.9

 Attachments: 7380-1.2.txt, 7380-2.0.txt


 On clients connecting to C* 1.2.15 using native protocol. We see that when 
 the client is bounced, the old connection is not going away
 On Thrift, there is the rpc_keepalive but there is no similar feature for the 
 native protocol



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7380) Native protocol needs keepalive, we should add it


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028672#comment-14028672
 ] 

Brandon Williams commented on CASSANDRA-7380:
-

SimpleClient is for stress, so I didn't bother since keepalive there shouldn't 
be needed.

 Native protocol needs keepalive, we should add it
 -

 Key: CASSANDRA-7380
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7380
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 1.2, 2.0
Reporter: Jose Martinez Poblete
Assignee: Brandon Williams
 Fix For: 1.2.17, 2.0.9

 Attachments: 7380-1.2.txt, 7380-2.0.txt


 On clients connecting to C* 1.2.15 using native protocol. We see that when 
 the client is bounced, the old connection is not going away
 On Thrift, there is the rpc_keepalive but there is no similar feature for the 
 native protocol



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7362) Be able to selectively mount a snapshot of a table as a read-only version of that table


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028674#comment-14028674
 ] 

Jonathan Ellis commented on CASSANDRA-7362:
---

Note that this requires 7190 or it will be very easy to mount snapshots that no 
longer match the current schema.

 Be able to selectively mount a snapshot of a table as a read-only version 
 of that table
 -

 Key: CASSANDRA-7362
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7362
 Project: Cassandra
  Issue Type: New Feature
  Components: Core, Tools
Reporter: Tupshin Harper
Priority: Minor
 Fix For: 3.0


 When doing batch jobs (thinking hive and shark as prominent examples) or 
 repeated analysis of the same data, it can be challenging to get a consistent 
 result if the data is changing under your feet. Rather than the low level 
 CASSANDRA-2527, I propose that we add the capability to take a named snapshot 
  (exact uuid in 2.1 and later), and be able to activate and deactivate it as 
 a regular sstable (e.g. myks.mytable snapshot could be activated as 
 myks.mytable-longuuid). That table would be queryable just like any other, 
 but would not be writable. Any attempt to insert or update would throw an 
 exception. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00628e70
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00628e70
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00628e70

Branch: refs/heads/cassandra-2.1
Commit: 00628e70a52f4cfa0ca6abfd295ba03f77298293
Parents: 99594cd 2df27c0
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Jun 12 03:11:39 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Jun 12 03:11:39 2014 +0200

--

--

git commit: Reference sstable before populating keycache after compaction

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 3e5f2bd64 - 2df27c090


Reference sstable before populating keycache after compaction

Patch by marcuse; reviewed by thobbs for CASSANDRA-7234


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2df27c09
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2df27c09
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2df27c09

Branch: refs/heads/cassandra-2.0
Commit: 2df27c090039166b3e3519688f5b7482b9d90f8c
Parents: 3e5f2bd
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Jun 9 13:16:32 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Jun 12 03:06:58 2014 +0200

--
 CHANGES.txt   |  1 +
 .../cassandra/db/compaction/CompactionTask.java   | 14 +-
 2 files changed, 14 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2df27c09/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 97ac75e..f4f7fc4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -14,6 +14,7 @@
  * Cqlsh counts non-empty lines for Blank lines warning (CASSANDRA-7325)
  * Make StreamSession#closeSession() idempotent (CASSANDRA-7262)
  * Fix infinite loop on exception while streaming (CASSANDRA-7330)
+ * Reference sstables before populating key cache (CASSANDRA-7234)
 Merged from 1.2:
  * Check internal addresses for seeds (CASSANDRA-6523)
  * Fix potential / by 0 in HHOM page size calculation (CASSANDRA-7354)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2df27c09/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 10c26db..5ef4aad 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -242,7 +242,19 @@ public class CompactionTask extends AbstractCompactionTask
 replaceCompactedSSTables(toCompact, sstables);
 // TODO: this doesn't belong here, it should be part of the reader to 
load when the tracker is wired up
 for (SSTableReader sstable : sstables)
-sstable.preheat(cachedKeyMap.get(sstable.descriptor));
+{
+if (sstable.acquireReference())
+{
+try
+{
+sstable.preheat(cachedKeyMap.get(sstable.descriptor));
+}
+finally
+{
+sstable.releaseReference();
+}
+}
+}
 
 // log a bunch of statistics about the result and save to system table 
compaction_history
 long dTime = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - start);

[1/2] git commit: Reference sstable before populating keycache after compaction

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 99594cd68 - 00628e70a


Reference sstable before populating keycache after compaction

Patch by marcuse; reviewed by thobbs for CASSANDRA-7234


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2df27c09
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2df27c09
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2df27c09

Branch: refs/heads/cassandra-2.1
Commit: 2df27c090039166b3e3519688f5b7482b9d90f8c
Parents: 3e5f2bd
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Jun 9 13:16:32 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Jun 12 03:06:58 2014 +0200

--
 CHANGES.txt   |  1 +
 .../cassandra/db/compaction/CompactionTask.java   | 14 +-
 2 files changed, 14 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2df27c09/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 97ac75e..f4f7fc4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -14,6 +14,7 @@
  * Cqlsh counts non-empty lines for Blank lines warning (CASSANDRA-7325)
  * Make StreamSession#closeSession() idempotent (CASSANDRA-7262)
  * Fix infinite loop on exception while streaming (CASSANDRA-7330)
+ * Reference sstables before populating key cache (CASSANDRA-7234)
 Merged from 1.2:
  * Check internal addresses for seeds (CASSANDRA-6523)
  * Fix potential / by 0 in HHOM page size calculation (CASSANDRA-7354)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2df27c09/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 10c26db..5ef4aad 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -242,7 +242,19 @@ public class CompactionTask extends AbstractCompactionTask
 replaceCompactedSSTables(toCompact, sstables);
 // TODO: this doesn't belong here, it should be part of the reader to 
load when the tracker is wired up
 for (SSTableReader sstable : sstables)
-sstable.preheat(cachedKeyMap.get(sstable.descriptor));
+{
+if (sstable.acquireReference())
+{
+try
+{
+sstable.preheat(cachedKeyMap.get(sstable.descriptor));
+}
+finally
+{
+sstable.releaseReference();
+}
+}
+}
 
 // log a bunch of statistics about the result and save to system table 
compaction_history
 long dTime = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - start);

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bd92b0f7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bd92b0f7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bd92b0f7

Branch: refs/heads/trunk
Commit: bd92b0f70cf1c455aa640e7c3591ad92166ae919
Parents: 5a9b1b4 00628e7
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Jun 12 03:12:14 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Jun 12 03:12:14 2014 +0200

--

--

[1/3] git commit: Reference sstable before populating keycache after compaction

Repository: cassandra
Updated Branches:
  refs/heads/trunk 5a9b1b4c1 - bd92b0f70


Reference sstable before populating keycache after compaction

Patch by marcuse; reviewed by thobbs for CASSANDRA-7234


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2df27c09
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2df27c09
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2df27c09

Branch: refs/heads/trunk
Commit: 2df27c090039166b3e3519688f5b7482b9d90f8c
Parents: 3e5f2bd
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Jun 9 13:16:32 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Jun 12 03:06:58 2014 +0200

--
 CHANGES.txt   |  1 +
 .../cassandra/db/compaction/CompactionTask.java   | 14 +-
 2 files changed, 14 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2df27c09/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 97ac75e..f4f7fc4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -14,6 +14,7 @@
  * Cqlsh counts non-empty lines for Blank lines warning (CASSANDRA-7325)
  * Make StreamSession#closeSession() idempotent (CASSANDRA-7262)
  * Fix infinite loop on exception while streaming (CASSANDRA-7330)
+ * Reference sstables before populating key cache (CASSANDRA-7234)
 Merged from 1.2:
  * Check internal addresses for seeds (CASSANDRA-6523)
  * Fix potential / by 0 in HHOM page size calculation (CASSANDRA-7354)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2df27c09/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 10c26db..5ef4aad 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -242,7 +242,19 @@ public class CompactionTask extends AbstractCompactionTask
 replaceCompactedSSTables(toCompact, sstables);
 // TODO: this doesn't belong here, it should be part of the reader to 
load when the tracker is wired up
 for (SSTableReader sstable : sstables)
-sstable.preheat(cachedKeyMap.get(sstable.descriptor));
+{
+if (sstable.acquireReference())
+{
+try
+{
+sstable.preheat(cachedKeyMap.get(sstable.descriptor));
+}
+finally
+{
+sstable.releaseReference();
+}
+}
+}
 
 // log a bunch of statistics about the result and save to system table 
compaction_history
 long dTime = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - start);

[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00628e70
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00628e70
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00628e70

Branch: refs/heads/trunk
Commit: 00628e70a52f4cfa0ca6abfd295ba03f77298293
Parents: 99594cd 2df27c0
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Jun 12 03:11:39 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Jun 12 03:11:39 2014 +0200

--

--

[jira] [Commented] (CASSANDRA-6602) Compaction improvements to optimize time series data


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028687#comment-14028687
 ] 

Jonathan Ellis commented on CASSANDRA-6602:
---

[~Bj0rn] is your thesis available as well?

 Compaction improvements to optimize time series data
 

 Key: CASSANDRA-6602
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6602
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Tupshin Harper
Assignee: Björn Hegerfors
  Labels: compaction, performance
 Fix For: 3.0

 Attachments: 
 cassandra-2.0-CASSANDRA-6602-DateTieredCompactionStrategy.txt, 
 cassandra-2.0-CASSANDRA-6602-DateTieredCompactionStrategy_v2.txt


 There are some unique characteristics of many/most time series use cases that 
 both provide challenges, as well as provide unique opportunities for 
 optimizations.
 One of the major challenges is in compaction. The existing compaction 
 strategies will tend to re-compact data on disk at least a few times over the 
 lifespan of each data point, greatly increasing the cpu and IO costs of that 
 write.
 Compaction exists to
 1) ensure that there aren't too many files on disk
 2) ensure that data that should be contiguous (part of the same partition) is 
 laid out contiguously
 3) deleting data due to ttls or tombstones
 The special characteristics of time series data allow us to optimize away all 
 three.
 Time series data
 1) tends to be delivered in time order, with relatively constrained exceptions
 2) often has a pre-determined and fixed expiration date
 3) Never gets deleted prior to TTL
 4) Has relatively predictable ingestion rates
 Note that I filed CASSANDRA-5561 and this ticket potentially replaces or 
 lowers the need for it. In that ticket, jbellis reasonably asks, how that 
 compaction strategy is better than disabling compaction.
 Taking that to heart, here is a compaction-strategy-less approach that could 
 be extremely efficient for time-series use cases that follow the above 
 pattern.
 (For context, I'm thinking of an example use case involving lots of streams 
 of time-series data with a 5GB per day ingestion rate, and a 1000 day 
 retention with TTL, resulting in an eventual steady state of 5TB per node)
 1) You have an extremely large memtable (preferably off heap, if/when doable) 
 for the table, and that memtable is sized to be able to hold a lengthy window 
 of time. A typical period might be one day. At the end of that period, you 
 flush the contents of the memtable to an sstable and move to the next one. 
 This is basically identical to current behaviour, but with thresholds 
 adjusted so that you can ensure flushing at predictable intervals. (Open 
 question is whether predictable intervals is actually necessary, or whether 
 just waiting until the huge memtable is nearly full is sufficient)
 2) Combine the behaviour with CASSANDRA-5228 so that sstables will be 
 efficiently dropped once all of the columns have. (Another side note, it 
 might be valuable to have a modified version of CASSANDRA-3974 that doesn't 
 bother storing per-column TTL since it is required that all columns have the 
 same TTL)
 3) Be able to mark column families as read/write only (no explicit deletes), 
 so no tombstones.
 4) Optionally add back an additional type of delete that would delete all 
 data earlier than a particular timestamp, resulting in immediate dropping of 
 obsoleted sstables.
 The result is that for in-order delivered data, Every cell will be laid out 
 optimally on disk on the first pass, and over the course of 1000 days and 5TB 
 of data, there will only be 1000 5GB sstables, so the number of filehandles 
 will be reasonable.
 For exceptions (out-of-order delivery), most cases will be caught by the 
 extended (24 hour+) memtable flush times and merged correctly automatically. 
 For those that were slightly askew at flush time, or were delivered so far 
 out of order that they go in the wrong sstable, there is relatively low 
 overhead to reading from two sstables for a time slice, instead of one, and 
 that overhead would be incurred relatively rarely unless out-of-order 
 delivery was the common case, in which case, this strategy should not be used.
 Another possible optimization to address out-of-order would be to maintain 
 more than one time-centric memtables in memory at a time (e.g. two 12 hour 
 ones), and then you always insert into whichever one of the two owns the 
 appropriate range of time. By delaying flushing the ahead one until we are 
 ready to roll writes over to a third one, we are able to avoid any 
 fragmentation as long as all deliveries come in no more than 12 hours late 
 (in this example, presumably tunable).
 Anything that triggers compactions

[jira] [Commented] (CASSANDRA-6696) Drive replacement in JBOD can cause data to reappear.

[
https://issues.apache.org/jira/browse/CASSANDRA-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028694#comment-14028694
]

Jonathan Ellis commented on CASSANDRA-6696:
---

Not serving stale data is good, but warning the other nodes when we blacklist a
disk to read those vnodes' data from other replicas would be even better. New
ticket for that?

Drive replacement in JBOD can cause data to reappear.
--

Key: CASSANDRA-6696
URL: https://issues.apache.org/jira/browse/CASSANDRA-6696
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: sankalp kohli
Assignee: Marcus Eriksson
Fix For: 3.0

In JBOD, when someone gets a bad drive, the bad drive is replaced with a new
empty one and repair is run.
This can cause deleted data to come back in some cases. Also this is true for
corrupt stables in which we delete the corrupt stable and run repair.
Here is an example:
Say we have 3 nodes A,B and C and RF=3 and GC grace=10days.
row=sankalp col=sankalp is written 20 days back and successfully went to all
three nodes.
Then a delete/tombstone was written successfully for the same row column 15
days back.
Since this tombstone is more than gc grace, it got compacted in Nodes A and B
since it got compacted with the actual data. So there is no trace of this row
column in node A and B.
Now in node C, say the original data is in drive1 and tombstone is in drive2.
Compaction has not yet reclaimed the data and tombstone.
Drive2 becomes corrupt and was replaced with new empty drive.
Due to the replacement, the tombstone in now gone and row=sankalp col=sankalp
has come back to life.
Now after replacing the drive we run repair. This data will be propagated to
all nodes.
Note: This is still a problem even if we run repair every gc grace.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7365) some compactions do not works under windows (file in use during rename)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-7365:
---

Attachment: 7365_v1.txt

I'm w/Benedict - I think it makes the most sense to disable this feature for 
the 2.1 release of Cassandra and re-enable it for 3.0 when our file I/O 
supports it.

Attaching a simple v1 that bypasses the maybeReopenEarly() if not on Unix.  
Seems to fix the issue w/my local testing - it's at a 100% reproduction rate 
without it, and 0 for about 5 runs of 10M on reproduction with it.

 some compactions do not works under windows (file in use during rename)
 ---

 Key: CASSANDRA-7365
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7365
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: jdk7, cassandra-2.1rc1, os windows 32 bit
Reporter: Radim Kolar
Assignee: Joshua McKenzie
  Labels: Windows
 Fix For: 2.1.1

 Attachments: 7365_v1.txt, cassandra.yaml, system.log


 compaction do not works under windows due to file rename fails: (Pro
 es nemß p°Ýstup k souboru, neboŁ jej prßvý vyu×Ývß jinř proces = process can 
 not access file because its in use by another process). Not all compactions 
 are broken. compactions done during server startup on system tables works 
 fine.
 INFO  18:30:27 Completed flushing 
 c:\cassandra-2.1\data\system\compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b\system-compactions_in_progress-ka-6-Dat.db
  (42 bytes) for commitlog position ReplayPosition(segmentId=1402165543361, 
 psition=8024611)
 ERROR 18:30:27 Exception in thread hread[CompactionExecutor:5,1,RMI Runtime]
 java.lang.RuntimeException: Failed to rename 
 c:\cassandra-2.1\data\test\sipdb-5
 f51090ee6511e3815625991ef2b954\test-sipdb-tmp-ka-7-Index.db to 
 c:\cassandra-2.1
 data\test\sipdb-58f51090ee6511e3815625991ef2b954\test-sipdb-ka-7-Index.db
 at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.j
 va:167) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.j
 va:151) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.j
 va:512) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.j
 va:504) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.ja
 a:479) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SST
 bleWriter.java:427) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SST
 bleWriter.java:422) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewrit
 r.java:312) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewrit
 r.java:306) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(Compaction
 ask.java:188) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAware
 unnable.java:48) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:
 8) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(Co
 pactionTask.java:74) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(Ab
 tractCompactionTask.java:59) ~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
 at 
 org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompa
 tionTask.run(CompactionManager.java:235) 
 ~[apache-cassandra-2.1.0-rc1.jar:2.1.0
 rc1]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:4
 1) ~[na:1.7.0_60]
 at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
 ~[na:1.7.0_
 0]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor
 java:1145) ~[na:1.7.0_60]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecuto
 .java:615) [na:1.7.0_60]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_60]
 Caused by: java.nio.file.FileSystemException: 
 c:\cassandra-2.1\data\test\sipdb-
 8f51090ee6511e3815625991ef2b954\test-sipdb-tmp-ka-7-Index.db - 
 c:\cassandra-2.
 \data\test\sipdb-58f51090ee6511e3815625991ef2b954\test-sipdb-ka-7-Index.db: 
 Pro
 es

buildbot success in ASF Buildbot on cassandra-2.1

2014-06-11 Thread buildbot

The Buildbot has detected a restored build on builder cassandra-2.1 while 
building cassandra.
Full details are available at:
 http://ci.apache.org/builders/cassandra-2.1/builds/135

Buildbot URL: http://ci.apache.org/

Buildslave for this Build: portunus_ubuntu

Build Reason: scheduler
Build Source Stamp: [branch cassandra-2.1] 
00628e70a52f4cfa0ca6abfd295ba03f77298293
Blamelist: Marcus Eriksson marc...@apache.org

Build succeeded!

sincerely,
 -The Buildbot

[jira] [Commented] (CASSANDRA-7348) Modify system tcp keepalive settings on Windows install scripts


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028735#comment-14028735
 ] 

Joshua McKenzie commented on CASSANDRA-7348:


So just to confirm - you're +1 on the minor_update patch and calling this ready 
for commit?

 Modify system tcp keepalive settings on Windows install scripts
 ---

 Key: CASSANDRA-7348
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7348
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
Priority: Minor
  Labels: Windows
 Fix For: 2.1.1

 Attachments: 7348-minor_update.txt, 7348_v1.txt, 7348_v2.txt


 Reference CASSANDRA-3569
 We need to change the powershell installation scripts to also update the 
 registry w/a 5 minute tcp keepalive timer timeout.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7335) GossipingPropertyFileSnitchTest fails on Windows


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-7335:
---

Reviewer: Jonathan Ellis

 GossipingPropertyFileSnitchTest fails on Windows
 

 Key: CASSANDRA-7335
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7335
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
Priority: Minor
  Labels: Windows
 Fix For: 3.0

 Attachments: 7335_v1.txt


 {code:failure message}
 [junit] Testcase: 
 testAutoReloadConfig(org.apache.cassandra.locator.GossipingPropertyFileSnitchTest):
Caused an ERROR
 [junit] Illegal char : at index 2: 
 /C:/vm-shared/src/cassandra/test/conf/cassandra-rackdc.properties
 [junit] java.nio.file.InvalidPathException: Illegal char : at index 2: 
 /C:/vm-shared/src/cassandra/test/conf/cassandra-rackdc.properties
 [junit] at 
 sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
 [junit] at 
 sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
 [junit] at 
 sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
 [junit] at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94)
 [junit] at 
 sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255)
 [junit] at java.nio.file.Paths.get(Paths.java:84)
 [junit] at 
 org.apache.cassandra.locator.GossipingPropertyFileSnitchTest.testAutoReloadConfig(GossipingPropertyFileSnitchTest.java:40)
 [junit]
 [junit]
 [junit] Test org.apache.cassandra.locator.GossipingPropertyFileSnitchTest 
 FAILED
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7316) Windows feature parity - lock JVM in memory to prevent swapping


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-7316:
---

Description: 
Similar to mlockall() in CLibrary.java for linux, it would be nice to lock the 
virtual address space on Windows to prevent page faults.

One option: Reference API:  
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366895(v=vs.85).aspx

  was:Similar to mlockall() in CLibrary.java for linux, it would be nice to 
lock the virtual address space on Windows to prevent page faults.  Reference 
API:  
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366895(v=vs.85).aspx

Summary: Windows feature parity - lock JVM in memory to prevent 
swapping  (was: Windows feature parity - lock virtual address space via 
VirtualLock)

 Windows feature parity - lock JVM in memory to prevent swapping
 ---

 Key: CASSANDRA-7316
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7316
 Project: Cassandra
  Issue Type: Improvement
Reporter: Joshua McKenzie
Assignee: Ala' Alkhaldi
Priority: Minor
  Labels: Windows, perfomance
 Fix For: 3.0


 Similar to mlockall() in CLibrary.java for linux, it would be nice to lock 
 the virtual address space on Windows to prevent page faults.
 One option: Reference API:  
 http://msdn.microsoft.com/en-us/library/windows/desktop/aa366895(v=vs.85).aspx



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6621) STCS fallback is not optimal when bootstrapping

2014-06-11 Thread Marcus Eriksson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028766#comment-14028766
 ] 

Marcus Eriksson commented on CASSANDRA-6621:


bq. We might want to also stream the stable level and can put stables coming 
from same level in one level on the bootstrapping node. The problem with this 
will be that we might end up with very few stable in higher levels violating 
the constrain that only last level can be less than limit.

This should be fine for short periods of time right? Problem will be that it 
will take a long time until the highest level gets compacted. What if we detect 
that, and include a couple of those high level sstables in lower lever 
compactions until the higher level is empty or starts doing real compactions?



 STCS fallback is not optimal when bootstrapping
 ---

 Key: CASSANDRA-6621
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6621
 Project: Cassandra
  Issue Type: Improvement
Reporter: Bartłomiej Romański
Priority: Minor

 The initial discussion started in (closed) CASSANDRA-5371. I've rewritten my 
 last comment here...
 After streaming (e.g. during boostrap) Cassandra places all sstables at L0. 
 At the end of the process we end up with huge number of sstables at the 
 lowest level. 
 Currently, Cassandra falls back to STCS until the number of sstables at L0 
 reaches the reasonable level (32 or something).
 I'm not sure if falling back to STCS is the best way to handle this 
 particular situation. I've read the comment in the code and I'm aware why it 
 is a good thing to do if we have to many sstables at L0 as a result of too 
 many random inserts. We have a lot of sstables, each of them covers the whole 
 ring, there's simply no better option.
 However, after the bootstrap situation looks a bit different. The loaded 
 sstables already have very small ranges! We just have to tidy up a bit and 
 everything should be OK. STCS ignores that completely and after a while we 
 have a bit less sstables but each of them covers the whole ring instead of 
 just a small part. I believe that in that case letting LCS do the job is a 
 better option that allowing STCS mix everything up before.
 Is there a way to disable STCS fallback? I'd like to test that scenario in 
 practice during our next bootstrap...
 Does Cassandra really have to put streamed sstables at L0? The only thing we 
 have to assure is that sstables at any given level do not overlap. If we 
 stream different regions from different nodes how can we get any overlaps?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7348) Modify system tcp keepalive settings on Windows install scripts

2014-06-11 Thread Philip Thompson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028785#comment-14028785
 ] 

Philip Thompson commented on CASSANDRA-7348:


Yes, +1. Sorry.

 Modify system tcp keepalive settings on Windows install scripts
 ---

 Key: CASSANDRA-7348
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7348
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
Priority: Minor
  Labels: Windows
 Fix For: 2.1.1

 Attachments: 7348-minor_update.txt, 7348_v1.txt, 7348_v2.txt


 Reference CASSANDRA-3569
 We need to change the powershell installation scripts to also update the 
 registry w/a 5 minute tcp keepalive timer timeout.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (CASSANDRA-6621) STCS fallback is not optimal when bootstrapping

[
https://issues.apache.org/jira/browse/CASSANDRA-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028820#comment-14028820
]

sankalp kohli edited comment on CASSANDRA-6621 at 6/12/14 4:49 AM:
---

What if we detect that, and include a couple of those high level sstables in
lower lever compactions until the higher level is empty or starts doing real
compactions?
Yes we can keep doing that till last level is the only level which is not full.
To minimize this, like I said we can do this:
Stream stables from the source by sorting them by level which will cause
streaming of stables in following order L1 to Lx and then finally L0.
I think it will help.

was (Author: kohlisankalp):
What if we detect that, and include a couple of those high level sstables in
lower lever compactions until the higher level is empty or starts doing real
compactions?
Yes we can keep doing that till last level is the only level which is not full.
To minimize this, like I said we can do this:
Stream stables from the source by sorting them by level which will cause
streaming of stables in following order L1 to Lx and then finally L0.
Sounds good?

STCS fallback is not optimal when bootstrapping
---

Key: CASSANDRA-6621
URL: https://issues.apache.org/jira/browse/CASSANDRA-6621
Project: Cassandra
Issue Type: Improvement
Reporter: Bartłomiej Romański
Priority: Minor

The initial discussion started in (closed) CASSANDRA-5371. I've rewritten my
last comment here...
After streaming (e.g. during boostrap) Cassandra places all sstables at L0.
At the end of the process we end up with huge number of sstables at the
lowest level.
Currently, Cassandra falls back to STCS until the number of sstables at L0
reaches the reasonable level (32 or something).
I'm not sure if falling back to STCS is the best way to handle this
particular situation. I've read the comment in the code and I'm aware why it
is a good thing to do if we have to many sstables at L0 as a result of too
many random inserts. We have a lot of sstables, each of them covers the whole
ring, there's simply no better option.
However, after the bootstrap situation looks a bit different. The loaded
sstables already have very small ranges! We just have to tidy up a bit and
everything should be OK. STCS ignores that completely and after a while we
have a bit less sstables but each of them covers the whole ring instead of
just a small part. I believe that in that case letting LCS do the job is a
better option that allowing STCS mix everything up before.
Is there a way to disable STCS fallback? I'd like to test that scenario in
practice during our next bootstrap...
Does Cassandra really have to put streamed sstables at L0? The only thing we
have to assure is that sstables at any given level do not overlap. If we
stream different regions from different nodes how can we get any overlaps?

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6621) STCS fallback is not optimal when bootstrapping

[
https://issues.apache.org/jira/browse/CASSANDRA-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028820#comment-14028820
]

sankalp kohli commented on CASSANDRA-6621:
--

STCS fallback is not optimal when bootstrapping
---

Key: CASSANDRA-6621
URL: https://issues.apache.org/jira/browse/CASSANDRA-6621
Project: Cassandra
Issue Type: Improvement
Reporter: Bartłomiej Romański
Priority: Minor

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7380) Native protocol needs keepalive, we should add it