date:20141009


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164905#comment-14164905
 ] 

kais commented on CASSANDRA-8081:
-

Sorry for my bad explanation, but it is indeed the length of the value that I 
mentioned, I also reproduced this error on 1.2.19 version 2.1.0, I know it is 
not a common case but the big worry is that this assertion block the 
flushwriter thread, and no longer flush gets done. Also any other mechanism 
like drain or repair are also blocked.

 AssertionError with 2ndary indexes 
 ---

 Key: CASSANDRA-8081
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8081
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: kais

 If you create a table with a column type text or blob and add a secondary 
 index on it. If you insert a value greater than 
 FBUtilities.MAX_UNSIGNED_SHORT, then flush, you get an assertion error
 {code}
 CREATE TABLE test_text (key text PRIMARY KEY, col text);
 CREATE INDEX test_text_col_idx ON test_text (col);
 {code}
 {code}
  INFO [FlushWriter:3] 2014-10-08 10:53:38,471 Memtable.java (line 331) 
 Writing Memtable-test_text.test_text_col_idx@849649959(15/150 serialized/live 
 bytes, 1 ops)
  INFO [FlushWriter:4] 2014-10-08 10:53:38,554 Memtable.java (line 331) 
 Writing Memtable-test_text@1448092010(100025/1000250 serialized/live bytes, 2 
 ops)
 ERROR [FlushWriter:3] 2014-10-08 10:53:38,554 CassandraDaemon.java (line 196) 
 Exception in thread Thread[FlushWriter:3,5,RMI Runtime]
 java.lang.AssertionError: 10
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithShortLength(ByteBufferUtil.java:342)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8081) AssertionError with 2ndary indexes

2014-10-09 Thread Rajanarayanan Thottuvaikkatumana (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

kais updated CASSANDRA-8081:

Reproduced In: 2.0.10, 1.2.19, 2.1.1  (was: 2.0.10)

 AssertionError with 2ndary indexes 
 ---

 Key: CASSANDRA-8081
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8081
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: kais

 If you create a table with a column type text or blob and add a secondary 
 index on it. If you insert a value greater than 
 FBUtilities.MAX_UNSIGNED_SHORT, then flush, you get an assertion error
 {code}
 CREATE TABLE test_text (key text PRIMARY KEY, col text);
 CREATE INDEX test_text_col_idx ON test_text (col);
 {code}
 {code}
  INFO [FlushWriter:3] 2014-10-08 10:53:38,471 Memtable.java (line 331) 
 Writing Memtable-test_text.test_text_col_idx@849649959(15/150 serialized/live 
 bytes, 1 ops)
  INFO [FlushWriter:4] 2014-10-08 10:53:38,554 Memtable.java (line 331) 
 Writing Memtable-test_text@1448092010(100025/1000250 serialized/live bytes, 2 
 ops)
 ERROR [FlushWriter:3] 2014-10-08 10:53:38,554 CassandraDaemon.java (line 196) 
 Exception in thread Thread[FlushWriter:3,5,RMI Runtime]
 java.lang.AssertionError: 10
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithShortLength(ByteBufferUtil.java:342)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8021) Improve cqlsh autocomplete for alter keyspace


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164907#comment-14164907
 ] 

Rajanarayanan Thottuvaikkatumana commented on CASSANDRA-8021:
-

The suggested changes have been done and the new patch is attached with this. 
The old patch has been removed to avoid confusion. This patch is for the 2.1.1 
branch and once this is certified, I will upload the patch for the other 
branch. Thanks Raj

 Improve cqlsh autocomplete for alter keyspace
 -

 Key: CASSANDRA-8021
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8021
 Project: Cassandra
  Issue Type: Improvement
Reporter: Philip Thompson
Assignee: Rajanarayanan Thottuvaikkatumana
Priority: Minor
  Labels: cqlsh, lhf
 Fix For: 2.0.11, 2.1.1

 Attachments: cassandra-2.1.1-8021.txt


 Cqlsh autocomplete stops giving suggestions for the statement
 {code}ALTER KEYSPACE k WITH REPLICATION { 'class' : 'SimpleStrategy', 
 'replication_factor' : 1'};{code} after the word WITH.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8081) AssertionError with 2ndary indexes


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164905#comment-14164905
 ] 

kais edited comment on CASSANDRA-8081 at 10/9/14 8:17 AM:
--

Sorry for my bad explanation, but it is indeed the length of the value that I 
mentioned, I also reproduced this error on 1.2.19 and 2.1.0 version, I know it 
is not a common case but the big worry is that this assertion block the 
flushwriter thread, and no longer flush gets done. Also any other mechanism 
like drain or repair are also blocked.


was (Author: kais):
Sorry for my bad explanation, but it is indeed the length of the value that I 
mentioned, I also reproduced this error on 1.2.19 version 2.1.0, I know it is 
not a common case but the big worry is that this assertion block the 
flushwriter thread, and no longer flush gets done. Also any other mechanism 
like drain or repair are also blocked.

 AssertionError with 2ndary indexes 
 ---

 Key: CASSANDRA-8081
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8081
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: kais

 If you create a table with a column type text or blob and add a secondary 
 index on it. If you insert a value greater than 
 FBUtilities.MAX_UNSIGNED_SHORT, then flush, you get an assertion error
 {code}
 CREATE TABLE test_text (key text PRIMARY KEY, col text);
 CREATE INDEX test_text_col_idx ON test_text (col);
 {code}
 {code}
  INFO [FlushWriter:3] 2014-10-08 10:53:38,471 Memtable.java (line 331) 
 Writing Memtable-test_text.test_text_col_idx@849649959(15/150 serialized/live 
 bytes, 1 ops)
  INFO [FlushWriter:4] 2014-10-08 10:53:38,554 Memtable.java (line 331) 
 Writing Memtable-test_text@1448092010(100025/1000250 serialized/live bytes, 2 
 ops)
 ERROR [FlushWriter:3] 2014-10-08 10:53:38,554 CassandraDaemon.java (line 196) 
 Exception in thread Thread[FlushWriter:3,5,RMI Runtime]
 java.lang.AssertionError: 10
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithShortLength(ByteBufferUtil.java:342)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8081) AssertionError with 2ndary indexes


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

kais updated CASSANDRA-8081:

Reproduced In: 2.1.0, 2.0.10, 1.2.19  (was: 1.2.19, 2.0.10, 2.1.1)

 AssertionError with 2ndary indexes 
 ---

 Key: CASSANDRA-8081
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8081
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: kais

 If you create a table with a column type text or blob and add a secondary 
 index on it. If you insert a value greater than 
 FBUtilities.MAX_UNSIGNED_SHORT, then flush, you get an assertion error
 {code}
 CREATE TABLE test_text (key text PRIMARY KEY, col text);
 CREATE INDEX test_text_col_idx ON test_text (col);
 {code}
 {code}
  INFO [FlushWriter:3] 2014-10-08 10:53:38,471 Memtable.java (line 331) 
 Writing Memtable-test_text.test_text_col_idx@849649959(15/150 serialized/live 
 bytes, 1 ops)
  INFO [FlushWriter:4] 2014-10-08 10:53:38,554 Memtable.java (line 331) 
 Writing Memtable-test_text@1448092010(100025/1000250 serialized/live bytes, 2 
 ops)
 ERROR [FlushWriter:3] 2014-10-08 10:53:38,554 CassandraDaemon.java (line 196) 
 Exception in thread Thread[FlushWriter:3,5,RMI Runtime]
 java.lang.AssertionError: 10
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithShortLength(ByteBufferUtil.java:342)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8081) AssertionError with 2ndary indexes


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164956#comment-14164956
 ] 

kais commented on CASSANDRA-8081:
-

I know it's different, but for mysql 

??BLOB and TEXT columns also can be indexed, but a prefix length must be 
given.??

{code}
CREATE INDEX test_text_col_idx ON test_text (col(10));
{code}

 AssertionError with 2ndary indexes 
 ---

 Key: CASSANDRA-8081
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8081
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: kais

 If you create a table with a column type text or blob and add a secondary 
 index on it. If you insert a value greater than 
 FBUtilities.MAX_UNSIGNED_SHORT, then flush, you get an assertion error
 {code}
 CREATE TABLE test_text (key text PRIMARY KEY, col text);
 CREATE INDEX test_text_col_idx ON test_text (col);
 {code}
 {code}
  INFO [FlushWriter:3] 2014-10-08 10:53:38,471 Memtable.java (line 331) 
 Writing Memtable-test_text.test_text_col_idx@849649959(15/150 serialized/live 
 bytes, 1 ops)
  INFO [FlushWriter:4] 2014-10-08 10:53:38,554 Memtable.java (line 331) 
 Writing Memtable-test_text@1448092010(100025/1000250 serialized/live bytes, 2 
 ops)
 ERROR [FlushWriter:3] 2014-10-08 10:53:38,554 CassandraDaemon.java (line 196) 
 Exception in thread Thread[FlushWriter:3,5,RMI Runtime]
 java.lang.AssertionError: 10
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithShortLength(ByteBufferUtil.java:342)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8081) AssertionError with 2ndary indexes


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164969#comment-14164969
 ] 

kais commented on CASSANDRA-8081:
-

If I may, I think the simplest solution would be if the length exceeds 65535 
characters: 

1 - index the first 65535 characters or nothing, the result would be the same 
2 - put a warning in the logs

 AssertionError with 2ndary indexes 
 ---

 Key: CASSANDRA-8081
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8081
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: kais

 If you create a table with a column type text or blob and add a secondary 
 index on it. If you insert a value greater than 
 FBUtilities.MAX_UNSIGNED_SHORT, then flush, you get an assertion error
 {code}
 CREATE TABLE test_text (key text PRIMARY KEY, col text);
 CREATE INDEX test_text_col_idx ON test_text (col);
 {code}
 {code}
  INFO [FlushWriter:3] 2014-10-08 10:53:38,471 Memtable.java (line 331) 
 Writing Memtable-test_text.test_text_col_idx@849649959(15/150 serialized/live 
 bytes, 1 ops)
  INFO [FlushWriter:4] 2014-10-08 10:53:38,554 Memtable.java (line 331) 
 Writing Memtable-test_text@1448092010(100025/1000250 serialized/live bytes, 2 
 ops)
 ERROR [FlushWriter:3] 2014-10-08 10:53:38,554 CassandraDaemon.java (line 196) 
 Exception in thread Thread[FlushWriter:3,5,RMI Runtime]
 java.lang.AssertionError: 10
   at 
 org.apache.cassandra.utils.ByteBufferUtil.writeWithShortLength(ByteBufferUtil.java:342)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8088) Notify subscribers when a column family is truncated

2014-10-09 Thread Sam Tunnicliffe (JIRA)

Sam Tunnicliffe created CASSANDRA-8088:
--

 Summary: Notify subscribers when a column family is truncated
 Key: CASSANDRA-8088
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8088
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Priority: Minor
 Fix For: 2.0.11, 2.1.1


Similarly to notifications regarding new or changed SSTable lists and memtable 
renewals, it can also be useful for interested classes to receive notifications 
when a truncate happens, or is about to happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8088) Notify subscribers when a column family is truncated

2014-10-09 Thread Sam Tunnicliffe (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-8088:
---
Attachment: 8808.txt

Patch adds a new INotification type which is delivered to registered consumers 
just before the actual truncation takes place


 Notify subscribers when a column family is truncated
 

 Key: CASSANDRA-8088
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8088
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Priority: Minor
 Fix For: 2.0.11, 2.1.1

 Attachments: 8808.txt


 Similarly to notifications regarding new or changed SSTable lists and 
 memtable renewals, it can also be useful for interested classes to receive 
 notifications when a truncate happens, or is about to happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8082) Support finer grained Modify CQL permissions


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165049#comment-14165049
 ] 

Aleksey Yeschenko commented on CASSANDRA-8082:
--

[~johnny15676] neither. But we'll consider it when rethinking Auth APIs for 
CASSANDRA-7653 in 3.0 and maybe add extra permissions for functions/types/users 
(CASSANDRA-7216).

 Support finer grained Modify CQL permissions
 

 Key: CASSANDRA-8082
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8082
 Project: Cassandra
  Issue Type: New Feature
Reporter: Johnny Miller

 Currently CQL permissions are grouped as:
 ALL   - All statements
 ALTER - ALTER KEYSPACE, ALTER TABLE, CREATE INDEX, DROP INDEX
 AUTHORIZE - GRANT, REVOKE
 CREATE - CREATE KEYSPACE, CREATE TABLE
 DROP - DROP KEYSPACE, DROP TABLE
 MODIFY - INSERT, DELETE, UPDATE, TRUNCATE
 SELECT -SELECT
 The MODIFY permission is too wide. There are plenty scenarios where a user 
 should not be to DELETE and TRUNCATE a table but should be able to INSERT and 
 UPDATE. 
 It would be great if Cassandra could either support defining permissions 
 dynamically or have additional finer grained MODIFY related permissions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-8061) tmplink files are not removed


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-8061:
-

Assignee: Ryan McGuire

Ryan, can your team reproduce?

 tmplink files are not removed
 -

 Key: CASSANDRA-8061
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8061
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Linux
Reporter: Gianluca Borello
Assignee: Ryan McGuire

 After installing 2.1.0, I'm experiencing a bunch of tmplink files that are 
 filling my disk. I found https://issues.apache.org/jira/browse/CASSANDRA-7803 
 and that is very similar, and I confirm it happens both on 2.1.0 as well as 
 from the latest commit on the cassandra-2.1 branch 
 (https://github.com/apache/cassandra/commit/aca80da38c3d86a40cc63d9a122f7d45258e4685
  from the cassandra-2.1)
 Even starting with a clean keyspace, after a few hours I get:
 $ sudo find /raid0 | grep tmplink | xargs du -hs
 2.7G  
 /raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Data.db
 13M   
 /raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Index.db
 1.8G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Data.db
 12M   
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Index.db
 5.2M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Index.db
 822M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Data.db
 7.3M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Index.db
 1.2G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Data.db
 6.7M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Index.db
 1.1G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Data.db
 11M   
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Index.db
 1.7G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Data.db
 812K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Index.db
 122M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Data.db
 744K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-739-Index.db
 660K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Index.db
 796K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Index.db
 137M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Data.db
 161M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Data.db
 139M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Data.db
 940K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Index.db
 936K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Index.db
 161M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Data.db
 672K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Index.db
 113M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Data.db
 116M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Data.db
 712K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Index.db
 127M

[jira] [Commented] (CASSANDRA-7256) Error when dropping keyspace.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165071#comment-14165071
 ] 

Jonathan Ellis commented on CASSANDRA-7256:
---

[~xedin] can you review the complete patch?  If you like it, perhaps it makes 
the most sense to standardize 2.0 and 2.1 on the same code.

 Error when dropping keyspace.  
 ---

 Key: CASSANDRA-7256
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7256
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: ubuntu 3 nodes (had 3 more in 2nd datacenter but removed 
 it)
Reporter: Steven Lowenthal
Assignee: Aleksey Yeschenko
  Labels: schema
 Fix For: 2.0.11, 2.1.1

 Attachments: 7256-complete.txt, 7256-minimal.txt


 created a 3 node datacenter  called existing.
 ran cassandra-stress:
 {{cassandra-stress -R NetworkTopologyStrategy -O existing:2 -d existing0 -n 
 200 -k}}
 Added a 2nd datacenter called new with 3 nodes started it with 
 {{auto_bootstrap: false}}
 {code}
 alter keyspace Keyspace1 with replication = 
 {'class':'NetworkTopologyStrategy','existing':2,'new':2};
 {code}
 I then discovered that cassandra-stress --operation=read failed with 
 LOCAL_QUORUM if a node was down in the local datacenter - this occured in 
 both, but should not have, so decided to try again.
 I shut down the new datacenter and removed all 3 nodes.  I then tried to drop 
 the Keyspace1 keyspace.  cqlsh disconnected, and the log shows the error 
 below.
 {code}
 ERROR [MigrationStage:1] 2014-05-16 23:57:03,085 CassandraDaemon.java (line 
 198) Exception in thread Thread[MigrationStage:1,5,main]
 java.lang.IllegalStateException: One row required, 0 found
 at org.apache.cassandra.cql3.UntypedResultSet.one(UntypedResultSet.java:53)
 at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:263)
 at org.apache.cassandra.db.DefsTables.mergeKeyspaces(DefsTables.java:227)
 at org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:182)
 at 
 org.apache.cassandra.service.MigrationManager$2.runMayThrow(MigrationManager.java:303)
 at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/6] git commit: Add sanity check of $JAVA on startup patch by Robert Stupp; reviewed by Michael Shuler for CASSANDRA-7676

Add sanity check of $JAVA on startup
patch by Robert Stupp; reviewed by Michael Shuler for CASSANDRA-7676


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fc0bb152
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fc0bb152
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fc0bb152

Branch: refs/heads/cassandra-2.1
Commit: fc0bb1520f6b74de834a5bd79440fa78bda913e8
Parents: 87cbcc0
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Oct 9 07:24:19 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Oct 9 07:24:19 2014 -0500

--
 CHANGES.txt   | 1 +
 bin/cassandra | 5 +
 2 files changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fc0bb152/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1bef399..7deea86 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * Add sanity check of $JAVA on startup (CASSANDRA-7676)
  * Schedule fat client schema pull on join (CASSANDRA-7993)
  * Don't reset nodes' versions when closing IncomingTcpConnections
(CASSANDRA-7734)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/fc0bb152/bin/cassandra
--
diff --git a/bin/cassandra b/bin/cassandra
index a950e0b..09fe013 100755
--- a/bin/cassandra
+++ b/bin/cassandra
@@ -98,6 +98,11 @@ else
 JAVA=java
 fi
 
+if [ -z $JAVA ] ; then
+echo Unable to find java executable. Check JAVA_HOME and PATH environment 
variables.  /dev/stderr
+exit 1;
+fi
+
 # If numactl is available, use it. For Cassandra, the priority is to
 # avoid disk I/O. Even for the purpose of CPU efficiency, we don't
 # really have CPU-data affinity anyway. Also, empirically test that numactl

[4/6] git commit: merge from 2.0

merge from 2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f92a457a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f92a457a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f92a457a

Branch: refs/heads/trunk
Commit: f92a457a3046fec099c232a64863d6f6a7cc78ea
Parents: dacddd2 fc0bb15
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Oct 9 07:25:51 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Oct 9 07:25:51 2014 -0500

--
 CHANGES.txt   | 1 +
 bin/cassandra | 5 +
 2 files changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f92a457a/CHANGES.txt
--
diff --cc CHANGES.txt
index 159979f,7deea86..f2fc9ae
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,71 -1,5 +1,72 @@@
 -2.0.11:
 +2.1.1
 + * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
 + * Don't duplicate sstables smaller than split size when using
 +   the sstablesplitter tool (CASSANDRA-7616)
 + * Avoid re-parsing already prepared statements (CASSANDRA-7923)
 + * Fix some Thrift slice deletions and updates of COMPACT STORAGE
 +   tables with some clustering columns omitted (CASSANDRA-7990)
 + * Fix filtering for CONTAINS on sets (CASSANDRA-8033)
 + * Properly track added size (CASSANDRA-7239)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Fix Assertion error on RangeTombstoneList diff (CASSANDRA-8013)
 + * Release references to overlapping sstables during compaction 
(CASSANDRA-7819)
 + * Send notification when opening compaction results early (CASSANDRA-8034)
 + * Make native server start block until properly bound (CASSANDRA-7885)
 + * (cqlsh) Fix IPv6 support (CASSANDRA-7988)
 + * Ignore fat clients when checking for endpoint collision (CASSANDRA-7939)
 + * Make sstablerepairedset take a list of files (CASSANDRA-7995)
 + * (cqlsh) Tab completeion for indexes on map keys (CASSANDRA-7972)
 + * (cqlsh) Fix UDT field selection in select clause (CASSANDRA-7891)
 + * Fix resource leak in event of corrupt sstable
 + * (cqlsh) Add command line option for cqlshrc file path (CASSANDRA-7131)
 + * Provide visibility into prepared statements churn (CASSANDRA-7921, 
CASSANDRA-7930)
 + * Invalidate prepared statements when their keyspace or table is
 +   dropped (CASSANDRA-7566)
 + * cassandra-stress: fix support for NetworkTopologyStrategy (CASSANDRA-7945)
 + * Fix saving caches when a table is dropped (CASSANDRA-7784)
 + * Add better error checking of new stress profile (CASSANDRA-7716)
 + * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom 
(CASSANDRA-7934)
 + * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
 + * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
 + * GCInspector more closely tracks GC; cassandra-stress and nodetool report 
it (CASSANDRA-7916)
 + * nodetool won't output bogus ownership info without a keyspace 
(CASSANDRA-7173)
 + * Add human readable option to nodetool commands (CASSANDRA-5433)
 + * Don't try to set repairedAt on old sstables (CASSANDRA-7913)
 + * Add metrics for tracking PreparedStatement use (CASSANDRA-7719)
 + * (cqlsh) tab-completion for triggers (CASSANDRA-7824)
 + * (cqlsh) Support for query paging (CASSANDRA-7514)
 + * (cqlsh) Show progress of COPY operations (CASSANDRA-7789)
 + * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
 + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
 + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
 + * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
 + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
 + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
 +   output (CASSANDRA-7659)
 + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
 + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
 + * Support list index operations with conditions (CASSANDRA-7499)
 + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
 + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
 + * (cqlsh) Error when tracing query (CASSANDRA-7613)
 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569)
 + * SSTableExport uses correct validator to create string representation of 
partition
 +   keys (CASSANDRA-7498)
 + * Avoid NPEs when receiving type changes for an unknown keyspace 
(CASSANDRA-7689)
 + * Add support for custom 2i validation (CASSANDRA-7575)
 + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454)
 + * Add listen_interface and rpc_interface options (CASSANDRA-7417)
 + * Improve schema merge performance (CASSANDRA-7444)
 + * Adjust MT depth based on # of partition

[3/6] git commit: Add sanity check of $JAVA on startup patch by Robert Stupp; reviewed by Michael Shuler for CASSANDRA-7676

Add sanity check of $JAVA on startup
patch by Robert Stupp; reviewed by Michael Shuler for CASSANDRA-7676


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fc0bb152
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fc0bb152
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fc0bb152

Branch: refs/heads/trunk
Commit: fc0bb1520f6b74de834a5bd79440fa78bda913e8
Parents: 87cbcc0
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Oct 9 07:24:19 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Oct 9 07:24:19 2014 -0500

--
 CHANGES.txt   | 1 +
 bin/cassandra | 5 +
 2 files changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fc0bb152/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1bef399..7deea86 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * Add sanity check of $JAVA on startup (CASSANDRA-7676)
  * Schedule fat client schema pull on join (CASSANDRA-7993)
  * Don't reset nodes' versions when closing IncomingTcpConnections
(CASSANDRA-7734)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/fc0bb152/bin/cassandra
--
diff --git a/bin/cassandra b/bin/cassandra
index a950e0b..09fe013 100755
--- a/bin/cassandra
+++ b/bin/cassandra
@@ -98,6 +98,11 @@ else
 JAVA=java
 fi
 
+if [ -z $JAVA ] ; then
+echo Unable to find java executable. Check JAVA_HOME and PATH environment 
variables.  /dev/stderr
+exit 1;
+fi
+
 # If numactl is available, use it. For Cassandra, the priority is to
 # avoid disk I/O. Even for the purpose of CPU efficiency, we don't
 # really have CPU-data affinity anyway. Also, empirically test that numactl

[5/6] git commit: merge from 2.0

merge from 2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f92a457a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f92a457a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f92a457a

Branch: refs/heads/cassandra-2.1
Commit: f92a457a3046fec099c232a64863d6f6a7cc78ea
Parents: dacddd2 fc0bb15
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Oct 9 07:25:51 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Oct 9 07:25:51 2014 -0500

--
 CHANGES.txt   | 1 +
 bin/cassandra | 5 +
 2 files changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f92a457a/CHANGES.txt
--
diff --cc CHANGES.txt
index 159979f,7deea86..f2fc9ae
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,71 -1,5 +1,72 @@@
 -2.0.11:
 +2.1.1
 + * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
 + * Don't duplicate sstables smaller than split size when using
 +   the sstablesplitter tool (CASSANDRA-7616)
 + * Avoid re-parsing already prepared statements (CASSANDRA-7923)
 + * Fix some Thrift slice deletions and updates of COMPACT STORAGE
 +   tables with some clustering columns omitted (CASSANDRA-7990)
 + * Fix filtering for CONTAINS on sets (CASSANDRA-8033)
 + * Properly track added size (CASSANDRA-7239)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Fix Assertion error on RangeTombstoneList diff (CASSANDRA-8013)
 + * Release references to overlapping sstables during compaction 
(CASSANDRA-7819)
 + * Send notification when opening compaction results early (CASSANDRA-8034)
 + * Make native server start block until properly bound (CASSANDRA-7885)
 + * (cqlsh) Fix IPv6 support (CASSANDRA-7988)
 + * Ignore fat clients when checking for endpoint collision (CASSANDRA-7939)
 + * Make sstablerepairedset take a list of files (CASSANDRA-7995)
 + * (cqlsh) Tab completeion for indexes on map keys (CASSANDRA-7972)
 + * (cqlsh) Fix UDT field selection in select clause (CASSANDRA-7891)
 + * Fix resource leak in event of corrupt sstable
 + * (cqlsh) Add command line option for cqlshrc file path (CASSANDRA-7131)
 + * Provide visibility into prepared statements churn (CASSANDRA-7921, 
CASSANDRA-7930)
 + * Invalidate prepared statements when their keyspace or table is
 +   dropped (CASSANDRA-7566)
 + * cassandra-stress: fix support for NetworkTopologyStrategy (CASSANDRA-7945)
 + * Fix saving caches when a table is dropped (CASSANDRA-7784)
 + * Add better error checking of new stress profile (CASSANDRA-7716)
 + * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom 
(CASSANDRA-7934)
 + * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
 + * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
 + * GCInspector more closely tracks GC; cassandra-stress and nodetool report 
it (CASSANDRA-7916)
 + * nodetool won't output bogus ownership info without a keyspace 
(CASSANDRA-7173)
 + * Add human readable option to nodetool commands (CASSANDRA-5433)
 + * Don't try to set repairedAt on old sstables (CASSANDRA-7913)
 + * Add metrics for tracking PreparedStatement use (CASSANDRA-7719)
 + * (cqlsh) tab-completion for triggers (CASSANDRA-7824)
 + * (cqlsh) Support for query paging (CASSANDRA-7514)
 + * (cqlsh) Show progress of COPY operations (CASSANDRA-7789)
 + * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
 + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
 + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
 + * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
 + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
 + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
 +   output (CASSANDRA-7659)
 + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
 + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
 + * Support list index operations with conditions (CASSANDRA-7499)
 + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
 + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
 + * (cqlsh) Error when tracing query (CASSANDRA-7613)
 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569)
 + * SSTableExport uses correct validator to create string representation of 
partition
 +   keys (CASSANDRA-7498)
 + * Avoid NPEs when receiving type changes for an unknown keyspace 
(CASSANDRA-7689)
 + * Add support for custom 2i validation (CASSANDRA-7575)
 + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454)
 + * Add listen_interface and rpc_interface options (CASSANDRA-7417)
 + * Improve schema merge performance (CASSANDRA-7444)
 + * Adjust MT depth based on # of

[1/6] git commit: Add sanity check of $JAVA on startup patch by Robert Stupp; reviewed by Michael Shuler for CASSANDRA-7676

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 87cbcc0fc - fc0bb1520
  refs/heads/cassandra-2.1 dacddd266 - f92a457a3
  refs/heads/trunk a848b08ce - 2fe5b9a82


Add sanity check of $JAVA on startup
patch by Robert Stupp; reviewed by Michael Shuler for CASSANDRA-7676


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fc0bb152
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fc0bb152
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fc0bb152

Branch: refs/heads/cassandra-2.0
Commit: fc0bb1520f6b74de834a5bd79440fa78bda913e8
Parents: 87cbcc0
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Oct 9 07:24:19 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Oct 9 07:24:19 2014 -0500

--
 CHANGES.txt   | 1 +
 bin/cassandra | 5 +
 2 files changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fc0bb152/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1bef399..7deea86 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * Add sanity check of $JAVA on startup (CASSANDRA-7676)
  * Schedule fat client schema pull on join (CASSANDRA-7993)
  * Don't reset nodes' versions when closing IncomingTcpConnections
(CASSANDRA-7734)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/fc0bb152/bin/cassandra
--
diff --git a/bin/cassandra b/bin/cassandra
index a950e0b..09fe013 100755
--- a/bin/cassandra
+++ b/bin/cassandra
@@ -98,6 +98,11 @@ else
 JAVA=java
 fi
 
+if [ -z $JAVA ] ; then
+echo Unable to find java executable. Check JAVA_HOME and PATH environment 
variables.  /dev/stderr
+exit 1;
+fi
+
 # If numactl is available, use it. For Cassandra, the priority is to
 # avoid disk I/O. Even for the purpose of CPU efficiency, we don't
 # really have CPU-data affinity anyway. Also, empirically test that numactl

[6/6] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2fe5b9a8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2fe5b9a8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2fe5b9a8

Branch: refs/heads/trunk
Commit: 2fe5b9a82dde6ce35128e1cf258d68128b1769c6
Parents: a848b08 f92a457
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Oct 9 07:26:03 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Oct 9 07:26:03 2014 -0500

--
 CHANGES.txt   | 1 +
 bin/cassandra | 5 +
 2 files changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2fe5b9a8/CHANGES.txt
--

[jira] [Commented] (CASSANDRA-7676) bin/cassandra should complain if $JAVA is empty or not an executable


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165077#comment-14165077
 ] 

Jonathan Ellis commented on CASSANDRA-7676:
---

committed

 bin/cassandra should complain if $JAVA is empty or not an executable
 

 Key: CASSANDRA-7676
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7676
 Project: Cassandra
  Issue Type: Improvement
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Trivial
  Labels: qa-resolved
 Fix For: 2.0.11, 2.1.1

 Attachments: 7676v2.txt, java_not_found-fix.txt


 If JAVA_HOME points to a non-existing directory, bin/cassandra just complains 
 with {{[node1 ERROR] /Users/snazy/.ccm/test/node1/bin/cassandra: line 151: 
 exec: : not found}} (via ccm).
 Patch checks if {{$JAVA}} is empty or does not point to an executable and 
 prints appropriate error message.
 (Patch against trunk)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6602) Compaction improvements to optimize time series data


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165082#comment-14165082
 ] 

Jonathan Ellis commented on CASSANDRA-6602:
---

I would have gone even further and said that since it's just a new strategy 
I'd be fine with letting people experiment even in 2.0.

 Compaction improvements to optimize time series data
 

 Key: CASSANDRA-6602
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6602
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Tupshin Harper
Assignee: Björn Hegerfors
  Labels: compaction, performance
 Fix For: 3.0

 Attachments: 1 week.txt, 8 weeks.txt, STCS 16 hours.txt, 
 TimestampViewer.java, 
 cassandra-2.0-CASSANDRA-6602-DateTieredCompactionStrategy.txt, 
 cassandra-2.0-CASSANDRA-6602-DateTieredCompactionStrategy_v2.txt, 
 cassandra-2.0-CASSANDRA-6602-DateTieredCompactionStrategy_v3.txt


 There are some unique characteristics of many/most time series use cases that 
 both provide challenges, as well as provide unique opportunities for 
 optimizations.
 One of the major challenges is in compaction. The existing compaction 
 strategies will tend to re-compact data on disk at least a few times over the 
 lifespan of each data point, greatly increasing the cpu and IO costs of that 
 write.
 Compaction exists to
 1) ensure that there aren't too many files on disk
 2) ensure that data that should be contiguous (part of the same partition) is 
 laid out contiguously
 3) deleting data due to ttls or tombstones
 The special characteristics of time series data allow us to optimize away all 
 three.
 Time series data
 1) tends to be delivered in time order, with relatively constrained exceptions
 2) often has a pre-determined and fixed expiration date
 3) Never gets deleted prior to TTL
 4) Has relatively predictable ingestion rates
 Note that I filed CASSANDRA-5561 and this ticket potentially replaces or 
 lowers the need for it. In that ticket, jbellis reasonably asks, how that 
 compaction strategy is better than disabling compaction.
 Taking that to heart, here is a compaction-strategy-less approach that could 
 be extremely efficient for time-series use cases that follow the above 
 pattern.
 (For context, I'm thinking of an example use case involving lots of streams 
 of time-series data with a 5GB per day ingestion rate, and a 1000 day 
 retention with TTL, resulting in an eventual steady state of 5TB per node)
 1) You have an extremely large memtable (preferably off heap, if/when doable) 
 for the table, and that memtable is sized to be able to hold a lengthy window 
 of time. A typical period might be one day. At the end of that period, you 
 flush the contents of the memtable to an sstable and move to the next one. 
 This is basically identical to current behaviour, but with thresholds 
 adjusted so that you can ensure flushing at predictable intervals. (Open 
 question is whether predictable intervals is actually necessary, or whether 
 just waiting until the huge memtable is nearly full is sufficient)
 2) Combine the behaviour with CASSANDRA-5228 so that sstables will be 
 efficiently dropped once all of the columns have. (Another side note, it 
 might be valuable to have a modified version of CASSANDRA-3974 that doesn't 
 bother storing per-column TTL since it is required that all columns have the 
 same TTL)
 3) Be able to mark column families as read/write only (no explicit deletes), 
 so no tombstones.
 4) Optionally add back an additional type of delete that would delete all 
 data earlier than a particular timestamp, resulting in immediate dropping of 
 obsoleted sstables.
 The result is that for in-order delivered data, Every cell will be laid out 
 optimally on disk on the first pass, and over the course of 1000 days and 5TB 
 of data, there will only be 1000 5GB sstables, so the number of filehandles 
 will be reasonable.
 For exceptions (out-of-order delivery), most cases will be caught by the 
 extended (24 hour+) memtable flush times and merged correctly automatically. 
 For those that were slightly askew at flush time, or were delivered so far 
 out of order that they go in the wrong sstable, there is relatively low 
 overhead to reading from two sstables for a time slice, instead of one, and 
 that overhead would be incurred relatively rarely unless out-of-order 
 delivery was the common case, in which case, this strategy should not be used.
 Another possible optimization to address out-of-order would be to maintain 
 more than one time-centric memtables in memory at a time (e.g. two 12 hour 
 ones), and then you always insert into whichever one of the two owns the 
 appropriate range of time. By delaying flushing the ahead one until we are

git commit: Notify DT subscribers when a column family is truncated

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 fc0bb1520 - cced4ba15


Notify DT subscribers when a column family is truncated

patch by Sam Tunnicliffe; reviewed by Aleksey Yeschenko for
CASSANDRA-8088


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cced4ba1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cced4ba1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cced4ba1

Branch: refs/heads/cassandra-2.0
Commit: cced4ba15bc5db7395f5981d01393757c3e0332e
Parents: fc0bb15
Author: Sam Tunnicliffe s...@beobal.com
Authored: Thu Oct 9 15:31:28 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Oct 9 15:33:05 2014 +0300

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  |  2 ++
 .../org/apache/cassandra/db/DataTracker.java|  7 +
 .../notifications/TruncationNotification.java   | 32 
 4 files changed, 42 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cced4ba1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7deea86..1c2231d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * Notify DT subscribers when a column family is truncated (CASSANDRA-8088)
  * Add sanity check of $JAVA on startup (CASSANDRA-7676)
  * Schedule fat client schema pull on join (CASSANDRA-7993)
  * Don't reset nodes' versions when closing IncomingTcpConnections

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cced4ba1/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 3da44de..d0ff951 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -2035,6 +2035,8 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 logger.debug(Discarding sstable data for truncated CF + 
indexes);
 
 final long truncatedAt = System.currentTimeMillis();
+data.notifyTruncated(truncatedAt);
+
 if (DatabaseDescriptor.isAutoSnapshot())
 snapshot(Keyspace.getTimestampedSnapshotName(name));
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cced4ba1/src/java/org/apache/cassandra/db/DataTracker.java
--
diff --git a/src/java/org/apache/cassandra/db/DataTracker.java 
b/src/java/org/apache/cassandra/db/DataTracker.java
index d41181b..f6d2c75 100644
--- a/src/java/org/apache/cassandra/db/DataTracker.java
+++ b/src/java/org/apache/cassandra/db/DataTracker.java
@@ -475,6 +475,13 @@ public class DataTracker
 subscriber.handleNotification(notification, this);
 }
 
+public void notifyTruncated(long truncatedAt)
+{
+INotification notification = new TruncationNotification(truncatedAt);
+for (INotificationConsumer subscriber : subscribers)
+subscriber.handleNotification(notification, this);
+}
+
 public void subscribe(INotificationConsumer consumer)
 {
 subscribers.add(consumer);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cced4ba1/src/java/org/apache/cassandra/notifications/TruncationNotification.java
--
diff --git 
a/src/java/org/apache/cassandra/notifications/TruncationNotification.java 
b/src/java/org/apache/cassandra/notifications/TruncationNotification.java
new file mode 100644
index 000..345dd17
--- /dev/null
+++ b/src/java/org/apache/cassandra/notifications/TruncationNotification.java
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.notifications;
+
+/**
+ * Fired during truncate,

[jira] [Commented] (CASSANDRA-8063) Consider removing UDF-as-class functionality


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165084#comment-14165084
 ] 

Jonathan Ellis commented on CASSANDRA-8063:
---

+1

 Consider removing UDF-as-class functionality
 

 Key: CASSANDRA-8063
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8063
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
  Labels: cql
 Fix For: 3.0


 We've introduce 2 ways to provide (scalar) UDFs: either by providing a 
 class+method name (and assuming C* can find such class and method in the 
 classpath), or by providing the body of the function directly in the {{CREATE 
 FUNCTION}} statement (with such body being able to be in either java or some 
 variety of scripting languages).
 I submit that we remove the first option: the declaration of functions from a 
 class+method name. I was the first to insist on adding it, but in hindsight I 
 think it adds more complexity/confusion than anything else. More 
 specifically, I think the UDF-as-class option is always inferior to 
 CASSANDRA-7562 because:
 # it's more error prone. You have to manually deploy the class containing the 
 function to every C* node (and make sure it is in the classpath). It's way 
 too easy to end up with some node not having the function due to simple 
 operator error.
 # it's not faster. In fact, post-CASSANDRA-7924, the UDF-as-java-source is 
 probably faster since it doesn't involve reflection while the UDF-as-class 
 option does.
 Overall, I think removing the UDF-as-class will simplify things without 
 really losing anything (of course, we can re-evaluate this in the future if 
 new arguments arises, but it's easier to add than to remove).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a33a9ad
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a33a9ad
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a33a9ad

Branch: refs/heads/trunk
Commit: 5a33a9ad23d2863b08f3ee5c3497cf96f0448d48
Parents: f92a457 cced4ba
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Oct 9 15:35:17 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Oct 9 15:35:17 2014 +0300

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  |  2 ++
 .../org/apache/cassandra/db/DataTracker.java|  7 +
 .../notifications/TruncationNotification.java   | 32 
 4 files changed, 42 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a33a9ad/CHANGES.txt
--
diff --cc CHANGES.txt
index f2fc9ae,1c2231d..054870b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,71 -1,5 +1,72 @@@
 -2.0.11:
 +2.1.1
 + * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
 + * Don't duplicate sstables smaller than split size when using
 +   the sstablesplitter tool (CASSANDRA-7616)
 + * Avoid re-parsing already prepared statements (CASSANDRA-7923)
 + * Fix some Thrift slice deletions and updates of COMPACT STORAGE
 +   tables with some clustering columns omitted (CASSANDRA-7990)
 + * Fix filtering for CONTAINS on sets (CASSANDRA-8033)
 + * Properly track added size (CASSANDRA-7239)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Fix Assertion error on RangeTombstoneList diff (CASSANDRA-8013)
 + * Release references to overlapping sstables during compaction 
(CASSANDRA-7819)
 + * Send notification when opening compaction results early (CASSANDRA-8034)
 + * Make native server start block until properly bound (CASSANDRA-7885)
 + * (cqlsh) Fix IPv6 support (CASSANDRA-7988)
 + * Ignore fat clients when checking for endpoint collision (CASSANDRA-7939)
 + * Make sstablerepairedset take a list of files (CASSANDRA-7995)
 + * (cqlsh) Tab completeion for indexes on map keys (CASSANDRA-7972)
 + * (cqlsh) Fix UDT field selection in select clause (CASSANDRA-7891)
 + * Fix resource leak in event of corrupt sstable
 + * (cqlsh) Add command line option for cqlshrc file path (CASSANDRA-7131)
 + * Provide visibility into prepared statements churn (CASSANDRA-7921, 
CASSANDRA-7930)
 + * Invalidate prepared statements when their keyspace or table is
 +   dropped (CASSANDRA-7566)
 + * cassandra-stress: fix support for NetworkTopologyStrategy (CASSANDRA-7945)
 + * Fix saving caches when a table is dropped (CASSANDRA-7784)
 + * Add better error checking of new stress profile (CASSANDRA-7716)
 + * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom 
(CASSANDRA-7934)
 + * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
 + * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
 + * GCInspector more closely tracks GC; cassandra-stress and nodetool report 
it (CASSANDRA-7916)
 + * nodetool won't output bogus ownership info without a keyspace 
(CASSANDRA-7173)
 + * Add human readable option to nodetool commands (CASSANDRA-5433)
 + * Don't try to set repairedAt on old sstables (CASSANDRA-7913)
 + * Add metrics for tracking PreparedStatement use (CASSANDRA-7719)
 + * (cqlsh) tab-completion for triggers (CASSANDRA-7824)
 + * (cqlsh) Support for query paging (CASSANDRA-7514)
 + * (cqlsh) Show progress of COPY operations (CASSANDRA-7789)
 + * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
 + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
 + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
 + * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
 + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
 + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
 +   output (CASSANDRA-7659)
 + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
 + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
 + * Support list index operations with conditions (CASSANDRA-7499)
 + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
 + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
 + * (cqlsh) Error when tracing query (CASSANDRA-7613)
 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569)
 + * SSTableExport uses correct validator to create string representation of 
partition
 +   keys (CASSANDRA-7498)
 + * Avoid NPEs when receiving type changes for an unknown keyspace 
(CASSANDRA-7689)
 + * Add support for

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a9bad890
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a9bad890
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a9bad890

Branch: refs/heads/trunk
Commit: a9bad890fede3fee534c9fd94340d960caf86b1c
Parents: 2fe5b9a 5a33a9a
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Oct 9 15:35:37 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Oct 9 15:35:37 2014 +0300

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  |  2 ++
 .../org/apache/cassandra/db/DataTracker.java|  7 +
 .../notifications/TruncationNotification.java   | 32 
 4 files changed, 42 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a9bad890/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a9bad890/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--

[1/3] git commit: Notify DT subscribers when a column family is truncated

Repository: cassandra
Updated Branches:
  refs/heads/trunk 2fe5b9a82 - a9bad890f


Notify DT subscribers when a column family is truncated

patch by Sam Tunnicliffe; reviewed by Aleksey Yeschenko for
CASSANDRA-8088


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cced4ba1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cced4ba1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cced4ba1

Branch: refs/heads/trunk
Commit: cced4ba15bc5db7395f5981d01393757c3e0332e
Parents: fc0bb15
Author: Sam Tunnicliffe s...@beobal.com
Authored: Thu Oct 9 15:31:28 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Oct 9 15:33:05 2014 +0300

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  |  2 ++
 .../org/apache/cassandra/db/DataTracker.java|  7 +
 .../notifications/TruncationNotification.java   | 32 
 4 files changed, 42 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cced4ba1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7deea86..1c2231d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * Notify DT subscribers when a column family is truncated (CASSANDRA-8088)
  * Add sanity check of $JAVA on startup (CASSANDRA-7676)
  * Schedule fat client schema pull on join (CASSANDRA-7993)
  * Don't reset nodes' versions when closing IncomingTcpConnections

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cced4ba1/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 3da44de..d0ff951 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -2035,6 +2035,8 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 logger.debug(Discarding sstable data for truncated CF + 
indexes);
 
 final long truncatedAt = System.currentTimeMillis();
+data.notifyTruncated(truncatedAt);
+
 if (DatabaseDescriptor.isAutoSnapshot())
 snapshot(Keyspace.getTimestampedSnapshotName(name));
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cced4ba1/src/java/org/apache/cassandra/db/DataTracker.java
--
diff --git a/src/java/org/apache/cassandra/db/DataTracker.java 
b/src/java/org/apache/cassandra/db/DataTracker.java
index d41181b..f6d2c75 100644
--- a/src/java/org/apache/cassandra/db/DataTracker.java
+++ b/src/java/org/apache/cassandra/db/DataTracker.java
@@ -475,6 +475,13 @@ public class DataTracker
 subscriber.handleNotification(notification, this);
 }
 
+public void notifyTruncated(long truncatedAt)
+{
+INotification notification = new TruncationNotification(truncatedAt);
+for (INotificationConsumer subscriber : subscribers)
+subscriber.handleNotification(notification, this);
+}
+
 public void subscribe(INotificationConsumer consumer)
 {
 subscribers.add(consumer);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cced4ba1/src/java/org/apache/cassandra/notifications/TruncationNotification.java
--
diff --git 
a/src/java/org/apache/cassandra/notifications/TruncationNotification.java 
b/src/java/org/apache/cassandra/notifications/TruncationNotification.java
new file mode 100644
index 000..345dd17
--- /dev/null
+++ b/src/java/org/apache/cassandra/notifications/TruncationNotification.java
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.notifications;
+
+/**
+ * Fired during truncate, after the memtable

[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a33a9ad
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a33a9ad
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a33a9ad

Branch: refs/heads/cassandra-2.1
Commit: 5a33a9ad23d2863b08f3ee5c3497cf96f0448d48
Parents: f92a457 cced4ba
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Oct 9 15:35:17 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Oct 9 15:35:17 2014 +0300

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  |  2 ++
 .../org/apache/cassandra/db/DataTracker.java|  7 +
 .../notifications/TruncationNotification.java   | 32 
 4 files changed, 42 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a33a9ad/CHANGES.txt
--
diff --cc CHANGES.txt
index f2fc9ae,1c2231d..054870b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,71 -1,5 +1,72 @@@
 -2.0.11:
 +2.1.1
 + * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
 + * Don't duplicate sstables smaller than split size when using
 +   the sstablesplitter tool (CASSANDRA-7616)
 + * Avoid re-parsing already prepared statements (CASSANDRA-7923)
 + * Fix some Thrift slice deletions and updates of COMPACT STORAGE
 +   tables with some clustering columns omitted (CASSANDRA-7990)
 + * Fix filtering for CONTAINS on sets (CASSANDRA-8033)
 + * Properly track added size (CASSANDRA-7239)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Fix Assertion error on RangeTombstoneList diff (CASSANDRA-8013)
 + * Release references to overlapping sstables during compaction 
(CASSANDRA-7819)
 + * Send notification when opening compaction results early (CASSANDRA-8034)
 + * Make native server start block until properly bound (CASSANDRA-7885)
 + * (cqlsh) Fix IPv6 support (CASSANDRA-7988)
 + * Ignore fat clients when checking for endpoint collision (CASSANDRA-7939)
 + * Make sstablerepairedset take a list of files (CASSANDRA-7995)
 + * (cqlsh) Tab completeion for indexes on map keys (CASSANDRA-7972)
 + * (cqlsh) Fix UDT field selection in select clause (CASSANDRA-7891)
 + * Fix resource leak in event of corrupt sstable
 + * (cqlsh) Add command line option for cqlshrc file path (CASSANDRA-7131)
 + * Provide visibility into prepared statements churn (CASSANDRA-7921, 
CASSANDRA-7930)
 + * Invalidate prepared statements when their keyspace or table is
 +   dropped (CASSANDRA-7566)
 + * cassandra-stress: fix support for NetworkTopologyStrategy (CASSANDRA-7945)
 + * Fix saving caches when a table is dropped (CASSANDRA-7784)
 + * Add better error checking of new stress profile (CASSANDRA-7716)
 + * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom 
(CASSANDRA-7934)
 + * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
 + * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
 + * GCInspector more closely tracks GC; cassandra-stress and nodetool report 
it (CASSANDRA-7916)
 + * nodetool won't output bogus ownership info without a keyspace 
(CASSANDRA-7173)
 + * Add human readable option to nodetool commands (CASSANDRA-5433)
 + * Don't try to set repairedAt on old sstables (CASSANDRA-7913)
 + * Add metrics for tracking PreparedStatement use (CASSANDRA-7719)
 + * (cqlsh) tab-completion for triggers (CASSANDRA-7824)
 + * (cqlsh) Support for query paging (CASSANDRA-7514)
 + * (cqlsh) Show progress of COPY operations (CASSANDRA-7789)
 + * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
 + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
 + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
 + * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
 + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
 + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
 +   output (CASSANDRA-7659)
 + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
 + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
 + * Support list index operations with conditions (CASSANDRA-7499)
 + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
 + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
 + * (cqlsh) Error when tracing query (CASSANDRA-7613)
 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569)
 + * SSTableExport uses correct validator to create string representation of 
partition
 +   keys (CASSANDRA-7498)
 + * Avoid NPEs when receiving type changes for an unknown keyspace 
(CASSANDRA-7689)
 + * Add

[1/2] git commit: Notify DT subscribers when a column family is truncated

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 f92a457a3 - 5a33a9ad2


Notify DT subscribers when a column family is truncated

patch by Sam Tunnicliffe; reviewed by Aleksey Yeschenko for
CASSANDRA-8088


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cced4ba1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cced4ba1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cced4ba1

Branch: refs/heads/cassandra-2.1
Commit: cced4ba15bc5db7395f5981d01393757c3e0332e
Parents: fc0bb15
Author: Sam Tunnicliffe s...@beobal.com
Authored: Thu Oct 9 15:31:28 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Oct 9 15:33:05 2014 +0300

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  |  2 ++
 .../org/apache/cassandra/db/DataTracker.java|  7 +
 .../notifications/TruncationNotification.java   | 32 
 4 files changed, 42 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cced4ba1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7deea86..1c2231d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.11:
+ * Notify DT subscribers when a column family is truncated (CASSANDRA-8088)
  * Add sanity check of $JAVA on startup (CASSANDRA-7676)
  * Schedule fat client schema pull on join (CASSANDRA-7993)
  * Don't reset nodes' versions when closing IncomingTcpConnections

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cced4ba1/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 3da44de..d0ff951 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -2035,6 +2035,8 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 logger.debug(Discarding sstable data for truncated CF + 
indexes);
 
 final long truncatedAt = System.currentTimeMillis();
+data.notifyTruncated(truncatedAt);
+
 if (DatabaseDescriptor.isAutoSnapshot())
 snapshot(Keyspace.getTimestampedSnapshotName(name));
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cced4ba1/src/java/org/apache/cassandra/db/DataTracker.java
--
diff --git a/src/java/org/apache/cassandra/db/DataTracker.java 
b/src/java/org/apache/cassandra/db/DataTracker.java
index d41181b..f6d2c75 100644
--- a/src/java/org/apache/cassandra/db/DataTracker.java
+++ b/src/java/org/apache/cassandra/db/DataTracker.java
@@ -475,6 +475,13 @@ public class DataTracker
 subscriber.handleNotification(notification, this);
 }
 
+public void notifyTruncated(long truncatedAt)
+{
+INotification notification = new TruncationNotification(truncatedAt);
+for (INotificationConsumer subscriber : subscribers)
+subscriber.handleNotification(notification, this);
+}
+
 public void subscribe(INotificationConsumer consumer)
 {
 subscribers.add(consumer);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cced4ba1/src/java/org/apache/cassandra/notifications/TruncationNotification.java
--
diff --git 
a/src/java/org/apache/cassandra/notifications/TruncationNotification.java 
b/src/java/org/apache/cassandra/notifications/TruncationNotification.java
new file mode 100644
index 000..345dd17
--- /dev/null
+++ b/src/java/org/apache/cassandra/notifications/TruncationNotification.java
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.notifications;
+
+/**
+ * Fired during truncate,

[jira] [Commented] (CASSANDRA-8063) Consider removing UDF-as-class functionality


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165090#comment-14165090
 ] 

Aleksey Yeschenko commented on CASSANDRA-8063:
--

+1

 Consider removing UDF-as-class functionality
 

 Key: CASSANDRA-8063
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8063
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
  Labels: cql
 Fix For: 3.0


 We've introduce 2 ways to provide (scalar) UDFs: either by providing a 
 class+method name (and assuming C* can find such class and method in the 
 classpath), or by providing the body of the function directly in the {{CREATE 
 FUNCTION}} statement (with such body being able to be in either java or some 
 variety of scripting languages).
 I submit that we remove the first option: the declaration of functions from a 
 class+method name. I was the first to insist on adding it, but in hindsight I 
 think it adds more complexity/confusion than anything else. More 
 specifically, I think the UDF-as-class option is always inferior to 
 CASSANDRA-7562 because:
 # it's more error prone. You have to manually deploy the class containing the 
 function to every C* node (and make sure it is in the classpath). It's way 
 too easy to end up with some node not having the function due to simple 
 operator error.
 # it's not faster. In fact, post-CASSANDRA-7924, the UDF-as-java-source is 
 probably faster since it doesn't involve reflection while the UDF-as-class 
 option does.
 Overall, I think removing the UDF-as-class will simplify things without 
 really losing anything (of course, we can re-evaluate this in the future if 
 new arguments arises, but it's easier to add than to remove).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-8063) Consider removing UDF-as-class functionality

2014-10-09 Thread Robert Stupp (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp reassigned CASSANDRA-8063:
---

Assignee: Robert Stupp

 Consider removing UDF-as-class functionality
 

 Key: CASSANDRA-8063
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8063
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Robert Stupp
  Labels: cql
 Fix For: 3.0


 We've introduce 2 ways to provide (scalar) UDFs: either by providing a 
 class+method name (and assuming C* can find such class and method in the 
 classpath), or by providing the body of the function directly in the {{CREATE 
 FUNCTION}} statement (with such body being able to be in either java or some 
 variety of scripting languages).
 I submit that we remove the first option: the declaration of functions from a 
 class+method name. I was the first to insist on adding it, but in hindsight I 
 think it adds more complexity/confusion than anything else. More 
 specifically, I think the UDF-as-class option is always inferior to 
 CASSANDRA-7562 because:
 # it's more error prone. You have to manually deploy the class containing the 
 function to every C* node (and make sure it is in the classpath). It's way 
 too easy to end up with some node not having the function due to simple 
 operator error.
 # it's not faster. In fact, post-CASSANDRA-7924, the UDF-as-java-source is 
 probably faster since it doesn't involve reflection while the UDF-as-class 
 option does.
 Overall, I think removing the UDF-as-class will simplify things without 
 really losing anything (of course, we can re-evaluate this in the future if 
 new arguments arises, but it's easier to add than to remove).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8019) Windows Unit tests and Dtests erroring due to sstable deleting task error


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165106#comment-14165106
 ] 

Jonathan Ellis commented on CASSANDRA-8019:
---

IIRC the current logic is, we refcount sstables on read.  Compaction + drop 
assumes that if refcount is zero it's safe to delete.  Meanwhile, 
markCompaction ensures we don't have multiple threads trying to compact the 
same sstable.

How are we getting into a situation where SSTableScanner (used by compaction) 
still has it open when it's deleted?

 Windows Unit tests and Dtests erroring due to sstable deleting task error
 -

 Key: CASSANDRA-8019
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8019
 Project: Cassandra
  Issue Type: Bug
 Environment: Windows 7
Reporter: Philip Thompson
Assignee: Joshua McKenzie
  Labels: windows
 Fix For: 2.1.1

 Attachments: 8019_aggressive_v1.txt, 8019_conservative_v1.txt, 
 8019_v2.txt


 Currently a large number of dtests and unit tests are erroring on windows 
 with the following error in the node log:
 {code}
 ERROR [NonPeriodicTasks:1] 2014-09-29 11:05:04,383 
 SSTableDeletingTask.java:89 - Unable to delete 
 c:\\users\\username\\appdata\\local\\temp\\dtest-vr6qgw\\test\\node1\\data\\system\\local-7ad54392bcdd35a684174e047860b377\\system-local-ka-4-Data.db
  (it will be removed on server restart; we'll also retry after GC)\n
 {code}
 git bisect points to the following commit:
 {code}
 0e831007760bffced8687f51b99525b650d7e193 is the first bad commit
 commit 0e831007760bffced8687f51b99525b650d7e193
 Author: Benedict Elliott Smith bened...@apache.org
 Date:  Fri Sep 19 18:17:19 2014 +0100
 Fix resource leak in event of corrupt sstable
 patch by benedict; review by yukim for CASSANDRA-7932
 :100644 100644 d3ee7d99179dce03307503a8093eb47bd0161681 
 f55e5d27c1c53db3485154cd16201fc5419f32df M  CHANGES.txt
 :04 04 194f4c0569b6be9cc9e129c441433c5c14de7249 
 3c62b53b2b2bd4b212ab6005eab38f8a8e228923 M  src
 :04 04 64f49266e328b9fdacc516c52ef1921fe42e994f 
 de2ca38232bee6d2a6a5e068ed9ee0fbbc5aaebe M  test
 {code}
 You can reproduce this by running simple_bootstrap_test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7105) SELECT with IN on final column of composite and compound primary key fails

2014-10-09 Thread Bill Mitchell (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165105#comment-14165105
 ] 

Bill Mitchell commented on CASSANDRA-7105:
--

Although I did not validate this when the fixed 2.0 driver was released, I have 
now verified that my test does not fail with the 2.1.1 driver.  

 SELECT with IN on final column of composite and compound primary key fails
 --

 Key: CASSANDRA-7105
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7105
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: DataStax Cassandra 2.0.7
 Windows dual-core laptop
Reporter: Bill Mitchell
Assignee: Sylvain Lebresne
 Fix For: 1.2.17

 Attachments: 7105-v2.txt, 7105.txt


 I have a failing sequence where I specify an IN constraint on the final int 
 column of the composite primary key and an IN constraint on the final String 
 column of the compound primary key and no rows are returned, when rows should 
 be returned.  
 {noformat}
 CREATE TABLE IF NOT EXISTS sr2 (siteID TEXT, partition INT, listID BIGINT, 
 emailAddr TEXT, emailCrypt TEXT, createDate TIMESTAMP, removeDate TIMESTAMP, 
 removeImportID BIGINT, properties TEXT, PRIMARY KEY ((siteID, listID, 
 partition), createDate, emailCrypt) ) WITH CLUSTERING ORDER BY (createDate 
 DESC, emailCrypt DESC)  AND compression = {'sstable_compression' : 
 'SnappyCompressor'} AND compaction = {'class' : 
 'SizeTieredCompactionStrategy'};
 insert into sr2 (siteID, listID, partition, emailAddr, emailCrypt, 
 createDate) values ('4ca4f79e-3ab2-41c5-ae42-c7009736f1d5', 34, 1, 'xyzzy', 
 '5fe7719229092cdde4526afbc65c900c', '2014-04-28T14:05:59.236-0500');
 insert into sr2 (siteID, listID, partition, emailAddr, emailCrypt, 
 createDate) values ('4ca4f79e-3ab2-41c5-ae42-c7009736f1d5', 34, 2, 'noname', 
 '97bf28af2ca9c498d6e47237bb8680bf', '2014-04-28T14:05:59.236-0500');
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 2 and 
 createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt = 
 '97bf28af2ca9c498d6e47237bb8680bf';
  emailcrypt   | emailaddr
 --+---
  97bf28af2ca9c498d6e47237bb8680bf |noname
 (1 rows)
 select emailCrypt, emailAddr  from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 1 and 
 createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt = 
 '5fe7719229092cdde4526afbc65c900c';
  emailcrypt   | emailaddr
 --+---
  5fe7719229092cdde4526afbc65c900c | xyzzy
 (1 rows)
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
 and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
 ('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');
 (0 rows)
 cqlsh:test_multiple_in select * from sr2;
  siteid   | listid | partition | createdate   
 | emailcrypt | emailaddr| 
 properties | removedate | re
 moveimportid
 --++---+--++--+++---
 -
  4ca4f79e-3ab2-41c5-ae42-c7009736f1d5 | 34 | 2 | 2014-04-28 
 14:05:59Central Daylight Time | noname | 97bf28af2ca9c498d6e47237bb8680bf 
 |   null |   null |
 null
  4ca4f79e-3ab2-41c5-ae42-c7009736f1d5 | 34 | 1 | 2014-04-28 
 14:05:59Central Daylight Time |  xyzzy | 5fe7719229092cdde4526afbc65c900c 
 |   null |   null |
 null
 (2 rows)
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
 and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
 ('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');
 (0 rows)
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 1 and 
 createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
 ('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');
 (0 rows)
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 2 and 
 createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
 ('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');
 (0 rows)
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and

[jira] [Created] (CASSANDRA-8089) Invalid tombstone warnings / exceptions

Andrew S created CASSANDRA-8089:
---

 Summary: Invalid tombstone warnings / exceptions
 Key: CASSANDRA-8089
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8089
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Casasandra 2.1.0
Debian 7.6, 3.2.0-4-amd64 GNU/Linux

java version 1.7.0_51
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
Reporter: Andrew S


Hey,

We are having a strange issue with tombstone warnings which look like this:

{code}
WARN  12:28:42 Read 129 live and 4113 tombstoned cells in XXX.xxx (see 
tombstone_warn_threshold). 500 columns was requested, 
slices=[31660a4e-4f94-11e4-ac1d-53f244a29642-0a8073aa-4f9f-11e4-87c7-5b3e253389d8:!],
 delInfo={deletedAt=-9223372036854775808, localDeletion=2147483647}
{code}

What is strange is that the row requested should not contain any tombstones as 
we never delete data from that row.

To debug the issue we have dumped the data for this row using sstable2json and 
the result does not contain any tombstones.

{code}
./sstable2json /var/lib/cassandra/data/XXX/xxx/XXX-xxx-ka-81524-Data.db -k 
xxx
{code}

We are getting the warnings after issuing a simple query:

{code}
select count(*) from xxx where key = 'x' and aggregate='x';
{code}

There are only ~500 cells but it issues a warning about scanning 1700 
tombstones.

We are very worried about this because for some of the queries we are hitting 
TombstoneOverwhelmingException for no obvious reason.

Here is the table definiion:

{code}
CREATE TABLE Xxxx.xxx (
key text,
aggregate text,
t timeuuid,
. {date fields }
PRIMARY KEY ((key, aggregate), t)
) WITH CLUSTERING ORDER BY (t ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = 'we love cassandra'
AND compaction = {'min_threshold': '6', 'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.SnappyCompressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 3600
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.1
AND speculative_retry = '99.0PERCENTILE';
{code}

Thanks,
Andrew



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8089) Invalid tombstone warnings / exceptions


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew S updated CASSANDRA-8089:

Environment: 
Cassandra 2.1.0
Debian 7.6, 3.2.0-4-amd64 GNU/Linux

java version 1.7.0_51
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)

  was:
Casasandra 2.1.0
Debian 7.6, 3.2.0-4-amd64 GNU/Linux

java version 1.7.0_51
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)


 Invalid tombstone warnings / exceptions
 ---

 Key: CASSANDRA-8089
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8089
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1.0
 Debian 7.6, 3.2.0-4-amd64 GNU/Linux
 java version 1.7.0_51
 Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
 Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
Reporter: Andrew S

 Hey,
 We are having a strange issue with tombstone warnings which look like this:
 {code}
 WARN  12:28:42 Read 129 live and 4113 tombstoned cells in XXX.xxx (see 
 tombstone_warn_threshold). 500 columns was requested, 
 slices=[31660a4e-4f94-11e4-ac1d-53f244a29642-0a8073aa-4f9f-11e4-87c7-5b3e253389d8:!],
  delInfo={deletedAt=-9223372036854775808, localDeletion=2147483647}
 {code}
 What is strange is that the row requested should not contain any tombstones 
 as we never delete data from that row. (We do delete data from other row in 
 the same column family)
 To debug the issue we have dumped the data for this row using sstable2json 
 and the result does not contain any tombstones.
 {code}
 ./sstable2json /var/lib/cassandra/data/XXX/xxx/XXX-xxx-ka-81524-Data.db -k 
 xxx
 {code}
 We are getting the warnings after issuing a simple query:
 {code}
 select count(*) from xxx where key = 'x' and aggregate='x';
 {code}
 There are only ~500 cells but it issues a warning about scanning 1700 
 tombstones.
 We are very worried about this because for some of the queries we are hitting 
 TombstoneOverwhelmingException for no obvious reason.
 Here is the table definiion:
 {code}
 CREATE TABLE Xxxx.xxx (
 key text,
 aggregate text,
 t timeuuid,
 . {date fields }
 PRIMARY KEY ((key, aggregate), t)
 ) WITH CLUSTERING ORDER BY (t ASC)
 AND bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = 'we love cassandra'
 AND compaction = {'min_threshold': '6', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.SnappyCompressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 3600
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.1
 AND speculative_retry = '99.0PERCENTILE';
 {code}
 Thanks,
 Andrew



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8089) Invalid tombstone warnings / exceptions


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew S updated CASSANDRA-8089:

Description: 
Hey,

We are having a strange issue with tombstone warnings which look like this:

{code}
WARN  12:28:42 Read 129 live and 4113 tombstoned cells in XXX.xxx (see 
tombstone_warn_threshold). 500 columns was requested, 
slices=[31660a4e-4f94-11e4-ac1d-53f244a29642-0a8073aa-4f9f-11e4-87c7-5b3e253389d8:!],
 delInfo={deletedAt=-9223372036854775808, localDeletion=2147483647}
{code}

What is strange is that the row requested should not contain any tombstones as 
we never delete data from that row. (We do delete data from other row in the 
same column family)

To debug the issue we have dumped the data for this row using sstable2json and 
the result does not contain any tombstones.

{code}
./sstable2json /var/lib/cassandra/data/XXX/xxx/XXX-xxx-ka-81524-Data.db -k 
xxx
{code}

We are getting the warnings after issuing a simple query:

{code}
select count(*) from xxx where key = 'x' and aggregate='x';
{code}

There are only ~500 cells but it issues a warning about scanning 1700 
tombstones.

We are very worried about this because for some of the queries we are hitting 
TombstoneOverwhelmingException for no obvious reason.

Here is the table definiion:

{code}
CREATE TABLE Xxxx.xxx (
key text,
aggregate text,
t timeuuid,
. {date fields }
PRIMARY KEY ((key, aggregate), t)
) WITH CLUSTERING ORDER BY (t ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = 'we love cassandra'
AND compaction = {'min_threshold': '6', 'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.SnappyCompressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 3600
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.1
AND speculative_retry = '99.0PERCENTILE';
{code}

Thanks,
Andrew

  was:
Hey,

We are having a strange issue with tombstone warnings which look like this:

{code}
WARN  12:28:42 Read 129 live and 4113 tombstoned cells in XXX.xxx (see 
tombstone_warn_threshold). 500 columns was requested, 
slices=[31660a4e-4f94-11e4-ac1d-53f244a29642-0a8073aa-4f9f-11e4-87c7-5b3e253389d8:!],
 delInfo={deletedAt=-9223372036854775808, localDeletion=2147483647}
{code}

What is strange is that the row requested should not contain any tombstones as 
we never delete data from that row.

To debug the issue we have dumped the data for this row using sstable2json and 
the result does not contain any tombstones.

{code}
./sstable2json /var/lib/cassandra/data/XXX/xxx/XXX-xxx-ka-81524-Data.db -k 
xxx
{code}

We are getting the warnings after issuing a simple query:

{code}
select count(*) from xxx where key = 'x' and aggregate='x';
{code}

There are only ~500 cells but it issues a warning about scanning 1700 
tombstones.

We are very worried about this because for some of the queries we are hitting 
TombstoneOverwhelmingException for no obvious reason.

Here is the table definiion:

{code}
CREATE TABLE Xxxx.xxx (
key text,
aggregate text,
t timeuuid,
. {date fields }
PRIMARY KEY ((key, aggregate), t)
) WITH CLUSTERING ORDER BY (t ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = 'we love cassandra'
AND compaction = {'min_threshold': '6', 'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.SnappyCompressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 3600
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.1
AND speculative_retry = '99.0PERCENTILE';
{code}

Thanks,
Andrew


 Invalid tombstone warnings / exceptions
 ---

 Key: CASSANDRA-8089
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8089
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Casasandra 2.1.0
 Debian 7.6, 3.2.0-4-amd64 GNU/Linux
 java version 1.7.0_51
 Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
 Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
Reporter: Andrew S

 Hey,
 We are having a strange issue with tombstone warnings which look like this:
 {code}
 WARN  12:28:42 Read 129 live and 4113 tombstoned cells in XXX.xxx (see 
 tombstone_warn_threshold). 500 columns was

[jira] [Updated] (CASSANDRA-8089) Invalid tombstone warnings / exceptions


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew S updated CASSANDRA-8089:

Description: 
Hey,

We are having a strange issue with tombstone warnings which look like this:

{code}
WARN  12:28:42 Read 129 live and 4113 tombstoned cells in XXX.xxx (see 
tombstone_warn_threshold). 500 columns was requested, 
slices=[31660a4e-4f94-11e4-ac1d-53f244a29642-0a8073aa-4f9f-11e4-87c7-5b3e253389d8:!],
 delInfo={deletedAt=-9223372036854775808, localDeletion=2147483647}
{code}

What is strange is that the row requested should not contain any tombstones as 
we never delete data from that row. (We do delete data from other row in the 
same column family)

To debug the issue we have dumped the data for this row using sstable2json and 
the result does not contain any tombstones. (We have done this on all nodes 
having the data and all sstables containing the key)

{code}
./sstable2json /var/lib/cassandra/data/XXX/xxx/XXX-xxx-ka-81524-Data.db -k 
xxx
{code}

We are getting the warnings after issuing a simple query:

{code}
select count(*) from xxx where key = 'x' and aggregate='x';
{code}

There are only ~500 cells but it issues a warning about scanning 1700 
tombstones.

We are very worried about this because for some of the queries we are hitting 
TombstoneOverwhelmingException for no obvious reason.

Here is the table definiion:

{code}
CREATE TABLE Xxxx.xxx (
key text,
aggregate text,
t timeuuid,
. {date fields }
PRIMARY KEY ((key, aggregate), t)
) WITH CLUSTERING ORDER BY (t ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = 'we love cassandra'
AND compaction = {'min_threshold': '6', 'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.SnappyCompressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 3600
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.1
AND speculative_retry = '99.0PERCENTILE';
{code}

Thanks,
Andrew

  was:
Hey,

We are having a strange issue with tombstone warnings which look like this:

{code}
WARN  12:28:42 Read 129 live and 4113 tombstoned cells in XXX.xxx (see 
tombstone_warn_threshold). 500 columns was requested, 
slices=[31660a4e-4f94-11e4-ac1d-53f244a29642-0a8073aa-4f9f-11e4-87c7-5b3e253389d8:!],
 delInfo={deletedAt=-9223372036854775808, localDeletion=2147483647}
{code}

What is strange is that the row requested should not contain any tombstones as 
we never delete data from that row. (We do delete data from other row in the 
same column family)

To debug the issue we have dumped the data for this row using sstable2json and 
the result does not contain any tombstones.

{code}
./sstable2json /var/lib/cassandra/data/XXX/xxx/XXX-xxx-ka-81524-Data.db -k 
xxx
{code}

We are getting the warnings after issuing a simple query:

{code}
select count(*) from xxx where key = 'x' and aggregate='x';
{code}

There are only ~500 cells but it issues a warning about scanning 1700 
tombstones.

We are very worried about this because for some of the queries we are hitting 
TombstoneOverwhelmingException for no obvious reason.

Here is the table definiion:

{code}
CREATE TABLE Xxxx.xxx (
key text,
aggregate text,
t timeuuid,
. {date fields }
PRIMARY KEY ((key, aggregate), t)
) WITH CLUSTERING ORDER BY (t ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = 'we love cassandra'
AND compaction = {'min_threshold': '6', 'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.SnappyCompressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 3600
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.1
AND speculative_retry = '99.0PERCENTILE';
{code}

Thanks,
Andrew


 Invalid tombstone warnings / exceptions
 ---

 Key: CASSANDRA-8089
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8089
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1.0
 Debian 7.6, 3.2.0-4-amd64 GNU/Linux
 java version 1.7.0_51
 Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
 Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
Reporter: Andrew S

 Hey,
 We are having a strange issue with tombstone warnings

[jira] [Commented] (CASSANDRA-6246) EPaxos

2014-10-09 Thread Blake Eggleston (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165151#comment-14165151
 ] 

Blake Eggleston commented on CASSANDRA-6246:


Since epaxos executes mutations at different times on each machine, each 
instance needs a serialized copy of the statement. The CQL3CasRequest.RowUpdate 
class keeps a reference to the actual ModificationStatement, and serializing 
that looks like it will involve implementing at least 50 (de)serializers. Since 
I’m not super familiar with the inner workings of the UpdateStatement and 
DeleteStatement, I thought I’d ask here to see if there’s a better solution I’m 
not seeing.

 EPaxos
 --

 Key: CASSANDRA-6246
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6246
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Blake Eggleston
Priority: Minor

 One reason we haven't optimized our Paxos implementation with Multi-paxos is 
 that Multi-paxos requires leader election and hence, a period of 
 unavailability when the leader dies.
 EPaxos is a Paxos variant that requires (1) less messages than multi-paxos, 
 (2) is particularly useful across multiple datacenters, and (3) allows any 
 node to act as coordinator: 
 http://sigops.org/sosp/sosp13/papers/p358-moraru.pdf
 However, there is substantial additional complexity involved if we choose to 
 implement it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7280) Hadoop support not respecting cassandra.input.split.size

2014-10-09 Thread Darla Baker (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165161#comment-14165161
 ] 

Darla Baker commented on CASSANDRA-7280:


Any chance this will be assigned soon?

 Hadoop support not respecting cassandra.input.split.size
 

 Key: CASSANDRA-7280
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7280
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Reporter: Jeremy Hanna

 Long ago (0.7), I tried to set the cassandra.input.split.size property and 
 never really got it to respect that property.  However the batch size was 
 useful for what I needed to affect the timeouts.
 Now with the cql record reader and the native paging, users can specify 
 queries potentially using allow filtering clauses.  The input split size is 
 more important because the server may have to scan through many many records 
 to get matching records.  If the user can effectively set the input split 
 size, then that gives a hard limit on how many records it will traverse.
 Currently it appears to be overriding the property, perhaps in the 
 client.describe_splits_ex method on the server side.
 It can be argued that users shouldn't be using allow filtering clauses in 
 their cql in the first place.  However it is still a bug that the input split 
 size is not honored.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8087) Multiple non-DISTINCT rows returned when page_size set


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-8087:
---
Tester: Philip Thompson

 Multiple non-DISTINCT rows returned when page_size set
 --

 Key: CASSANDRA-8087
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8087
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Adam Holmberg
Priority: Minor
 Fix For: 2.0.11


 Using the following statements to reproduce:
 {code}
 CREATE TABLE test (
 k int,
 p int,
 s int static,
 PRIMARY KEY (k, p)
 );
 INSERT INTO test (k, p) VALUES (1, 1);
 INSERT INTO test (k, p) VALUES (1, 2);
 SELECT DISTINCT k, s FROM test ;
 {code}
 Native clients that set result_page_size in the query message receive 
 multiple non-distinct rows back (one per clustered value p in row k).
 This is only reproduced on 2.0.10. Does not appear in 2.1.0
 It does not appear in cqlsh for 2.0.10 because thrift.
 See https://datastax-oss.atlassian.net/browse/PYTHON-164 for background



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6246) EPaxos


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165177#comment-14165177
 ] 

T Jake Luciani commented on CASSANDRA-6246:
---

Can't you just call .getMutations on the statements and serialize the actual 
RowMutations?

 EPaxos
 --

 Key: CASSANDRA-6246
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6246
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Blake Eggleston
Priority: Minor

 One reason we haven't optimized our Paxos implementation with Multi-paxos is 
 that Multi-paxos requires leader election and hence, a period of 
 unavailability when the leader dies.
 EPaxos is a Paxos variant that requires (1) less messages than multi-paxos, 
 (2) is particularly useful across multiple datacenters, and (3) allows any 
 node to act as coordinator: 
 http://sigops.org/sosp/sosp13/papers/p358-moraru.pdf
 However, there is substantial additional complexity involved if we choose to 
 implement it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6246) EPaxos

2014-10-09 Thread Blake Eggleston (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165185#comment-14165185
 ] 

Blake Eggleston commented on CASSANDRA-6246:


That would work most of the time, but a few operations do a read before a write.

I suppose I could narrow the serialization support down to just the operations 
and terms that are involved in those statements, but I'd like to avoid special 
casing specific operations if possible.

 EPaxos
 --

 Key: CASSANDRA-6246
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6246
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Blake Eggleston
Priority: Minor

 One reason we haven't optimized our Paxos implementation with Multi-paxos is 
 that Multi-paxos requires leader election and hence, a period of 
 unavailability when the leader dies.
 EPaxos is a Paxos variant that requires (1) less messages than multi-paxos, 
 (2) is particularly useful across multiple datacenters, and (3) allows any 
 node to act as coordinator: 
 http://sigops.org/sosp/sosp13/papers/p358-moraru.pdf
 However, there is substantial additional complexity involved if we choose to 
 implement it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8061) tmplink files are not removed

2014-10-09 Thread Ryan McGuire (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165198#comment-14165198
 ] 

Ryan McGuire commented on CASSANDRA-8061:
-

[~gianlucaborello] What kind of workload are you running (quantity of 
writes/reads, number of CFs)? You say 'Even starting with a clean keyspace', 
are you saying that a completely fresh instance with no data inserted produces 
this, or only sometime after you start to actually put data in it?

 tmplink files are not removed
 -

 Key: CASSANDRA-8061
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8061
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Linux
Reporter: Gianluca Borello
Assignee: Ryan McGuire

 After installing 2.1.0, I'm experiencing a bunch of tmplink files that are 
 filling my disk. I found https://issues.apache.org/jira/browse/CASSANDRA-7803 
 and that is very similar, and I confirm it happens both on 2.1.0 as well as 
 from the latest commit on the cassandra-2.1 branch 
 (https://github.com/apache/cassandra/commit/aca80da38c3d86a40cc63d9a122f7d45258e4685
  from the cassandra-2.1)
 Even starting with a clean keyspace, after a few hours I get:
 $ sudo find /raid0 | grep tmplink | xargs du -hs
 2.7G  
 /raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Data.db
 13M   
 /raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Index.db
 1.8G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Data.db
 12M   
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Index.db
 5.2M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Index.db
 822M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Data.db
 7.3M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Index.db
 1.2G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Data.db
 6.7M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Index.db
 1.1G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Data.db
 11M   
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Index.db
 1.7G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Data.db
 812K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Index.db
 122M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Data.db
 744K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-739-Index.db
 660K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Index.db
 796K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Index.db
 137M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Data.db
 161M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Data.db
 139M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Data.db
 940K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Index.db
 936K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Index.db
 161M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Data.db
 672K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Index.db
 113M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Data.db
 116M

[jira] [Created] (CASSANDRA-8090) NullPointerException when using prepared statements

2014-10-09 Thread Carl Yeksigian (JIRA)

Carl Yeksigian created CASSANDRA-8090:
-

 Summary: NullPointerException when using prepared statements
 Key: CASSANDRA-8090
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8090
 Project: Cassandra
  Issue Type: Bug
Reporter: Carl Yeksigian


Due to the changes in CASSANDRA-4914, using a prepared statement from multiple 
threads leads to a race condition where the simple selection may be reset from 
a different thread, causing the following NPE:

{noformat}
java.lang.NullPointerException: null
at org.apache.cassandra.cql3.ResultSet.addRow(ResultSet.java:63) 
~[main/:na]
at 
org.apache.cassandra.cql3.statements.Selection$ResultSetBuilder.build(Selection.java:372)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:1120)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:283)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:260)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:213)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:63)
 ~[main/:na]
at 
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:226)
 ~[main/:na]
at 
org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:481)
 ~[main/:na]
at 
org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:133)
 ~[main/:na]
at 
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:438)
 [main/:na]
at 
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:334)
 [main/:na]
at 
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
[na:1.7.0_67]
at 
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163)
 [main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) 
[main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
{noformat}

Reproduced this using the stress tool:
{noformat}
 ./tools/bin/cassandra-stress user profile=tools/cqlstress-example.yaml 
ops\(insert=1,simple1=1\)
{noformat}

You'll need to change the {noformat}select:{noformat} line to be /1000 to 
prevent the illegal query exceptions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8091) Stress tool creates too large batches

2014-10-09 Thread Carl Yeksigian (JIRA)

Carl Yeksigian created CASSANDRA-8091:
-

 Summary: Stress tool creates too large batches
 Key: CASSANDRA-8091
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8091
 Project: Cassandra
  Issue Type: Bug
Reporter: Carl Yeksigian


With CASSANDRA-8011, the stress tool now gets exceptions because its batches 
are too large. We should change the default behaviour to not create batches so 
large, probably using individual inserts instead of an unlogged batch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-8090) NullPointerException when using prepared statements

2014-10-09 Thread Benjamin Lerer (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer reassigned CASSANDRA-8090:
-

Assignee: Benjamin Lerer

 NullPointerException when using prepared statements
 ---

 Key: CASSANDRA-8090
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8090
 Project: Cassandra
  Issue Type: Bug
Reporter: Carl Yeksigian
Assignee: Benjamin Lerer

 Due to the changes in CASSANDRA-4914, using a prepared statement from 
 multiple threads leads to a race condition where the simple selection may be 
 reset from a different thread, causing the following NPE:
 {noformat}
 java.lang.NullPointerException: null
   at org.apache.cassandra.cql3.ResultSet.addRow(ResultSet.java:63) 
 ~[main/:na]
   at 
 org.apache.cassandra.cql3.statements.Selection$ResultSetBuilder.build(Selection.java:372)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:1120)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:283)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:260)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:213)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:63)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:226)
  ~[main/:na]
   at 
 org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:481)
  ~[main/:na]
   at 
 org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:133)
  ~[main/:na]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:438)
  [main/:na]
   at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:334)
  [main/:na]
   at 
 io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_67]
   at 
 org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163)
  [main/:na]
   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) 
 [main/:na]
   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
 {noformat}
 Reproduced this using the stress tool:
 {noformat}
  ./tools/bin/cassandra-stress user profile=tools/cqlstress-example.yaml 
 ops\(insert=1,simple1=1\)
 {noformat}
 You'll need to change the {noformat}select:{noformat} line to be /1000 to 
 prevent the illegal query exceptions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-5586) Remove cli usage from dtests


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-5586:
---
Labels:   (was: qa-resolved)

 Remove cli usage from dtests
 

 Key: CASSANDRA-5586
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5586
 Project: Cassandra
  Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Ryan McGuire
Priority: Minor

 The dtests in some situations fork the cli.  With the cli essentially 
 stagnant now, there's no need to do this when the same thing can be 
 accomplished with a thrift or cql call. (ccm's convenience api for invoking 
 the cli could probably also be removed at this point)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8091) Stress tool creates too large batches


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165241#comment-14165241
 ] 

T Jake Luciani commented on CASSANDRA-8091:
---

Maybe a maxbatchsize option?

 Stress tool creates too large batches
 -

 Key: CASSANDRA-8091
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8091
 Project: Cassandra
  Issue Type: Bug
Reporter: Carl Yeksigian
  Labels: stress

 With CASSANDRA-8011, the stress tool now gets exceptions because its batches 
 are too large. We should change the default behaviour to not create batches 
 so large, probably using individual inserts instead of an unlogged batch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-5586) Remove cli usage from dtests


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson reopened CASSANDRA-5586:


 Remove cli usage from dtests
 

 Key: CASSANDRA-5586
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5586
 Project: Cassandra
  Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Ryan McGuire
Priority: Minor

 The dtests in some situations fork the cli.  With the cli essentially 
 stagnant now, there's no need to do this when the same thing can be 
 accomplished with a thrift or cql call. (ccm's convenience api for invoking 
 the cli could probably also be removed at this point)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-5586) Remove cli usage from dtests


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165244#comment-14165244
 ] 

Philip Thompson commented on CASSANDRA-5586:


CLI has been removed in 3.0. We still have tests that are using cli and should 
be changed to use thrift instead.

 Remove cli usage from dtests
 

 Key: CASSANDRA-5586
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5586
 Project: Cassandra
  Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Ryan McGuire
Priority: Minor

 The dtests in some situations fork the cli.  With the cli essentially 
 stagnant now, there's no need to do this when the same thing can be 
 accomplished with a thrift or cql call. (ccm's convenience api for invoking 
 the cli could probably also be removed at this point)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7188) Wrong class type: class org.apache.cassandra.db.Column in CounterColumn.reconcile


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-7188:
-
Attachment: 7188.txt

 Wrong class type: class org.apache.cassandra.db.Column in 
 CounterColumn.reconcile
 -

 Key: CASSANDRA-7188
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7188
 Project: Cassandra
  Issue Type: Bug
Reporter: Nicolas Lalevée
Assignee: Aleksey Yeschenko
 Attachments: 7188.txt


 When migrating a cluster of 6 nodes from 1.2.11 to 2.0.7, we started to see 
 on the first migrated node this error:
 {noformat}
 ERROR [ReplicateOnWriteStage:1] 2014-05-07 11:26:59,779 CassandraDaemon.java 
 (line 198) Exception in thread Thread[ReplicateOnWriteStage:1,5,main]
 java.lang.AssertionError: Wrong class type: class 
 org.apache.cassandra.db.Column
 at 
 org.apache.cassandra.db.CounterColumn.reconcile(CounterColumn.java:159)
 at 
 org.apache.cassandra.db.filter.QueryFilter$1.reduce(QueryFilter.java:109)
 at 
 org.apache.cassandra.db.filter.QueryFilter$1.reduce(QueryFilter.java:103)
 at 
 org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:112)
 at 
 org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
 at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
 at 
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
 at 
 org.apache.cassandra.db.filter.NamesQueryFilter.collectReducedColumns(NamesQueryFilter.java:98)
 at 
 org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
 at 
 org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
 at 
 org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
 at 
 org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
 at 
 org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1540)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1369)
 at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:327)
 at 
 org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:55)
 at 
 org.apache.cassandra.db.CounterMutation.makeReplicationMutation(CounterMutation.java:100)
 at 
 org.apache.cassandra.service.StorageProxy$8$1.runMayThrow(StorageProxy.java:1085)
 at 
 org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1916)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 {noformat}
 We then saw on the other 5 nodes, still on 1.2.x, this error:
 {noformat}
 ERROR [MutationStage:2793] 2014-05-07 11:46:12,301 CassandraDaemon.java (line 
 191) Exception in thread Thread[MutationStage:2793,5,main]
 java.lang.AssertionError: Wrong class type: class 
 org.apache.cassandra.db.Column
 at 
 org.apache.cassandra.db.CounterColumn.reconcile(CounterColumn.java:165)
 at 
 org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:378)
 at 
 org.apache.cassandra.db.AtomicSortedColumns.addColumn(AtomicSortedColumns.java:166)
 at 
 org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:119)
 at org.apache.cassandra.db.SuperColumn.addColumn(SuperColumn.java:218)
 at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:229)
 at 
 org.apache.cassandra.db.ThreadSafeSortedColumns.addColumnInternal(ThreadSafeSortedColumns.java:108)
 at 
 org.apache.cassandra.db.ThreadSafeSortedColumns.addAllWithSizeDelta(ThreadSafeSortedColumns.java:138)
 at 
 org.apache.cassandra.db.AbstractColumnContainer.addAllWithSizeDelta(AbstractColumnContainer.java:99)
 at org.apache.cassandra.db.Memtable.resolve(Memtable.java:205)
 at org.apache.cassandra.db.Memtable.put(Memtable.java:168)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:742)
 at org.apache.cassandra.db.Table.apply(Table.java:388)
 at org.apache.cassandra.db.Table.apply(Table.java:353)
 at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:280)
 at 
 org.apache.cassandra.db.CounterMutation.apply(CounterMutation.java:137)
 at

[jira] [Commented] (CASSANDRA-5586) Remove cli usage from dtests

2014-10-09 Thread Brandon Williams (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165245#comment-14165245
 ] 

Brandon Williams commented on CASSANDRA-5586:
-

bq. should be changed to use thrift instead

Aren't we on the native protocol now? :)

 Remove cli usage from dtests
 

 Key: CASSANDRA-5586
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5586
 Project: Cassandra
  Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Ryan McGuire
Priority: Minor

 The dtests in some situations fork the cli.  With the cli essentially 
 stagnant now, there's no need to do this when the same thing can be 
 accomplished with a thrift or cql call. (ccm's convenience api for invoking 
 the cli could probably also be removed at this point)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7188) Wrong class type: class org.apache.cassandra.db.Column in CounterColumn.reconcile


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165249#comment-14165249
 ] 

Aleksey Yeschenko commented on CASSANDRA-7188:
--

bq. Maybe it is related to the fact that CQL doesn't support well super column 
families?

Nope, CQL supports super columns very well. The bug here is in the 
composites-supercolumns conversion, caused by CASSANDRA-3237.

A very trivial one, too - to serialize a CounterMutation for forwarding it to a 
1.2 node, if that one is chosen a leader by the 2.0 coordinator, as part of the 
process we call Column#withUpdatedName() - in SuperColumns#groupSuperColumns(), 
in this case. And CounterUpdateColumn was missing a withUpdatedName() override, 
so when serializing, it used the implementation in Column, and didn't set the 
correct flag. The 1.2 node, upon receiving the serialized mutation, read those 
columns as regular Column, and that's what's been causing your reconcile assert.

 Wrong class type: class org.apache.cassandra.db.Column in 
 CounterColumn.reconcile
 -

 Key: CASSANDRA-7188
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7188
 Project: Cassandra
  Issue Type: Bug
Reporter: Nicolas Lalevée
Assignee: Aleksey Yeschenko
 Attachments: 7188.txt


 When migrating a cluster of 6 nodes from 1.2.11 to 2.0.7, we started to see 
 on the first migrated node this error:
 {noformat}
 ERROR [ReplicateOnWriteStage:1] 2014-05-07 11:26:59,779 CassandraDaemon.java 
 (line 198) Exception in thread Thread[ReplicateOnWriteStage:1,5,main]
 java.lang.AssertionError: Wrong class type: class 
 org.apache.cassandra.db.Column
 at 
 org.apache.cassandra.db.CounterColumn.reconcile(CounterColumn.java:159)
 at 
 org.apache.cassandra.db.filter.QueryFilter$1.reduce(QueryFilter.java:109)
 at 
 org.apache.cassandra.db.filter.QueryFilter$1.reduce(QueryFilter.java:103)
 at 
 org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:112)
 at 
 org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
 at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
 at 
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
 at 
 org.apache.cassandra.db.filter.NamesQueryFilter.collectReducedColumns(NamesQueryFilter.java:98)
 at 
 org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
 at 
 org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
 at 
 org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
 at 
 org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
 at 
 org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1540)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1369)
 at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:327)
 at 
 org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:55)
 at 
 org.apache.cassandra.db.CounterMutation.makeReplicationMutation(CounterMutation.java:100)
 at 
 org.apache.cassandra.service.StorageProxy$8$1.runMayThrow(StorageProxy.java:1085)
 at 
 org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1916)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 {noformat}
 We then saw on the other 5 nodes, still on 1.2.x, this error:
 {noformat}
 ERROR [MutationStage:2793] 2014-05-07 11:46:12,301 CassandraDaemon.java (line 
 191) Exception in thread Thread[MutationStage:2793,5,main]
 java.lang.AssertionError: Wrong class type: class 
 org.apache.cassandra.db.Column
 at 
 org.apache.cassandra.db.CounterColumn.reconcile(CounterColumn.java:165)
 at 
 org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:378)
 at 
 org.apache.cassandra.db.AtomicSortedColumns.addColumn(AtomicSortedColumns.java:166)
 at 
 org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:119)
 at org.apache.cassandra.db.SuperColumn.addColumn(SuperColumn.java:218)
 at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:229)
 at

[jira] [Commented] (CASSANDRA-5586) Remove cli usage from dtests


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165256#comment-14165256
 ] 

Philip Thompson commented on CASSANDRA-5586:


Yes, we are, but I believe we have one or two stragglers that I didn't know of 
a cql equivalent to the thrift usage but I have someone else look into those.

 Remove cli usage from dtests
 

 Key: CASSANDRA-5586
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5586
 Project: Cassandra
  Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Ryan McGuire
Priority: Minor

 The dtests in some situations fork the cli.  With the cli essentially 
 stagnant now, there's no need to do this when the same thing can be 
 accomplished with a thrift or cql call. (ccm's convenience api for invoking 
 the cli could probably also be removed at this point)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7188) Wrong class type: class org.apache.cassandra.db.Column in CounterColumn.reconcile


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165257#comment-14165257
 ] 

Philip Thompson commented on CASSANDRA-7188:


We have specific tests for upgrading with supercolumns. There appear to be none 
that use counter columns, I'll go ahead and add some.

 Wrong class type: class org.apache.cassandra.db.Column in 
 CounterColumn.reconcile
 -

 Key: CASSANDRA-7188
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7188
 Project: Cassandra
  Issue Type: Bug
Reporter: Nicolas Lalevée
Assignee: Aleksey Yeschenko
 Fix For: 2.0.11

 Attachments: 7188.txt


 When migrating a cluster of 6 nodes from 1.2.11 to 2.0.7, we started to see 
 on the first migrated node this error:
 {noformat}
 ERROR [ReplicateOnWriteStage:1] 2014-05-07 11:26:59,779 CassandraDaemon.java 
 (line 198) Exception in thread Thread[ReplicateOnWriteStage:1,5,main]
 java.lang.AssertionError: Wrong class type: class 
 org.apache.cassandra.db.Column
 at 
 org.apache.cassandra.db.CounterColumn.reconcile(CounterColumn.java:159)
 at 
 org.apache.cassandra.db.filter.QueryFilter$1.reduce(QueryFilter.java:109)
 at 
 org.apache.cassandra.db.filter.QueryFilter$1.reduce(QueryFilter.java:103)
 at 
 org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:112)
 at 
 org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
 at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
 at 
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
 at 
 org.apache.cassandra.db.filter.NamesQueryFilter.collectReducedColumns(NamesQueryFilter.java:98)
 at 
 org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
 at 
 org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
 at 
 org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
 at 
 org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
 at 
 org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1540)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1369)
 at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:327)
 at 
 org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:55)
 at 
 org.apache.cassandra.db.CounterMutation.makeReplicationMutation(CounterMutation.java:100)
 at 
 org.apache.cassandra.service.StorageProxy$8$1.runMayThrow(StorageProxy.java:1085)
 at 
 org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1916)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 {noformat}
 We then saw on the other 5 nodes, still on 1.2.x, this error:
 {noformat}
 ERROR [MutationStage:2793] 2014-05-07 11:46:12,301 CassandraDaemon.java (line 
 191) Exception in thread Thread[MutationStage:2793,5,main]
 java.lang.AssertionError: Wrong class type: class 
 org.apache.cassandra.db.Column
 at 
 org.apache.cassandra.db.CounterColumn.reconcile(CounterColumn.java:165)
 at 
 org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:378)
 at 
 org.apache.cassandra.db.AtomicSortedColumns.addColumn(AtomicSortedColumns.java:166)
 at 
 org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:119)
 at org.apache.cassandra.db.SuperColumn.addColumn(SuperColumn.java:218)
 at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:229)
 at 
 org.apache.cassandra.db.ThreadSafeSortedColumns.addColumnInternal(ThreadSafeSortedColumns.java:108)
 at 
 org.apache.cassandra.db.ThreadSafeSortedColumns.addAllWithSizeDelta(ThreadSafeSortedColumns.java:138)
 at 
 org.apache.cassandra.db.AbstractColumnContainer.addAllWithSizeDelta(AbstractColumnContainer.java:99)
 at org.apache.cassandra.db.Memtable.resolve(Memtable.java:205)
 at org.apache.cassandra.db.Memtable.put(Memtable.java:168)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:742)
 at org.apache.cassandra.db.Table.apply(Table.java:388)
 at org.apache.cassandra.db.Table.apply(Table.java:353)

[jira] [Updated] (CASSANDRA-7188) Wrong class type: class org.apache.cassandra.db.Column in CounterColumn.reconcile


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-7188:
---
Tester: Philip Thompson

 Wrong class type: class org.apache.cassandra.db.Column in 
 CounterColumn.reconcile
 -

 Key: CASSANDRA-7188
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7188
 Project: Cassandra
  Issue Type: Bug
Reporter: Nicolas Lalevée
Assignee: Aleksey Yeschenko
 Fix For: 2.0.11

 Attachments: 7188.txt


 When migrating a cluster of 6 nodes from 1.2.11 to 2.0.7, we started to see 
 on the first migrated node this error:
 {noformat}
 ERROR [ReplicateOnWriteStage:1] 2014-05-07 11:26:59,779 CassandraDaemon.java 
 (line 198) Exception in thread Thread[ReplicateOnWriteStage:1,5,main]
 java.lang.AssertionError: Wrong class type: class 
 org.apache.cassandra.db.Column
 at 
 org.apache.cassandra.db.CounterColumn.reconcile(CounterColumn.java:159)
 at 
 org.apache.cassandra.db.filter.QueryFilter$1.reduce(QueryFilter.java:109)
 at 
 org.apache.cassandra.db.filter.QueryFilter$1.reduce(QueryFilter.java:103)
 at 
 org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:112)
 at 
 org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
 at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
 at 
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
 at 
 org.apache.cassandra.db.filter.NamesQueryFilter.collectReducedColumns(NamesQueryFilter.java:98)
 at 
 org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
 at 
 org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
 at 
 org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
 at 
 org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
 at 
 org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1540)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1369)
 at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:327)
 at 
 org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:55)
 at 
 org.apache.cassandra.db.CounterMutation.makeReplicationMutation(CounterMutation.java:100)
 at 
 org.apache.cassandra.service.StorageProxy$8$1.runMayThrow(StorageProxy.java:1085)
 at 
 org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1916)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 {noformat}
 We then saw on the other 5 nodes, still on 1.2.x, this error:
 {noformat}
 ERROR [MutationStage:2793] 2014-05-07 11:46:12,301 CassandraDaemon.java (line 
 191) Exception in thread Thread[MutationStage:2793,5,main]
 java.lang.AssertionError: Wrong class type: class 
 org.apache.cassandra.db.Column
 at 
 org.apache.cassandra.db.CounterColumn.reconcile(CounterColumn.java:165)
 at 
 org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:378)
 at 
 org.apache.cassandra.db.AtomicSortedColumns.addColumn(AtomicSortedColumns.java:166)
 at 
 org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:119)
 at org.apache.cassandra.db.SuperColumn.addColumn(SuperColumn.java:218)
 at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:229)
 at 
 org.apache.cassandra.db.ThreadSafeSortedColumns.addColumnInternal(ThreadSafeSortedColumns.java:108)
 at 
 org.apache.cassandra.db.ThreadSafeSortedColumns.addAllWithSizeDelta(ThreadSafeSortedColumns.java:138)
 at 
 org.apache.cassandra.db.AbstractColumnContainer.addAllWithSizeDelta(AbstractColumnContainer.java:99)
 at org.apache.cassandra.db.Memtable.resolve(Memtable.java:205)
 at org.apache.cassandra.db.Memtable.put(Memtable.java:168)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:742)
 at org.apache.cassandra.db.Table.apply(Table.java:388)
 at org.apache.cassandra.db.Table.apply(Table.java:353)
 at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:280)
 at 
 org.apache.cassandra.db.CounterMutation.apply(CounterMutation.java:137)

[jira] [Commented] (CASSANDRA-7970) JSON support for CQL

2014-10-09 Thread Sylvain Lebresne (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165276#comment-14165276
 ] 

Sylvain Lebresne commented on CASSANDRA-7970:
-

I'll note that there is imo 2 slightly separate case to handle. Not only do we 
want to translate full JSON objects to CQL rows and vice-versa as in the 
example of the description. But we should also allow to convert CQL values to 
JSON and vice-versa, because well, the row to/from json will really just be a 
little extension of the cql value to/from json. Concretely, we'd want something 
along the lines of:
{noformat}
UPDATE users SET addresses = addresses + fromJson($${work : { ... }}$$) WHERE 
id = ...
{noformat}
and I don't see why we wouldn't allow stuffs like:
{noformat}
SELECT id, toJson(addresses) FROM users;
{noformat}
since again, we will have to implement such functions internally. For those 
cases, I do think the functional syntax work better than a specific syntax 
because well, those are really plain old functions.

That said, I'd be personally be fine with adding toJson/fromJson functions for 
the conversion of CQL value to and from json, but having specific syntax like 
the one in the description to apply the conversion to full rows (I do agree 
that the functional syntax is somewhat inconsistent with normal functions 
when it comes to applying it to row).


 JSON support for CQL
 

 Key: CASSANDRA-7970
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7970
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Jonathan Ellis
Assignee: Tyler Hobbs
 Fix For: 3.0


 JSON is popular enough that not supporting it is becoming a competitive 
 weakness.  We can add JSON support in a way that is compatible with our 
 performance goals by *mapping* JSON to an existing schema: one JSON documents 
 maps to one CQL row.
 Thus, it is NOT a goal to support schemaless documents, which is a misfeature 
 [1] [2] [3].  Rather, it is to allow a convenient way to easily turn a JSON 
 document from a service or a user into a CQL row, with all the validation 
 that entails.
 Since we are not looking to support schemaless documents, we will not be 
 adding a JSON data type (CASSANDRA-6833) a la postgresql.  Rather, we will 
 map the JSON to UDT, collections, and primitive CQL types.
 Here's how this might look:
 {code}
 CREATE TYPE address (
   street text,
   city text,
   zip_code int,
   phones settext
 );
 CREATE TABLE users (
   id uuid PRIMARY KEY,
   name text,
   addresses maptext, address
 );
 INSERT INTO users JSON
 {‘id’: 4b856557-7153,
‘name’: ‘jbellis’,
‘address’: {“home”: {“street”: “123 Cassandra Dr”,
 “city”: “Austin”,
 “zip_code”: 78747,
 “phones”: [2101234567]}}};
 SELECT JSON id, address FROM users;
 {code}
 (We would also want to_json and from_json functions to allow mapping a single 
 column's worth of data.  These would not require extra syntax.)
 [1] http://rustyrazorblade.com/2014/07/the-myth-of-schema-less/
 [2] https://blog.compose.io/schema-less-is-usually-a-lie/
 [3] http://dl.acm.org/citation.cfm?id=2481247



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] git commit: Shutdown JVM on OOM

2014-10-09 Thread jmckenzie

Repository: cassandra
Updated Branches:
  refs/heads/trunk a9bad890f - 50734f207


Shutdown JVM on OOM

patch by Josh McKenzie; reviewed by Aleksey Yeschenko for CASSANDRA-7507


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/14521347
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/14521347
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/14521347

Branch: refs/heads/trunk
Commit: 1452134707e2a143def3c03d1db416b1c8197565
Parents: 5a33a9a
Author: Joshua McKenzie josh.mcken...@datastax.com
Authored: Thu Oct 9 10:40:57 2014 -0500
Committer: Joshua McKenzie josh.mcken...@datastax.com
Committed: Thu Oct 9 10:40:57 2014 -0500

--
 CHANGES.txt |  1 +
 .../cassandra/auth/CassandraAuthorizer.java |  6 +--
 .../AbstractTracingAwareExecutorService.java|  2 +
 .../DebuggableScheduledThreadPoolExecutor.java  |  7 ++-
 .../apache/cassandra/concurrent/SEPWorker.java  |  3 ++
 .../db/commitlog/CommitLogReplayer.java |  1 +
 .../db/commitlog/CommitLogSegmentManager.java   |  2 +
 .../db/compaction/CompactionManager.java|  7 +--
 .../cassandra/db/compaction/Scrubber.java   |  2 +
 src/java/org/apache/cassandra/gms/Gossiper.java |  2 +
 .../io/sstable/SSTableSimpleUnsortedWriter.java |  2 +
 .../org/apache/cassandra/io/util/FileUtils.java |  5 ++-
 .../cassandra/service/CassandraDaemon.java  | 20 ++---
 .../cassandra/service/StorageService.java   | 18 ++--
 .../cassandra/streaming/ConnectionHandler.java  |  6 ++-
 .../streaming/messages/IncomingFileMessage.java |  6 ++-
 .../org/apache/cassandra/tools/NodeProbe.java   |  6 ++-
 .../cassandra/tools/SSTableLevelResetter.java   |  6 ++-
 .../org/apache/cassandra/transport/Message.java |  6 ++-
 .../org/apache/cassandra/utils/CLibrary.java|  1 +
 .../cassandra/utils/FastByteOperations.java |  4 +-
 .../cassandra/utils/JVMStabilityInspector.java  | 45 
 .../apache/cassandra/utils/ResourceWatcher.java |  1 +
 23 files changed, 117 insertions(+), 42 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/14521347/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 054870b..8d64d34 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.1
+ * Shutdown JVM on OOM (CASSANDRA-7507)
  * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
  * Don't duplicate sstables smaller than split size when using
the sstablesplitter tool (CASSANDRA-7616)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/14521347/src/java/org/apache/cassandra/auth/CassandraAuthorizer.java
--
diff --git a/src/java/org/apache/cassandra/auth/CassandraAuthorizer.java 
b/src/java/org/apache/cassandra/auth/CassandraAuthorizer.java
index 9b22e22..20060c0 100644
--- a/src/java/org/apache/cassandra/auth/CassandraAuthorizer.java
+++ b/src/java/org/apache/cassandra/auth/CassandraAuthorizer.java
@@ -185,7 +185,7 @@ public class CassandraAuthorizer implements IAuthorizer
 {
 process(String.format(DELETE FROM %s.%s WHERE username = '%s', 
Auth.AUTH_KS, PERMISSIONS_CF, escape(droppedUser)));
 }
-catch (Throwable e)
+catch (RequestExecutionException e)
 {
 logger.warn(CassandraAuthorizer failed to revoke all permissions 
of {}: {}, droppedUser, e);
 }
@@ -204,7 +204,7 @@ public class CassandraAuthorizer implements IAuthorizer
  PERMISSIONS_CF,
  escape(droppedResource.getName(;
 }
-catch (Throwable e)
+catch (RequestExecutionException e)
 {
 logger.warn(CassandraAuthorizer failed to revoke all permissions 
on {}: {}, droppedResource, e);
 return;
@@ -220,7 +220,7 @@ public class CassandraAuthorizer implements IAuthorizer
   escape(row.getString(USERNAME)),
   escape(droppedResource.getName(;
 }
-catch (Throwable e)
+catch (RequestExecutionException e)
 {
 logger.warn(CassandraAuthorizer failed to revoke all 
permissions on {}: {}, droppedResource, e);
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/14521347/src/java/org/apache/cassandra/concurrent/AbstractTracingAwareExecutorService.java
--
diff --git 
a/src/java/org/apache/cassandra/concurrent/AbstractTracingAwareExecutorService.java
 
b/src/java/org/apache/cassandra/concurrent/AbstractTracingAwareExecutorService.java

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

2014-10-09 Thread jmckenzie

Merge branch 'cassandra-2.1' into trunk

Conflicts:
src/java/org/apache/cassandra/db/compaction/CompactionManager.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/50734f20
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/50734f20
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/50734f20

Branch: refs/heads/trunk
Commit: 50734f2073e3155e3b82de64ff078fad03251809
Parents: a9bad89 1452134
Author: Joshua McKenzie josh.mcken...@datastax.com
Authored: Thu Oct 9 10:44:35 2014 -0500
Committer: Joshua McKenzie josh.mcken...@datastax.com
Committed: Thu Oct 9 10:44:35 2014 -0500

--
 CHANGES.txt |  1 +
 .../cassandra/auth/CassandraAuthorizer.java |  6 +--
 .../AbstractTracingAwareExecutorService.java|  2 +
 .../DebuggableScheduledThreadPoolExecutor.java  |  7 ++-
 .../apache/cassandra/concurrent/SEPWorker.java  |  3 ++
 .../db/commitlog/CommitLogReplayer.java |  1 +
 .../db/commitlog/CommitLogSegmentManager.java   |  2 +
 .../db/compaction/CompactionManager.java|  6 +--
 .../cassandra/db/compaction/Scrubber.java   |  2 +
 src/java/org/apache/cassandra/gms/Gossiper.java |  2 +
 .../io/sstable/SSTableSimpleUnsortedWriter.java |  2 +
 .../org/apache/cassandra/io/util/FileUtils.java |  5 ++-
 .../cassandra/service/CassandraDaemon.java  | 20 ++---
 .../cassandra/service/StorageService.java   | 18 ++--
 .../cassandra/streaming/ConnectionHandler.java  |  6 ++-
 .../streaming/messages/IncomingFileMessage.java |  6 ++-
 .../org/apache/cassandra/tools/NodeProbe.java   |  6 ++-
 .../cassandra/tools/SSTableLevelResetter.java   |  6 ++-
 .../org/apache/cassandra/transport/Message.java |  6 ++-
 .../org/apache/cassandra/utils/CLibrary.java|  1 +
 .../cassandra/utils/FastByteOperations.java |  4 +-
 .../cassandra/utils/JVMStabilityInspector.java  | 45 
 .../apache/cassandra/utils/ResourceWatcher.java |  1 +
 23 files changed, 117 insertions(+), 41 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/50734f20/CHANGES.txt
--
diff --cc CHANGES.txt
index 49f6006,8d64d34..16b450a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,36 -1,5 +1,37 @@@
 +3.0
 + * Support for scripting languages in user-defined functions (CASSANDRA-7526)
 + * Support for aggregation functions (CASSANDRA-4914)
 + * Improve query to read paxos table on propose (CASSANDRA-7929)
 + * Remove cassandra-cli (CASSANDRA-7920)
 + * Optimize java source-based UDF invocation (CASSANDRA-7924)
 + * Accept dollar quoted strings in CQL (CASSANDRA-7769)
 + * Make assassinate a first class command (CASSANDRA-7935)
 + * Support IN clause on any clustering column (CASSANDRA-4762)
 + * Improve compaction logging (CASSANDRA-7818)
 + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917)
 + * Support Java source code for user-defined functions (CASSANDRA-7562)
 + * Require arg types to disambiguate UDF drops (CASSANDRA-7812)
 + * Do anticompaction in groups (CASSANDRA-6851)
 + * Verify that UDF class methods are static (CASSANDRA-7781)
 + * Support pure user-defined functions (CASSANDRA-7395, 7740)
 + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416)
 + * Move sstable RandomAccessReader to nio2, which allows using the
 +   FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050)
 + * Remove CQL2 (CASSANDRA-5918)
 + * Add Thrift get_multi_slice call (CASSANDRA-6757)
 + * Optimize fetching multiple cells by name (CASSANDRA-6933)
 + * Allow compilation in java 8 (CASSANDRA-7028)
 + * Make incremental repair default (CASSANDRA-7250)
 + * Enable code coverage thru JaCoCo (CASSANDRA-7226)
 + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) 
 + * Shorten SSTable path (CASSANDRA-6962)
 + * Use unsafe mutations for most unit tests (CASSANDRA-6969)
 + * Fix race condition during calculation of pending ranges (CASSANDRA-7390)
 + * Fail on very large batch sizes (CASSANDRA-8011)
 +
 +
  2.1.1
+  * Shutdown JVM on OOM (CASSANDRA-7507)
   * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
   * Don't duplicate sstables smaller than split size when using
 the sstablesplitter tool (CASSANDRA-7616)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/50734f20/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 0f8acba,51f45b8..b08668e
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@@ -1040,74 -986,64 +1037,75 @@@ public class

git commit: Shutdown JVM on OOM

2014-10-09 Thread jmckenzie

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 5a33a9ad2 - 145213470


Shutdown JVM on OOM

patch by Josh McKenzie; reviewed by Aleksey Yeschenko for CASSANDRA-7507


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/14521347
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/14521347
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/14521347

Branch: refs/heads/cassandra-2.1
Commit: 1452134707e2a143def3c03d1db416b1c8197565
Parents: 5a33a9a
Author: Joshua McKenzie josh.mcken...@datastax.com
Authored: Thu Oct 9 10:40:57 2014 -0500
Committer: Joshua McKenzie josh.mcken...@datastax.com
Committed: Thu Oct 9 10:40:57 2014 -0500

--
 CHANGES.txt |  1 +
 .../cassandra/auth/CassandraAuthorizer.java |  6 +--
 .../AbstractTracingAwareExecutorService.java|  2 +
 .../DebuggableScheduledThreadPoolExecutor.java  |  7 ++-
 .../apache/cassandra/concurrent/SEPWorker.java  |  3 ++
 .../db/commitlog/CommitLogReplayer.java |  1 +
 .../db/commitlog/CommitLogSegmentManager.java   |  2 +
 .../db/compaction/CompactionManager.java|  7 +--
 .../cassandra/db/compaction/Scrubber.java   |  2 +
 src/java/org/apache/cassandra/gms/Gossiper.java |  2 +
 .../io/sstable/SSTableSimpleUnsortedWriter.java |  2 +
 .../org/apache/cassandra/io/util/FileUtils.java |  5 ++-
 .../cassandra/service/CassandraDaemon.java  | 20 ++---
 .../cassandra/service/StorageService.java   | 18 ++--
 .../cassandra/streaming/ConnectionHandler.java  |  6 ++-
 .../streaming/messages/IncomingFileMessage.java |  6 ++-
 .../org/apache/cassandra/tools/NodeProbe.java   |  6 ++-
 .../cassandra/tools/SSTableLevelResetter.java   |  6 ++-
 .../org/apache/cassandra/transport/Message.java |  6 ++-
 .../org/apache/cassandra/utils/CLibrary.java|  1 +
 .../cassandra/utils/FastByteOperations.java |  4 +-
 .../cassandra/utils/JVMStabilityInspector.java  | 45 
 .../apache/cassandra/utils/ResourceWatcher.java |  1 +
 23 files changed, 117 insertions(+), 42 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/14521347/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 054870b..8d64d34 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.1
+ * Shutdown JVM on OOM (CASSANDRA-7507)
  * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
  * Don't duplicate sstables smaller than split size when using
the sstablesplitter tool (CASSANDRA-7616)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/14521347/src/java/org/apache/cassandra/auth/CassandraAuthorizer.java
--
diff --git a/src/java/org/apache/cassandra/auth/CassandraAuthorizer.java 
b/src/java/org/apache/cassandra/auth/CassandraAuthorizer.java
index 9b22e22..20060c0 100644
--- a/src/java/org/apache/cassandra/auth/CassandraAuthorizer.java
+++ b/src/java/org/apache/cassandra/auth/CassandraAuthorizer.java
@@ -185,7 +185,7 @@ public class CassandraAuthorizer implements IAuthorizer
 {
 process(String.format(DELETE FROM %s.%s WHERE username = '%s', 
Auth.AUTH_KS, PERMISSIONS_CF, escape(droppedUser)));
 }
-catch (Throwable e)
+catch (RequestExecutionException e)
 {
 logger.warn(CassandraAuthorizer failed to revoke all permissions 
of {}: {}, droppedUser, e);
 }
@@ -204,7 +204,7 @@ public class CassandraAuthorizer implements IAuthorizer
  PERMISSIONS_CF,
  escape(droppedResource.getName(;
 }
-catch (Throwable e)
+catch (RequestExecutionException e)
 {
 logger.warn(CassandraAuthorizer failed to revoke all permissions 
on {}: {}, droppedResource, e);
 return;
@@ -220,7 +220,7 @@ public class CassandraAuthorizer implements IAuthorizer
   escape(row.getString(USERNAME)),
   escape(droppedResource.getName(;
 }
-catch (Throwable e)
+catch (RequestExecutionException e)
 {
 logger.warn(CassandraAuthorizer failed to revoke all 
permissions on {}: {}, droppedResource, e);
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/14521347/src/java/org/apache/cassandra/concurrent/AbstractTracingAwareExecutorService.java
--
diff --git 
a/src/java/org/apache/cassandra/concurrent/AbstractTracingAwareExecutorService.java

[jira] [Created] (CASSANDRA-8092) Check for swallowed Throwable in CI environment

2014-10-09 Thread Joshua McKenzie (JIRA)

Joshua McKenzie created CASSANDRA-8092:
--

 Summary: Check for swallowed Throwable in CI environment
 Key: CASSANDRA-8092
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8092
 Project: Cassandra
  Issue Type: Test
Reporter: Joshua McKenzie
Assignee: Michael Shuler
Priority: Minor
 Fix For: 2.1.1
 Attachments: 7507_check_swallowed.txt

Reference CASSANDRA-7507.  We don't want to swallow Throwable or Exception's in 
the code-base but rather delegate them to the JVMStabilityInspector added in 
that ticket.  Attached patch adds a python script to tools\bin (maybe incorrect 
placement) that'll check for Throwable and return count of swallowed.  It will 
also do the same for Exception if you pass it the flag for that.

Ideally we'd integrate checking our code-base on build in the CI environment 
and fail if we have count  0 on Throwable for now, and update it to include 
Exception as well after CASSANDRA-7579.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8066) High Heap Consumption due to high number of SSTableReader


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-8066:
--
Assignee: T Jake Luciani

 High Heap Consumption due to high number of SSTableReader
 -

 Key: CASSANDRA-8066
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8066
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1.0
Reporter: Benoit Lacelle
Assignee: T Jake Luciani
 Fix For: 2.1.1


 Given a workload with quite a lot of reads, I recently encountered high heap 
 memory consumption. Given 2GB of Heap, it appears I have 750.000+ tasks in 
 SSTableReader.syncExecutor, consuming more than 1.2GB. These tasks have type 
 SSTableReader$5, which I guess corresponds to :
 {code}
 readMeterSyncFuture = syncExecutor.scheduleAtFixedRate(new Runnable()
 {
 public void run()
 {
 if (!isCompacted.get())
 {
 meterSyncThrottle.acquire();
 SystemKeyspace.persistSSTableReadMeter(desc.ksname, desc.cfname, 
 desc.generation, readMeter);
 }
 }
 }, 1, 5, TimeUnit.MINUTES);
 {code}
 I do not have have to the environment right now, but I could provide a 
 threaddump later if necessary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8066) High Heap Consumption due to high number of SSTableReader


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-8066:
--
Fix Version/s: 2.1.1

 High Heap Consumption due to high number of SSTableReader
 -

 Key: CASSANDRA-8066
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8066
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1.0
Reporter: Benoit Lacelle
Assignee: T Jake Luciani
 Fix For: 2.1.1


 Given a workload with quite a lot of reads, I recently encountered high heap 
 memory consumption. Given 2GB of Heap, it appears I have 750.000+ tasks in 
 SSTableReader.syncExecutor, consuming more than 1.2GB. These tasks have type 
 SSTableReader$5, which I guess corresponds to :
 {code}
 readMeterSyncFuture = syncExecutor.scheduleAtFixedRate(new Runnable()
 {
 public void run()
 {
 if (!isCompacted.get())
 {
 meterSyncThrottle.acquire();
 SystemKeyspace.persistSSTableReadMeter(desc.ksname, desc.cfname, 
 desc.generation, readMeter);
 }
 }
 }, 1, 5, TimeUnit.MINUTES);
 {code}
 I do not have have to the environment right now, but I could provide a 
 threaddump later if necessary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] git commit: Allow concurrent use of CQLBulkOutputFormat patch by Paul Pak; reviewed by Piotr Kolaczkowski for CASSANDRA-7776

Allow concurrent use of CQLBulkOutputFormat
patch by Paul Pak; reviewed by Piotr Kolaczkowski for CASSANDRA-7776


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/708adca7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/708adca7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/708adca7

Branch: refs/heads/trunk
Commit: 708adca7d702ce2466e80c4bf83f3c5b7cecca46
Parents: 1452134
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Oct 9 11:10:20 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Oct 9 11:10:20 2014 -0500

--
 CHANGES.txt   | 1 +
 .../org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/708adca7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8d64d34..417e8c1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.1
+ * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)
  * Shutdown JVM on OOM (CASSANDRA-7507)
  * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
  * Don't duplicate sstables smaller than split size when using

http://git-wip-us.apache.org/repos/asf/cassandra/blob/708adca7/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
index 7a75bb4..0a7b9f1 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
@@ -23,6 +23,7 @@ import java.nio.ByteBuffer;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.UUID;
 
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.exceptions.InvalidRequestException;
@@ -150,7 +151,7 @@ public class CqlBulkRecordWriter extends 
AbstractBulkRecordWriterObject, ListB
 
 private File getColumnFamilyDirectory() throws IOException
 {
-File dir = new File(String.format(%s%s%s%s%s, getOutputLocation(), 
File.separator, keyspace, File.separator, columnFamily));
+File dir = new File(String.format(%s%s%s%s%s-%s, 
getOutputLocation(), File.separator, keyspace, File.separator, columnFamily, 
UUID.randomUUID().toString()));
 
 if (!dir.exists()  !dir.mkdirs())
 {

[1/3] git commit: Allow concurrent use of CQLBulkOutputFormat patch by Paul Pak; reviewed by Piotr Kolaczkowski for CASSANDRA-7776

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 145213470 - 708adca7d
  refs/heads/trunk 50734f207 - 7892b8355


Allow concurrent use of CQLBulkOutputFormat
patch by Paul Pak; reviewed by Piotr Kolaczkowski for CASSANDRA-7776


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/708adca7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/708adca7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/708adca7

Branch: refs/heads/cassandra-2.1
Commit: 708adca7d702ce2466e80c4bf83f3c5b7cecca46
Parents: 1452134
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Oct 9 11:10:20 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Oct 9 11:10:20 2014 -0500

--
 CHANGES.txt   | 1 +
 .../org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/708adca7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8d64d34..417e8c1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.1
+ * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)
  * Shutdown JVM on OOM (CASSANDRA-7507)
  * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
  * Don't duplicate sstables smaller than split size when using

http://git-wip-us.apache.org/repos/asf/cassandra/blob/708adca7/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
index 7a75bb4..0a7b9f1 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java
@@ -23,6 +23,7 @@ import java.nio.ByteBuffer;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.UUID;
 
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.exceptions.InvalidRequestException;
@@ -150,7 +151,7 @@ public class CqlBulkRecordWriter extends 
AbstractBulkRecordWriterObject, ListB
 
 private File getColumnFamilyDirectory() throws IOException
 {
-File dir = new File(String.format(%s%s%s%s%s, getOutputLocation(), 
File.separator, keyspace, File.separator, columnFamily));
+File dir = new File(String.format(%s%s%s%s%s-%s, 
getOutputLocation(), File.separator, keyspace, File.separator, columnFamily, 
UUID.randomUUID().toString()));
 
 if (!dir.exists()  !dir.mkdirs())
 {

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7892b835
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7892b835
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7892b835

Branch: refs/heads/trunk
Commit: 7892b8355ca640055420e030b1c06e39b065452a
Parents: 50734f2 708adca
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Oct 9 11:10:29 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Oct 9 11:10:29 2014 -0500

--
 CHANGES.txt   | 1 +
 .../org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7892b835/CHANGES.txt
--
diff --cc CHANGES.txt
index 16b450a,417e8c1..5ec542c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,36 -1,5 +1,37 @@@
 +3.0
 + * Support for scripting languages in user-defined functions (CASSANDRA-7526)
 + * Support for aggregation functions (CASSANDRA-4914)
 + * Improve query to read paxos table on propose (CASSANDRA-7929)
 + * Remove cassandra-cli (CASSANDRA-7920)
 + * Optimize java source-based UDF invocation (CASSANDRA-7924)
 + * Accept dollar quoted strings in CQL (CASSANDRA-7769)
 + * Make assassinate a first class command (CASSANDRA-7935)
 + * Support IN clause on any clustering column (CASSANDRA-4762)
 + * Improve compaction logging (CASSANDRA-7818)
 + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917)
 + * Support Java source code for user-defined functions (CASSANDRA-7562)
 + * Require arg types to disambiguate UDF drops (CASSANDRA-7812)
 + * Do anticompaction in groups (CASSANDRA-6851)
 + * Verify that UDF class methods are static (CASSANDRA-7781)
 + * Support pure user-defined functions (CASSANDRA-7395, 7740)
 + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416)
 + * Move sstable RandomAccessReader to nio2, which allows using the
 +   FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050)
 + * Remove CQL2 (CASSANDRA-5918)
 + * Add Thrift get_multi_slice call (CASSANDRA-6757)
 + * Optimize fetching multiple cells by name (CASSANDRA-6933)
 + * Allow compilation in java 8 (CASSANDRA-7028)
 + * Make incremental repair default (CASSANDRA-7250)
 + * Enable code coverage thru JaCoCo (CASSANDRA-7226)
 + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) 
 + * Shorten SSTable path (CASSANDRA-6962)
 + * Use unsafe mutations for most unit tests (CASSANDRA-6969)
 + * Fix race condition during calculation of pending ranges (CASSANDRA-7390)
 + * Fail on very large batch sizes (CASSANDRA-8011)
 +
 +
  2.1.1
+  * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)
   * Shutdown JVM on OOM (CASSANDRA-7507)
   * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
   * Don't duplicate sstables smaller than split size when using

[jira] [Commented] (CASSANDRA-7776) Allow multiple MR jobs to concurrently write to the same column family from the same node using CqlBulkOutputFormat


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165309#comment-14165309
 ] 

Jonathan Ellis commented on CASSANDRA-7776:
---

committed

 Allow multiple MR jobs to concurrently write to the same column family from 
 the same node using CqlBulkOutputFormat
 ---

 Key: CASSANDRA-7776
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7776
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Paul Pak
Assignee: Paul Pak
Priority: Minor
  Labels: cql3, hadoop
 Attachments: trunk-7776-v1.txt


 After sstable files are written, all files in the specified output directory 
 are loaded (transferred) to the remote cassandra cluster. If multiple writes 
 occur on a node to the same table (i.e. directory), then the multiple load 
 processes end up transferring the same sstable files multiple times. 
 Furthermore, if directory cleanup of successful outputs is set to occur 
 ([CASSANDRA-|https://issues.apache.org/jira/browse/CASSANDRA-]), then 
 there could be errors caused by write/load contention.
 This can be simply remedied by using unique output directories for each MR 
 job.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-5586) Remove cli usage from dtests


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165316#comment-14165316
 ] 

Aleksey Yeschenko commented on CASSANDRA-5586:
--

Would be best to convert tests using CLI to pure Thrift, not native proto/cql. 
That extra Thrift coverage is good to have.

 Remove cli usage from dtests
 

 Key: CASSANDRA-5586
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5586
 Project: Cassandra
  Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Ryan McGuire
Priority: Minor

 The dtests in some situations fork the cli.  With the cli essentially 
 stagnant now, there's no need to do this when the same thing can be 
 accomplished with a thrift or cql call. (ccm's convenience api for invoking 
 the cli could probably also be removed at this point)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-5586) Remove cli usage from dtests


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165320#comment-14165320
 ] 

Aleksey Yeschenko commented on CASSANDRA-5586:
--

(Even if there is a CQL equivalent)

 Remove cli usage from dtests
 

 Key: CASSANDRA-5586
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5586
 Project: Cassandra
  Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Ryan McGuire
Priority: Minor

 The dtests in some situations fork the cli.  With the cli essentially 
 stagnant now, there's no need to do this when the same thing can be 
 accomplished with a thrift or cql call. (ccm's convenience api for invoking 
 the cli could probably also be removed at this point)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8075) Make netty pipeline fully async


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165317#comment-14165317
 ] 

Jonathan Ellis commented on CASSANDRA-8075:
---

What is the difference between 5239 and the fully async you describe here?

 Make netty pipeline fully async
 ---

 Key: CASSANDRA-8075
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8075
 Project: Cassandra
  Issue Type: Improvement
Reporter: T Jake Luciani
  Labels: performance
 Fix For: 3.0


 Related to CASSANDRA-5239.  Netty does much better if we remove the request 
 thread pool jump. 
 Here is a benchmark of our netty pipeline returning a static value (so no 
 actual cassandra processing).  When the thread pool is removed we see a 
 dramatic improvement.
 http://cstar.datastax.com/graph?stats=64ed5186-4e51-11e4-a8d5-bc764e04482cmetric=op_rateoperation=2_usersmoothing=1show_aggregates=truexmin=0xmax=228.25ymin=0ymax=150217.1
 On my laptop using the async branch can match redis bench on the same machine 
 ~160k ops/sec.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8075) Make netty pipeline fully async


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165322#comment-14165322
 ] 

T Jake Luciani commented on CASSANDRA-8075:
---

Remove the request thread pool used in netty.  Also ideally remove the read 
stage thread pool

 Make netty pipeline fully async
 ---

 Key: CASSANDRA-8075
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8075
 Project: Cassandra
  Issue Type: Improvement
Reporter: T Jake Luciani
  Labels: performance
 Fix For: 3.0


 Related to CASSANDRA-5239.  Netty does much better if we remove the request 
 thread pool jump. 
 Here is a benchmark of our netty pipeline returning a static value (so no 
 actual cassandra processing).  When the thread pool is removed we see a 
 dramatic improvement.
 http://cstar.datastax.com/graph?stats=64ed5186-4e51-11e4-a8d5-bc764e04482cmetric=op_rateoperation=2_usersmoothing=1show_aggregates=truexmin=0xmax=228.25ymin=0ymax=150217.1
 On my laptop using the async branch can match redis bench on the same machine 
 ~160k ops/sec.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7777) Ability to clean up local sstable files after they've been loaded by the CqlBulkRecordWriter

2014-10-09 Thread Paul Pak (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Pak updated CASSANDRA-:

Attachment: trunk--v3.txt

 Ability to clean up local sstable files after they've been loaded by the 
 CqlBulkRecordWriter
 

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Paul Pak
Assignee: Paul Pak
Priority: Minor
  Labels: cql3, hadoop
 Attachments: trunk--v1.txt, trunk--v2.txt, trunk--v3.txt


 Deleting the source files should most likely be the default behavior with the 
 ability to disable it via config.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7979) Acceptable time skew for C*

[
https://issues.apache.org/jira/browse/CASSANDRA-7979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Ellis updated CASSANDRA-7979:
--
Reviewer: Joshua McKenzie

Benedict is out for a month, I'll ask [~JoshuaMcKenzie] to review.

Acceptable time skew for C*
---

Key: CASSANDRA-7979
URL: https://issues.apache.org/jira/browse/CASSANDRA-7979
Project: Cassandra
Issue Type: Improvement
Reporter: sankalp kohli
Assignee: sankalp kohli
Priority: Minor
Attachments: 2.0_7979.diff, trunk_7979.diff

It is very hard to know the bounds on clock skew required for C* to work
properly. Since the resolution is based on time and is at thrift column
level, it depends on the application. How fast is the application updating
the same column. If you update a column say after 5 millisecond and the clock
skew is more than that, you might not see the updates in correct order.
In this JIRA, I am proposing a change which will answer this question: How
much clock skew is acceptable for a given application. This will help answer
the question whether the system needs some alternate NTP algorithms to keep
time in sync.
If we measure the time difference between two updates to the same column, we
will be able to answer the question on clock skew.
We can implement this in memtable(AtomicSortedColumns.addColumn). If we find
that a column is updated within say 100 millisecond, add the diff to a
histogram. Since this might have performance issues, we might want to have
some throttling like randomization or only enable it for a small time via
nodetool.
With this histogram, we will know what is an acceptable clock skew.
Also apart from column resolution, is there any other area which will be
affected by clock skew?
Note: For the sake of argument, I am not talking about back date deletes or
application modified timestamps.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7777) Ability to clean up local sstable files after they've been loaded by the CqlBulkRecordWriter

2014-10-09 Thread Paul Pak (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165336#comment-14165336
 ] 

Paul Pak commented on CASSANDRA-:
-

Updated the configuration name to cassandra.output.delete.source. I suppose 
keeping the helper methods to set the config on CqlBulkOutputFormat is the 
indication that this is applicable to this OutputFormat. If this functionality 
is generalized in the future, the helper methods should probably be moved to 
CqlConfigHelper.

 Ability to clean up local sstable files after they've been loaded by the 
 CqlBulkRecordWriter
 

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Paul Pak
Assignee: Paul Pak
Priority: Minor
  Labels: cql3, hadoop
 Attachments: trunk--v1.txt, trunk--v2.txt, trunk--v3.txt


 Deleting the source files should most likely be the default behavior with the 
 ability to disable it via config.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8075) Make netty pipeline fully async


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-8075:
--
Description: 
Related to CASSANDRA-5239.  Netty does much better if we remove the request 
thread pool jump. 

Here is a benchmark of our netty pipeline returning a static value (so no 
actual cassandra processing).  When the thread pool is removed we see a 
dramatic improvement.

http://cstar.datastax.com/graph?stats=64ed5186-4e51-11e4-a8d5-bc764e04482cmetric=op_rateoperation=2_usersmoothing=1show_aggregates=truexmin=0xmax=228.25ymin=0ymax=150217.1

On my laptop using the async branch can match redis bench on the same machine 
~160k ops/sec.



  was:
Related to CASSANDRA-7761.  Netty does much better if we remove the request 
thread pool jump. 

Here is a benchmark of our netty pipeline returning a static value (so no 
actual cassandra processing).  When the thread pool is removed we see a 
dramatic improvement.

http://cstar.datastax.com/graph?stats=64ed5186-4e51-11e4-a8d5-bc764e04482cmetric=op_rateoperation=2_usersmoothing=1show_aggregates=truexmin=0xmax=228.25ymin=0ymax=150217.1

On my laptop using the async branch can match redis bench on the same machine 
~160k ops/sec.




 Make netty pipeline fully async
 ---

 Key: CASSANDRA-8075
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8075
 Project: Cassandra
  Issue Type: Improvement
Reporter: T Jake Luciani
  Labels: performance
 Fix For: 3.0


 Related to CASSANDRA-5239.  Netty does much better if we remove the request 
 thread pool jump. 
 Here is a benchmark of our netty pipeline returning a static value (so no 
 actual cassandra processing).  When the thread pool is removed we see a 
 dramatic improvement.
 http://cstar.datastax.com/graph?stats=64ed5186-4e51-11e4-a8d5-bc764e04482cmetric=op_rateoperation=2_usersmoothing=1show_aggregates=truexmin=0xmax=228.25ymin=0ymax=150217.1
 On my laptop using the async branch can match redis bench on the same machine 
 ~160k ops/sec.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8075) Make netty pipeline fully async


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-8075:
--
Description: 
Related to CASSANDRA-7761.  Netty does much better if we remove the request 
thread pool jump. 

Here is a benchmark of our netty pipeline returning a static value (so no 
actual cassandra processing).  When the thread pool is removed we see a 
dramatic improvement.

http://cstar.datastax.com/graph?stats=64ed5186-4e51-11e4-a8d5-bc764e04482cmetric=op_rateoperation=2_usersmoothing=1show_aggregates=truexmin=0xmax=228.25ymin=0ymax=150217.1

On my laptop using the async branch can match redis bench on the same machine 
~160k ops/sec.



  was:
Related to CASSANDRA-5239.  Netty does much better if we remove the request 
thread pool jump. 

Here is a benchmark of our netty pipeline returning a static value (so no 
actual cassandra processing).  When the thread pool is removed we see a 
dramatic improvement.

http://cstar.datastax.com/graph?stats=64ed5186-4e51-11e4-a8d5-bc764e04482cmetric=op_rateoperation=2_usersmoothing=1show_aggregates=truexmin=0xmax=228.25ymin=0ymax=150217.1

On my laptop using the async branch can match redis bench on the same machine 
~160k ops/sec.




 Make netty pipeline fully async
 ---

 Key: CASSANDRA-8075
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8075
 Project: Cassandra
  Issue Type: Improvement
Reporter: T Jake Luciani
  Labels: performance
 Fix For: 3.0


 Related to CASSANDRA-7761.  Netty does much better if we remove the request 
 thread pool jump. 
 Here is a benchmark of our netty pipeline returning a static value (so no 
 actual cassandra processing).  When the thread pool is removed we see a 
 dramatic improvement.
 http://cstar.datastax.com/graph?stats=64ed5186-4e51-11e4-a8d5-bc764e04482cmetric=op_rateoperation=2_usersmoothing=1show_aggregates=truexmin=0xmax=228.25ymin=0ymax=150217.1
 On my laptop using the async branch can match redis bench on the same machine 
 ~160k ops/sec.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Integrate Sigar library and add basic OS performance checks on startup

2014-10-09 Thread jake

Repository: cassandra
Updated Branches:
  refs/heads/trunk 7892b8355 - 861c4cf2d


Integrate Sigar library and add basic OS performance checks on startup

Patch by Amit Kumar and Josh McKenzie; reviewed by tjake for (CASSANDRA-7838)


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/861c4cf2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/861c4cf2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/861c4cf2

Branch: refs/heads/trunk
Commit: 861c4cf2de9075c721945fb7ef84a23030a4f35a
Parents: 7892b83
Author: Joshua McKenzie josh.mcken...@datastax.com
Authored: Wed Oct 8 17:08:11 2014 -0500
Committer: Jake Luciani j...@apache.org
Committed: Thu Oct 9 12:32:57 2014 -0400

--
 CHANGES.txt |   1 +
 NOTICE.txt  |   3 +
 bin/cassandra.bat   |   5 +-
 bin/cassandra.in.bat|   3 +
 bin/cassandra.in.sh |   3 +
 build.xml   |   3 +
 conf/cassandra-env.ps1  |   3 +
 conf/cassandra-env.sh   |   5 +
 debian/cassandra.install|   1 +
 lib/licenses/sigar-1.6.4.txt| 201 +++
 lib/sigar-1.6.4.jar | Bin 0 - 428580 bytes
 lib/sigar-bin/libsigar-amd64-freebsd-6.so   | Bin 0 - 210641 bytes
 lib/sigar-bin/libsigar-amd64-linux.so   | Bin 0 - 246605 bytes
 lib/sigar-bin/libsigar-amd64-solaris.so | Bin 0 - 251360 bytes
 lib/sigar-bin/libsigar-ia64-hpux-11.sl  | Bin 0 - 577452 bytes
 lib/sigar-bin/libsigar-ia64-linux.so| Bin 0 - 494929 bytes
 lib/sigar-bin/libsigar-pa-hpux-11.sl| Bin 0 - 516096 bytes
 lib/sigar-bin/libsigar-ppc-aix-5.so | Bin 0 - 400925 bytes
 lib/sigar-bin/libsigar-ppc-linux.so | Bin 0 - 258547 bytes
 lib/sigar-bin/libsigar-ppc64-aix-5.so   | Bin 0 - 425077 bytes
 lib/sigar-bin/libsigar-ppc64-linux.so   | Bin 0 - 330767 bytes
 lib/sigar-bin/libsigar-s390x-linux.so   | Bin 0 - 269932 bytes
 lib/sigar-bin/libsigar-sparc-solaris.so | Bin 0 - 285004 bytes
 lib/sigar-bin/libsigar-sparc64-solaris.so   | Bin 0 - 261896 bytes
 lib/sigar-bin/libsigar-universal-macosx.dylib   | Bin 0 - 377668 bytes
 lib/sigar-bin/libsigar-universal64-macosx.dylib | Bin 0 - 397440 bytes
 lib/sigar-bin/libsigar-x86-freebsd-5.so | Bin 0 - 179751 bytes
 lib/sigar-bin/libsigar-x86-freebsd-6.so | Bin 0 - 179379 bytes
 lib/sigar-bin/libsigar-x86-linux.so | Bin 0 - 233385 bytes
 lib/sigar-bin/libsigar-x86-solaris.so   | Bin 0 - 242880 bytes
 lib/sigar-bin/sigar-amd64-winnt.dll | Bin 0 - 402432 bytes
 lib/sigar-bin/sigar-x86-winnt.dll   | Bin 0 - 266240 bytes
 lib/sigar-bin/sigar-x86-winnt.lib   | Bin 0 - 99584 bytes
 .../cassandra/service/CassandraDaemon.java  |  20 +-
 .../apache/cassandra/utils/SigarLibrary.java| 181 +
 35 files changed, 422 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/861c4cf2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5ec542c..be80ee3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Add Sigar library and perform basic OS settings check on startup 
(CASSANDRA-7838)
  * Support for scripting languages in user-defined functions (CASSANDRA-7526)
  * Support for aggregation functions (CASSANDRA-4914)
  * Improve query to read paxos table on propose (CASSANDRA-7929)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/861c4cf2/NOTICE.txt
--
diff --git a/NOTICE.txt b/NOTICE.txt
index bbe21d7..2fe15f5 100644
--- a/NOTICE.txt
+++ b/NOTICE.txt
@@ -63,3 +63,6 @@ HLL++ support provided by stream-lib
 
 Javassist
 (http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/)
+
+SIGAR
+http://sigar.hyperic.com/

http://git-wip-us.apache.org/repos/asf/cassandra/blob/861c4cf2/bin/cassandra.bat
--
diff --git a/bin/cassandra.bat b/bin/cassandra.bat
index 1606dcc..cba69c6 100644
--- a/bin/cassandra.bat
+++ b/bin/cassandra.bat
@@ -18,7 +18,6 @@
 if %OS% == Windows_NT setlocal
 
 set ARG=%1
-if /i %ARG% == LEGACY goto runLegacy
 set INSTALL=INSTALL
 set UNINSTALL=UNINSTALL
 
@@ -26,6 +25,7 @@ pushd %~dp0..
 if NOT DEFINED CASSANDRA_HOME set CASSANDRA_HOME=%CD%
 popd
 
+if /i %ARG% == LEGACY goto runLegacy
 REM 
-
 REM See if we have access to run

[jira] [Commented] (CASSANDRA-7838) Warn user when OS settings are poor / integrate sigar


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165352#comment-14165352
 ] 

T Jake Luciani commented on CASSANDRA-7838:
---

Thanks [~kumaramit01]!

 Warn user when OS settings are poor / integrate sigar
 -

 Key: CASSANDRA-7838
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7838
 Project: Cassandra
  Issue Type: Improvement
Reporter: T Jake Luciani
Assignee: AMIT KUMAR
Priority: Minor
  Labels: bootcamp, lhf
 Fix For: 3.0

 Attachments: 
 0005-CASSANDRA-7838-log-warning-for-networked-drives.patch, 
 0006-added-missing-winnt-native.patch, 
 0007-CASSANDRA-7838-WIP-adding-a-few-other-checks.patch, 
 0008-CASSANDRA-7838-Removed-network-check-and-read-latenc.patch, 
 0009-Bug-fix-swap-check-was-incorrect-fixed-logging-state.patch, 
 0010-Minor-log-change-happy-path.patch


 The Sigar project let's you probe os/cpu/filesystems across the major 
 platforms.
 https://github.com/hyperic/sigar
 It would be nice on start-up to use this to warn users if they are running 
 with settings that will make them sad, like Network drive or EBS on Ec2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8093) Add Metrics for OS and hardware usage with Sigar

T Jake Luciani created CASSANDRA-8093:
-

 Summary: Add Metrics for OS and hardware usage with Sigar
 Key: CASSANDRA-8093
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8093
 Project: Cassandra
  Issue Type: Improvement
Reporter: T Jake Luciani
 Fix For: 3.0


Now that CASSANDRA-7838 is in we can expose many OS and hardware specific 
metrics which will be useful for operators who want a holistic view of their 
nodes from Cassandra.

- Disk
- CPU
- Memory
- Network





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8061) tmplink files are not removed

2014-10-09 Thread Gianluca Borello (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165362#comment-14165362
 ] 

Gianluca Borello commented on CASSANDRA-8061:
-

I installed 2.1 on our staging environment. About 20 CFs, mixed workload 
depending on the kind of traffic we test at a specific moment, usually ranging 
from a total of 100 to 1 writes/s, and say no more than 50 reads/s. We 
heavily use TTLs, most of the rows completely expire in less than 24 hours. All 
the Cassandra metrics are always within reasonable values, the only thing to 
notice is that when we push the number of writes towards the upper end, we can 
get a considerable number of pending compactions (around 100-200). In normal 
conditions, pending compactions are just a few.

I am observing the tmplink files on a different number of CFs, but for your 
reference one that causes it is for example:

create table mounted_fs_by_agent1 (
customer int,
base bigint,
ts bigint,
agent int,
weight bigint,
value blob,
primary key ((customer, base), agent, ts)
) WITH
gc_grace_seconds = 3600 AND
compaction = {'class': 'LeveledCompactionStrategy'} AND
compression = { 'sstable_compression' : 'LZ4Compressor' } AND
CLUSTERING ORDER BY (agent DESC) AND
CLUSTERING ORDER BY (ts DESC);

Where blob are small blobs of ~1000-2000 bytes.

I rolled back to 2.0 because this bug is effectively a blocker for us, as the 
disk space is completely filled up in just a day or two because of this (unless 
I restart Cassandra), but if you are unable to replicate it I can set it up 
again and try to answer more questions you might have.

When I said fresh instance, I meant fresh install and sometimes after I 
actually start putting data.


 tmplink files are not removed
 -

 Key: CASSANDRA-8061
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8061
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Linux
Reporter: Gianluca Borello
Assignee: Ryan McGuire

 After installing 2.1.0, I'm experiencing a bunch of tmplink files that are 
 filling my disk. I found https://issues.apache.org/jira/browse/CASSANDRA-7803 
 and that is very similar, and I confirm it happens both on 2.1.0 as well as 
 from the latest commit on the cassandra-2.1 branch 
 (https://github.com/apache/cassandra/commit/aca80da38c3d86a40cc63d9a122f7d45258e4685
  from the cassandra-2.1)
 Even starting with a clean keyspace, after a few hours I get:
 $ sudo find /raid0 | grep tmplink | xargs du -hs
 2.7G  
 /raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Data.db
 13M   
 /raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Index.db
 1.8G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Data.db
 12M   
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Index.db
 5.2M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Index.db
 822M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Data.db
 7.3M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Index.db
 1.2G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Data.db
 6.7M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Index.db
 1.1G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Data.db
 11M   
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Index.db
 1.7G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Data.db
 812K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Index.db
 122M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Data.db
 744K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-739-Index.db
 660K

[jira] [Created] (CASSANDRA-8094) Heavy writes in RangeSlice read requests

2014-10-09 Thread Minh Do (JIRA)

Minh Do created CASSANDRA-8094:
--

 Summary: Heavy writes in RangeSlice read  requests 
 Key: CASSANDRA-8094
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8094
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Minh Do
Assignee: Minh Do
 Fix For: 2.0.11


RangeSlice requests always do a scheduler read repair when coordinators try to 
resolve replicats' responses no matter read_repair_chance is set or not.

Because of this, in low writes and high reads clusters, there are very high 
write requests going on between nodes.  

We should have an option to turn this off and this can be different than the 
read_repair_chance.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8094) Heavy writes in RangeSlice read requests

2014-10-09 Thread Minh Do (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Minh Do updated CASSANDRA-8094:
---
Description:
RangeSlice requests always do a scheduled read repair when coordinators try to
resolve replicas' responses no matter read_repair_chance is set or not.

Because of this, in low writes and high reads clusters, there are very high
write requests going on between nodes.

We should have an option to turn this off and this can be different than the
read_repair_chance.

was:
RangeSlice requests always do a scheduler read repair when coordinators try to
resolve replicats' responses no matter read_repair_chance is set or not.

Because of this, in low writes and high reads clusters, there are very high
write requests going on between nodes.

We should have an option to turn this off and this can be different than the
read_repair_chance.

Heavy writes in RangeSlice read requests
--

Key: CASSANDRA-8094
URL: https://issues.apache.org/jira/browse/CASSANDRA-8094
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Minh Do
Assignee: Minh Do
Fix For: 2.0.11

RangeSlice requests always do a scheduled read repair when coordinators try
to resolve replicas' responses no matter read_repair_chance is set or not.
Because of this, in low writes and high reads clusters, there are very high
write requests going on between nodes.
We should have an option to turn this off and this can be different than the
read_repair_chance.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-8084) GossipFilePropertySnitch and EC2MultiRegionSnitch when used in AWS/GCE clusters doesnt use the PRIVATE IPS for Intra-DC communications - When running nodetool repair

2014-10-09 Thread Yuki Morishita (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita reassigned CASSANDRA-8084:
-

Assignee: Yuki Morishita

 GossipFilePropertySnitch and EC2MultiRegionSnitch when used in AWS/GCE 
 clusters doesnt use the PRIVATE IPS for Intra-DC communications - When 
 running nodetool repair
 -

 Key: CASSANDRA-8084
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8084
 Project: Cassandra
  Issue Type: Bug
  Components: Config
 Environment: Tested this in GCE and AWS clusters. Created multi 
 region and multi dc cluster once in GCE and once in AWS and ran into the same 
 problem. 
 DISTRIB_ID=Ubuntu
 DISTRIB_RELEASE=12.04
 DISTRIB_CODENAME=precise
 DISTRIB_DESCRIPTION=Ubuntu 12.04.3 LTS
 NAME=Ubuntu
 VERSION=12.04.3 LTS, Precise Pangolin
 ID=ubuntu
 ID_LIKE=debian
 PRETTY_NAME=Ubuntu precise (12.04.3 LTS)
 VERSION_ID=12.04
 Tried to install Apache Cassandra version ReleaseVersion: 2.0.10 and also 
 latest DSE version which is 4.5 and which corresponds to 2.0.8.39.
Reporter: Jana
Assignee: Yuki Morishita
  Labels: features
 Fix For: 2.0.10


 Neither of these snitches(GossipFilePropertySnitch and EC2MultiRegionSnitch ) 
 used the PRIVATE IPS for communication between INTRA-DC nodes in my 
 multi-region multi-dc cluster in cloud(on both AWS and GCE) when I ran 
 nodetool repair -local. It works fine during regular reads.
  Here are the various cluster flavors I tried and failed- 
 AWS + Multi-REGION + Multi-DC + GossipPropertyFileSnitch + 
 (Prefer_local=true) in rackdc-properties file. 
 AWS + Multi-REGION + Multi-DC + EC2MultiRegionSnitch + (Prefer_local=true) in 
 rackdc-properties file. 
 GCE + Multi-REGION + Multi-DC + GossipPropertyFileSnitch + 
 (Prefer_local=true) in rackdc-properties file. 
 GCE + Multi-REGION + Multi-DC + EC2MultiRegionSnitch + (Prefer_local=true) in 
 rackdc-properties file. 
 I am expecting with the above setup all of my nodes in a given DC all 
 communicate via private ips since the cloud providers dont charge us for 
 using the private ips and they charge for using public ips.
 But they can use PUBLIC IPs for INTER-DC communications which is working as 
 expected. 
 Here is a snippet from my log files when I ran the nodetool repair -local - 
 Node responding to 'node running repair' 
 INFO [AntiEntropyStage:1] 2014-10-08 14:47:51,628 Validator.java (line 254) 
 [repair #1439f290-4efa-11e4-bf3a-df845ecf54f8] Sending completed merkle tree 
 to /54.172.118.222 for system_traces/sessions
  INFO [AntiEntropyStage:1] 2014-10-08 14:47:51,741 Validator.java (line 254) 
 [repair #1439f290-4efa-11e4-bf3a-df845ecf54f8] Sending completed merkle tree 
 to /54.172.118.222 for system_traces/events
 Node running repair - 
 INFO [AntiEntropyStage:1] 2014-10-08 14:47:51,927 RepairSession.java (line 
 166) [repair #1439f290-4efa-11e4-bf3a-df845ecf54f8] Received merkle tree for 
 events from /54.172.118.222
 Note: The IPs its communicating is all PUBLIC Ips and it should have used the 
 PRIVATE IPs starting with 172.x.x.x
 YAML file values : 
 The listen address is set to: PRIVATE IP
 The broadcast address is set to: PUBLIC IP
 The SEEDs address is set to: PUBLIC IPs from both DCs
 The SNITCHES tried: GPFS and EC2MultiRegionSnitch
 RACK-DC: Had prefer_local set to true. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8084) GossipFilePropertySnitch and EC2MultiRegionSnitch when used in AWS/GCE clusters doesnt use the PRIVATE IPS for Intra-DC communications - When running nodetool repair

2014-10-09 Thread Yuki Morishita (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-8084:
--
Attachment: 8084-2.0.txt

Attached patch looks up system table for 'preferred' IP to stream for an 
endpoint.
(Note: this does not change sstableloader behavior. Only Cassandra noe itself.)

Can you test if this works?

 GossipFilePropertySnitch and EC2MultiRegionSnitch when used in AWS/GCE 
 clusters doesnt use the PRIVATE IPS for Intra-DC communications - When 
 running nodetool repair
 -

 Key: CASSANDRA-8084
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8084
 Project: Cassandra
  Issue Type: Bug
  Components: Config
 Environment: Tested this in GCE and AWS clusters. Created multi 
 region and multi dc cluster once in GCE and once in AWS and ran into the same 
 problem. 
 DISTRIB_ID=Ubuntu
 DISTRIB_RELEASE=12.04
 DISTRIB_CODENAME=precise
 DISTRIB_DESCRIPTION=Ubuntu 12.04.3 LTS
 NAME=Ubuntu
 VERSION=12.04.3 LTS, Precise Pangolin
 ID=ubuntu
 ID_LIKE=debian
 PRETTY_NAME=Ubuntu precise (12.04.3 LTS)
 VERSION_ID=12.04
 Tried to install Apache Cassandra version ReleaseVersion: 2.0.10 and also 
 latest DSE version which is 4.5 and which corresponds to 2.0.8.39.
Reporter: Jana
Assignee: Yuki Morishita
  Labels: features
 Fix For: 2.0.11

 Attachments: 8084-2.0.txt


 Neither of these snitches(GossipFilePropertySnitch and EC2MultiRegionSnitch ) 
 used the PRIVATE IPS for communication between INTRA-DC nodes in my 
 multi-region multi-dc cluster in cloud(on both AWS and GCE) when I ran 
 nodetool repair -local. It works fine during regular reads.
  Here are the various cluster flavors I tried and failed- 
 AWS + Multi-REGION + Multi-DC + GossipPropertyFileSnitch + 
 (Prefer_local=true) in rackdc-properties file. 
 AWS + Multi-REGION + Multi-DC + EC2MultiRegionSnitch + (Prefer_local=true) in 
 rackdc-properties file. 
 GCE + Multi-REGION + Multi-DC + GossipPropertyFileSnitch + 
 (Prefer_local=true) in rackdc-properties file. 
 GCE + Multi-REGION + Multi-DC + EC2MultiRegionSnitch + (Prefer_local=true) in 
 rackdc-properties file. 
 I am expecting with the above setup all of my nodes in a given DC all 
 communicate via private ips since the cloud providers dont charge us for 
 using the private ips and they charge for using public ips.
 But they can use PUBLIC IPs for INTER-DC communications which is working as 
 expected. 
 Here is a snippet from my log files when I ran the nodetool repair -local - 
 Node responding to 'node running repair' 
 INFO [AntiEntropyStage:1] 2014-10-08 14:47:51,628 Validator.java (line 254) 
 [repair #1439f290-4efa-11e4-bf3a-df845ecf54f8] Sending completed merkle tree 
 to /54.172.118.222 for system_traces/sessions
  INFO [AntiEntropyStage:1] 2014-10-08 14:47:51,741 Validator.java (line 254) 
 [repair #1439f290-4efa-11e4-bf3a-df845ecf54f8] Sending completed merkle tree 
 to /54.172.118.222 for system_traces/events
 Node running repair - 
 INFO [AntiEntropyStage:1] 2014-10-08 14:47:51,927 RepairSession.java (line 
 166) [repair #1439f290-4efa-11e4-bf3a-df845ecf54f8] Received merkle tree for 
 events from /54.172.118.222
 Note: The IPs its communicating is all PUBLIC Ips and it should have used the 
 PRIVATE IPs starting with 172.x.x.x
 YAML file values : 
 The listen address is set to: PRIVATE IP
 The broadcast address is set to: PUBLIC IP
 The SEEDs address is set to: PUBLIC IPs from both DCs
 The SNITCHES tried: GPFS and EC2MultiRegionSnitch
 RACK-DC: Had prefer_local set to true. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-8061) tmplink files are not removed

2014-10-09 Thread Ryan McGuire (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire reassigned CASSANDRA-8061:
---

Assignee: Shawn Kumar  (was: Ryan McGuire)

 tmplink files are not removed
 -

 Key: CASSANDRA-8061
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8061
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Linux
Reporter: Gianluca Borello
Assignee: Shawn Kumar

 After installing 2.1.0, I'm experiencing a bunch of tmplink files that are 
 filling my disk. I found https://issues.apache.org/jira/browse/CASSANDRA-7803 
 and that is very similar, and I confirm it happens both on 2.1.0 as well as 
 from the latest commit on the cassandra-2.1 branch 
 (https://github.com/apache/cassandra/commit/aca80da38c3d86a40cc63d9a122f7d45258e4685
  from the cassandra-2.1)
 Even starting with a clean keyspace, after a few hours I get:
 $ sudo find /raid0 | grep tmplink | xargs du -hs
 2.7G  
 /raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Data.db
 13M   
 /raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Index.db
 1.8G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Data.db
 12M   
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Index.db
 5.2M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Index.db
 822M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Data.db
 7.3M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Index.db
 1.2G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Data.db
 6.7M  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Index.db
 1.1G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Data.db
 11M   
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Index.db
 1.7G  
 /raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Data.db
 812K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Index.db
 122M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Data.db
 744K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-739-Index.db
 660K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Index.db
 796K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Index.db
 137M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Data.db
 161M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Data.db
 139M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Data.db
 940K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Index.db
 936K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Index.db
 161M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Data.db
 672K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Index.db
 113M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Data.db
 116M  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Data.db
 712K  
 /raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Index.db
 127M

[jira] [Commented] (CASSANDRA-8066) High Heap Consumption due to high number of SSTableReader


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165441#comment-14165441
 ] 

T Jake Luciani commented on CASSANDRA-8066:
---

I suspect the early opening and re-opening of  sstables during compaction is 
creating many entries in the queue.

We should perhaps postpone this read meter tracking until the sstable has been 
fully written.



 High Heap Consumption due to high number of SSTableReader
 -

 Key: CASSANDRA-8066
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8066
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1.0
Reporter: Benoit Lacelle
Assignee: T Jake Luciani
 Fix For: 2.1.1


 Given a workload with quite a lot of reads, I recently encountered high heap 
 memory consumption. Given 2GB of Heap, it appears I have 750.000+ tasks in 
 SSTableReader.syncExecutor, consuming more than 1.2GB. These tasks have type 
 SSTableReader$5, which I guess corresponds to :
 {code}
 readMeterSyncFuture = syncExecutor.scheduleAtFixedRate(new Runnable()
 {
 public void run()
 {
 if (!isCompacted.get())
 {
 meterSyncThrottle.acquire();
 SystemKeyspace.persistSSTableReadMeter(desc.ksname, desc.cfname, 
 desc.generation, readMeter);
 }
 }
 }, 1, 5, TimeUnit.MINUTES);
 {code}
 I do not have have to the environment right now, but I could provide a 
 threaddump later if necessary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8095) create table query not working in concurrent

2014-10-09 Thread Katsutoshi Nagaoka (JIRA)

Katsutoshi Nagaoka created CASSANDRA-8095:
-

 Summary: create table query not working in concurrent
 Key: CASSANDRA-8095
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8095
 Project: Cassandra
  Issue Type: Bug
 Environment: 2.1.0 on Ubuntu 14.04
Reporter: Katsutoshi Nagaoka


After 2.1.0, I have encountered the strange behavior of create table.

When I executed create table query in concurrent, 

{code}
package cassandra.test;

import com.datastax.driver.core.Cluster;
import com.datastax.driver.core.Session;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;

public class ConcurrentCreateTable {

public static void main(String[] args) {
ConcurrentCreateTable instance = new ConcurrentCreateTable();
try {
instance.execute();
} catch (Exception e) {
e.printStackTrace();
}
}

private final Cluster cluster;

public ConcurrentCreateTable() {
this.cluster = Cluster.builder().addContactPoint(server1).build();
}

private void execute() throws Exception {
Session session = cluster.newSession();
session.execute(DROP KEYSPACE IF EXISTS ks);
session.execute(CREATE KEYSPACE IF NOT EXISTS ks WITH REPLICATION = 
{'class':'SimpleStrategy','replication_factor':3});
ExecutorService executor = Executors.newFixedThreadPool(10);
ListFuture? futures = new ArrayList();
futures.add(executor.submit(new CreateTableRunner(text)));
futures.add(executor.submit(new CreateTableRunner(int)));
futures.add(executor.submit(new CreateTableRunner(boolean)));
futures.add(executor.submit(new CreateTableRunner(bigint)));
executor.shutdown();
executor.awaitTermination(1L, TimeUnit.MINUTES);
session.execute(DROP KEYSPACE ks);
session.close();
cluster.close();
}

private class CreateTableRunner implements Runnable {

private final Session session;
private final String type;

public CreateTableRunner(String type) {
this.session = cluster.newSession();
this.type = type;
}

@Override
public void run() {
session.execute(CREATE TABLE IF NOT EXISTS ks.tb (key  + type + 
, value  + type + , PRIMARY KEY (key)));
session.close();
}
}
}
{code}

sometimes each node has different table schema or no table.

{code}
[on server1] cqlsh DESCRIBE TABLE ks.tb;

CREATE TABLE ks.tb (
key int PRIMARY KEY,
value int
) WITH bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';

[on server2] cqlsh DESCRIBE TABLE ks.tb;

CREATE TABLE ks.tb (
key bigint PRIMARY KEY,
value bigint
) WITH bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';

[on server3] cqlsh DESCRIBE TABLE ks.tb;

Column family 'tb' not found
{code}

I can reproduce this issue on three nodes cluster in 2.1.0. and the test works 
fine in 2.0.10.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8019) Windows Unit tests and Dtests erroring due to sstable deleting task error

2014-10-09 Thread Joshua McKenzie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165484#comment-14165484
 ] 

Joshua McKenzie commented on CASSANDRA-8019:


{quote}Compaction + drop assumes that if refcount is zero it's safe to 
delete.{quote}
It does, however unless we can guarantee that all SSTableScanners are closed 
with handles to the underlying files this is an incorrect assumption (on 
Windows, pre 3.0)
{quote}How are we getting into a situation where SSTableScanner (used by 
compaction) still has it open when it's deleted?{quote}
Previously (before CASSANDRA-7932) we used a CloseableIterator and closed both 
that and the CompactionController prior to 
DataTracker.markCompactedSSTablesReplaced. Currently we're managing the 
controller and scanners via scoped-resource management within CompactionTask 
and calling markCompactedSSTablesReplaced before either are closed out.  This 
marks the sstables obsolete, decrements ref count, and attempts to delete them 
while we still have the index and data file explicitly open in the scanners.

Fixing the ordering in CompactionTask fixes the error this ticket was opened 
for but doesn't address all instances of these types of errors in unit tests on 
the 2.1 branch on Windows.  I can play whac-a-mole tracking all of these down 
but there's nothing stopping us from re-introducing further errors of this type 
since there's no contract between the readers and scanners as far as references 
to underlying files is concerned.  On 2.1+linux or trunk+either, you'll never 
see anything indicating that this ordering problem has occurred.

 Windows Unit tests and Dtests erroring due to sstable deleting task error
 -

 Key: CASSANDRA-8019
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8019
 Project: Cassandra
  Issue Type: Bug
 Environment: Windows 7
Reporter: Philip Thompson
Assignee: Joshua McKenzie
  Labels: windows
 Fix For: 2.1.1

 Attachments: 8019_aggressive_v1.txt, 8019_conservative_v1.txt, 
 8019_v2.txt


 Currently a large number of dtests and unit tests are erroring on windows 
 with the following error in the node log:
 {code}
 ERROR [NonPeriodicTasks:1] 2014-09-29 11:05:04,383 
 SSTableDeletingTask.java:89 - Unable to delete 
 c:\\users\\username\\appdata\\local\\temp\\dtest-vr6qgw\\test\\node1\\data\\system\\local-7ad54392bcdd35a684174e047860b377\\system-local-ka-4-Data.db
  (it will be removed on server restart; we'll also retry after GC)\n
 {code}
 git bisect points to the following commit:
 {code}
 0e831007760bffced8687f51b99525b650d7e193 is the first bad commit
 commit 0e831007760bffced8687f51b99525b650d7e193
 Author: Benedict Elliott Smith bened...@apache.org
 Date:  Fri Sep 19 18:17:19 2014 +0100
 Fix resource leak in event of corrupt sstable
 patch by benedict; review by yukim for CASSANDRA-7932
 :100644 100644 d3ee7d99179dce03307503a8093eb47bd0161681 
 f55e5d27c1c53db3485154cd16201fc5419f32df M  CHANGES.txt
 :04 04 194f4c0569b6be9cc9e129c441433c5c14de7249 
 3c62b53b2b2bd4b212ab6005eab38f8a8e228923 M  src
 :04 04 64f49266e328b9fdacc516c52ef1921fe42e994f 
 de2ca38232bee6d2a6a5e068ed9ee0fbbc5aaebe M  test
 {code}
 You can reproduce this by running simple_bootstrap_test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Fix requests with skipMetadata=false returning no metadata

2014-10-09 Thread slebresne

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 708adca7d - 0249363e8


Fix requests with skipMetadata=false returning no metadata

patch by slebresne; reviewed by thobbs for CASSANDRA-8054


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0249363e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0249363e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0249363e

Branch: refs/heads/cassandra-2.1
Commit: 0249363e89e4b860c9486816445cea360380f2ea
Parents: 708adca
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Thu Oct 9 20:41:32 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Thu Oct 9 20:41:32 2014 +0200

--
 CHANGES.txt |  2 ++
 .../org/apache/cassandra/cql3/ResultSet.java| 30 
 .../cql3/statements/SelectStatement.java|  5 +++-
 .../cassandra/cql3/statements/Selection.java|  2 +-
 .../transport/messages/ResultMessage.java   |  5 
 5 files changed, 19 insertions(+), 25 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0249363e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 417e8c1..688dc32 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 2.1.1
+ * Fix EXECUTE request with skipMetadata=false returning no metadata
+   (CASSANDRA-8054)
  * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)
  * Shutdown JVM on OOM (CASSANDRA-7507)
  * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0249363e/src/java/org/apache/cassandra/cql3/ResultSet.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ResultSet.java 
b/src/java/org/apache/cassandra/cql3/ResultSet.java
index 25635fa..3928060 100644
--- a/src/java/org/apache/cassandra/cql3/ResultSet.java
+++ b/src/java/org/apache/cassandra/cql3/ResultSet.java
@@ -92,16 +92,6 @@ public class ResultSet
 }
 }
 
-public ResultSet withPagingState(PagingState state)
-{
-if (state == null)
-return this;
-
-// The metadata is shared by all execution of a given statement. So if 
there is a paging state
-// we need to copy the metadata
-return new ResultSet(metadata.withPagingState(state), rows);
-}
-
 public ResultSet makeCountResult(ColumnIdentifier alias)
 {
 assert metadata.names != null;
@@ -250,14 +240,14 @@ public class ResultSet
 
 public static final Metadata EMPTY = new 
Metadata(EnumSet.of(Flag.NO_METADATA), null, 0, null);
 
-public final EnumSetFlag flags;
+private final EnumSetFlag flags;
 // Please note that columnCount can actually be smaller than names, 
even if names is not null. This is
 // used to include columns in the resultSet that we need to do 
post-query re-orderings
 // (SelectStatement.orderResults) but that shouldn't be sent to the 
user as they haven't been requested
 // (CASSANDRA-4911). So the serialization code will exclude any 
columns in name whose index is = columnCount.
 public final ListColumnSpecification names;
-public final int columnCount;
-public final PagingState pagingState;
+private final int columnCount;
+private PagingState pagingState;
 
 public Metadata(ListColumnSpecification names)
 {
@@ -274,6 +264,11 @@ public class ResultSet
 this.pagingState = pagingState;
 }
 
+public Metadata copy()
+{
+return new Metadata(flags, names, columnCount, pagingState);
+}
+
 // The maximum number of values that the ResultSet can hold. This can 
be bigger than columnCount due to CASSANDRA-4911
 public int valueCount()
 {
@@ -305,14 +300,13 @@ public class ResultSet
 return true;
 }
 
-public Metadata withPagingState(PagingState pagingState)
+public void setHasMorePages(PagingState pagingState)
 {
 if (pagingState == null)
-return this;
+return;
 
-EnumSetFlag newFlags = EnumSet.copyOf(flags);
-newFlags.add(Flag.HAS_MORE_PAGES);
-return new Metadata(newFlags, names, columnCount, pagingState);
+flags.add(Flag.HAS_MORE_PAGES);
+this.pagingState = pagingState;
 }
 
 public void setSkipMetadata()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0249363e/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff

[jira] [Updated] (CASSANDRA-8063) Consider removing UDF-as-class functionality

2014-10-09 Thread Robert Stupp (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-8063:

Attachment: 8063.txt

patch to remove reflection based UDF

 Consider removing UDF-as-class functionality
 

 Key: CASSANDRA-8063
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8063
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Robert Stupp
  Labels: cql
 Fix For: 3.0

 Attachments: 8063.txt


 We've introduce 2 ways to provide (scalar) UDFs: either by providing a 
 class+method name (and assuming C* can find such class and method in the 
 classpath), or by providing the body of the function directly in the {{CREATE 
 FUNCTION}} statement (with such body being able to be in either java or some 
 variety of scripting languages).
 I submit that we remove the first option: the declaration of functions from a 
 class+method name. I was the first to insist on adding it, but in hindsight I 
 think it adds more complexity/confusion than anything else. More 
 specifically, I think the UDF-as-class option is always inferior to 
 CASSANDRA-7562 because:
 # it's more error prone. You have to manually deploy the class containing the 
 function to every C* node (and make sure it is in the classpath). It's way 
 too easy to end up with some node not having the function due to simple 
 operator error.
 # it's not faster. In fact, post-CASSANDRA-7924, the UDF-as-java-source is 
 probably faster since it doesn't involve reflection while the UDF-as-class 
 option does.
 Overall, I think removing the UDF-as-class will simplify things without 
 really losing anything (of course, we can re-evaluate this in the future if 
 new arguments arises, but it's easier to add than to remove).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8063) Consider removing UDF-as-class functionality

2014-10-09 Thread Sylvain Lebresne (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-8063:

Reviewer: Benjamin Lerer

 Consider removing UDF-as-class functionality
 

 Key: CASSANDRA-8063
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8063
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Robert Stupp
  Labels: cql
 Fix For: 3.0

 Attachments: 8063.txt


 We've introduce 2 ways to provide (scalar) UDFs: either by providing a 
 class+method name (and assuming C* can find such class and method in the 
 classpath), or by providing the body of the function directly in the {{CREATE 
 FUNCTION}} statement (with such body being able to be in either java or some 
 variety of scripting languages).
 I submit that we remove the first option: the declaration of functions from a 
 class+method name. I was the first to insist on adding it, but in hindsight I 
 think it adds more complexity/confusion than anything else. More 
 specifically, I think the UDF-as-class option is always inferior to 
 CASSANDRA-7562 because:
 # it's more error prone. You have to manually deploy the class containing the 
 function to every C* node (and make sure it is in the classpath). It's way 
 too easy to end up with some node not having the function due to simple 
 operator error.
 # it's not faster. In fact, post-CASSANDRA-7924, the UDF-as-java-source is 
 probably faster since it doesn't involve reflection while the UDF-as-class 
 option does.
 Overall, I think removing the UDF-as-class will simplify things without 
 really losing anything (of course, we can re-evaluate this in the future if 
 new arguments arises, but it's easier to add than to remove).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7927) Kill daemon on any disk error

2014-10-09 Thread Joshua McKenzie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165600#comment-14165600
 ] 

Joshua McKenzie commented on CASSANDRA-7927:


Sorry for the delay on this -  I have a version rebased to 2.1 head [available 
here|https://github.com/josh-mckenzie/cassandra/compare/7927?expand=1]
* Added support for die policy to CommitLog exception handling
* Removed 'killMeNow' method in StabilityInspector
* Migrated the FileUtil killing logic into the StabilityInspector
* Slight refactor on JVMStabilityInspector to keep it single-point-of-entry 
(hand Throwable to it, let it deal with it)
* Updated the unit tests to work w/the new structure
* Removed erroneous added entries from Config.CommitFailurePolicy
* Reverted ordering on enums in Config to just append the new entry on the end

Regarding migrating the logic into the JVMStabilityInspector: I expect we're 
going to have very few exception conditions that will cause us to mark the JVM 
as unstable and kill it, so I'd prefer to keep that class as simple as possible 
and nest that logic inside it rather than distributing it throughout by opening 
the interface to a 'killMeNow' type method.  Hand a throwable to it, it'll kill 
things if they need to be killed.

[~jdsumsion]: could you review the revised branch posted above?  Thanks!

 Kill daemon on any disk error
 -

 Key: CASSANDRA-7927
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7927
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
 Environment: aws, stock cassandra or dse
Reporter: John Sumsion
Assignee: John Sumsion
  Labels: bootcamp, lhf
 Fix For: 2.1.1

 Attachments: 7927-v1-die.patch


 We got a disk read error on 1.2.13 that didn't trigger the disk failure 
 policy, and I'm trying to hunt down why, but in doing so, I saw that there is 
 no disk_failure_policy option for just killing the daemon.
 If we ever get a corrupt sstable, we want to replace the node anyway, because 
 some aws instance store disks just go bad.
 I want to use the JVMStabilityInspector from CASSANDRA-7507 to kill so that 
 remains standard, so I will base my patch on CASSANDRA-7507.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8075) Make netty pipeline fully async

2014-10-09 Thread Sylvain Lebresne (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165616#comment-14165616
 ] 

Sylvain Lebresne commented on CASSANDRA-8075:
-

bq. Remove the request thread pool used in netty

That's part of CASSANDRA-5239 really, at least that was the intent.

 Make netty pipeline fully async
 ---

 Key: CASSANDRA-8075
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8075
 Project: Cassandra
  Issue Type: Improvement
Reporter: T Jake Luciani
  Labels: performance
 Fix For: 3.0


 Related to CASSANDRA-5239.  Netty does much better if we remove the request 
 thread pool jump. 
 Here is a benchmark of our netty pipeline returning a static value (so no 
 actual cassandra processing).  When the thread pool is removed we see a 
 dramatic improvement.
 http://cstar.datastax.com/graph?stats=64ed5186-4e51-11e4-a8d5-bc764e04482cmetric=op_rateoperation=2_usersmoothing=1show_aggregates=truexmin=0xmax=228.25ymin=0ymax=150217.1
 On my laptop using the async branch can match redis bench on the same machine 
 ~160k ops/sec.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8075) Make netty pipeline fully async