[jira] [Updated] (CASSANDRA-10707) Add support for Group By to Select statement

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10707:
---
Component/s: CQL

> Add support for Group By to Select statement
> 
>
> Key: CASSANDRA-10707
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10707
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>
> Now that Cassandra support aggregate functions, it makes sense to support 
> {{GROUP BY}} on the {{SELECT}} statements.
> It should be possible to group either at the partition level or at the 
> clustering column level.
> {code}
> SELECT partitionKey, max(value) FROM myTable GROUP BY partitionKey;
> SELECT partitionKey, clustering0, clustering1, max(value) FROM myTable GROUP 
> BY partitionKey, clustering0, clustering1; 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10801) Unexplained inconsistent data with Cassandra 2.1

[Maor Cohen (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061757#comment-15061757
 ] 

Maor Cohen edited comment on CASSANDRA-10801 at 12/17/15 9:09 AM:
--

Tracing result of 3 query executions. First and last queries returned one row 
as expected (seems to be found by key cache). Second returned nothing.

See tracing.log


was (Author: maor.cohen):
Tracing result of 3 query executions. First and last queries returned one row 
as expected (seems to be found by key cache). Second returned nothing.

> Unexplained inconsistent data with Cassandra 2.1
> 
>
> Key: CASSANDRA-10801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10801
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Imri Zvik
> Fix For: 2.1.x
>
> Attachments: tracing.log
>
>
> We are experiencing weird behavior which we cannot explain.
> We have a CF, with RF=3, and we are writing and reading data to it with 
> consistency level of ONE.
> For some reason, we see inconsistent results when querying for data.
> Even for rows that were written a day ago, we're seeing inconsistent results 
> (1 replca has the data, the two others don't).
> Now, I would expect to see timeouts/dropped mutations, but all relevant 
> counters are not advancing, and I would also expect hints to fix this 
> inconsistency within minutes, but yet it doesn't.
> {code}
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
>  writetime(last_update) | site_id   | tree_id | individual_id | last_update
> +---+-+---+-
>1448988343028000 | 229673621 |   9 |   9032483 |  1380912397
> (1 rows)
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
> site_id   | tree_id | individual_id | last_update
> ---+-+---+-
> (0 rows)
> cqlsh:testing> SELECT dateof(now()) FROM system.local ;
>  dateof(now())
> --
>  2015-12-02 14:48:44+
> (1 rows)
> {code}
> We are running with Cassandra 2.1.11 with Oracle Java 1.8.0_65-b17



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10801) Unexplained inconsistent data with Cassandra 2.1

[Maor Cohen (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061757#comment-15061757
 ] 

Maor Cohen edited comment on CASSANDRA-10801 at 12/17/15 9:08 AM:
--

Tracing result of 3 query executions. First and last queries returned one row 
as expected (seems to be found by key cache). Second returned nothing.


was (Author: maor.cohen):
Tracing result of 3 query executions. First and last queries return 1 row as 
expected (seems to be found by key cache). Second returned nothing.

> Unexplained inconsistent data with Cassandra 2.1
> 
>
> Key: CASSANDRA-10801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10801
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Imri Zvik
> Fix For: 2.1.x
>
> Attachments: tracing.log
>
>
> We are experiencing weird behavior which we cannot explain.
> We have a CF, with RF=3, and we are writing and reading data to it with 
> consistency level of ONE.
> For some reason, we see inconsistent results when querying for data.
> Even for rows that were written a day ago, we're seeing inconsistent results 
> (1 replca has the data, the two others don't).
> Now, I would expect to see timeouts/dropped mutations, but all relevant 
> counters are not advancing, and I would also expect hints to fix this 
> inconsistency within minutes, but yet it doesn't.
> {code}
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
>  writetime(last_update) | site_id   | tree_id | individual_id | last_update
> +---+-+---+-
>1448988343028000 | 229673621 |   9 |   9032483 |  1380912397
> (1 rows)
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
> site_id   | tree_id | individual_id | last_update
> ---+-+---+-
> (0 rows)
> cqlsh:testing> SELECT dateof(now()) FROM system.local ;
>  dateof(now())
> --
>  2015-12-02 14:48:44+
> (1 rows)
> {code}
> We are running with Cassandra 2.1.11 with Oracle Java 1.8.0_65-b17



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10784) ZonedDateTime as new CQL data type

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061734#comment-15061734
 ] 

Robert Stupp commented on CASSANDRA-10784:
--

[~jasonstack], the problem with time zone (and locales) is that the reference 
data needs to be in sync in the whole cluster. But (unfortunately) time zones 
(and also locales & currencies) change. Companies that have to deal with TZs, 
locales and/or currencies in their application build their own data set (or buy 
that from vendors) to be both up-to-date and consistent - where "up-to-date" 
also means that you get the correct information for any specific timestamp for 
any location.
IMO that's not something Cassandra as a database can or should provide. If we 
would go that route, it would open a can of worms.

Keep in mind that borders change and political decisions can change everything 
- if such changes occur, your master data may become worthless if not 
completely wrong. The only source or truth we have is the JRE (or any other 
library). Keeping that information in sync in a distributed system with 
x-hundred nodes is hard if no impossible (upgrading the JRE on all nodes at 
once... no way). Just look at the Java release notes - these things change even 
between minor Java releases.

Obviously, you can store time zone information in a separate column, use UDTs, 
whatever. But still, IMO it can only be handled correctly in the application - 
probably storing the reference data in C*.

Don't get me wrong - I'm not saying that having time zone (or locales or 
currencies) is bad - just want to say that IMO it's extremely hard to implement 
and maintain right.

> ZonedDateTime as new CQL data type
> --
>
> Key: CASSANDRA-10784
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10784
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL, Documentation and Website
>Reporter: ZhaoYang
>Assignee: ZhaoYang
> Fix For: 3.x
>
>
> For globalization, ZonedDateTime is very important. Timestamp is not enough 
> and Zone info should be included. So Cassandra should support ZonedDateTime 
> as native data type and ordering using ZonedDateTime's utc value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10801) Unexplained inconsistent data with Cassandra 2.1

[Maor Cohen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maor Cohen updated CASSANDRA-10801:
---
Attachment: tracing.log

Tracing result of 3 query executions. First and last queries return 1 row as 
expected (seems to be found by key cache). Second returned nothing.

> Unexplained inconsistent data with Cassandra 2.1
> 
>
> Key: CASSANDRA-10801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10801
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Imri Zvik
> Fix For: 2.1.x
>
> Attachments: tracing.log
>
>
> We are experiencing weird behavior which we cannot explain.
> We have a CF, with RF=3, and we are writing and reading data to it with 
> consistency level of ONE.
> For some reason, we see inconsistent results when querying for data.
> Even for rows that were written a day ago, we're seeing inconsistent results 
> (1 replca has the data, the two others don't).
> Now, I would expect to see timeouts/dropped mutations, but all relevant 
> counters are not advancing, and I would also expect hints to fix this 
> inconsistency within minutes, but yet it doesn't.
> {code}
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
>  writetime(last_update) | site_id   | tree_id | individual_id | last_update
> +---+-+---+-
>1448988343028000 | 229673621 |   9 |   9032483 |  1380912397
> (1 rows)
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
> site_id   | tree_id | individual_id | last_update
> ---+-+---+-
> (0 rows)
> cqlsh:testing> SELECT dateof(now()) FROM system.local ;
>  dateof(now())
> --
>  2015-12-02 14:48:44+
> (1 rows)
> {code}
> We are running with Cassandra 2.1.11 with Oracle Java 1.8.0_65-b17



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10801) Unexplained inconsistent data with Cassandra 2.1

[Maor Cohen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maor Cohen updated CASSANDRA-10801:
---
Attachment: tracing.log

Updated trace log. Last query shows read repair. From that point, row should 
always return.

> Unexplained inconsistent data with Cassandra 2.1
> 
>
> Key: CASSANDRA-10801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10801
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Imri Zvik
> Fix For: 2.1.x
>
> Attachments: tracing.log
>
>
> We are experiencing weird behavior which we cannot explain.
> We have a CF, with RF=3, and we are writing and reading data to it with 
> consistency level of ONE.
> For some reason, we see inconsistent results when querying for data.
> Even for rows that were written a day ago, we're seeing inconsistent results 
> (1 replca has the data, the two others don't).
> Now, I would expect to see timeouts/dropped mutations, but all relevant 
> counters are not advancing, and I would also expect hints to fix this 
> inconsistency within minutes, but yet it doesn't.
> {code}
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
>  writetime(last_update) | site_id   | tree_id | individual_id | last_update
> +---+-+---+-
>1448988343028000 | 229673621 |   9 |   9032483 |  1380912397
> (1 rows)
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
> site_id   | tree_id | individual_id | last_update
> ---+-+---+-
> (0 rows)
> cqlsh:testing> SELECT dateof(now()) FROM system.local ;
>  dateof(now())
> --
>  2015-12-02 14:48:44+
> (1 rows)
> {code}
> We are running with Cassandra 2.1.11 with Oracle Java 1.8.0_65-b17



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10888) Tombstone error warning does not log partition key

[Ole Pedersen (JIRA)]

Ole Pedersen created CASSANDRA-10888:


 Summary: Tombstone error warning does not log partition key
 Key: CASSANDRA-10888
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10888
 Project: Cassandra
  Issue Type: Improvement
 Environment: DSE 4.8.2
Reporter: Ole Pedersen


Log partition key if read fails due to the ERROR threshold on read tombstoned 
cells.

Right now I can specify a warning and an error threshold for C* when reading 
from a partition with many tombstones.

If the query reads more than the “warning threshold” then C* writes a warning 
to the log with the partition key.

But if a query reads more than the “error threshold” then C* aborts the query 
and writes to the log – but not the partition key, this time. 

What I am missing is: Could C* also please write the partition key in case of 
query abort due to tombstones?




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10801) Unexplained inconsistent data with Cassandra 2.1

[Maor Cohen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maor Cohen updated CASSANDRA-10801:
---
Attachment: (was: tracing.log)

> Unexplained inconsistent data with Cassandra 2.1
> 
>
> Key: CASSANDRA-10801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10801
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Imri Zvik
> Fix For: 2.1.x
>
>
> We are experiencing weird behavior which we cannot explain.
> We have a CF, with RF=3, and we are writing and reading data to it with 
> consistency level of ONE.
> For some reason, we see inconsistent results when querying for data.
> Even for rows that were written a day ago, we're seeing inconsistent results 
> (1 replca has the data, the two others don't).
> Now, I would expect to see timeouts/dropped mutations, but all relevant 
> counters are not advancing, and I would also expect hints to fix this 
> inconsistency within minutes, but yet it doesn't.
> {code}
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
>  writetime(last_update) | site_id   | tree_id | individual_id | last_update
> +---+-+---+-
>1448988343028000 | 229673621 |   9 |   9032483 |  1380912397
> (1 rows)
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
> site_id   | tree_id | individual_id | last_update
> ---+-+---+-
> (0 rows)
> cqlsh:testing> SELECT dateof(now()) FROM system.local ;
>  dateof(now())
> --
>  2015-12-02 14:48:44+
> (1 rows)
> {code}
> We are running with Cassandra 2.1.11 with Oracle Java 1.8.0_65-b17



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-10886) test_read_invalid_text dtest fails

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania reassigned CASSANDRA-10886:


Assignee: Stefania

> test_read_invalid_text dtest fails 
> ---
>
> Key: CASSANDRA-10886
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10886
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Stefania
> Fix For: 3.0.x
>
>
> {{cqlsh_tests/cqlsh_copy_tests.py:CqlshCopyTest.test_read_invalid_text}} 
> seems to fail hard on Windows on trunk. These were only recently unskipped 
> when CASSANDRA-9302 was closed, so I don't know if they fail on other 
> branches, and I won't know until those jobs run.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-10858) test_copy_to_with_more_failures_than_max_attempts is failing

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania reassigned CASSANDRA-10858:


Assignee: Stefania

> test_copy_to_with_more_failures_than_max_attempts is failing
> 
>
> Key: CASSANDRA-10858
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10858
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Testing, Tools
>Reporter: Philip Thompson
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> {{cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_copy_to_with_more_failures_than_max_attempts}}
>  which was introduced for CASSANDRA-9306 is failing when run on clusters 
> without vnodes. To reproduce, simply run the test with the environment 
> variable DISABLE_VNODES=true. 
> Here is the entire debug output:
> {code}
> dtest: DEBUG: cluster ccm directory: 
> /var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/dtest-iJQECY
> dtest: DEBUG: removing ccm cluster test at: 
> /var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/dtest-iJQECY
> dtest: DEBUG: clearing ssl stores from 
> [/var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/dtest-iJQECY] directory
> dtest: DEBUG: cluster ccm directory: 
> /var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/dtest-CnOgA8
> dtest: DEBUG: Running stress
> dtest: DEBUG: Exporting to csv file: 
> /var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/tmpZYqB01 with 
> {"failing_range": {"start": 0, "end": 500, "num_failures": 
> 5}} and 3 max attempts
> dtest: DEBUG:
> Starting copy of keyspace1.standard1 with columns ['key', 'C0', 'C1', 'C2', 
> 'C3', 'C4'].
> Processed 1 rows; Written: 10503.508303 rows/s
> Processed 2 rows; Written: 11860.954046 rows/s
> Processed 3 rows; Written: 13068.388704 rows/s
> Processed 4 rows; Written: 16941.628006 rows/s
> Processed 5 rows; Written: 17609.109488 rows/s
> Processed 6 rows; Written: 19475.156238 rows/s
> Processed 7 rows; Written: 19976.978154 rows/s
> Processed 8 rows; Written: 19992.329090 rows/s
> Processed 9 rows; Written: 20623.291907 rows/s
> Processed 10 rows; Written: 21644.815363 rows/s
> Processed 10 rows; Written: 10822.407682 rows/s
> 10 rows exported in 5.816 seconds.
> {code}
> I assume this is related to the failure injection code in cqlsh not handling 
> fewer token ranges.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10787) OutOfMemoryError after few hours from node restart

[Piotr Westfalewicz (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061795#comment-15061795
 ] 

Piotr Westfalewicz edited comment on CASSANDRA-10787 at 12/17/15 9:40 AM:
--

Hey guys,

Here is the continuation of the story:
0. Taking your advice, I've decided to create more powerful cluster
1. I've created a new cluster, on 2x m1.xlarge instances (4 vCPU, 64-bit, 15GB 
RAM, Raid0 4x420GB HDD Disk), and changed RF to 2
2. Took the snapshot of the data (keyspace.table=logs.group) on one of the old 
nodes
3. scp between old node and the new node. From snapshot to the 
cassandra/data/kayspace/tablename folder
4. Loaded the data without restarting the server - by nodetool refresh
5. Triggered nodetool repair

After few hours - the server went down. I've attached the logs. This is the 
case 5 files. Maybe that's because of the size of sstables? In my case one of 
them was around 50GB.

I've also migrated the rest of the data (not "big" logs.group, but another 
tables from 500M to 5GB) the same way and the server was working fine and the 
data was accessible.


was (Author: piotrwest):
Hey guys,

Here is the continuation of the story:
0. Taking your advice, I've decided to create more powerful cluster
1. I've created a new cluster, on 2x m1.xlarge instances (4 vCPU, 64-bit, 15GB 
RAM, Raid0 4x420GB HDD Disk), and changed RF to 2
2. Took the snapshot of the data (keyspace.table=logs.group) on one of the old 
nodes
3. scp between old node and the new node. From snapshot to the 
cassandra/data/kayspace/tablename folder
4. Loaded the data without restarting the server - by nodetool refresh
5. Triggered nodetool repair

After few hours - the server went down. I've attached the logs. This is the 
case 5 files. Maybe that's because of the size of sstables? In my case one of 
them was around 50GB.

I've also migrated the rest of the data (not "big" logs.group, but another 
tables from 500M to 5GB) the same way and the server was working fine.

> OutOfMemoryError after few hours from node restart
> --
>
> Key: CASSANDRA-10787
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10787
> Project: Cassandra
>  Issue Type: Bug
> Environment: Amazon DataStax Auto-Clustering AMI 2.6.3-1404-pv
> on 2x m1.large instances (2 vCPU, 64-bit, 7.5GB RAM, Raid0 2x420GB Disk)
> [cqlsh 5.0.1 | Cassandra 2.2.3 | CQL spec 3.3.1 | Native protocol v4]
> RF=3
>Reporter: Piotr Westfalewicz
> Fix For: 2.2.x
>
> Attachments: case2_debuglog_head.txt, case2_debuglog_tail.txt, 
> case2_systemlog.txt, case3_debuglog_tail.txt, case3_systemlog_tail.txt, 
> case4_debuglog_tail.txt, case4_systemlog.txt, case5_debuglog.txt, 
> case5_systemlog.txt
>
>
> Cassandra Cluster was operating flawessly for around 3 months. Lately I've 
> got a critical problem with it - after few hours of running clients are 
> disconnected permanently (that may be Datastax C# Driver problem, though), 
> however few more hours later (with smaller load), on all 2 nodes there is 
> thrown an exception (details in files):
> bq. java.lang.OutOfMemoryError: Java heap space
> Cases description:
> Case 2 (heavy load):
> - 2015-11-26 16:09:40,834 Restarted all nodes in cassandra cluster
>   - 2015-11-26 17:03:46,774 First client disconnected permanently
>   - 2015-11-26 22:17:02,327 Node shutdown
>   Case 3 (unknown load, different node):
>   - 2015-11-26 02:19:49,585 Node shutdown (visible only in 
> systemlog, I don't know why not in debug log)
>   Case 4 (low load):
>   - 2015-11-27 13:00:24,994 Node restart
>   - 2015-11-27 22:26:56,131 Node shutdown
> Is that a software issue or I am using too weak Amazon instances? If so, how 
> can the required amount of memory be calculated?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10801) Unexplained inconsistent data with Cassandra 2.1

[Maor Cohen (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061762#comment-15061762
 ] 

Maor Cohen edited comment on CASSANDRA-10801 at 12/17/15 9:52 AM:
--

Updated trace log. Last query shows read repair. From that point, row should 
always return.
It's also worth mentioning that we read in QUORUM consistency.


was (Author: maor.cohen):
Updated trace log. Last query shows read repair. From that point, row should 
always return.

> Unexplained inconsistent data with Cassandra 2.1
> 
>
> Key: CASSANDRA-10801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10801
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Imri Zvik
> Fix For: 2.1.x
>
> Attachments: tracing.log
>
>
> We are experiencing weird behavior which we cannot explain.
> We have a CF, with RF=3, and we are writing and reading data to it with 
> consistency level of ONE.
> For some reason, we see inconsistent results when querying for data.
> Even for rows that were written a day ago, we're seeing inconsistent results 
> (1 replca has the data, the two others don't).
> Now, I would expect to see timeouts/dropped mutations, but all relevant 
> counters are not advancing, and I would also expect hints to fix this 
> inconsistency within minutes, but yet it doesn't.
> {code}
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
>  writetime(last_update) | site_id   | tree_id | individual_id | last_update
> +---+-+---+-
>1448988343028000 | 229673621 |   9 |   9032483 |  1380912397
> (1 rows)
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
> site_id   | tree_id | individual_id | last_update
> ---+-+---+-
> (0 rows)
> cqlsh:testing> SELECT dateof(now()) FROM system.local ;
>  dateof(now())
> --
>  2015-12-02 14:48:44+
> (1 rows)
> {code}
> We are running with Cassandra 2.1.11 with Oracle Java 1.8.0_65-b17



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10423) Paxos/LWT failures when moving node

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061831#comment-15061831
 ] 

Sylvain Lebresne commented on CASSANDRA-10423:
--

bq. Why will there be more contention when there are more participants?

That's actually not what I'm saying. What I'm saying is that for a fixed amount 
of contention, having more participants make it more likely for Paxos rounds to 
interleave and thus having to restart new rounds and thus for any particular 
update to be slower. Or to put it another way, I'm just saying that conditional 
update performance is generally negatively impacted by bigger RF (all other 
things being equal), and moving happen to temporarily bump the RF as far Paxos 
is concerned.

But again, I'm not pretending that this explain the problem here at all as the 
difference in performance observed by Roger during a move is more dramatic than 
what I would expect this explanation to account for. I just don't have other 
idea of why a move would impact Paxos that badly and so we'd need to be able to 
reproduce this easily to find the problem.

> Paxos/LWT failures when moving node
> ---
>
> Key: CASSANDRA-10423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10423
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra version: 2.0.14
> Java-driver version: 2.0.11
>Reporter: Roger Schildmeijer
> Fix For: 2.1.x
>
>
> While moving a node (nodetool move ) we noticed that lwt started 
> failing for some (~50%) requests. The java-driver (version 2.0.11) returned 
> com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout 
> during write query at consistency SERIAL (7 replica were required but only 0 
> acknowledged the write). The cluster was not under heavy load.
> I noticed that the failed lwt requests all took just above 1s. That 
> information and the WriteTimeoutException could indicate that this happens:
> https://github.com/apache/cassandra/blob/cassandra-2.0.14/src/java/org/apache/cassandra/service/StorageProxy.java#L268
> I can't explain why though. Why would there be more cas contention just 
> because a node is moving?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9258) Range movement causes CPU & performance impact

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061842#comment-15061842
 ] 

Sylvain Lebresne commented on CASSANDRA-9258:
-

bq. Sylvain, are you happy with the provided data or do you insist on writing a 
jmh bench for regression testing?

I think we still should write a jmh regression test for, well, regression 
testing, but also because it'll give us a good baseline for future 
improvements. That said, the provided data (and general reasoning) is 
convincing enough that this is a net improvement that if no-one has time to 
write that jmh bench shortly, I won't hold back on committing since this is 
potentially serious for some users. Still, let the record show that I dislike 
the idea of postponing the write of such regression test because history so far 
shows that this is codename for "never getting it done".

> Range movement causes CPU & performance impact
> --
>
> Key: CASSANDRA-9258
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9258
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.4
>Reporter: Rick Branson
>Assignee: Dikang Gu
> Fix For: 2.1.x
>
> Attachments: 0001-pending-ranges-map.patch, Screenshot 2015-12-16 
> 16.11.36.png, Screenshot 2015-12-16 16.11.51.png
>
>
> Observing big CPU & latency regressions when doing range movements on 
> clusters with many tens of thousands of vnodes. See CPU usage increase by 
> ~80% when a single node is being replaced.
> Top methods are:
> 1) Ljava/math/BigInteger;.compareTo in 
> Lorg/apache/cassandra/dht/ComparableObjectToken;.compareTo 
> 2) Lcom/google/common/collect/AbstractMapBasedMultimap;.wrapCollection in 
> Lcom/google/common/collect/AbstractMapBasedMultimap$AsMap$AsMapIterator;.next
> 3) Lorg/apache/cassandra/db/DecoratedKey;.compareTo in 
> Lorg/apache/cassandra/dht/Range;.contains
> Here's a sample stack from a thread dump:
> {code}
> "Thrift:50673" daemon prio=10 tid=0x7f2f20164800 nid=0x3a04af runnable 
> [0x7f2d878d]
>java.lang.Thread.State: RUNNABLE
>   at org.apache.cassandra.dht.Range.isWrapAround(Range.java:260)
>   at org.apache.cassandra.dht.Range.contains(Range.java:51)
>   at org.apache.cassandra.dht.Range.contains(Range.java:110)
>   at 
> org.apache.cassandra.locator.TokenMetadata.pendingEndpointsFor(TokenMetadata.java:916)
>   at 
> org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:775)
>   at 
> org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:541)
>   at 
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:616)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1101)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1083)
>   at 
> org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:976)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3996)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3980)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>   at 
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:205)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745){code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9258) Range movement causes CPU & performance impact

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061779#comment-15061779
 ] 

Branimir Lambov commented on CASSANDRA-9258:


The new version looks good to me. Updated the branch, test results should be 
available shortly: [unit 
tests|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-dikang85-9258-testall/]
 and 
[dtests|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-dikang85-9258-dtest/].

Sylvain, are you happy with the provided data or do you insist on writing a 
{{jmh}} bench for regression testing?

> Range movement causes CPU & performance impact
> --
>
> Key: CASSANDRA-9258
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9258
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.4
>Reporter: Rick Branson
>Assignee: Dikang Gu
> Fix For: 2.1.x
>
> Attachments: 0001-pending-ranges-map.patch, Screenshot 2015-12-16 
> 16.11.36.png, Screenshot 2015-12-16 16.11.51.png
>
>
> Observing big CPU & latency regressions when doing range movements on 
> clusters with many tens of thousands of vnodes. See CPU usage increase by 
> ~80% when a single node is being replaced.
> Top methods are:
> 1) Ljava/math/BigInteger;.compareTo in 
> Lorg/apache/cassandra/dht/ComparableObjectToken;.compareTo 
> 2) Lcom/google/common/collect/AbstractMapBasedMultimap;.wrapCollection in 
> Lcom/google/common/collect/AbstractMapBasedMultimap$AsMap$AsMapIterator;.next
> 3) Lorg/apache/cassandra/db/DecoratedKey;.compareTo in 
> Lorg/apache/cassandra/dht/Range;.contains
> Here's a sample stack from a thread dump:
> {code}
> "Thrift:50673" daemon prio=10 tid=0x7f2f20164800 nid=0x3a04af runnable 
> [0x7f2d878d]
>java.lang.Thread.State: RUNNABLE
>   at org.apache.cassandra.dht.Range.isWrapAround(Range.java:260)
>   at org.apache.cassandra.dht.Range.contains(Range.java:51)
>   at org.apache.cassandra.dht.Range.contains(Range.java:110)
>   at 
> org.apache.cassandra.locator.TokenMetadata.pendingEndpointsFor(TokenMetadata.java:916)
>   at 
> org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:775)
>   at 
> org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:541)
>   at 
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:616)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1101)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1083)
>   at 
> org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:976)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3996)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3980)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>   at 
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:205)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745){code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10777) LegacySSTableTest Test-case failure, Issue with test\data\legacy-sstables\jb\Keyspace1\Keyspace1-Standard1-jb-0-Summary.db file read on BIG ENDian systems

[Nirav Thakkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nirav Thakkar updated CASSANDRA-10777:
--
Priority: Critical  (was: Major)

> LegacySSTableTest Test-case failure,  Issue with 
> test\data\legacy-sstables\jb\Keyspace1\Keyspace1-Standard1-jb-0-Summary.db  
> file read on BIG ENDian systems
> 
>
> Key: CASSANDRA-10777
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10777
> Project: Cassandra
>  Issue Type: Bug
> Environment: OS:RHEL7 and BIG ENDian system
>Reporter: Nirav Thakkar
>Priority: Critical
> Attachments: IndexSummaryBE.jpg, IndexSummaryLE.jpg, 
> SampleTable_Summary.db_BE.jpg, SampleTable_Summary.db_LE.jpg
>
>
> I am building Cassandra 2.2.3 on BigEndian System. Cassandra builds 
> successfully. However there are quite a few test case failures. One of the 
> failing test case is LegacySSTableTest. This test case reads the file 
> test\data\legacy-sstables\jb\Keyspace1\Keyspace1-Standard1-jb-0-Summary.db
> Are these existing test data files, specific to Little Endian systems?
> I have observed differences in values read on a LE Vs BE systems.
> I use the following command to run the test case. ant test 
> -Dtest.name=LegacySSTableTest
> I get the following error:
> [junit] - Standard Error -
> [junit] Failed to read jb
> [junit] -  ---
> [junit] Testcase: 
> testStreaming(org.apache.cassandra.io.sstable.LegacySSTableTest): FAILED
> [junit] null
> [junit] junit.framework.AssertionFailedError
> [junit] at 
> org.apache.cassandra.io.sstable.IndexSummary.(IndexSummary.java:80)
> [junit] at 
> org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:317)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:877)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:730)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:692)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:480)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:376)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:371)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:363)
> [junit] at 
> org.apache.cassandra.io.sstable.LegacySSTableTest.testStreaming(LegacySSTableTest.java:115)
> [junit] at 
> org.apache.cassandra.io.sstable.LegacySSTableTest.testStreaming(LegacySSTableTest.java:110)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10787) OutOfMemoryError after few hours from node restart

[Piotr Westfalewicz (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Piotr Westfalewicz updated CASSANDRA-10787:
---
Attachment: case5_systemlog.txt
case5_debuglog.txt

Hey guys,

Here is the continuation of the story:
0. Taking your advice, I've decided to create more powerful cluster
1. I've created a new cluster, on 2x m1.xlarge instances (4 vCPU, 64-bit, 15GB 
RAM, Raid0 4x420GB HDD Disk), and changed RF to 2
2. Took the snapshot of the data (keyspace.table=logs.group) on one of the old 
nodes
3. scp between old node and the new node. From snapshot to the 
cassandra/data/kayspace/tablename folder
4. Loaded the data without restarting the server - by nodetool refresh
5. Triggered nodetool repair

After few hours - the server went down. I've attached the logs. This is the 
case 5 files. Maybe that's because of the size of sstables? In my case one of 
them was around 50GB.

I've also migrated the rest of the data (not "big" logs.group, but another 
tables from 500M to 5GB) the same way and the server was working fine.

> OutOfMemoryError after few hours from node restart
> --
>
> Key: CASSANDRA-10787
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10787
> Project: Cassandra
>  Issue Type: Bug
> Environment: Amazon DataStax Auto-Clustering AMI 2.6.3-1404-pv
> on 2x m1.large instances (2 vCPU, 64-bit, 7.5GB RAM, Raid0 2x420GB Disk)
> [cqlsh 5.0.1 | Cassandra 2.2.3 | CQL spec 3.3.1 | Native protocol v4]
> RF=3
>Reporter: Piotr Westfalewicz
> Fix For: 2.2.x
>
> Attachments: case2_debuglog_head.txt, case2_debuglog_tail.txt, 
> case2_systemlog.txt, case3_debuglog_tail.txt, case3_systemlog_tail.txt, 
> case4_debuglog_tail.txt, case4_systemlog.txt, case5_debuglog.txt, 
> case5_systemlog.txt
>
>
> Cassandra Cluster was operating flawessly for around 3 months. Lately I've 
> got a critical problem with it - after few hours of running clients are 
> disconnected permanently (that may be Datastax C# Driver problem, though), 
> however few more hours later (with smaller load), on all 2 nodes there is 
> thrown an exception (details in files):
> bq. java.lang.OutOfMemoryError: Java heap space
> Cases description:
> Case 2 (heavy load):
> - 2015-11-26 16:09:40,834 Restarted all nodes in cassandra cluster
>   - 2015-11-26 17:03:46,774 First client disconnected permanently
>   - 2015-11-26 22:17:02,327 Node shutdown
>   Case 3 (unknown load, different node):
>   - 2015-11-26 02:19:49,585 Node shutdown (visible only in 
> systemlog, I don't know why not in debug log)
>   Case 4 (low load):
>   - 2015-11-27 13:00:24,994 Node restart
>   - 2015-11-27 22:26:56,131 Node shutdown
> Is that a software issue or I am using too weak Amazon instances? If so, how 
> can the required amount of memory be calculated?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10777) LegacySSTableTest Test-case failure, Issue with test\data\legacy-sstables\jb\Keyspace1\Keyspace1-Standard1-jb-0-Summary.db file read on BIG ENDian systems

[Nirav Thakkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061841#comment-15061841
 ] 

Nirav Thakkar commented on CASSANDRA-10777:
---

Is it possible to give me the steps or just tell me how these files are 
generated, so I can try generating the files on the environment I am using. If 
there is a difference then we can work on it.

> LegacySSTableTest Test-case failure,  Issue with 
> test\data\legacy-sstables\jb\Keyspace1\Keyspace1-Standard1-jb-0-Summary.db  
> file read on BIG ENDian systems
> 
>
> Key: CASSANDRA-10777
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10777
> Project: Cassandra
>  Issue Type: Bug
> Environment: OS:RHEL7 and BIG ENDian system
>Reporter: Nirav Thakkar
> Attachments: IndexSummaryBE.jpg, IndexSummaryLE.jpg, 
> SampleTable_Summary.db_BE.jpg, SampleTable_Summary.db_LE.jpg
>
>
> I am building Cassandra 2.2.3 on BigEndian System. Cassandra builds 
> successfully. However there are quite a few test case failures. One of the 
> failing test case is LegacySSTableTest. This test case reads the file 
> test\data\legacy-sstables\jb\Keyspace1\Keyspace1-Standard1-jb-0-Summary.db
> Are these existing test data files, specific to Little Endian systems?
> I have observed differences in values read on a LE Vs BE systems.
> I use the following command to run the test case. ant test 
> -Dtest.name=LegacySSTableTest
> I get the following error:
> [junit] - Standard Error -
> [junit] Failed to read jb
> [junit] -  ---
> [junit] Testcase: 
> testStreaming(org.apache.cassandra.io.sstable.LegacySSTableTest): FAILED
> [junit] null
> [junit] junit.framework.AssertionFailedError
> [junit] at 
> org.apache.cassandra.io.sstable.IndexSummary.(IndexSummary.java:80)
> [junit] at 
> org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:317)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:877)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:730)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:692)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:480)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:376)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:371)
> [junit] at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:363)
> [junit] at 
> org.apache.cassandra.io.sstable.LegacySSTableTest.testStreaming(LegacySSTableTest.java:115)
> [junit] at 
> org.apache.cassandra.io.sstable.LegacySSTableTest.testStreaming(LegacySSTableTest.java:110)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10880) Paging state between 2.2 and 3.0 are incompatible on protocol v4

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061862#comment-15061862
 ] 

Sylvain Lebresne commented on CASSANDRA-10880:
--

bq. we lose UDF/UDA support in the driver, don't we?

Well, not really. We really only lose the _specific_ schema change events, but 
changes are still notified (as keyspace update if memory serves) so drivers can 
still deal with it (albeit in a slightly less efficient manner since they don't 
know what function has been added/updated and they may have to read more to 
figure that out, but that is unlikely to have much visible impact to users in 
practice).

But let me be clear that I'm not pretending any of the solutions above are 
great, they're not. And if someone has some idea I haven't think of that is 
much better, please do share. Short of that, I think "you shouldn't use the 
nice-but-not-absolutely-necessary additions of the v4 protocol until 3.0" is 
better than "you cannot upgrade to 3.x".

> Paging state between 2.2 and 3.0 are incompatible on protocol v4
> 
>
> Key: CASSANDRA-10880
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10880
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Sylvain Lebresne
>Priority: Critical
>  Labels: client-impacting
> Fix For: 3.x
>
>
> In CASSANDRA-10254, the paging states generated by 3.0 for the native 
> protocol v4 were made 3.0 specific. This was done because the paging state in 
> pre-3.0 versions contains a serialized cell name, but 3.0 doesn't talk in 
> term of cells internally (at least not the pre-3.0 ones) and so using an 
> old-format cell name when we only have 3.0 nodes is inefficient and inelegant.
> Unfortunately that change was made on the assumption than the protocol v4 was 
> 3.0 only but it's not, it ended up being released with 2.2 and that 
> completely slipped my mind. So in practice, you can't properly have a mixed 
> 2.2/3.0 cluster if your driver is using the protocol v4.
> And unfortunately, I don't think there is an easy way to fix that without 
> breaking something. Concretely, I can see 3 choices:
> # we change 3.0 so that it generates old-format paging states on the v4 
> protocol. The 2 main downsides are that 1) this breaks 3.0 upgrades if the 
> driver is using the v4 protocol, and at least on the java side the only 
> driver versions that support 3.0 will use v4 by default and 2) we're signing 
> off on having sub-optimal paging state until the protocol v5 ships (probably 
> not too soon).
> # we remove the v4 protocol from 2.2. This means 2.2 will have to use v3 
> before upgrade at the risk of breaking upgrade. This is also bad, but I'm not 
> sure the driver version using the v4 protocol are quite ready yet (at least 
> the java driver is not GA yet) so if we work with the drivers teams to make 
> sure the v3 protocol gets prefered by default on 2.2 in the GA versions of 
> these driver, this might be somewhat transparent to users.
> # we don't change anything code-wise, but we document clearly that you can't 
> upgrade from 2.2 to 3.0 if your clients use protocol v4 (so we leave upgrade 
> broken if the v4 protocol is used as it is currently). This is not great, but 
> we can work with the drivers teams here again to make sure drivers prefer the 
> v3 version for 2.2 nodes so most people don't notice in practice.
> I think I'm leaning towards solution 3). It's not great but at least we break 
> no minor upgrades (neither on 2.2, nor on 3.0) which is probably the most 
> important. We'd basically be just adding a new condition on 2.2->3.0 
> upgrades.  We could additionally make 3.0 node completely refuse v4 
> connections if they know a 2.2 nodes is in the cluster for extra safety.
> Ping [~omichallat], [~adutra] and [~aholmber] as you might want to be aware 
> of that ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10892) Can we run the batch to create views?

[Sandeep Gopalasetty (JIRA)]

Sandeep Gopalasetty created CASSANDRA-10892:
---

 Summary: Can we run the batch to create views?
 Key: CASSANDRA-10892
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10892
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
 Environment: Windows
Reporter: Sandeep Gopalasetty
 Fix For: 3.0.0


Can we run the batch to create views?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10875) cqlsh fails to decode utf-8 characters for text typed columns.

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062096#comment-15062096
 ] 

Paulo Motta commented on CASSANDRA-10875:
-

By default cqlsh uses the system-configured encoding. You can override this by 
using the {{--encoding}} option.

In order to find out your system's default encoding used by cqlsh, you can type 
{{bin/cqlsh \-\-help}} and check what's the default value of the {{--encoding}} 
option. Can you check that please?

> cqlsh fails to decode utf-8 characters for text typed columns.
> --
>
> Key: CASSANDRA-10875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10875
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Yasuharu Goto
>Assignee: Yasuharu Goto
>Priority: Minor
> Fix For: 2.1.13, 3.1
>
> Attachments: 10875-2.1.12.txt, 10875-3.1.txt
>
>
> Hi, we've found a bug that cqlsh can't handle unicode text in select 
> conditions even if it were text type.
> {noformat}
> $ ./bin/cqlsh
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.2-SNAPSHOT | CQL spec 3.3.1 | Native protocol v4]
> Use HELP for help.
> cqlsh> create KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 1};
> cqlsh> create table test.test(txt text primary key);
> cqlsh> insert into test.test (txt) values('日本語');
> cqlsh> select * from test.test where txt='日本語';
> 'ascii' codec can't decode byte 0xe6 in position 35: ordinal not in range(128)
> cqlsh> 
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10893) super_counter_test.py:TestSuperCounterClusterRestart.functional_test dtest failing on C* trunk on Windows

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10893:


 Summary: 
super_counter_test.py:TestSuperCounterClusterRestart.functional_test dtest 
failing on C* trunk on Windows
 Key: CASSANDRA-10893
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10893
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Jim Witschey
 Fix For: 3.x


The {{super_counter_test.py:TestSuperCounterClusterRestart.functional_test}} 
dtest fails consistently on trunk on Windows:

http://cassci.datastax.com/view/win32/job/trunk_dtest_win32/lastCompletedBuild/testReport/super_counter_test/TestSuperCounterClusterRestart/functional_test/history/

Sometimes the failure results from a {{NotFoundException}}:

http://cassci.datastax.com/job/trunk_dtest_win32/315/testReport/super_counter_test/TestSuperCounterClusterRestart/functional_test/

and other times it fails because some of the counters do not have the expected 
value:

http://cassci.datastax.com/view/win32/job/trunk_dtest_win32/312/testReport/junit/super_counter_test/TestSuperCounterClusterRestart/functional_test/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10829) cleanup + repair generates a lot of logs

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062136#comment-15062136
 ] 

Marcus Eriksson edited comment on CASSANDRA-10829 at 12/17/15 2:52 PM:
---

Problem is that we use the sstables marked as 'compacting' to create the 
canonical view (to get the original sstable instances) and when running 
cleanup, scrub, upgradesstables and anticompaction we mark each sstable as 
compacted as we finish it, but we keep it as compacting until the entire 
operation is done. This means that we have sstables which are marked as 
compacted in the CANONICAL_SSTABLES and we cannot reference them and it makes 
us log the 'spinning ... ' message.

There is also an issue with the log rate limiting, it was meant to only log 
every 100ms, but it logs every iteration.

Patch that unmarks compacting once the sstable finished and fixes the rate 
limiting: 

https://github.com/krummas/cassandra/commits/marcuse/10829
and tests:
https://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10829-dtest/
https://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10829-testall/

seems only the rate limiting fix applies to 2.2+ (edit: the other issues are 
already fixed with LifecycleTransaction)


was (Author: krummas):
Problem is that we use the sstables marked as 'compacting' to create the 
canonical view (to get the original sstable instances) and when running 
cleanup, scrub, upgradesstables and anticompaction we mark each sstable as 
compacted as we finish it, but we keep it as compacting until the entire 
operation is done. This means that we have sstables which are marked as 
compacted in the CANONICAL_SSTABLES and we cannot reference them and it makes 
us log the 'spinning ... ' message.

There is also an issue with the log rate limiting, it was meant to only log 
every 100ms, but it logs every iteration.

Patch that unmarks compacting once the sstable finished and fixes the rate 
limiting: 

https://github.com/krummas/cassandra/commits/marcuse/10829
and tests:
https://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10829-dtest/
https://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10829-testall/

seems only the rate limiting fix applies to 2.2+

> cleanup + repair generates a lot of logs
> 
>
> Key: CASSANDRA-10829
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10829
> Project: Cassandra
>  Issue Type: Bug
> Environment: 5 nodes on Cassandra 2.1.11 (on Debian)
>Reporter: Fabien Rousseau
>Assignee: Marcus Eriksson
>
> One of our node generates a lot of cassandra logs (int the 10 MB/s) and CPU 
> usage has increased (by a factor 2-3).
> This was most probably triggered by a "nodetool snapshot" while a cleanup was 
> already running on this node.
> An example of those logs:
> 2015-12-08 09:15:17,794 INFO  
> [ValidationExecutor:689]ColumnFamilyStore.java:1923 Spinning trying to 
> capture released readers [...]
> 2015-12-08 09:15:17,794 INFO  
> [ValidationExecutor:689]ColumnFamilyStore.java:1924 Spinning trying to 
> capture all readers [...]
> 2015-12-08 09:15:17,795 INFO  
> [ValidationExecutor:689]ColumnFamilyStore.java:1923 Spinning trying to 
> capture released readers [...]
> 2015-12-08 09:15:17,795 INFO  
> [ValidationExecutor:689]ColumnFamilyStore.java:1924 Spinning trying to 
> capture all readers [...]
> (I removed SSTableReader information because it's rather long... I can share 
> it privately if needed)
> Note that the date has not been changed (only 1ms between logs)
> It should not generate that gigantic amount of logs :)
> This is probably linked to: 
> https://issues.apache.org/jira/browse/CASSANDRA-9637



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-10837) Cluster/session should be closed in Cassandra Hadoop Input/Output classes

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer reassigned CASSANDRA-10837:
--

Assignee: Benjamin Lerer  (was: Alex Liu)

> Cluster/session should be closed in Cassandra Hadoop Input/Output classes
> -
>
> Key: CASSANDRA-10837
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10837
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Alex Liu
>Assignee: Benjamin Lerer
> Fix For: 3.0.x
>
> Attachments: 10837-3.0-branch.txt, 10837-3.0-v4.txt, 
> 10837-v2-3.0-branch.txt, 10837-v3-3.0-branch.txt
>
>
> See a lot of following warnings during Hadoop job running
> {code}
> ERROR 11:37:45 LEAK: You are creating too many HashedWheelTimer instances.  
> HashedWheelTimer is a shared resource that must be reused across the JVM,so 
> that only a few instances are created.
> {code}
> Each cluster/session needs be closed and a shared HashedWheelTimer may reduce 
> the resource leakage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10859) AssertionError in nodetool cfhistograms

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10859:

Reviewer: Paulo Motta

> AssertionError in nodetool cfhistograms
> ---
>
> Key: CASSANDRA-10859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10859
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Jim Witschey
>Assignee: Yuki Morishita
> Fix For: 3.0.x, 3.x
>
>
> {{nodetool cfhistograms}} raises an {{AssertionError}} on the CassCI 
> {{trunk}} jobs:
> http://cassci.datastax.com/job/trunk_dtest/lastCompletedBuild/testReport/jmx_test/TestJMX/cfhistograms_test/history/
> This regression started in this build on CassCI and has failed consistently 
> since then:
> http://cassci.datastax.com/job/trunk_dtest/806/testReport/junit/jmx_test/TestJMX/cfhistograms_test/
> I can't find any recent dtest changes to this method, so I believe it's a 
> Cassandra bug. Here's the changeset for the first failing build:
> http://cassci.datastax.com/job/trunk_dtest/806/changes
> Maybe something in the changes to the scripts here introduced the bug:
> https://github.com/apache/cassandra/commit/b5240204d7aa2a32c6649d19da2b961c856cde28
> [~jjordan] could you have a look at this please?
> This appears to only affect {{trunk}} at present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10837) Cluster/session should be closed in Cassandra Hadoop Input/Output classes

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10837:
--
Fix Version/s: (was: 3.0.x)
   3.0.3
   3.2

> Cluster/session should be closed in Cassandra Hadoop Input/Output classes
> -
>
> Key: CASSANDRA-10837
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10837
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Alex Liu
>Assignee: Alex Liu
> Fix For: 3.2, 3.0.3
>
> Attachments: 10837-3.0-branch.txt, 10837-3.0-v4.txt, 
> 10837-v2-3.0-branch.txt, 10837-v3-3.0-branch.txt
>
>
> See a lot of following warnings during Hadoop job running
> {code}
> ERROR 11:37:45 LEAK: You are creating too many HashedWheelTimer instances.  
> HashedWheelTimer is a shared resource that must be reused across the JVM,so 
> that only a few instances are created.
> {code}
> Each cluster/session needs be closed and a shared HashedWheelTimer may reduce 
> the resource leakage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10797) Bootstrap new node fails with OOM when streaming nodes contains thousands of sstables

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-10797:
-
External issue ID:   (was: 21205)

> Bootstrap new node fails with OOM when streaming nodes contains thousands of 
> sstables
> -
>
> Key: CASSANDRA-10797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10797
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: Cassandra 2.1.8.621 w/G1GC
>Reporter: Jose Martinez Poblete
>Assignee: Paulo Motta
> Fix For: 3.0.x, 3.x
>
> Attachments: 10797-nonpatched.png, 10797-patched.png, 
> 10798-nonpatched-500M.png, 10798-patched-500M.png, 112415_system.log, 
> Heapdump_OOM.zip, Screen Shot 2015-12-01 at 7.34.40 PM.png, dtest.tar.gz
>
>
> When adding a new node to an existing DC, it runs OOM after 25-45 minutes
> Upon heapdump revision, it is found the sending nodes are streaming thousands 
> of sstables which in turns blows the bootstrapping node heap 
> {noformat}
> ERROR [RMI Scheduler(0)] 2015-11-24 10:10:44,585 
> JVMStabilityInspector.java:94 - JVM state determined to be unstable.  Exiting 
> forcefully due to:
> java.lang.OutOfMemoryError: Java heap space
> ERROR [STREAM-IN-/173.36.28.148] 2015-11-24 10:10:44,585 
> StreamSession.java:502 - [Stream #0bb13f50-92cb-11e5-bc8d-f53b7528ffb4] 
> Streaming error occurred
> java.lang.IllegalStateException: Shutdown in progress
> at 
> java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82) 
> ~[na:1.8.0_65]
> at java.lang.Runtime.removeShutdownHook(Runtime.java:239) 
> ~[na:1.8.0_65]
> at 
> org.apache.cassandra.service.StorageService.removeShutdownHook(StorageService.java:747)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.utils.JVMStabilityInspector$Killer.killCurrentJVM(JVMStabilityInspector.java:95)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:64)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:66)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:38)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:250)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
> ERROR [RMI TCP Connection(idle)] 2015-11-24 10:10:44,585 
> JVMStabilityInspector.java:94 - JVM state determined to be unstable.  Exiting 
> forcefully due to:
> java.lang.OutOfMemoryError: Java heap space
> ERROR [OptionalTasks:1] 2015-11-24 10:10:44,585 CassandraDaemon.java:223 - 
> Exception in thread Thread[OptionalTasks:1,5,main]
> java.lang.IllegalStateException: Shutdown in progress
> {noformat}
> Attached is the Eclipse MAT report as a zipped web page



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10797) Bootstrap new node fails with OOM when streaming nodes contains thousands of sstables

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-10797:
-
External issue URL:   (was: 
https://datastax.zendesk.com/agent/tickets/21205)

> Bootstrap new node fails with OOM when streaming nodes contains thousands of 
> sstables
> -
>
> Key: CASSANDRA-10797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10797
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: Cassandra 2.1.8.621 w/G1GC
>Reporter: Jose Martinez Poblete
>Assignee: Paulo Motta
> Fix For: 3.0.x, 3.x
>
> Attachments: 10797-nonpatched.png, 10797-patched.png, 
> 10798-nonpatched-500M.png, 10798-patched-500M.png, 112415_system.log, 
> Heapdump_OOM.zip, Screen Shot 2015-12-01 at 7.34.40 PM.png, dtest.tar.gz
>
>
> When adding a new node to an existing DC, it runs OOM after 25-45 minutes
> Upon heapdump revision, it is found the sending nodes are streaming thousands 
> of sstables which in turns blows the bootstrapping node heap 
> {noformat}
> ERROR [RMI Scheduler(0)] 2015-11-24 10:10:44,585 
> JVMStabilityInspector.java:94 - JVM state determined to be unstable.  Exiting 
> forcefully due to:
> java.lang.OutOfMemoryError: Java heap space
> ERROR [STREAM-IN-/173.36.28.148] 2015-11-24 10:10:44,585 
> StreamSession.java:502 - [Stream #0bb13f50-92cb-11e5-bc8d-f53b7528ffb4] 
> Streaming error occurred
> java.lang.IllegalStateException: Shutdown in progress
> at 
> java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82) 
> ~[na:1.8.0_65]
> at java.lang.Runtime.removeShutdownHook(Runtime.java:239) 
> ~[na:1.8.0_65]
> at 
> org.apache.cassandra.service.StorageService.removeShutdownHook(StorageService.java:747)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.utils.JVMStabilityInspector$Killer.killCurrentJVM(JVMStabilityInspector.java:95)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:64)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:66)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:38)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:250)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
> ERROR [RMI TCP Connection(idle)] 2015-11-24 10:10:44,585 
> JVMStabilityInspector.java:94 - JVM state determined to be unstable.  Exiting 
> forcefully due to:
> java.lang.OutOfMemoryError: Java heap space
> ERROR [OptionalTasks:1] 2015-11-24 10:10:44,585 CassandraDaemon.java:223 - 
> Exception in thread Thread[OptionalTasks:1,5,main]
> java.lang.IllegalStateException: Shutdown in progress
> {noformat}
> Attached is the Eclipse MAT report as a zipped web page



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10745) Deprecate PropertyFileSnitch

[Sean Durity (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062037#comment-15062037
 ] 

Sean Durity commented on CASSANDRA-10745:
-

Thank you for considering my comment. I don't have any preference on the exact 
implementation details. Either of the options above still lets me distribute 
the same configuration files to every node, which is my request. I appreciate 
your work!

> Deprecate PropertyFileSnitch
> 
>
> Key: CASSANDRA-10745
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10745
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Distributed Metadata
>Reporter: Paulo Motta
>Priority: Minor
>
> Opening this ticket to discuss deprecating PropertyFileSnitch, since it's 
> error-prone and more snitch code to maintain (See CASSANDRA-10243). Migration 
> from existing cluster with PropertyFileSnitch to GossipingPropertyFileSnitch 
> is straightforward.
> Is there any useful use case that can be achieved only with 
> PropertyFileSnitch?
> If not objections, we would add deprecation warnings in 2.2.x, 3.0.x, 3.2 and 
> deprecate in 3.4 or 3.6.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10892) Can we run the batch to create views?

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-10892.
--
Resolution: Invalid

I'll refer you to my comment on your other equally invalid ticket.

> Can we run the batch to create views?
> -
>
> Key: CASSANDRA-10892
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10892
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows
>Reporter: Sandeep Gopalasetty
> Fix For: 3.0.0
>
>
> Can we run the batch to create views?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10837) Cluster/session should be closed in Cassandra Hadoop Input/Output classes

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10837:
---
Assignee: Alex Liu  (was: Benjamin Lerer)

> Cluster/session should be closed in Cassandra Hadoop Input/Output classes
> -
>
> Key: CASSANDRA-10837
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10837
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Alex Liu
>Assignee: Alex Liu
> Fix For: 3.0.x
>
> Attachments: 10837-3.0-branch.txt, 10837-3.0-v4.txt, 
> 10837-v2-3.0-branch.txt, 10837-v3-3.0-branch.txt
>
>
> See a lot of following warnings during Hadoop job running
> {code}
> ERROR 11:37:45 LEAK: You are creating too many HashedWheelTimer instances.  
> HashedWheelTimer is a shared resource that must be reused across the JVM,so 
> that only a few instances are created.
> {code}
> Each cluster/session needs be closed and a shared HashedWheelTimer may reduce 
> the resource leakage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10891) Cassandra driver doesn't allow to insert null as a value in map

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-10891.
--
Resolution: Invalid

As indicated by the stacktrace, this is a driver exception and you may open a 
ticket on java driver JIRA if you wish to: 
https://datastax-oss.atlassian.net/browse/JAVA.

I will note however that {{null}} values are not supported by CQL so you simply 
cannot do what you are trying to do: you need to make sure you have no nulls. 
Whether the driver should throw a more meaningful exception in that case is 
something I'll left to the drivers maintainer to answer.

> Cassandra driver doesn't allow to insert null as a value in map
> ---
>
> Key: CASSANDRA-10891
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10891
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tomasz Szlek
>
> When cassandra has column with type map and I try to insert entry 
> with null value cassandra driver throws following exception:
> {code}
> java.lang.NullPointerException
> at 
> com.datastax.driver.core.TypeCodec$StringCodec.serialize(TypeCodec.java:405)
> at 
> com.datastax.driver.core.TypeCodec$StringCodec.serialize(TypeCodec.java:350)
> at 
> com.datastax.driver.core.TypeCodec$MapCodec.serialize(TypeCodec.java:1169)
> at 
> com.datastax.driver.core.TypeCodec$MapCodec.serialize(TypeCodec.java:1085)
> at com.datastax.driver.core.DataType.serialize(DataType.java:622)
> at com.datastax.driver.core.DataType.serializeValue(DataType.java:727)
> at com.datastax.driver.core.querybuilder.Utils.convert(Utils.java:80)
> at 
> com.datastax.driver.core.querybuilder.BuiltStatement.getValues(BuiltStatement.java:172)
> at 
> com.datastax.driver.core.SessionManager.makeRequestMessage(SessionManager.java:496)
> at 
> com.datastax.driver.core.SessionManager.makeRequestMessage(SessionManager.java:465)
> at 
> com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:119)
> at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10859) AssertionError in nodetool cfhistograms

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062087#comment-15062087
 ] 

Paulo Motta commented on CASSANDRA-10859:
-

Test is fixed, but can't we improve this a bit by creating a constructor that 
receives only the bucket data and creates the bucket offsets internally? So, 
this:
{code:java}
long[] rowSizeBucketOffsets = new 
EstimatedHistogram(estimatedRowSize.length - 1).getBucketOffsets();
long[] columnCountBucketOffsets = new 
EstimatedHistogram(estimatedColumnCount.length - 1).getBucketOffsets();
EstimatedHistogram rowSizeHist = new 
EstimatedHistogram(rowSizeBucketOffsets, estimatedRowSize);
EstimatedHistogram columnCountHist = new 
EstimatedHistogram(columnCountBucketOffsets, estimatedColumnCount);
{code}
would become:
{code:java}
EstimatedHistogram rowSizeHist = new 
EstimatedHistogram(estimatedRowSize);
EstimatedHistogram columnCountHist = new 
EstimatedHistogram(estimatedColumnCount);
{code}


> AssertionError in nodetool cfhistograms
> ---
>
> Key: CASSANDRA-10859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10859
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Jim Witschey
>Assignee: Yuki Morishita
> Fix For: 3.0.x, 3.x
>
>
> {{nodetool cfhistograms}} raises an {{AssertionError}} on the CassCI 
> {{trunk}} jobs:
> http://cassci.datastax.com/job/trunk_dtest/lastCompletedBuild/testReport/jmx_test/TestJMX/cfhistograms_test/history/
> This regression started in this build on CassCI and has failed consistently 
> since then:
> http://cassci.datastax.com/job/trunk_dtest/806/testReport/junit/jmx_test/TestJMX/cfhistograms_test/
> I can't find any recent dtest changes to this method, so I believe it's a 
> Cassandra bug. Here's the changeset for the first failing build:
> http://cassci.datastax.com/job/trunk_dtest/806/changes
> Maybe something in the changes to the scripts here introduced the bug:
> https://github.com/apache/cassandra/commit/b5240204d7aa2a32c6649d19da2b961c856cde28
> [~jjordan] could you have a look at this please?
> This appears to only affect {{trunk}} at present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10891) Cassandra driver doesn't allow to insert null as a value in map

[Tomasz Szlek (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomasz Szlek updated CASSANDRA-10891:
-
Summary: Cassandra driver doesn't allow to insert null as a value in map  
(was: Cassandra driver do not allow to insert null as a value in map)

> Cassandra driver doesn't allow to insert null as a value in map
> ---
>
> Key: CASSANDRA-10891
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10891
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tomasz Szlek
>
> When cassandra has column with type map and I try to insert entry 
> with null value cassandra driver throws following exception:
> {code}
> java.lang.NullPointerException
> at 
> com.datastax.driver.core.TypeCodec$StringCodec.serialize(TypeCodec.java:405)
> at 
> com.datastax.driver.core.TypeCodec$StringCodec.serialize(TypeCodec.java:350)
> at 
> com.datastax.driver.core.TypeCodec$MapCodec.serialize(TypeCodec.java:1169)
> at 
> com.datastax.driver.core.TypeCodec$MapCodec.serialize(TypeCodec.java:1085)
> at com.datastax.driver.core.DataType.serialize(DataType.java:622)
> at com.datastax.driver.core.DataType.serializeValue(DataType.java:727)
> at com.datastax.driver.core.querybuilder.Utils.convert(Utils.java:80)
> at 
> com.datastax.driver.core.querybuilder.BuiltStatement.getValues(BuiltStatement.java:172)
> at 
> com.datastax.driver.core.SessionManager.makeRequestMessage(SessionManager.java:496)
> at 
> com.datastax.driver.core.SessionManager.makeRequestMessage(SessionManager.java:465)
> at 
> com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:119)
> at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10889) How can we combine two tables? Can we do this with JOINS? like INNER JOINS or other?

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062105#comment-15062105
 ] 

Sylvain Lebresne commented on CASSANDRA-10889:
--

JIRA is not a support forum. Please, pretty please, stop opening tickets for 
beginner questions and use the mailing list instead. You can find link to join 
the user mailing list if you haven't yet at the bottom of 
http://cassandra.apache.org/.

> How can we combine two tables? Can we do this with JOINS? like INNER JOINS or 
> other?
> 
>
> Key: CASSANDRA-10889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10889
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows/Linux
>Reporter: Sandeep Gopalasetty
> Fix For: 3.0.x
>
>
> How can we combine two tables? Can we do this with JOINS? like INNER JOINS or 
> other?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10889) How can we combine two tables? Can we do this with JOINS? like INNER JOINS or other?

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-10889.
--
Resolution: Invalid

> How can we combine two tables? Can we do this with JOINS? like INNER JOINS or 
> other?
> 
>
> Key: CASSANDRA-10889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10889
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows/Linux
>Reporter: Sandeep Gopalasetty
> Fix For: 3.0.x
>
>
> How can we combine two tables? Can we do this with JOINS? like INNER JOINS or 
> other?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10891) Cassandra driver do not allow to insert null as a value in map

[Tomasz Szlek (JIRA)]

Tomasz Szlek created CASSANDRA-10891:


 Summary: Cassandra driver do not allow to insert null as a value 
in map
 Key: CASSANDRA-10891
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10891
 Project: Cassandra
  Issue Type: Bug
Reporter: Tomasz Szlek


When cassandra has column with type map and I try to insert entry 
with null value cassandra driver throws following exception:

{code}
java.lang.NullPointerException
at 
com.datastax.driver.core.TypeCodec$StringCodec.serialize(TypeCodec.java:405)
at 
com.datastax.driver.core.TypeCodec$StringCodec.serialize(TypeCodec.java:350)
at 
com.datastax.driver.core.TypeCodec$MapCodec.serialize(TypeCodec.java:1169)
at 
com.datastax.driver.core.TypeCodec$MapCodec.serialize(TypeCodec.java:1085)
at com.datastax.driver.core.DataType.serialize(DataType.java:622)
at com.datastax.driver.core.DataType.serializeValue(DataType.java:727)
at com.datastax.driver.core.querybuilder.Utils.convert(Utils.java:80)
at 
com.datastax.driver.core.querybuilder.BuiltStatement.getValues(BuiltStatement.java:172)
at 
com.datastax.driver.core.SessionManager.makeRequestMessage(SessionManager.java:496)
at 
com.datastax.driver.core.SessionManager.makeRequestMessage(SessionManager.java:465)
at 
com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:119)
at 
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)

{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10829) cleanup + repair generates a lot of logs

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062136#comment-15062136
 ] 

Marcus Eriksson commented on CASSANDRA-10829:
-

Problem is that we use the sstables marked as 'compacting' to create the 
canonical view (to get the original sstable instances) and when running 
cleanup, scrub, upgradesstables and anticompaction we mark each sstable as 
compacted as we finish it, but we keep it as compacting until the entire 
operation is done. This means that we have sstables which are marked as 
compacted in the CANONICAL_SSTABLES and we cannot reference them and it makes 
us log the 'spinning ... ' message.

There is also an issue with the log rate limiting, it was meant to only log 
every 100ms, but it logs every iteration.

Patch that unmarks compacting once the sstable finished and fixes the rate 
limiting: 

https://github.com/krummas/cassandra/commits/marcuse/10829
and tests:
https://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10829-dtest/
https://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10829-testall/

seems only the rate limiting fix applies to 2.2+

> cleanup + repair generates a lot of logs
> 
>
> Key: CASSANDRA-10829
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10829
> Project: Cassandra
>  Issue Type: Bug
> Environment: 5 nodes on Cassandra 2.1.11 (on Debian)
>Reporter: Fabien Rousseau
>Assignee: Marcus Eriksson
>
> One of our node generates a lot of cassandra logs (int the 10 MB/s) and CPU 
> usage has increased (by a factor 2-3).
> This was most probably triggered by a "nodetool snapshot" while a cleanup was 
> already running on this node.
> An example of those logs:
> 2015-12-08 09:15:17,794 INFO  
> [ValidationExecutor:689]ColumnFamilyStore.java:1923 Spinning trying to 
> capture released readers [...]
> 2015-12-08 09:15:17,794 INFO  
> [ValidationExecutor:689]ColumnFamilyStore.java:1924 Spinning trying to 
> capture all readers [...]
> 2015-12-08 09:15:17,795 INFO  
> [ValidationExecutor:689]ColumnFamilyStore.java:1923 Spinning trying to 
> capture released readers [...]
> 2015-12-08 09:15:17,795 INFO  
> [ValidationExecutor:689]ColumnFamilyStore.java:1924 Spinning trying to 
> capture all readers [...]
> (I removed SSTableReader information because it's rather long... I can share 
> it privately if needed)
> Note that the date has not been changed (only 1ms between logs)
> It should not generate that gigantic amount of logs :)
> This is probably linked to: 
> https://issues.apache.org/jira/browse/CASSANDRA-9637



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9291) Too many tombstones in schema_columns from creating too many CFs

[Thierry Schellenbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062258#comment-15062258
 ] 

Thierry Schellenbach commented on CASSANDRA-9291:
-

Thierry from getstream.io here. We're getting this error message:

WARN [ReadStage:447081] 2015-12-17 16:18:16,908 SliceQueryFilter.java (line 
231) Read 3981 live and 12251 tombstone cells in system.schema_columns (see 
tombstone_warn_threshold). 2147483357 columns was requested, slices=[-]

CPU on 3 of our Cassandra machines is at 100%. Our infrastructure is down as a 
result.

nodetool resetlocalschema doesn't fix the issue.

> Too many tombstones in schema_columns from creating too many CFs
> 
>
> Key: CASSANDRA-9291
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9291
> Project: Cassandra
>  Issue Type: Bug
> Environment: Production Cluster with 2 DCs of 3 nodes each and 1 DC 
> of 7 nodes, running on dedicated Xeon hexacore, 96GB ram, RAID for Data and 
> SSF for commitlog, running Debian 7 (with Java 1.7.0_76-b13 64-Bit, 8GB and 
> 16GB of heap tested).
> Dev Cluster with 1 DC with 3 nodes and 1 DC with 1 node, running on 
> virtualized env., Ubuntu 12.04.5 (with Java 1.7.0_72-b14 64-Bit 1GB, 4GB 
> heap) 
>Reporter: Luis Correia
>Priority: Blocker
> Attachments: after_schema.txt, before_schema.txt, schemas500.cql
>
>
> When creating lots of columnfamilies (about 200) the system.schema_columns 
> gets filled with tombstones and therefore prevents clients using the binary 
> protocol of connecting.
> Clients already connected continue normal operation (reading and inserting).
> Log messages are:
> For the first tries (sorry for the lack of precision):
> bq. ERROR [main] 2015-04-22 00:01:38,527 SliceQueryFilter.java (line 200) 
> Scanned over 10 tombstones in system.schema_columns; query aborted (see 
> tombstone_failure_threshold)
> For each client that tries to connect but fails with timeout:
>  bq. WARN [ReadStage:35] 2015-04-27 15:40:10,600 SliceQueryFilter.java (line 
> 231) Read 395 live and 1217 tombstoned cells in system.schema_columns (see 
> tombstone_warn_threshold). 2147283441 columns was requested, slices=[-]
> bq. WARN [ReadStage:40] 2015-04-27 15:40:10,609 SliceQueryFilter.java (line 
> 231) Read 395 live and 1217 tombstoned cells in system.schema_columns (see 
> tombstone_warn_threshold). 2147283441 columns was requested, slices=[-]
> bq. WARN [ReadStage:61] 2015-04-27 15:40:10,670 SliceQueryFilter.java (line 
> 231) Read 395 live and 1217 tombstoned cells in system.schema_columns (see 
> tombstone_warn_threshold). 2147283441 columns was requested, slices=[-]
> bq. WARN [ReadStage:51] 2015-04-27 15:40:10,670 SliceQueryFilter.java (line 
> 231) Read 395 live and 1217 tombstoned cells in system.schema_columns (see 
> tombstone_warn_threshold). 2147283441 columns was requested, slices=[-]
> bq. WARN [ReadStage:55] 2015-04-27 15:40:10,675 SliceQueryFilter.java (line 
> 231) Read 395 live and 1217 tombstoned cells in system.schema_columns (see 
> tombstone_warn_threshold). 2147283441 columns was requested, slices=[-]
> bq. WARN [ReadStage:35] 2015-04-27 15:40:10,707 SliceQueryFilter.java (line 
> 231) Read 1146 live and 3534 tombstoned cells in system.schema_columns (see 
> tombstone_warn_threshold). 2147282894 columns was requested, slices=[-]
> bq. WARN [ReadStage:40] 2015-04-27 15:40:10,708 SliceQueryFilter.java (line 
> 231) Read 1146 live and 3534 tombstoned cells in system.schema_columns (see 
> tombstone_warn_threshold). 2147282894 columns was requested, slices=[-]
> bq. WARN [ReadStage:43] 2015-04-27 15:40:10,715 SliceQueryFilter.java (line 
> 231) Read 395 live and 1217 tombstoned cells in system.schema_columns (see 
> tombstone_warn_threshold). 2147283441 columns was requested, slices=[-]
> bq. WARN [ReadStage:51] 2015-04-27 15:40:10,736 SliceQueryFilter.java (line 
> 231) Read 1146 live and 3534 tombstoned cells in system.schema_columns (see 
> tombstone_warn_threshold). 2147282894 columns was requested, slices=[-]
> bq. WARN [ReadStage:61] 2015-04-27 15:40:10,736 SliceQueryFilter.java (line 
> 231) Read 1146 live and 3534 tombstoned cells in system.schema_columns (see 
> tombstone_warn_threshold). 2147282894 columns was requested, slices=[-]
> bq. WARN [ReadStage:35] 2015-04-27 15:40:10,750 SliceQueryFilter.java (line 
> 231) Read 864 live and 2664 tombstoned cells in system.schema_columns (see 
> tombstone_warn_threshold). 2147281748 columns was requested, slices=[-]
> bq. WARN [ReadStage:40] 2015-04-27 15:40:10,751 SliceQueryFilter.java (line 
> 231) Read 864 live and 2664 tombstoned cells in system.schema_columns (see 
> tombstone_warn_threshold). 2147281748 columns was requested, slices=[-]
> bq. WARN [ReadStage:55] 2015-04-27 15:40:10,759 

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062162#comment-15062162
 ] 

Ariel Weisberg commented on CASSANDRA-9318:
---

To summarize what I have been experimenting with. I have been limiting disk 
throughput with a rate limiter both with and without limiting the commit log. I 
am excluding the commitlog because I don't want to bottleneck the ingest rate. 
I have also been experimenting with limiting the rate at which remote delivered 
messages are consumed which more or less forces the mutations to time out.

I added hint backpressure and since then I have been unable to make the server 
fail or OOM from slow disk or slow remote nodes. What I have found is that I am 
unable to get the server to start enough concurrent requests to OOM in the disk 
bound case because the request stage is bounded to 128 requests. 

In the remote mutation bound case I think I am running out of ingest capacity 
on a single node and that is preventing me from running up more requests in 
memory faster than they can be evicted by the 2 second timeout. With gigabit 
ethernet the reason is obvious. With 10-gigabit ethernet I would still be 
limited to 1.6 gigabytes of data assuming timeouts are prompt. That's a hefty 
quantity, but something that could fit in a larger heap. I should be able to 
test that soon.

I noticed OutboundTcpConnection is another potential source of OOM since it can 
only drop messages if the thread isn't blocked writing to the socket. I am 
allowing it to wakeup regularly in my testing, but if something is unavailable 
we would have to not OOM before failure detection kicks in which is a longer 
timespan. It might make sense to have another thread periodically check if it 
should expire messages from the queue. This isn't something I can test for 
without access to 10-gig.
have coordinate several hundred (or thousand?) megabytes of transaction data to 
OOM. This assumes you can get past the ingest limitations of gigabit ethernet.

For code changes I think that disabling reads based on memory pressure is 
mostly harmless, but I can't demonstrate that it adds a lot of value given how 
things behave with a large enough heap and the network level bounds on ingest 
rate.

I do want to add backpressure to hints and the compressed commit log because 
that is an easy path to OOM. I'll package up those changes up today.

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 2.1.x, 2.2.x
>
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10894) sstablesplit_test.py:TestSSTableSplit.split_test failing on Windows

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10894:


 Summary: sstablesplit_test.py:TestSSTableSplit.split_test failing 
on Windows
 Key: CASSANDRA-10894
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10894
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Jim Witschey


This test fails consistently on Windows on C* trunk:

http://cassci.datastax.com/job/trunk_dtest_win32/315/testReport/sstablesplit_test/TestSSTableSplit/split_test/history/

It fails when {{sstablesplit}} leaves behind a single SSTable, rather than 6, 
as is expected:

http://cassci.datastax.com/job/trunk_dtest_win32/315/testReport/sstablesplit_test/TestSSTableSplit/split_test/

I have not seen this on other versions or in other environments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9303) Match cassandra-loader options in COPY FROM

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062302#comment-15062302
 ] 

Stefania commented on CASSANDRA-9303:
-



bq. I'd suggest the following \[copy(:ks.table)\] (global and per-table copy 
(to and from) options), \[copy-from(:ks.table)\] (global and per-table 
copy-from options), \[copy-to(:ks.table)\] (global and per-table copy-to 
options) where (:ks.table) is optional. so you can have \[copy\], \[copy-to\], 
\[copy-from\], \[copy-to:ks.table\], \[copy-from:ks.table\].

Done, I cleaned up the options a bit as well and removed the helper methods in 
the main cqlsh files.

bq. maybe we could just add an unique suffix to avoid appending to an existing 
file from a previous execution?

If a file from a previous execution exists it will be ranamed to 
.MMDD_HHMMSS.

bq. We can address if it won't take too much time, otherwise we can address it 
separately. Can we maybe improve it by making batchsize adaptive = 
min(batchsize, ingest_rate - current_record) or something more complicated will 
be needed?

Done, adaptive chunk size and retries needed changing.
 
bq. Move SKIPCOLS to COPY_COMMON_OPTIONS since it can be used in both copy-to 
and copy-from.

Actually it should be a COPY FROM only option, see more below.

bq. Regarding the beahvior of SKIPCOLS with COPY FROM, right now it only 
supports having fewer columns in the CSV. Should we also support actually 
skipping columns in the CSV even if they are present?

I think the sematic I chose, to use SKIPCOLS to subtract from the set of 
columns specified in the command line, is not as advantageous as the ability to 
skip columns in the file. Providing both features with the same option would be 
confusing. So, I converted SKIPCOLS to a COPY FROM option and changed its 
semantic to just skip columns that exist in the file. If in future the need 
arises to specify "all columns except" in the command line, we can introduce a 
regex like extression (^col_name) in the columns part of the COPY cmd.

bq. Another related feature to have in the future would be to pick only 
specific columnms from the csv and allowing custom orderings of columns, but we 
can leave that for later if there's a need.

I think reordering columns is not as useful as skipping them so I tend to agree 
to leave this as a future development if the need arises.

bq. After those are addressed you can probably start making 2.2+ patches.

I changed a lot of code today and I've run out of time anyway, so I'll wait for 
one more round of review before up-merging.


> Match cassandra-loader options in COPY FROM
> ---
>
> Key: CASSANDRA-9303
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9303
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Critical
> Fix For: 2.1.x
>
>
> https://github.com/brianmhess/cassandra-loader added a bunch of options to 
> handle real world requirements, we should match those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10897) Avoid building PartitionUpdate in toString()

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-10897:

Priority: Minor  (was: Major)

> Avoid building PartitionUpdate in toString()
> 
>
> Key: CASSANDRA-10897
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10897
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Minor
> Fix For: 3.0.x, 3.x
>
>
> In {{AbstractBTreePartition.toString()}} (which {{PartitionUpdate}} extends), 
> we iterate over the rows in the partition.  This triggers {{maybeBuild()}} in 
> the {{PartitionUpdate}}.  If the {{PartitionUpdate}} gets updated after the 
> {{toString()}} call, it will result in an {{IllegalStateException}} with the 
> message "An update should not be written again once it has been read".
> As a result, logging or using a debugger can trigger spurious errors, which 
> makes debugging difficult or impossible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10801) Unexplained inconsistent data with Cassandra 2.1

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062199#comment-15062199
 ] 

Paulo Motta commented on CASSANDRA-10801:
-

A funny thing is that on case 2 coordinator (10.84.30.222) sends a read message 
to 10.84.30.236 on instant 24.718, and that message is received on instant 
24.716, so a few ms before the message was sent.

How is data being written to the cluster? Individually or in batches? Are there 
multiple updates to the same key? Are there any data deletions? Are nodes 
clocks synchronized? What's the delay between data write and read? What is the 
gc_grace_seconds of the testcf table? Are there any ERROR or WARN in the logs?

No specific suspicion, just general questions to see if we can infer something. 
Some more specific reproduction steps would be nice too.

> Unexplained inconsistent data with Cassandra 2.1
> 
>
> Key: CASSANDRA-10801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10801
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Imri Zvik
> Fix For: 2.1.x
>
> Attachments: tracing.log
>
>
> We are experiencing weird behavior which we cannot explain.
> We have a CF, with RF=3, and we are writing and reading data to it with 
> consistency level of ONE.
> For some reason, we see inconsistent results when querying for data.
> Even for rows that were written a day ago, we're seeing inconsistent results 
> (1 replca has the data, the two others don't).
> Now, I would expect to see timeouts/dropped mutations, but all relevant 
> counters are not advancing, and I would also expect hints to fix this 
> inconsistency within minutes, but yet it doesn't.
> {code}
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
>  writetime(last_update) | site_id   | tree_id | individual_id | last_update
> +---+-+---+-
>1448988343028000 | 229673621 |   9 |   9032483 |  1380912397
> (1 rows)
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
> site_id   | tree_id | individual_id | last_update
> ---+-+---+-
> (0 rows)
> cqlsh:testing> SELECT dateof(now()) FROM system.local ;
>  dateof(now())
> --
>  2015-12-02 14:48:44+
> (1 rows)
> {code}
> We are running with Cassandra 2.1.11 with Oracle Java 1.8.0_65-b17



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10801) Unexplained inconsistent data with Cassandra 2.1

[Maor Cohen (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062242#comment-15062242
 ] 

Maor Cohen commented on CASSANDRA-10801:


Thanks Paulo.

I'll try to answer you questions:
* We tried writing in batches and using individual updates. Problem appeared in 
both methods.
* We don't update existing rows (using "update" query), instead we run another 
"insert" query that overwrites the row.
* We have very few deletions. Less than 1% of queries.
* Reading in production env is done using range query, although as you can see 
in the example it happens also with simple queries.
* In the attached example the data was written 4 days before the read (write 
consistency level was "ALL" in the write time).
* gc_grace_seconds = 864000
* No special errors or warnings in the log file. There are some dropped 
mutations but that's all.
* Clock is synchronized on all servers

I wish I had a way to reproduce this. This seems to be quite random. I hope the 
information I added can give you a lead.

Let me know if there is more information I can add.


> Unexplained inconsistent data with Cassandra 2.1
> 
>
> Key: CASSANDRA-10801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10801
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Imri Zvik
> Fix For: 2.1.x
>
> Attachments: tracing.log
>
>
> We are experiencing weird behavior which we cannot explain.
> We have a CF, with RF=3, and we are writing and reading data to it with 
> consistency level of ONE.
> For some reason, we see inconsistent results when querying for data.
> Even for rows that were written a day ago, we're seeing inconsistent results 
> (1 replca has the data, the two others don't).
> Now, I would expect to see timeouts/dropped mutations, but all relevant 
> counters are not advancing, and I would also expect hints to fix this 
> inconsistency within minutes, but yet it doesn't.
> {code}
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
>  writetime(last_update) | site_id   | tree_id | individual_id | last_update
> +---+-+---+-
>1448988343028000 | 229673621 |   9 |   9032483 |  1380912397
> (1 rows)
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id, 
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND 
> individual_id = 9032483;
> site_id   | tree_id | individual_id | last_update
> ---+-+---+-
> (0 rows)
> cqlsh:testing> SELECT dateof(now()) FROM system.local ;
>  dateof(now())
> --
>  2015-12-02 14:48:44+
> (1 rows)
> {code}
> We are running with Cassandra 2.1.11 with Oracle Java 1.8.0_65-b17



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10896) Fix skipping logic on upgrade tests in dtest

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10896:


 Summary: Fix skipping logic on upgrade tests in dtest
 Key: CASSANDRA-10896
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10896
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Jim Witschey
Assignee: Jim Witschey


This will be a general ticket for upgrade dtests that fail because of bad logic 
surrounding skipping tests. We need a better system in place for skipping tests 
that are not intended to work on certain versions of Cassandra; at present, we 
run the upgrade tests with {{SKIP=false}} because, again, the built-in skipping 
logic is bad.

One such test is test_v2_protocol_IN_with_tuples:

http://cassci.datastax.com/job/storage_engine_upgrade_dtest-22_tarball-311/lastCompletedBuild/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3/test_v2_protocol_IN_with_tuples/

This shouldn't be run on clusters that include nodes running 3.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10897) Avoid building PartitionUpdate in toString()

[Tyler Hobbs (JIRA)]

Tyler Hobbs created CASSANDRA-10897:
---

 Summary: Avoid building PartitionUpdate in toString()
 Key: CASSANDRA-10897
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10897
 Project: Cassandra
  Issue Type: Bug
  Components: Coordination
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
 Fix For: 3.0.x, 3.x


In {{AbstractBTreePartition.toString()}} (which {{PartitionUpdate}} extends), 
we iterate over the rows in the partition.  This triggers {{maybeBuild()}} in 
the {{PartitionUpdate}}.  If the {{PartitionUpdate}} gets updated after the 
{{toString()}} call, it will result in an {{IllegalStateException}} with the 
message "An update should not be written again once it has been read".

As a result, logging or using a debugger can trigger spurious errors, which 
makes debugging difficult or impossible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10897) Avoid building PartitionUpdate in toString()

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062318#comment-15062318
 ] 

Tyler Hobbs commented on CASSANDRA-10897:
-

Patch and pending CI runs:

||branch||testall||dtest||
|[CASSANDRA-10897|https://github.com/thobbs/cassandra/tree/CASSANDRA-10897]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10897-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10897-dtest]|
|[CASSANDRA-10897-trunk|https://github.com/thobbs/cassandra/tree/CASSANDRA-10897-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10897-trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10897-trunk-dtest]|

> Avoid building PartitionUpdate in toString()
> 
>
> Key: CASSANDRA-10897
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10897
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
> Fix For: 3.0.x, 3.x
>
>
> In {{AbstractBTreePartition.toString()}} (which {{PartitionUpdate}} extends), 
> we iterate over the rows in the partition.  This triggers {{maybeBuild()}} in 
> the {{PartitionUpdate}}.  If the {{PartitionUpdate}} gets updated after the 
> {{toString()}} call, it will result in an {{IllegalStateException}} with the 
> message "An update should not be written again once it has been read".
> As a result, logging or using a debugger can trigger spurious errors, which 
> makes debugging difficult or impossible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10895) The return of CASSANDRA-8718?

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10895:
-
Assignee: Joshua McKenzie

> The return of CASSANDRA-8718?
> -
>
> Key: CASSANDRA-10895
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10895
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Cassandra 2.1.9 running on:
> ubuntu@ip-10-250-1-225:~$ uname -a
> Linux ip-10-250-1-225 3.13.0-65-generic #105-Ubuntu SMP Mon Sep 21 18:50:58 
> UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
> ubuntu@ip-10-250-1-225:~$ java -version
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
> ubuntu@ip-10-250-1-225:~$
>Reporter: Pete Ehlke
>Assignee: Joshua McKenzie
> Attachments: hs_err_pid317.log
>
>
> Repeatable every time I run 'nodetool repair -inc -par' on this host:
> 2015-12-17_14:32:30.78574 INFO  14:32:30 Writing 
> Memtable-sstable_activity@907762948(95.060KiB serialized bytes, 17729 ops, 
> 0%/0% of on/off-heap limi
> t)
> 2015-12-17_14:32:30.80120 INFO  14:32:30 Completed flushing 
> /data/cassandra/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/system-ssta
> ble_activity-tmp-ka-2126-Data.db (75.676KiB) for commitlog position 
> ReplayPosition(segmentId=1449246749652, position=3079886)
> 2015-12-17_14:32:32.67513 #
> 2015-12-17_14:32:32.67516 # A fatal error has been detected by the Java 
> Runtime Environment:
> 2015-12-17_14:32:32.67517 #
> 2015-12-17_14:32:32.67518 #  SIGSEGV (0xb) at pc=0x7f477699117a, pid=317, 
> tid=139942572054272
> 2015-12-17_14:32:32.67525 #
> 2015-12-17_14:32:32.67526 # JRE version: Java(TM) SE Runtime Environment 
> (8.0_60-b27) (build 1.8.0_60-b27)
> 2015-12-17_14:32:32.67527 # Java VM: Java HotSpot(TM) 64-Bit Server VM 
> (25.60-b23 mixed mode linux-amd64 compressed oops)
> 2015-12-17_14:32:32.67528 # Problematic frame:
> 2015-12-17_14:32:32.67529 # J 21341 C2 
> org.apache.cassandra.io.sstable.IndexSummary.binarySearch(Lorg/apache/cassandra/db/RowPosition;)I
>  (88 bytes) @ 0x7f477699117a [0x7f4776991020+0x15a]
> 2015-12-17_14:32:32.67532 #
> 2015-12-17_14:32:32.67533 # Failed to write core dump. Core dumps have been 
> disabled. To enable core dumping, try "ulimit -c unlimited" before starting 
> Java again
> 2015-12-17_14:32:32.67535 #
> 2015-12-17_14:32:32.67587 # An error report file with more information is 
> saved as:
> 2015-12-17_14:32:32.67587 # /tmp/hs_err_pid317.log
> 2015-12-17_14:32:32.69831 #
> 2015-12-17_14:32:32.69832 # If you would like to submit a bug report, please 
> visit:
> 2015-12-17_14:32:32.69832 #   http://bugreport.java.com/bugreport/crash.jsp
> 2015-12-17_14:32:32.69832 #
> This looks exactly like CASSANDRA-8718, but that was marked as fixed in 2.0.13



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10893) super_counter_test.py:TestSuperCounterClusterRestart.functional_test dtest failing on C* trunk on Windows

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey resolved CASSANDRA-10893.
--
Resolution: Invalid

Sorry, this ticket is based on old information. This _was_ failing, I don't 
know if it is anymore. Will reopen this ticket if I can get more current 
information.

> super_counter_test.py:TestSuperCounterClusterRestart.functional_test dtest 
> failing on C* trunk on Windows
> -
>
> Key: CASSANDRA-10893
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10893
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
> Fix For: 3.x
>
>
> The {{super_counter_test.py:TestSuperCounterClusterRestart.functional_test}} 
> dtest fails consistently on trunk on Windows:
> http://cassci.datastax.com/view/win32/job/trunk_dtest_win32/lastCompletedBuild/testReport/super_counter_test/TestSuperCounterClusterRestart/functional_test/history/
> Sometimes the failure results from a {{NotFoundException}}:
> http://cassci.datastax.com/job/trunk_dtest_win32/315/testReport/super_counter_test/TestSuperCounterClusterRestart/functional_test/
> and other times it fails because some of the counters do not have the 
> expected value:
> http://cassci.datastax.com/view/win32/job/trunk_dtest_win32/312/testReport/junit/super_counter_test/TestSuperCounterClusterRestart/functional_test/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10894) sstablesplit_test.py:TestSSTableSplit.split_test failing on Windows

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey resolved CASSANDRA-10894.
--
Resolution: Invalid

Sorry, this ticket is based on old information. This _was_ failing, I don't 
know if it is anymore. Will reopen this ticket if I can get more current 
information.

> sstablesplit_test.py:TestSSTableSplit.split_test failing on Windows
> ---
>
> Key: CASSANDRA-10894
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10894
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
> Fix For: 3.0.x
>
>
> This test fails consistently on Windows on C* trunk:
> http://cassci.datastax.com/job/trunk_dtest_win32/315/testReport/sstablesplit_test/TestSSTableSplit/split_test/history/
> It fails when {{sstablesplit}} leaves behind a single SSTable, rather than 6, 
> as is expected:
> http://cassci.datastax.com/job/trunk_dtest_win32/315/testReport/sstablesplit_test/TestSSTableSplit/split_test/
> I have not seen this on other versions or in other environments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10895) The return of CASSANDRA-8718?

[Pete Ehlke (JIRA)]

Pete Ehlke created CASSANDRA-10895:
--

 Summary: The return of CASSANDRA-8718?
 Key: CASSANDRA-10895
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10895
 Project: Cassandra
  Issue Type: Bug
  Components: Compaction
 Environment: Cassandra 2.1.9 running on:

ubuntu@ip-10-250-1-225:~$ uname -a
Linux ip-10-250-1-225 3.13.0-65-generic #105-Ubuntu SMP Mon Sep 21 18:50:58 UTC 
2015 x86_64 x86_64 x86_64 GNU/Linux
ubuntu@ip-10-250-1-225:~$ java -version
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
ubuntu@ip-10-250-1-225:~$
Reporter: Pete Ehlke
 Attachments: hs_err_pid317.log

Repeatable every time I run 'nodetool repair -inc -par' on this host:

2015-12-17_14:32:30.78574 INFO  14:32:30 Writing 
Memtable-sstable_activity@907762948(95.060KiB serialized bytes, 17729 ops, 
0%/0% of on/off-heap limi
t)
2015-12-17_14:32:30.80120 INFO  14:32:30 Completed flushing 
/data/cassandra/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/system-ssta
ble_activity-tmp-ka-2126-Data.db (75.676KiB) for commitlog position 
ReplayPosition(segmentId=1449246749652, position=3079886)
2015-12-17_14:32:32.67513 #
2015-12-17_14:32:32.67516 # A fatal error has been detected by the Java Runtime 
Environment:
2015-12-17_14:32:32.67517 #
2015-12-17_14:32:32.67518 #  SIGSEGV (0xb) at pc=0x7f477699117a, pid=317, 
tid=139942572054272
2015-12-17_14:32:32.67525 #
2015-12-17_14:32:32.67526 # JRE version: Java(TM) SE Runtime Environment 
(8.0_60-b27) (build 1.8.0_60-b27)
2015-12-17_14:32:32.67527 # Java VM: Java HotSpot(TM) 64-Bit Server VM 
(25.60-b23 mixed mode linux-amd64 compressed oops)
2015-12-17_14:32:32.67528 # Problematic frame:
2015-12-17_14:32:32.67529 # J 21341 C2 
org.apache.cassandra.io.sstable.IndexSummary.binarySearch(Lorg/apache/cassandra/db/RowPosition;)I
 (88 bytes) @ 0x7f477699117a [0x7f4776991020+0x15a]
2015-12-17_14:32:32.67532 #
2015-12-17_14:32:32.67533 # Failed to write core dump. Core dumps have been 
disabled. To enable core dumping, try "ulimit -c unlimited" before starting 
Java again
2015-12-17_14:32:32.67535 #
2015-12-17_14:32:32.67587 # An error report file with more information is saved 
as:
2015-12-17_14:32:32.67587 # /tmp/hs_err_pid317.log
2015-12-17_14:32:32.69831 #
2015-12-17_14:32:32.69832 # If you would like to submit a bug report, please 
visit:
2015-12-17_14:32:32.69832 #   http://bugreport.java.com/bugreport/crash.jsp
2015-12-17_14:32:32.69832 #

This looks exactly like CASSANDRA-8718, but that was marked as fixed in 2.0.13



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10895) The return of CASSANDRA-8718?

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062216#comment-15062216
 ] 

Sylvain Lebresne commented on CASSANDRA-10895:
--

It does seem that we've closed CASSANDRA-8718 without really finding out what 
was going wrong. Assigning [~JoshuaMcKenzie] just because he was assigned 
previously, but feel free to re-assign if you don't have any cycles on this.

> The return of CASSANDRA-8718?
> -
>
> Key: CASSANDRA-10895
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10895
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Cassandra 2.1.9 running on:
> ubuntu@ip-10-250-1-225:~$ uname -a
> Linux ip-10-250-1-225 3.13.0-65-generic #105-Ubuntu SMP Mon Sep 21 18:50:58 
> UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
> ubuntu@ip-10-250-1-225:~$ java -version
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
> ubuntu@ip-10-250-1-225:~$
>Reporter: Pete Ehlke
>Assignee: Joshua McKenzie
> Attachments: hs_err_pid317.log
>
>
> Repeatable every time I run 'nodetool repair -inc -par' on this host:
> 2015-12-17_14:32:30.78574 INFO  14:32:30 Writing 
> Memtable-sstable_activity@907762948(95.060KiB serialized bytes, 17729 ops, 
> 0%/0% of on/off-heap limi
> t)
> 2015-12-17_14:32:30.80120 INFO  14:32:30 Completed flushing 
> /data/cassandra/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/system-ssta
> ble_activity-tmp-ka-2126-Data.db (75.676KiB) for commitlog position 
> ReplayPosition(segmentId=1449246749652, position=3079886)
> 2015-12-17_14:32:32.67513 #
> 2015-12-17_14:32:32.67516 # A fatal error has been detected by the Java 
> Runtime Environment:
> 2015-12-17_14:32:32.67517 #
> 2015-12-17_14:32:32.67518 #  SIGSEGV (0xb) at pc=0x7f477699117a, pid=317, 
> tid=139942572054272
> 2015-12-17_14:32:32.67525 #
> 2015-12-17_14:32:32.67526 # JRE version: Java(TM) SE Runtime Environment 
> (8.0_60-b27) (build 1.8.0_60-b27)
> 2015-12-17_14:32:32.67527 # Java VM: Java HotSpot(TM) 64-Bit Server VM 
> (25.60-b23 mixed mode linux-amd64 compressed oops)
> 2015-12-17_14:32:32.67528 # Problematic frame:
> 2015-12-17_14:32:32.67529 # J 21341 C2 
> org.apache.cassandra.io.sstable.IndexSummary.binarySearch(Lorg/apache/cassandra/db/RowPosition;)I
>  (88 bytes) @ 0x7f477699117a [0x7f4776991020+0x15a]
> 2015-12-17_14:32:32.67532 #
> 2015-12-17_14:32:32.67533 # Failed to write core dump. Core dumps have been 
> disabled. To enable core dumping, try "ulimit -c unlimited" before starting 
> Java again
> 2015-12-17_14:32:32.67535 #
> 2015-12-17_14:32:32.67587 # An error report file with more information is 
> saved as:
> 2015-12-17_14:32:32.67587 # /tmp/hs_err_pid317.log
> 2015-12-17_14:32:32.69831 #
> 2015-12-17_14:32:32.69832 # If you would like to submit a bug report, please 
> visit:
> 2015-12-17_14:32:32.69832 #   http://bugreport.java.com/bugreport/crash.jsp
> 2015-12-17_14:32:32.69832 #
> This looks exactly like CASSANDRA-8718, but that was marked as fixed in 2.0.13



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10896) Fix skipping logic on upgrade tests in dtest

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062268#comment-15062268
 ] 

Jim Witschey commented on CASSANDRA-10896:
--

/cc [~rhatch]

> Fix skipping logic on upgrade tests in dtest
> 
>
> Key: CASSANDRA-10896
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10896
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Jim Witschey
> Fix For: 3.0.x
>
>
> This will be a general ticket for upgrade dtests that fail because of bad 
> logic surrounding skipping tests. We need a better system in place for 
> skipping tests that are not intended to work on certain versions of 
> Cassandra; at present, we run the upgrade tests with {{SKIP=false}} because, 
> again, the built-in skipping logic is bad.
> One such test is test_v2_protocol_IN_with_tuples:
> http://cassci.datastax.com/job/storage_engine_upgrade_dtest-22_tarball-311/lastCompletedBuild/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3/test_v2_protocol_IN_with_tuples/
> This shouldn't be run on clusters that include nodes running 3.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6696) Partition sstables by token range

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062333#comment-15062333
 ] 

Carl Yeksigian commented on CASSANDRA-6696:
---

Forgot to add myself as a watcher, so I didn't see the comments.

{quote}
Idea is that we have one thread per disk writing, but I guess the thread count 
should be DatabaseDescriptor.getFlushWriters() per disk and the flushExecutor 
thread count should be 1 - we want to quickly hand off to the single 
flushExecutor when flushing and then run the per disk writing in the 
perDiskFlushExecutor. Do you have any other suggestion on how to model this?
{quote}
I think the change to have {{flushWriters}} per disk makes sense, but we should 
set the default to 1 instead of # of disks; we should also update the comment 
in the cassandra.yaml.

{quote}
We wont split on incoming streams based on token range in CASSANDRA-10540 - 
remote node will most likely already have sstables split based on its local 
ranges and those should match any ranges we own, so we can simply write it to 
disk, then the new sstable will get added to the correct compaction strategy 
(if it fits, otherwise it does a round in "L0")
{quote}
Makes sense to me.

{quote}
What do you mean? Splitter for a partition?
{quote}
Reading that took me awhile to figure out what I was trying to say; it was 
about sstableofflinerelevel. Looking over it again, I see that it is handling 
the different ranges correctly, so we can ignore that.

Other than the slight changes around flush writers, the rest of it looks good 
to me.

> Partition sstables by token range
> -
>
> Key: CASSANDRA-6696
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6696
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Marcus Eriksson
>  Labels: compaction, correctness, dense-storage, 
> jbod-aware-compaction, performance
> Fix For: 3.2
>
>
> In JBOD, when someone gets a bad drive, the bad drive is replaced with a new 
> empty one and repair is run. 
> This can cause deleted data to come back in some cases. Also this is true for 
> corrupt stables in which we delete the corrupt stable and run repair. 
> Here is an example:
> Say we have 3 nodes A,B and C and RF=3 and GC grace=10days. 
> row=sankalp col=sankalp is written 20 days back and successfully went to all 
> three nodes. 
> Then a delete/tombstone was written successfully for the same row column 15 
> days back. 
> Since this tombstone is more than gc grace, it got compacted in Nodes A and B 
> since it got compacted with the actual data. So there is no trace of this row 
> column in node A and B.
> Now in node C, say the original data is in drive1 and tombstone is in drive2. 
> Compaction has not yet reclaimed the data and tombstone.  
> Drive2 becomes corrupt and was replaced with new empty drive. 
> Due to the replacement, the tombstone in now gone and row=sankalp col=sankalp 
> has come back to life. 
> Now after replacing the drive we run repair. This data will be propagated to 
> all nodes. 
> Note: This is still a problem even if we run repair every gc grace. 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10829) cleanup + repair generates a lot of logs

[Fabien Rousseau (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062366#comment-15062366
 ] 

Fabien Rousseau commented on CASSANDRA-10829:
-

If I undertstand well:
 - log rate limit is now once every 100ms
 - rate limit is already on 2.2+ (not related to this ticket)
 - compacting sstables are now released sooner (as soon as it has been 
compacted, and no more at the end of the whole operation)

So, thanks a lot for the fix.


> cleanup + repair generates a lot of logs
> 
>
> Key: CASSANDRA-10829
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10829
> Project: Cassandra
>  Issue Type: Bug
> Environment: 5 nodes on Cassandra 2.1.11 (on Debian)
>Reporter: Fabien Rousseau
>Assignee: Marcus Eriksson
>
> One of our node generates a lot of cassandra logs (int the 10 MB/s) and CPU 
> usage has increased (by a factor 2-3).
> This was most probably triggered by a "nodetool snapshot" while a cleanup was 
> already running on this node.
> An example of those logs:
> 2015-12-08 09:15:17,794 INFO  
> [ValidationExecutor:689]ColumnFamilyStore.java:1923 Spinning trying to 
> capture released readers [...]
> 2015-12-08 09:15:17,794 INFO  
> [ValidationExecutor:689]ColumnFamilyStore.java:1924 Spinning trying to 
> capture all readers [...]
> 2015-12-08 09:15:17,795 INFO  
> [ValidationExecutor:689]ColumnFamilyStore.java:1923 Spinning trying to 
> capture released readers [...]
> 2015-12-08 09:15:17,795 INFO  
> [ValidationExecutor:689]ColumnFamilyStore.java:1924 Spinning trying to 
> capture all readers [...]
> (I removed SSTableReader information because it's rather long... I can share 
> it privately if needed)
> Note that the date has not been changed (only 1ms between logs)
> It should not generate that gigantic amount of logs :)
> This is probably linked to: 
> https://issues.apache.org/jira/browse/CASSANDRA-9637



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10795) Improve Failure Detector Unknown EP message

[Anthony Cozzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Cozzie updated CASSANDRA-10795:
---
Attachment: trunk-10795-2.txt

> Improve Failure Detector Unknown EP message
> ---
>
> Key: CASSANDRA-10795
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10795
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
>Priority: Minor
> Fix For: 3.2
>
> Attachments: trunk-10795-2.txt, trunk-10795.txt
>
>
> When the failure detector is asked whether an unknown endpoint is alive, it 
> prints an uninformative error message.  This patch adds a stack trace to the 
> print statement.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10726) Read repair inserts should not be blocking

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062414#comment-15062414
 ] 

Aleksey Yeschenko commented on CASSANDRA-10726:
---

Need to think about it more, but I wouldn't mind, in principle, dropping the 
guarantee. Arguably the behaviour seen by Richard is more confusing/painful 
than CASSANDRA-2494 issue. Should at the very least make an option to disable 
this.

> Read repair inserts should not be blocking
> --
>
> Key: CASSANDRA-10726
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10726
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination
>Reporter: Richard Low
>
> Today, if there’s a digest mismatch in a foreground read repair, the insert 
> to update out of date replicas is blocking. This means, if it fails, the read 
> fails with a timeout. If a node is dropping writes (maybe it is overloaded or 
> the mutation stage is backed up for some other reason), all reads to a 
> replica set could fail. Further, replicas dropping writes get more out of 
> sync so will require more read repair.
> The comment on the code for why the writes are blocking is:
> {code}
> // wait for the repair writes to be acknowledged, to minimize impact on any 
> replica that's
> // behind on writes in case the out-of-sync row is read multiple times in 
> quick succession
> {code}
> but the bad side effect is that reads timeout. Either the writes should not 
> be blocking or we should return success for the read even if the write times 
> out.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10795) Improve Failure Detector Unknown EP message

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062406#comment-15062406
 ] 

Carl Yeksigian commented on CASSANDRA-10795:


+1

> Improve Failure Detector Unknown EP message
> ---
>
> Key: CASSANDRA-10795
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10795
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
>Priority: Minor
> Fix For: 3.2
>
> Attachments: trunk-10795-2.txt, trunk-10795.txt
>
>
> When the failure detector is asked whether an unknown endpoint is alive, it 
> prints an uninformative error message.  This patch adds a stack trace to the 
> print statement.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10795) Improve Failure Detector Unknown EP message

[Anthony Cozzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062390#comment-15062390
 ] 

Anthony Cozzie commented on CASSANDRA-10795:


[~carlyeks], I removed the Objects.toString() call and attached a new patch.

> Improve Failure Detector Unknown EP message
> ---
>
> Key: CASSANDRA-10795
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10795
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
>Priority: Minor
> Fix For: 3.2
>
> Attachments: trunk-10795-2.txt, trunk-10795.txt
>
>
> When the failure detector is asked whether an unknown endpoint is alive, it 
> prints an uninformative error message.  This patch adds a stack trace to the 
> print statement.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10850) v4 spec has tons of grammatical mistakes

[Sandeep Tamhankar (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062395#comment-15062395
 ] 

Sandeep Tamhankar commented on CASSANDRA-10850:
---

About my outstanding questions: will the reviewer update the doc to answer 
those, or answer here (and I'll update the doc accordingly and submit a new 
patch)?

> v4 spec has tons of grammatical mistakes
> 
>
> Key: CASSANDRA-10850
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10850
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation and Website
>Reporter: Sandeep Tamhankar
> Fix For: 3.0.2
>
> Attachments: v4-protocol.patch
>
>
> https://github.com/apache/cassandra/blob/cassandra-3.0/doc/native_protocol_v4.spec
> I notice the following in the first section of the spec and then gave up:
> "The list of allowed opcode is defined Section 2.3" => "The list of allowed 
> opcode*s* is defined in Section 2.3"
> "the details of each corresponding message is described Section 4" => "the 
> details of each corresponding message are described in Section 4" since the 
> subject is details, not message.
> "Requests are those frame sent by" => "Requests are those frame*s* sent by"
> I think someone should go through the whole spec and fix all the mistakes 
> rather than me pointing out the ones I notice piece-meal. I found the grammar 
> errors to be rather distracting.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10888) Tombstone error warning does not log partition key

[Sebastian Estevez (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062778#comment-15062778
 ] 

Sebastian Estevez commented on CASSANDRA-10888:
---

We probably should have added this in CASSANDRA-8561, it makes sense to have 
the ERROR log the same details as the WARNING. 

> Tombstone error warning does not log partition key
> --
>
> Key: CASSANDRA-10888
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10888
> Project: Cassandra
>  Issue Type: Improvement
> Environment: DSE 4.8.2
>Reporter: Ole Pedersen
>
> Log partition key if read fails due to the ERROR threshold on read tombstoned 
> cells.
> Right now I can specify a warning and an error threshold for C* when reading 
> from a partition with many tombstones.
> If the query reads more than the “warning threshold” then C* writes a warning 
> to the log with the partition key.
> But if a query reads more than the “error threshold” then C* aborts the query 
> and writes to the log – but not the partition key, this time. 
> What I am missing is: Could C* also please write the partition key in case of 
> query abort due to tombstones?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10822) SSTable data loss when upgrading with row tombstone present

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-10822:
---
Fix Version/s: 3.1.1

> SSTable data loss when upgrading with row tombstone present
> ---
>
> Key: CASSANDRA-10822
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10822
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Branimir Lambov
>Priority: Critical
> Fix For: 3.0.2, 3.2, 3.1.1
>
>
> I ran into an issue when upgrading between 2.1.11 to 3.0.0 (and also 
> cassandra-3.0 branch) where subsequent rows were lost within a partition 
> where there is a row tombstone present.
> Here's a scenario that reproduces the issue.
> Using ccm create a single node cluster at 2.1.11:
> {{ccm create -n 1 -v 2.1.11 -s financial}}
> Run the following queries to create schema, populate some data and then 
> delete some data for november:
> {noformat}
> drop keyspace if exists financial;
> create keyspace if not exists financial with replication = {'class': 
> 'SimpleStrategy', 'replication_factor' : 1 };
> create table if not exists financial.symbol_history (
>   symbol text,
>   name text static,
>   year int,
>   month int,
>   day int,
>   volume bigint,
>   close double,
>   open double,
>   low double,
>   high double,
>   primary key((symbol, year), month, day)
> ) with CLUSTERING ORDER BY (month desc, day desc);
> insert into financial.symbol_history (symbol, name, year, month, day, volume) 
> values ('CORP', 'MegaCorp', 2004, 1, 1, 100);
> insert into financial.symbol_history (symbol, name, year, month, day, volume) 
> values ('CORP', 'MegaCorp', 2004, 2, 1, 100);
> insert into financial.symbol_history (symbol, name, year, month, day, volume) 
> values ('CORP', 'MegaCorp', 2004, 3, 1, 100);
> insert into financial.symbol_history (symbol, name, year, month, day, volume) 
> values ('CORP', 'MegaCorp', 2004, 4, 1, 100);
> insert into financial.symbol_history (symbol, name, year, month, day, volume) 
> values ('CORP', 'MegaCorp', 2004, 5, 1, 100);
> insert into financial.symbol_history (symbol, name, year, month, day, volume) 
> values ('CORP', 'MegaCorp', 2004, 6, 1, 100);
> insert into financial.symbol_history (symbol, name, year, month, day, volume) 
> values ('CORP', 'MegaCorp', 2004, 7, 1, 100);
> insert into financial.symbol_history (symbol, name, year, month, day, volume) 
> values ('CORP', 'MegaCorp', 2004, 8, 1, 100);
> insert into financial.symbol_history (symbol, name, year, month, day, volume) 
> values ('CORP', 'MegaCorp', 2004, 9, 1, 100);
> insert into financial.symbol_history (symbol, name, year, month, day, volume) 
> values ('CORP', 'MegaCorp', 2004, 10, 1, 100);
> insert into financial.symbol_history (symbol, name, year, month, day, volume) 
> values ('CORP', 'MegaCorp', 2004, 11, 1, 100);
> insert into financial.symbol_history (symbol, name, year, month, day, volume) 
> values ('CORP', 'MegaCorp', 2004, 12, 1, 100);
> delete from financial.symbol_history where symbol='CORP' and year = 2004 and 
> month=11;
> {noformat}
> Flush and run sstable2json on the sole Data.db file:
> {noformat}
> ccm node1 flush
> sstable2json /path/to/file.db
> {noformat}
> The output should look like the following:
> {code}
> [
> {"key": "CORP:2004",
>  "cells": [["::name","MegaCorp",1449457517033030],
>["12:1:","",1449457517033030],
>["12:1:volume","100",1449457517033030],
>["11:_","11:!",1449457564983269,"t",1449457564],
>["10:1:","",1449457516313738],
>["10:1:volume","100",1449457516313738],
>["9:1:","",1449457516310205],
>["9:1:volume","100",1449457516310205],
>["8:1:","",1449457516235664],
>["8:1:volume","100",1449457516235664],
>["7:1:","",1449457516233535],
>["7:1:volume","100",1449457516233535],
>["6:1:","",1449457516231458],
>["6:1:volume","100",1449457516231458],
>["5:1:","",1449457516228307],
>["5:1:volume","100",1449457516228307],
>["4:1:","",1449457516225415],
>["4:1:volume","100",1449457516225415],
>["3:1:","",1449457516222811],
>["3:1:volume","100",1449457516222811],
>["2:1:","",1449457516220301],
>["2:1:volume","100",1449457516220301],
>["1:1:","",1449457516210758],
>["1:1:volume","100",1449457516210758]]}
> ]
> {code}
> Prepare for upgrade
> {noformat}
> ccm node1 nodetool snapshot financial
> ccm node1 nodetool drain
> ccm node1 stop
> {noformat}
> Upgrade to cassandra-3.0 and start the node
> {noformat}
> ccm node1 setdir -v git:cassandra-3.0
> ccm node1 start
> {noformat}
> Run command in cqlsh and observe only 1 row is returned!  It 

[jira] [Created] (CASSANDRA-10898) Migrate Compaction Strategy Node by Node

[Andrew From (JIRA)]

Andrew From created CASSANDRA-10898:
---

 Summary: Migrate Compaction Strategy Node by Node
 Key: CASSANDRA-10898
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10898
 Project: Cassandra
  Issue Type: Wish
  Components: Compaction, Tools
Reporter: Andrew From


It would be a great feature to be able to slowly change the compaction strategy 
of a ColumnFamily node-by-node instead of cluster wide. Currently if you change 
it cluster wide there's no good way to predict how long it will take. Thus the 
process could run for days while you still need the live data, but the cluster 
responds much more slowly due to the compaction strategy migration.

I stumbled across 
http://blog.alteroot.org/articles/2015-04-20/change-cassandra-compaction-strategy-on-production-cluster.html
 which gave me the idea. I was thinking this would be a nice feature to add to 
NodeTool, provided that the strategy in the blog is sound I wouldn't mind going 
ahead with the dev work to automate it. If not I would love to hear other ideas 
on how to best make this happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

svn commit: r1720682 - /cassandra/site/publish/doc/cql3/CQL.html

[tylerhobbs]

Author: tylerhobbs
Date: Thu Dec 17 22:18:28 2015
New Revision: 1720682

URL: http://svn.apache.org/viewvc?rev=1720682=rev
Log:
Point CQL.html at the latest cql docs

Added:
cassandra/site/publish/doc/cql3/CQL.html   (with props)

Added: cassandra/site/publish/doc/cql3/CQL.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/doc/cql3/CQL.html?rev=1720682=auto
==
--- cassandra/site/publish/doc/cql3/CQL.html (added)
+++ cassandra/site/publish/doc/cql3/CQL.html Thu Dec 17 22:18:28 2015
@@ -0,0 +1 @@
+link CQL-2.2.html
\ No newline at end of file

Propchange: cassandra/site/publish/doc/cql3/CQL.html
--
svn:special = *

[jira] [Created] (CASSANDRA-10899) CQL-3.0.html spec is not published yet

[Wei Deng (JIRA)]

Wei Deng created CASSANDRA-10899:


 Summary: CQL-3.0.html spec is not published yet
 Key: CASSANDRA-10899
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10899
 Project: Cassandra
  Issue Type: Improvement
Reporter: Wei Deng
Assignee: Tyler Hobbs
Priority: Minor


We have https://cassandra.apache.org/doc/cql3/CQL-2.2.html but CQL-3.0.html 
doesn't exist yet and needs to be published, since Cassandra 3.0 is now 
officially GA.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10901) Per client TCP/IP connection quota

[Wei Deng (JIRA)]

Wei Deng created CASSANDRA-10901:


 Summary: Per client TCP/IP connection quota
 Key: CASSANDRA-10901
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10901
 Project: Cassandra
  Issue Type: New Feature
  Components: Coordination
Reporter: Wei Deng


Now that we started working on CASSANDRA-9318, could we also consider pushing 
it one step further to accommodate some per user/session quota? This will be 
useful in a multi-tenant Cassandra deployment.

My intention is to keep it simple, i.e. only enforcing quota at the node level, 
with protocol v3+ most likely you only need one TCP/IP connection (to 
accommodate 32K stream ids) so just enforcing some sort of in-flight request 
limit based on each TCP/IP connection looks good enough for me; if any client 
would ever need to add another TCP/IP connection (as they run out of the 32k 
stream ids), there should be no limit/quota existing on them already before 
that's allowed (the assumption here is that each client is connected to the 
cluster using only one Cassandra user).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9428) Implement hints compression

[Blake Eggleston (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062888#comment-15062888
 ] 

Blake Eggleston commented on CASSANDRA-9428:


Here's my initial implementation: 
https://github.com/bdeggleston/cassandra/commits/9428

A few things to note:
* The behavior of HintsWriter.Session.append has been altered slightly. When 
the size of a hint is larger than the remaining buffer, the buffer is first 
flushed to disk, then the hint is written into the newly cleared buffer. 
Previously, both the hint and the previous data were flushed to disk. I did 
this to avoid writing very small compressed blocks to disk or having to 
concatenate the two byte buffers.
* When opening readers, an uncompressed reader is always instantiated first, 
then upgraded to a compressed reader if the file is compressed. This is because 
the hints descriptor containing the compressor metadata is written to the head 
of the file, and it needs to be written uncompressed.

> Implement hints compression
> ---
>
> Key: CASSANDRA-9428
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9428
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Blake Eggleston
> Fix For: 3.0.x
>
>
> CASSANDRA-6230 is being implemented with compression in mind, but it's not 
> going to be implemented by the original ticket.
> Adding it on top should be relatively straight-forward, and important, since 
> there are several users in the wild that use compression interface for 
> encryption purposes. DSE is one of them (but isn't the only one). Losing 
> encryption capabilities would be a regression.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10900) CQL3 spec should clarify that now() function is calculated on the coordinator node

[Wei Deng (JIRA)]

Wei Deng created CASSANDRA-10900:


 Summary: CQL3 spec should clarify that now() function is 
calculated on the coordinator node
 Key: CASSANDRA-10900
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10900
 Project: Cassandra
  Issue Type: Improvement
  Components: Documentation and Website
Reporter: Wei Deng
Assignee: Wei Deng
Priority: Trivial


DataStax CQL v3.3 document on now() function here 
(http://docs.datastax.com/en/cql/3.3/cql/cql_reference/timeuuid_functions_r.html)
 states that it is generated on the coordinator, but this is not reflected on 
the apache cassandra CQL3 spec. Since it is a minor addition, there's no 
urgency to add it and the commit can stay in trunk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10784) ZonedDateTime as new CQL data type

[ZhaoYang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063564#comment-15063564
 ] 

ZhaoYang commented on CASSANDRA-10784:
--

[~snazy] Hi Robert, thanks for the reply.  I understand it's very bad to let 
Cassandra synchronized time zone info and these country-related regulations may 
change over time. 

Converting ZonedDateTime data into 2 column( 1 utc column and 1 zone column) 
can be done by client side logic. but It will be more convenient if cassandra 
can get the ZonedDateTime inserted by users and sorted on utc value. Cassandra 
doesn't need to know about reference data. only store utc and zone internally. 

BTW, what's the reason to implement a datastax.LocalDate instead of using 
java.LocalDate. Both use ISO-8601. Is it similarly because of changes between 
java releases.

> ZonedDateTime as new CQL data type
> --
>
> Key: CASSANDRA-10784
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10784
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL, Documentation and Website
>Reporter: ZhaoYang
>Assignee: ZhaoYang
> Fix For: 3.x
>
>
> For globalization, ZonedDateTime is very important. Timestamp is not enough 
> and Zone info should be included. So Cassandra should support ZonedDateTime 
> as native data type and ordering using ZonedDateTime's utc value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10902) Skip saved cache directory when checking SSTables at startup

[Carl Yeksigian (JIRA)]

Carl Yeksigian created CASSANDRA-10902:
--

 Summary: Skip saved cache directory when checking SSTables at 
startup
 Key: CASSANDRA-10902
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10902
 Project: Cassandra
  Issue Type: Bug
  Components: Configuration
Reporter: Carl Yeksigian
Assignee: Carl Yeksigian


The SSTable StartupCheck looks for all files which end with "*.db" and compares 
the version. This causes problems if {{saved_cache_directory}} is a 
subdirectory of a {{data_file_directories}}. We should make sure that we are 
not checking any subdirectory where we might be writing *.db files.

This is the cause of not being able to restart in CASSANDRA-10821.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10903) AssertionError while reading sstable

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-10903:

Description: 
I'm able to consistently reproduce the following error:

{noformat}
1  WARN [SharedPool-Worker-1] 2015-12-17 17:37:00,196  
AbstractTracingAwareExecutorService.java (line 169) Uncaught exception on 
thread Thread[SharedPool-Worker-1,5,main]: {}
java.lang.AssertionError: null
at 
org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.updateBlock(AbstractSSTableIterator.java:463)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.columniterator.SSTableIterator$ForwardIndexedReader.computeNext(SSTableIterator.java:268)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:158)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:352)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:219)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:32)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:369)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:189)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:158)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:426)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:286)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:108) 
~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:298)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:136)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:128)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:65) 
~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:288) 
~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1692)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2346)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_45]
at 
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
[cassandra-all-3.0.0.740.jar:3.0.0.740]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
{noformat}

This occurs while executing a query like:

{noformat}
SELECT static_column FROM table WHERE partition_key = ?
{noformat}

on a table with one partition key and multiple clustering keys.  I'm currently 
trying to create a more isolated reproduction.

  was:
I'm able to consistently reproduce the following error:

{noformat}
1  WARN [SharedPool-Worker-1] 2015-12-17 

[jira] [Commented] (CASSANDRA-10866) Column Family should expose count metrics for dropped mutations.

[Anubhav Kale (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063020#comment-15063020
 ] 

Anubhav Kale commented on CASSANDRA-10866:
--

I have provided a patch on top of 
https://issues.apache.org/jira/secure/attachment/12777927/Trunk-All-Comments.patch

I can add Unit Tests in messaging service but I did not see a pattern for 
checking metrics thorough unit tests so skipping those. The change is verified 
on a local installation through visual VM. 

It is possible to optimize the KS/CF lookup in MessagingService.java (maybe 
through a Map), but I am hoping that's not necessary.

> Column Family should expose count metrics for dropped mutations.
> 
>
> Key: CASSANDRA-10866
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10866
> Project: Cassandra
>  Issue Type: Improvement
> Environment: PROD
>Reporter: Anubhav Kale
>Priority: Minor
> Attachments: 0001-CFCount.patch
>
>
> Please take a look at the discussion in CASSANDRA-10580. This is opened so 
> that the latency on dropped mutations is exposed as a metric on column 
> families.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10821) OOM Killer terminates Cassandra when Compactions use too much memory then won't restart

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063055#comment-15063055
 ] 

Carl Yeksigian commented on CASSANDRA-10821:


The restart issue you are having is caused by thinking that a cache is an 
SSTable.This is probably caused by {{saved_cache_directory}} being a 
subdirectory of one of the {{data_file_directories}}. I've created 
CASSANDRA-10902 to track that issue.

That doesn't explain the OOM, though.

> OOM Killer terminates Cassandra when Compactions use too much memory then 
> won't restart
> ---
>
> Key: CASSANDRA-10821
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10821
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: EC2 32 x i2.xlarge split between us-east-1a,c and 
> us-west 2a,b
> Linux  4.1.10-17.31.amzn1.x86_64 #1 SMP Sat Oct 24 01:31:37 UTC 2015 x86_64 
> x86_64 x86_64 GNU/Linux
> Java(TM) SE Runtime Environment (build 1.8.0_65-b17)
> Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode)
> Cassandra version: 2.2.3
>Reporter: Thom Bartold
>
> We were writing to the DB from EC2 instances in us-east-1 at a rate of about 
> 3000 per second, replication us-east:2 us-west:2, LeveledCompaction and 
> DeflateCompressor.
> After about 48 hours some nodes had over 800 pending compactions and a few of 
> them started getting killed for Linux OOM. Priam attempts to restart the 
> nodes, but they fail because of corrupted saved_cahce files.
> Loading has finished, and the cluster is mostly idle, but 6 of the nodes were 
> killed again last night by OOM.
> This is the log message where the node won't restart:
> ERROR [main] 2015-12-05 13:59:13,754 CassandraDaemon.java:635 - Detected 
> unreadable sstables /media/ephemeral0/cassandra/saved_caches/KeyCache-ca.db, 
> please check NEWS.txt and ensure that you have upgraded through all required 
> intermediate versions, running upgradesstables
> This is the dmesg where the node is terminated:
> [360803.234422] Out of memory: Kill process 10809 (java) score 949 or 
> sacrifice child
> [360803.237544] Killed process 10809 (java) total-vm:438484092kB, 
> anon-rss:29228012kB, file-rss:107576kB
> This is what Compaction Stats look like currently:
> pending tasks: 1096
>  id   compaction type  keyspace   
>tablecompleted  totalunit   progress
>93eb3200-9b58-11e5-b9f1-ffef1041ec45Compaction   overlordpreprod   
> document   8670748796   839129219651   bytes  1.03%
>Compactionsystem   
>hints   30 1921326518   bytes  0.00%
> Active compaction remaining time :  27h33m47s
> Only 6 of the 32 nodes have compactions pending, and all on the order of 1000.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10797) Bootstrap new node fails with OOM when streaming nodes contains thousands of sstables

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063143#comment-15063143
 ] 

Paulo Motta commented on CASSANDRA-10797:
-

Fix on 3.0 was much smoother and straightforward with the new transactional 
system. The only change was to finish {{SSTableWriters}} as soon as they're 
received and update the transactions with the corresponding {{SSTableReaders}}. 
Thanks for the tip!

I also added tests to verify transactions with mixed {{SStableWriters}} and 
{{SStableReaders}} are cleaned up correctly if transaction is aborted. Since 
there is no longer rename when the sstable transitions from {{SStableWriter}} 
to {{SStableReader}}, uncommitted/unaborted transactions due to node failures 
in the middle of a transaction will have its sstables cleaned up as usual on 
next startup by {{LifecycleTransaction.removeUnfinishedLeftovers(metadata)}} on 
{{ColumnFamilyStore.scrubDataDirectories()}} (this is thoroughly tested on 
{{LogTransactionTest}}).

Below are branch and tests (not yet finished):

||3.0||trunk||
|[branch|https://github.com/apache/cassandra/compare/cassandra-3.0...pauloricardomg:3.0-10797]|[branch|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-10797]|
|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-10797-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-10797-testall/lastCompletedBuild/testReport/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-10797-dtest/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-10797-dtest/lastCompletedBuild/testReport/]|

> Bootstrap new node fails with OOM when streaming nodes contains thousands of 
> sstables
> -
>
> Key: CASSANDRA-10797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10797
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: Cassandra 2.1.8.621 w/G1GC
>Reporter: Jose Martinez Poblete
>Assignee: Paulo Motta
> Fix For: 3.0.x, 3.x
>
> Attachments: 10797-nonpatched.png, 10797-patched.png, 
> 10798-nonpatched-500M.png, 10798-patched-500M.png, 112415_system.log, 
> Heapdump_OOM.zip, Screen Shot 2015-12-01 at 7.34.40 PM.png, dtest.tar.gz
>
>
> When adding a new node to an existing DC, it runs OOM after 25-45 minutes
> Upon heapdump revision, it is found the sending nodes are streaming thousands 
> of sstables which in turns blows the bootstrapping node heap 
> {noformat}
> ERROR [RMI Scheduler(0)] 2015-11-24 10:10:44,585 
> JVMStabilityInspector.java:94 - JVM state determined to be unstable.  Exiting 
> forcefully due to:
> java.lang.OutOfMemoryError: Java heap space
> ERROR [STREAM-IN-/173.36.28.148] 2015-11-24 10:10:44,585 
> StreamSession.java:502 - [Stream #0bb13f50-92cb-11e5-bc8d-f53b7528ffb4] 
> Streaming error occurred
> java.lang.IllegalStateException: Shutdown in progress
> at 
> java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82) 
> ~[na:1.8.0_65]
> at java.lang.Runtime.removeShutdownHook(Runtime.java:239) 
> ~[na:1.8.0_65]
> at 
> org.apache.cassandra.service.StorageService.removeShutdownHook(StorageService.java:747)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.utils.JVMStabilityInspector$Killer.killCurrentJVM(JVMStabilityInspector.java:95)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:64)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:66)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:38)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:250)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
> ERROR [RMI TCP Connection(idle)] 2015-11-24 10:10:44,585 
> JVMStabilityInspector.java:94 - JVM state determined to be unstable.  Exiting 
> forcefully due to:
> java.lang.OutOfMemoryError: Java heap space
> ERROR [OptionalTasks:1] 2015-11-24 10:10:44,585 CassandraDaemon.java:223 - 
> Exception in thread Thread[OptionalTasks:1,5,main]
> 

[jira] [Updated] (CASSANDRA-10866) Column Family should expose count metrics for dropped mutations.

[Anubhav Kale (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anubhav Kale updated CASSANDRA-10866:
-
Attachment: 0001-CFCount.patch

> Column Family should expose count metrics for dropped mutations.
> 
>
> Key: CASSANDRA-10866
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10866
> Project: Cassandra
>  Issue Type: Improvement
> Environment: PROD
>Reporter: Anubhav Kale
>Priority: Minor
> Attachments: 0001-CFCount.patch
>
>
> Please take a look at the discussion in CASSANDRA-10580. This is opened so 
> that the latency on dropped mutations is exposed as a metric on column 
> families.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10903) AssertionError while reading sstable

[Tyler Hobbs (JIRA)]

Tyler Hobbs created CASSANDRA-10903:
---

 Summary: AssertionError while reading sstable
 Key: CASSANDRA-10903
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10903
 Project: Cassandra
  Issue Type: Bug
  Components: Local Write-Read Paths
Reporter: Tyler Hobbs


I'm able to consistently reproduce the following error:

{noformat}
1  WARN [SharedPool-Worker-1] 2015-12-17 17:37:00,196  
AbstractTracingAwareExecutorService.java (line 169) Uncaught exception on 
thread Thread[SharedPool-Worker-1,5,main]: {}
java.lang.AssertionError: null
at 
org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.updateBlock(AbstractSSTableIterator.java:463)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.columniterator.SSTableIterator$ForwardIndexedReader.computeNext(SSTableIterator.java:268)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:158)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:352)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:219)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:32)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:369)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:189)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:158)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:426)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:286)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:108) 
~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:298)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:136)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:128)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:65) 
~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:288) 
~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1692)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at 
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2346)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_45]
at 
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
 ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
[cassandra-all-3.0.0.740.jar:3.0.0.740]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
{noformat}

This occurs while executing a query like:

{noformat}
SELECT regular_column FROM table WHERE partition_key = ?
{noformat}

on a table with one partition key and multiple clustering keys.  I'm currently 
trying 

[jira] [Updated] (CASSANDRA-10903) AssertionError while reading sstable when querying static column

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-10903:

Summary: AssertionError while reading sstable when querying static column  
(was: AssertionError while reading sstable)

> AssertionError while reading sstable when querying static column
> 
>
> Key: CASSANDRA-10903
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10903
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Tyler Hobbs
>
> I'm able to consistently reproduce the following error:
> {noformat}
> 1  WARN [SharedPool-Worker-1] 2015-12-17 17:37:00,196  
> AbstractTracingAwareExecutorService.java (line 169) Uncaught exception on 
> thread Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.AssertionError: null
> at 
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.updateBlock(AbstractSSTableIterator.java:463)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardIndexedReader.computeNext(SSTableIterator.java:268)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:158)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:352)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:219)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:32)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:369)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:189)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:158)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:426)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:286)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:108) 
> ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:298)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:136)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:128)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:65) 
> ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:288) 
> ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1692)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2346)
>  ~[cassandra-all-3.0.0.740.jar:3.0.0.740]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_45]
> at 
> 

[jira] [Commented] (CASSANDRA-10887) Pending range calculator gives wrong pending ranges for moves

[Brandon Williams (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063120#comment-15063120
 ] 

Brandon Williams commented on CASSANDRA-10887:
--

Ping [~slebresne]

> Pending range calculator gives wrong pending ranges for moves
> -
>
> Key: CASSANDRA-10887
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10887
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Richard Low
>Priority: Critical
>
> My understanding is the PendingRangeCalculator is meant to calculate who 
> should receive extra writes during range movements. However, it adds the 
> wrong ranges for moves. An extreme example of this can be seen in the 
> following reproduction. Create a 5 node cluster (I did this on 2.0.16 and 
> 2.2.4) and a keyspace RF=3 and a simple table. Then start moving a node and 
> immediately kill -9 it. Now you see a node as down and moving in the ring. 
> Try a quorum write for a partition that is stored on that node - it will fail 
> with a timeout. Further, all CAS reads or writes fail immediately with 
> unavailable exception because they attempt to include the moving node twice. 
> This is likely to be the cause of CASSANDRA-10423.
> In my example I had this ring:
> 127.0.0.1  rack1   Up Normal  170.97 KB   20.00%  
> -9223372036854775808
> 127.0.0.2  rack1   Up Normal  124.06 KB   20.00%  
> -5534023222112865485
> 127.0.0.3  rack1   Down   Moving  108.7 KB40.00%  
> 1844674407370955160
> 127.0.0.4  rack1   Up Normal  142.58 KB   0.00%   
> 1844674407370955161
> 127.0.0.5  rack1   Up Normal  118.64 KB   20.00%  
> 5534023222112865484
> Node 3 was moving to -1844674407370955160. I added logging to print the 
> pending and natural endpoints. For ranges owned by node 3, node 3 appeared in 
> pending and natural endpoints. The blockFor is increased to 3 so we’re 
> effectively doing CL.ALL operations. This manifests as write timeouts and CAS 
> unavailables when the node is down.
> The correct pending range for this scenario is node 1 is gaining the range 
> (-1844674407370955160, 1844674407370955160). So node 1 should be added as a 
> destination for writes and CAS for this range, not node 3.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10873) Allow sstableloader to work with 3rd party authentication providers

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063149#comment-15063149
 ] 

Yuki Morishita commented on CASSANDRA-10873:


Thanks for the patch!
Besides [this 
change|https://github.com/mike-tr-adamson/cassandra/commit/2d4364be2c94b402721a738b7eecedf955f13f04#commitcomment-15055697],
 patch looks good to me.
Can you update your branch?

> Allow sstableloader to work with 3rd party authentication providers
> ---
>
> Key: CASSANDRA-10873
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10873
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.x
>
>
> When sstableloader was changed to use native protocol instead of thrift there 
> was a regression in that now sstableloader (BulkLoader) only takes 
> {{username}} and {{password}} as credentials so only works with the 
> {{PlainTextAuthProvider}} provided by the java driver.
> Previously it allowed 3rd party auth providers to be used, we need to add 
> back that ability by allowing the full classname of the auth provider to be 
> passed as a parameter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10890) I am using centos 6. I upgraded from cassandra 2.0.8 to 3.0.1 when is run cqlsh, it shows 'cql shell requires python 2.7' . i installed python 2.7 but still my cql s

[Naveen Achyuta (JIRA)]

Naveen Achyuta created CASSANDRA-10890:
--

 Summary: I am using centos 6. I upgraded from cassandra 2.0.8 to 
3.0.1 when is run cqlsh, it shows 'cql shell requires python 2.7' . i installed 
python 2.7 but still my cql shell is not working. 
 Key: CASSANDRA-10890
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10890
 Project: Cassandra
  Issue Type: Test
  Components: CQL
 Environment: centOS 6 
Reporter: Naveen Achyuta
 Fix For: 3.0.1






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Close Clusters and Sessions in Hadoop Input/Output classes

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 37986e825 -> 50f7e0204


Close Clusters and Sessions in Hadoop Input/Output classes

patch by Alex Liu; reviewed by Benjamin Lerer for CASSANDRA-10837


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2da3c9db
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2da3c9db
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2da3c9db

Branch: refs/heads/trunk
Commit: 2da3c9db154449e15d5a2c2072db77b65c9e931a
Parents: ed96322
Author: Alex Liu 
Authored: Thu Dec 17 12:18:46 2015 +0100
Committer: blerer 
Committed: Thu Dec 17 12:18:46 2015 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/hadoop/cql3/CqlInputFormat.java   | 46 +++---
 .../cassandra/hadoop/cql3/CqlRecordWriter.java  | 63 +++-
 3 files changed, 46 insertions(+), 64 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2da3c9db/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7677e38..a2951a8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.3
+ * (Hadoop) Close Clusters and Sessions in Hadoop Input/Output classes 
(CASSANDRA-1837)
  * Fix sstableloader not working with upper case keyspace name 
(CASSANDRA-10806)
 Merged from 2.2:
  * Add new types to Stress (CASSANDRA-9556)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2da3c9db/src/java/org/apache/cassandra/hadoop/cql3/CqlInputFormat.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlInputFormat.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlInputFormat.java
index 534e66d..a426532 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlInputFormat.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlInputFormat.java
@@ -21,12 +21,14 @@ import java.io.IOException;
 import java.util.*;
 import java.util.concurrent.*;
 
+import com.datastax.driver.core.Cluster;
 import com.datastax.driver.core.Host;
 import com.datastax.driver.core.Metadata;
 import com.datastax.driver.core.ResultSet;
 import com.datastax.driver.core.Row;
 import com.datastax.driver.core.Session;
 import com.datastax.driver.core.TokenRange;
+
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.mapred.InputSplit;
 import org.apache.hadoop.mapred.JobConf;
@@ -37,12 +39,12 @@ import org.apache.hadoop.mapreduce.TaskAttemptContext;
 import org.apache.hadoop.mapreduce.TaskAttemptID;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
-
 import org.apache.cassandra.db.SystemKeyspace;
 import org.apache.cassandra.dht.*;
 import org.apache.cassandra.thrift.KeyRange;
 import org.apache.cassandra.hadoop.*;
 
+import static java.util.stream.Collectors.toMap;
 
 /**
  * Hadoop InputFormat allowing map/reduce against Cassandra rows within one 
ColumnFamily.
@@ -72,7 +74,6 @@ public class CqlInputFormat extends 
org.apache.hadoop.mapreduce.InputFormat getRecordReader(InputSplit split, JobConf 
jobConf, final Reporter reporter)
 throws IOException
@@ -123,14 +124,12 @@ public class CqlInputFormat extends 
org.apache.hadoop.mapreduce.InputFormat> masterRangeNodes = getRangeMap(conf, 
keyspace);
-
 // canonical ranges, split into pieces, fetching the splits in parallel
 ExecutorService executor = new ThreadPoolExecutor(0, 128, 60L, 
TimeUnit.SECONDS, new LinkedBlockingQueue());
 List splits = new 
ArrayList<>();
 
-try
+try (Cluster cluster = 
CqlConfigHelper.getInputCluster(ConfigHelper.getInputInitialAddress(conf).split(","),
 conf);
+ Session session = cluster.connect())
 {
 List> 
splitfutures = new ArrayList<>();
 KeyRange jobKeyRange = ConfigHelper.getInputKeyRange(conf);
@@ -159,15 +158,17 @@ public class CqlInputFormat extends 
org.apache.hadoop.mapreduce.InputFormat> masterRangeNodes = 
getRangeMap(keyspace, metadata);
 
 for (TokenRange range : masterRangeNodes.keySet())
 {
 if (jobRange == null)
 {
 // for each tokenRange, pick a live owner and ask it to 
compute bite-sized splits
-splitfutures.add(executor.submit(new SplitCallable(range, 
masterRangeNodes.get(range), conf)));
+splitfutures.add(executor.submit(new SplitCallable(range, 
masterRangeNodes.get(range), conf, session)));
 }
 else
 {
@@ -177,7 +178,7 @@ public class CqlInputFormat extends 
org.apache.hadoop.mapreduce.InputFormat getSubSplits(String keyspace, String cfName, 
TokenRange 

[2/2] cassandra git commit: Merge branch cassandra-3.0 into trunk

[blerer]

Merge branch cassandra-3.0 into trunk

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/50f7e020
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/50f7e020
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/50f7e020

Branch: refs/heads/trunk
Commit: 50f7e020490ec20608ce2b4321a7fdc3d8943a92
Parents: 37986e8 2da3c9d
Author: blerer 
Authored: Thu Dec 17 12:55:55 2015 +0100
Committer: blerer 
Committed: Thu Dec 17 12:55:55 2015 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/hadoop/cql3/CqlInputFormat.java   | 46 +++---
 .../cassandra/hadoop/cql3/CqlRecordWriter.java  | 63 +++-
 3 files changed, 46 insertions(+), 64 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/50f7e020/CHANGES.txt
--
diff --cc CHANGES.txt
index 4b4a596,a2951a8..d49a75f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,25 -1,5 +1,26 @@@
 -3.0.3
 +3.2
 + * Establish bootstrap stream sessions sequentially (CASSANDRA-6992)
 + * Sort compactionhistory output by timestamp (CASSANDRA-10464)
 + * More efficient BTree removal (CASSANDRA-9991)
 + * Make tablehistograms accept the same syntax as tablestats (CASSANDRA-10149)
 + * Group pending compactions based on table (CASSANDRA-10718)
 + * Add compressor name in sstablemetadata output (CASSANDRA-9879)
 + * Fix type casting for counter columns (CASSANDRA-10824)
 + * Prevent running Cassandra as root (CASSANDRA-8142)
 + * bound maximum in-flight commit log replay mutation bytes to 64 megabytes 
(CASSANDRA-8639)
 + * Normalize all scripts (CASSANDRA-10679)
 + * Make compression ratio much more accurate (CASSANDRA-10225)
 + * Optimize building of Clustering object when only one is created 
(CASSANDRA-10409)
 + * Make index building pluggable (CASSANDRA-10681)
 + * Add sstable flush observer (CASSANDRA-10678)
 + * Improve NTS endpoints calculation (CASSANDRA-10200)
 + * Improve performance of the folderSize function (CASSANDRA-10677)
 + * Add support for type casting in selection clause (CASSANDRA-10310)
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +Merged from 3.0:
+  * (Hadoop) Close Clusters and Sessions in Hadoop Input/Output classes 
(CASSANDRA-1837)
   * Fix sstableloader not working with upper case keyspace name 
(CASSANDRA-10806)
  Merged from 2.2:
   * Add new types to Stress (CASSANDRA-9556)

[jira] [Created] (CASSANDRA-10889) How can we combine two tables? Can we do this JOINS? like INNER JOINS or other?

[Sandeep Gopalasetty (JIRA)]

Sandeep Gopalasetty created CASSANDRA-10889:
---

 Summary: How can we combine two tables? Can we do this JOINS? like 
INNER JOINS or other?
 Key: CASSANDRA-10889
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10889
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
 Environment: Windows/Linux
Reporter: Sandeep Gopalasetty
 Fix For: 3.0.x


How can we combine two tables? Can we do this with JOINS? like INNER JOINS or 
other?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10889) How can we combine two tables? Can we do this with JOINS? like INNER JOINS or other?

[Sandeep Gopalasetty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sandeep Gopalasetty updated CASSANDRA-10889:

Summary: How can we combine two tables? Can we do this with JOINS? like 
INNER JOINS or other?  (was: How can we combine two tables? Can we do this 
JOINS? like INNER JOINS or other?)

> How can we combine two tables? Can we do this with JOINS? like INNER JOINS or 
> other?
> 
>
> Key: CASSANDRA-10889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10889
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows/Linux
>Reporter: Sandeep Gopalasetty
> Fix For: 3.0.x
>
>
> How can we combine two tables? Can we do this with JOINS? like INNER JOINS or 
> other?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Close Clusters and Sessions in Hadoop Input/Output classes

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 ed96322a0 -> 2da3c9db1


Close Clusters and Sessions in Hadoop Input/Output classes

patch by Alex Liu; reviewed by Benjamin Lerer for CASSANDRA-10837


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2da3c9db
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2da3c9db
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2da3c9db

Branch: refs/heads/cassandra-3.0
Commit: 2da3c9db154449e15d5a2c2072db77b65c9e931a
Parents: ed96322
Author: Alex Liu 
Authored: Thu Dec 17 12:18:46 2015 +0100
Committer: blerer 
Committed: Thu Dec 17 12:18:46 2015 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/hadoop/cql3/CqlInputFormat.java   | 46 +++---
 .../cassandra/hadoop/cql3/CqlRecordWriter.java  | 63 +++-
 3 files changed, 46 insertions(+), 64 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2da3c9db/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7677e38..a2951a8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.3
+ * (Hadoop) Close Clusters and Sessions in Hadoop Input/Output classes 
(CASSANDRA-1837)
  * Fix sstableloader not working with upper case keyspace name 
(CASSANDRA-10806)
 Merged from 2.2:
  * Add new types to Stress (CASSANDRA-9556)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2da3c9db/src/java/org/apache/cassandra/hadoop/cql3/CqlInputFormat.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlInputFormat.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlInputFormat.java
index 534e66d..a426532 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlInputFormat.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlInputFormat.java
@@ -21,12 +21,14 @@ import java.io.IOException;
 import java.util.*;
 import java.util.concurrent.*;
 
+import com.datastax.driver.core.Cluster;
 import com.datastax.driver.core.Host;
 import com.datastax.driver.core.Metadata;
 import com.datastax.driver.core.ResultSet;
 import com.datastax.driver.core.Row;
 import com.datastax.driver.core.Session;
 import com.datastax.driver.core.TokenRange;
+
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.mapred.InputSplit;
 import org.apache.hadoop.mapred.JobConf;
@@ -37,12 +39,12 @@ import org.apache.hadoop.mapreduce.TaskAttemptContext;
 import org.apache.hadoop.mapreduce.TaskAttemptID;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
-
 import org.apache.cassandra.db.SystemKeyspace;
 import org.apache.cassandra.dht.*;
 import org.apache.cassandra.thrift.KeyRange;
 import org.apache.cassandra.hadoop.*;
 
+import static java.util.stream.Collectors.toMap;
 
 /**
  * Hadoop InputFormat allowing map/reduce against Cassandra rows within one 
ColumnFamily.
@@ -72,7 +74,6 @@ public class CqlInputFormat extends 
org.apache.hadoop.mapreduce.InputFormat getRecordReader(InputSplit split, JobConf 
jobConf, final Reporter reporter)
 throws IOException
@@ -123,14 +124,12 @@ public class CqlInputFormat extends 
org.apache.hadoop.mapreduce.InputFormat> masterRangeNodes = getRangeMap(conf, 
keyspace);
-
 // canonical ranges, split into pieces, fetching the splits in parallel
 ExecutorService executor = new ThreadPoolExecutor(0, 128, 60L, 
TimeUnit.SECONDS, new LinkedBlockingQueue());
 List splits = new 
ArrayList<>();
 
-try
+try (Cluster cluster = 
CqlConfigHelper.getInputCluster(ConfigHelper.getInputInitialAddress(conf).split(","),
 conf);
+ Session session = cluster.connect())
 {
 List> 
splitfutures = new ArrayList<>();
 KeyRange jobKeyRange = ConfigHelper.getInputKeyRange(conf);
@@ -159,15 +158,17 @@ public class CqlInputFormat extends 
org.apache.hadoop.mapreduce.InputFormat> masterRangeNodes = 
getRangeMap(keyspace, metadata);
 
 for (TokenRange range : masterRangeNodes.keySet())
 {
 if (jobRange == null)
 {
 // for each tokenRange, pick a live owner and ask it to 
compute bite-sized splits
-splitfutures.add(executor.submit(new SplitCallable(range, 
masterRangeNodes.get(range), conf)));
+splitfutures.add(executor.submit(new SplitCallable(range, 
masterRangeNodes.get(range), conf, session)));
 }
 else
 {
@@ -177,7 +178,7 @@ public class CqlInputFormat extends 
org.apache.hadoop.mapreduce.InputFormat getSubSplits(String keyspace, String cfName, 

[jira] [Commented] (CASSANDRA-10875) cqlsh fails to decode utf-8 characters for text typed columns.

[Yasuharu Goto (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063294#comment-15063294
 ] 

Yasuharu Goto commented on CASSANDRA-10875:
---

Thank you for your response!

I didn't notice --encoding option.
I checked --help and --encoding option.

In cassandra-2.1.9, cqlsh doesn't have --encoding option.
{noformat}

$ cqlsh --help
Usage: cqlsh [options] [host [port]]

CQL Shell for Apache Cassandra

Options:
  --version show program's version number and exit
  -h, --helpshow this help message and exit
  -C, --color   Always use color output
  --no-colorNever use color output
  -u USERNAME, --username=USERNAME
Authenticate as user.
  -p PASSWORD, --password=PASSWORD
Authenticate using password.
  -k KEYSPACE, --keyspace=KEYSPACE
Authenticate to the given keyspace.
  -f FILE, --file=FILE  Execute commands from FILE, then exit
  -t TRANSPORT_FACTORY, --transport-factory=TRANSPORT_FACTORY
Use the provided Thrift transport factory function.
  --debug   Show additional debugging information
  --cqlversion=CQLVERSION
Specify a particular CQL version (default: 3).
Examples: "2", "3.0.0-beta1"
  -2, --cql2Shortcut notation for --cqlversion=2
  -3, --cql3Shortcut notation for --cqlversion=3

Connects to localhost:9160 by default. These defaults can be changed by
setting $CQLSH_HOST and/or $CQLSH_PORT. When a host (and optional port number)
are given on the command line, they take precedence over any defaults.

$ cqlsh --encode=utf8
Usage: cqlsh [options] [host [port]]

cqlsh: error: no such option: --encode
{noformat}

In Cassandra-3.0.0, cqlsh has it. But the help says encoding is utf8 already.
{noformat}
./cqlsh --help
Usage: cqlsh.py [options] [host [port]]

CQL Shell for Apache Cassandra

Options:
  --version show program's version number and exit
  -h, --helpshow this help message and exit
  -C, --color   Always use color output
  --no-colorNever use color output
  --ssl Use SSL
  -u USERNAME, --username=USERNAME
Authenticate as user.
  -p PASSWORD, --password=PASSWORD
Authenticate using password.
  -k KEYSPACE, --keyspace=KEYSPACE
Authenticate to the given keyspace.
  -f FILE, --file=FILE  Execute commands from FILE, then exit
  --debug   Show additional debugging information
  --encoding=ENCODING   Specify a non-default encoding for output.  If you are
experiencing problems with unicode characters, using
utf8 may fix the problem. (Default from system
preferences: UTF-8)
  --cqlshrc=CQLSHRC Specify an alternative cqlshrc file location.
  --cqlversion=CQLVERSION
Specify a particular CQL version (default: 3.3.1).
Examples: "3.0.3", "3.1.0"
  -e EXECUTE, --execute=EXECUTE
Execute the statement and quit.
  --connect-timeout=CONNECT_TIMEOUT
Specify the connection timeout in seconds (default: 5
seconds).

Connects to 127.0.0.1:9042 by default. These defaults can be changed by
setting $CQLSH_HOST and/or $CQLSH_PORT. When a host (and optional port number)
are given on the command line, they take precedence over any defaults.
{noformat}

But,  cqlsh --encoding=utf8 doesn't seem to work correctly.

{noformat}
./cqlsh --encoding=utf8
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.0.0 | CQL spec 3.3.1 | Native protocol v4]
Use HELP for help.
cqlsh> select * from test where id='日本語';
'ascii' codec can't decode byte 0xe6 in position 29: ordinal not in range(128)
cqlsh> 
{noformat}

> cqlsh fails to decode utf-8 characters for text typed columns.
> --
>
> Key: CASSANDRA-10875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10875
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Yasuharu Goto
>Assignee: Yasuharu Goto
>Priority: Minor
> Fix For: 2.1.13, 3.1
>
> Attachments: 10875-2.1.12.txt, 10875-3.1.txt
>
>
> Hi, we've found a bug that cqlsh can't handle unicode text in select 
> conditions even if it were text type.
> {noformat}
> $ ./bin/cqlsh
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.2-SNAPSHOT | CQL spec 3.3.1 | Native protocol v4]
> Use HELP for help.
> cqlsh> create KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 1};
> cqlsh> create table test.test(txt text 

[jira] [Comment Edited] (CASSANDRA-10875) cqlsh fails to decode utf-8 characters for text typed columns.

[Yasuharu Goto (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063294#comment-15063294
 ] 

Yasuharu Goto edited comment on CASSANDRA-10875 at 12/18/15 2:14 AM:
-

Thank you for your response!

I didn't notice --encoding option.
I checked --help and --encoding option.

In cassandra-2.1.9, cqlsh doesn't have --encoding option.
{noformat}

$ cqlsh --help
Usage: cqlsh [options] [host [port]]

CQL Shell for Apache Cassandra

Options:
  --version show program's version number and exit
  -h, --helpshow this help message and exit
  -C, --color   Always use color output
  --no-colorNever use color output
  -u USERNAME, --username=USERNAME
Authenticate as user.
  -p PASSWORD, --password=PASSWORD
Authenticate using password.
  -k KEYSPACE, --keyspace=KEYSPACE
Authenticate to the given keyspace.
  -f FILE, --file=FILE  Execute commands from FILE, then exit
  -t TRANSPORT_FACTORY, --transport-factory=TRANSPORT_FACTORY
Use the provided Thrift transport factory function.
  --debug   Show additional debugging information
  --cqlversion=CQLVERSION
Specify a particular CQL version (default: 3).
Examples: "2", "3.0.0-beta1"
  -2, --cql2Shortcut notation for --cqlversion=2
  -3, --cql3Shortcut notation for --cqlversion=3

Connects to localhost:9160 by default. These defaults can be changed by
setting $CQLSH_HOST and/or $CQLSH_PORT. When a host (and optional port number)
are given on the command line, they take precedence over any defaults.

$ cqlsh --encode=utf8
Usage: cqlsh [options] [host [port]]

cqlsh: error: no such option: --encode
{noformat}

In Cassandra-3.0.0, cqlsh has it. But the help says encoding is utf8 already.
{noformat}
./cqlsh --help
Usage: cqlsh.py [options] [host [port]]

CQL Shell for Apache Cassandra

Options:
  --version show program's version number and exit
  -h, --helpshow this help message and exit
  -C, --color   Always use color output
  --no-colorNever use color output
  --ssl Use SSL
  -u USERNAME, --username=USERNAME
Authenticate as user.
  -p PASSWORD, --password=PASSWORD
Authenticate using password.
  -k KEYSPACE, --keyspace=KEYSPACE
Authenticate to the given keyspace.
  -f FILE, --file=FILE  Execute commands from FILE, then exit
  --debug   Show additional debugging information
  --encoding=ENCODING   Specify a non-default encoding for output.  If you are
experiencing problems with unicode characters, using
utf8 may fix the problem. (Default from system
preferences: UTF-8)
  --cqlshrc=CQLSHRC Specify an alternative cqlshrc file location.
  --cqlversion=CQLVERSION
Specify a particular CQL version (default: 3.3.1).
Examples: "3.0.3", "3.1.0"
  -e EXECUTE, --execute=EXECUTE
Execute the statement and quit.
  --connect-timeout=CONNECT_TIMEOUT
Specify the connection timeout in seconds (default: 5
seconds).

Connects to 127.0.0.1:9042 by default. These defaults can be changed by
setting $CQLSH_HOST and/or $CQLSH_PORT. When a host (and optional port number)
are given on the command line, they take precedence over any defaults.
{noformat}

Furthermore,  cqlsh --encoding=utf8 doesn't seem to work correctly.

{noformat}
./cqlsh --encoding=utf8
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.0.0 | CQL spec 3.3.1 | Native protocol v4]
Use HELP for help.
cqlsh> select * from test where id='日本語';
'ascii' codec can't decode byte 0xe6 in position 29: ordinal not in range(128)
cqlsh> 
{noformat}


was (Author: yasuharu):
Thank you for your response!

I didn't notice --encoding option.
I checked --help and --encoding option.

In cassandra-2.1.9, cqlsh doesn't have --encoding option.
{noformat}

$ cqlsh --help
Usage: cqlsh [options] [host [port]]

CQL Shell for Apache Cassandra

Options:
  --version show program's version number and exit
  -h, --helpshow this help message and exit
  -C, --color   Always use color output
  --no-colorNever use color output
  -u USERNAME, --username=USERNAME
Authenticate as user.
  -p PASSWORD, --password=PASSWORD
Authenticate using password.
  -k KEYSPACE, --keyspace=KEYSPACE
Authenticate to the given keyspace.
  -f FILE, --file=FILE  Execute commands from FILE, then exit
  -t TRANSPORT_FACTORY, --transport-factory=TRANSPORT_FACTORY
   

[jira] [Commented] (CASSANDRA-7464) Replace sstable2json and json2sstable

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063308#comment-15063308
 ] 

Chris Lohfink commented on CASSANDRA-7464:
--

In meantime for people wanting a sstable2json tool, [~andrew.tolbert] and I 
have a version here: https://github.com/tolbertam/sstable-tools that supports 
the 3.x versions currently. 

bq. A key differentiator between the storage format between older verisons of 
Cassandra and Cassandra 3.0 is that an SSTable was previously a representation 
of partitions and their cells (identified by their clustering and column name) 
whereas with Cassandra 3.0 an SSTable now represents partitions and their rows. 
You can read about these changes in more detail by visiting this blog post. 
Additional improvements over the sstable2json tool includes no longer requiring 
the cassandra.yaml in classpath with the schema of the sstables loaded. Also by 
running in client mode this tool will not write to system tables or your commit 
log. It can safely be run as any user anywhere with no side effects.

Its a little easier to run then older version as well. We are using this place 
as a playground but it may be a good starter if updating tool in C* as well.

> Replace sstable2json and json2sstable
> -
>
> Key: CASSANDRA-7464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7464
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Priority: Minor
> Fix For: 3.x
>
>
> Both tools are pretty awful. They are primarily meant for debugging (there is 
> much more efficient and convenient ways to do import/export data), but their 
> output manage to be hard to handle both for humans and for tools (especially 
> as soon as you have modern stuff like composites).
> There is value to having tools to export sstable contents into a format that 
> is easy to manipulate by human and tools for debugging, small hacks and 
> general tinkering, but sstable2json and json2sstable are not that.  
> So I propose that we deprecate those tools and consider writing better 
> replacements. It shouldn't be too hard to come up with an output format that 
> is more aware of modern concepts like composites, UDTs, 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10580) Add latency metrics for dropped messages

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063187#comment-15063187
 ] 

Paulo Motta commented on CASSANDRA-10580:
-

2.2 patch looks good. if this goes on 2.2 it will probably need to go on 3.0.x, 
so you'll also need to provide patches for the cassandra-3.0 branch 
(unfortunately none of the previous patches merge on 3.0).

Below are test results for 2.2 and trunk:

||2.2||trunk||
|[branch|https://github.com/apache/cassandra/compare/cassandra-2.2...pauloricardomg:2.2-10580]|[branch|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-10580]|
|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-10580-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-10580-testall/lastCompletedBuild/testReport/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-10580-dtest/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-10580-dtest/lastCompletedBuild/testReport/]|


> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10859) AssertionError in nodetool cfhistograms

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063190#comment-15063190
 ] 

Yuki Morishita commented on CASSANDRA-10859:


Thanks for review.
Updated branch and running tests one more time.

> AssertionError in nodetool cfhistograms
> ---
>
> Key: CASSANDRA-10859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10859
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Jim Witschey
>Assignee: Yuki Morishita
> Fix For: 3.0.x, 3.x
>
>
> {{nodetool cfhistograms}} raises an {{AssertionError}} on the CassCI 
> {{trunk}} jobs:
> http://cassci.datastax.com/job/trunk_dtest/lastCompletedBuild/testReport/jmx_test/TestJMX/cfhistograms_test/history/
> This regression started in this build on CassCI and has failed consistently 
> since then:
> http://cassci.datastax.com/job/trunk_dtest/806/testReport/junit/jmx_test/TestJMX/cfhistograms_test/
> I can't find any recent dtest changes to this method, so I believe it's a 
> Cassandra bug. Here's the changeset for the first failing build:
> http://cassci.datastax.com/job/trunk_dtest/806/changes
> Maybe something in the changes to the scripts here introduced the bug:
> https://github.com/apache/cassandra/commit/b5240204d7aa2a32c6649d19da2b961c856cde28
> [~jjordan] could you have a look at this please?
> This appears to only affect {{trunk}} at present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10580) Add latency metrics for dropped messages

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063243#comment-15063243
 ] 

Paulo Motta commented on CASSANDRA-10580:
-

[~tjake] since you're more familiar with metrics, would you mind having a 
second look on this? if this approach makes sense then the assignee will make 
patches for 3.0. thanks!

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9258) Range movement causes CPU & performance impact

[Dikang Gu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dikang Gu updated CASSANDRA-9258:
-
Attachment: 0001-pending-ranges-maps-for-2.2.patch

[~slebresne] [~blambov], I ported to 2.2 branch and wrote a micro bench for it.

> Range movement causes CPU & performance impact
> --
>
> Key: CASSANDRA-9258
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9258
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.4
>Reporter: Rick Branson
>Assignee: Dikang Gu
> Fix For: 2.1.x
>
> Attachments: 0001-pending-ranges-map.patch, 
> 0001-pending-ranges-maps-for-2.2.patch, Screenshot 2015-12-16 16.11.36.png, 
> Screenshot 2015-12-16 16.11.51.png
>
>
> Observing big CPU & latency regressions when doing range movements on 
> clusters with many tens of thousands of vnodes. See CPU usage increase by 
> ~80% when a single node is being replaced.
> Top methods are:
> 1) Ljava/math/BigInteger;.compareTo in 
> Lorg/apache/cassandra/dht/ComparableObjectToken;.compareTo 
> 2) Lcom/google/common/collect/AbstractMapBasedMultimap;.wrapCollection in 
> Lcom/google/common/collect/AbstractMapBasedMultimap$AsMap$AsMapIterator;.next
> 3) Lorg/apache/cassandra/db/DecoratedKey;.compareTo in 
> Lorg/apache/cassandra/dht/Range;.contains
> Here's a sample stack from a thread dump:
> {code}
> "Thrift:50673" daemon prio=10 tid=0x7f2f20164800 nid=0x3a04af runnable 
> [0x7f2d878d]
>java.lang.Thread.State: RUNNABLE
>   at org.apache.cassandra.dht.Range.isWrapAround(Range.java:260)
>   at org.apache.cassandra.dht.Range.contains(Range.java:51)
>   at org.apache.cassandra.dht.Range.contains(Range.java:110)
>   at 
> org.apache.cassandra.locator.TokenMetadata.pendingEndpointsFor(TokenMetadata.java:916)
>   at 
> org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:775)
>   at 
> org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:541)
>   at 
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:616)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1101)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1083)
>   at 
> org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:976)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3996)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3980)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>   at 
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:205)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745){code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10580) Add latency metrics for dropped messages

[Anubhav Kale (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063212#comment-15063212
 ] 

Anubhav Kale commented on CASSANDRA-10580:
--

Thanks. When will the decision about 2.2 be made ? I think its better to 
produce 3.0 patch at that point to avoid merge conflicts agaun. What do you 
think ?

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)