[jira] [Comment Edited] (CASSANDRA-7945) new cassanrda-stress does not work with NetworkTopologyStrategy

[Dinesh Kumar (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203619#comment-15203619
 ] 

Dinesh Kumar edited comment on CASSANDRA-7945 at 3/21/16 1:40 AM:
--

Hi, So what is the command to be used for assigning the replication_factor for 
multiple DCs (like DC1=2,DC2=1,DC=3) in stress along with NTS through CCM? 
Thanks, Dinesh


was (Author: dins2k2):
Hi, So what is the command to be used for assigning the replication_factor for 
multiple DCs (like DC1=2,DC2=1,DC=3) in stress along with NTS? Thanks, Dinesh

> new cassanrda-stress does not work with NetworkTopologyStrategy
> ---
>
> Key: CASSANDRA-7945
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7945
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Yuki Morishita
>Assignee: Benedict
>Priority: Trivial
>  Labels: stress
> Fix For: 2.1.1
>
> Attachments: 7945.txt, 7945v2.txt
>
>
> {code}
> $ cassandra-stress write -schema 
> 'replication(strategy=NetworkTopologyStrategy,DC1=2,DC2=1)'
> {code}
> prints error:
> {code}
> Unable to create stress keyspace: Error constructing replication strategy 
> class
> {code}
> Looks like it is caused by passing 'replication_factor' to NTS, which is 
> illegal option for NTS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7945) new cassanrda-stress does not work with NetworkTopologyStrategy

[Dinesh Kumar (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203619#comment-15203619
 ] 

Dinesh Kumar commented on CASSANDRA-7945:
-

Hi, So what is the command to be used for assigning the replication_factor for 
multiple DCs (like DC1=2,DC2=1,DC=3) in stress along with NTS? Thanks, Dinesh

> new cassanrda-stress does not work with NetworkTopologyStrategy
> ---
>
> Key: CASSANDRA-7945
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7945
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Yuki Morishita
>Assignee: Benedict
>Priority: Trivial
>  Labels: stress
> Fix For: 2.1.1
>
> Attachments: 7945.txt, 7945v2.txt
>
>
> {code}
> $ cassandra-stress write -schema 
> 'replication(strategy=NetworkTopologyStrategy,DC1=2,DC2=1)'
> {code}
> prints error:
> {code}
> Unable to create stress keyspace: Error constructing replication strategy 
> class
> {code}
> Looks like it is caused by passing 'replication_factor' to NTS, which is 
> illegal option for NTS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11206) Support large partitions on the 3.0 sstable format

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203372#comment-15203372
 ] 

Jonathan Ellis commented on CASSANDRA-11206:


Very promising!

> Support large partitions on the 3.0 sstable format
> --
>
> Key: CASSANDRA-11206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11206
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
> Fix For: 3.x
>
> Attachments: 11206-gc.png, trunk-gc.png
>
>
> Cassandra saves a sample of IndexInfo objects that store the offset within 
> each partition of every 64KB (by default) range of rows.  To find a row, we 
> binary search this sample, then scan the partition of the appropriate range.
> The problem is that this scales poorly as partitions grow: on a cache miss, 
> we deserialize the entire set of IndexInfo, which both creates a lot of GC 
> overhead (as noted in CASSANDRA-9754) but is also non-negligible i/o activity 
> (relative to reading a single 64KB row range) as partitions get truly large.
> We introduced an "offset map" in CASSANDRA-10314 that allows us to perform 
> the IndexInfo bsearch while only deserializing IndexInfo that we need to 
> compare against, i.e. log(N) deserializations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10091) Integrated JMX authn & authz

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201779#comment-15201779
 ] 

T Jake Luciani commented on CASSANDRA-10091:


This is a serious improvement over the current JMX.

Looking at the code I have a couple comments:

CassandraLoginModule:
  * All the cleanup code is duplicated for logout/abort/login/etc. please 
centralize.

JxmServerUtils:
  *  We should be using the broadcast interface for this vs 0.0.0.0

I also got this error at one point.
{quote}
  Mar 18, 2016 12:13:57 PM RMIConnectionImpl RMIServerCommunicatorAdmin-doStop
  WARNING: Failed to close: java.rmi.NoSuchObjectException: object not exported
{code}

The ability to grant permissions on individual beans is really awesome, I 
tested this and it worked great.

What kind of dtests can we add for this?  Can you kick the tests off once you 
address the above and update NEWS.txt?


> Integrated JMX authn & authz
> 
>
> Key: CASSANDRA-10091
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10091
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jan Karlsson
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 3.x
>
>
> It would be useful to authenticate with JMX through Cassandra's internal 
> authentication. This would reduce the overhead of keeping passwords in files 
> on the machine and would consolidate passwords to one location. It would also 
> allow the possibility to handle JMX permissions in Cassandra.
> It could be done by creating our own JMX server and setting custom classes 
> for the authenticator and authorizer. We could then add some parameters where 
> the user could specify what authenticator and authorizer to use in case they 
> want to make their own.
> This could also be done by creating a premain method which creates a jmx 
> server. This would give us the feature without changing the Cassandra code 
> itself. However I believe this would be a good feature to have in Cassandra.
> I am currently working on a solution which creates a JMX server and uses a 
> custom authenticator and authorizer. It is currently build as a premain, 
> however it would be great if we could put this in Cassandra instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11341) dtest failure in upgrade_tests.cql_tests.TestCQLNodes3RF3_2_1_HEAD_UpTo_2_2.whole_list_conditional_test

[Russ Hatch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Russ Hatch updated CASSANDRA-11341:
---
Status: Ready to Commit  (was: Patch Available)

> dtest failure in 
> upgrade_tests.cql_tests.TestCQLNodes3RF3_2_1_HEAD_UpTo_2_2.whole_list_conditional_test
> ---
>
> Key: CASSANDRA-11341
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11341
> Project: Cassandra
>  Issue Type: Test
>Reporter: Jim Witschey
>Assignee: Russ Hatch
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/upgrade_tests-all/22/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3_2_1_HEAD_UpTo_2_2/whole_list_conditional_test
> Failed on CassCI build upgrade_tests-all #22
> There's only one flap in the history currently. This was the failure:
> {code}
> Expected [[0, ['foo', 'bar', 'foobar']]] from SELECT * FROM tlist, but got 
> [[0, None]]
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: /mnt/tmp/dtest-SF2dOV
> dtest: DEBUG: Custom init_config not found. Setting defaults.
> dtest: DEBUG: Done setting configuration options:
> {   'initial_token': None,
> 'num_tokens': '32',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> dtest: DEBUG: upgrading node1 to 2.2.5
> dtest: DEBUG: Querying upgraded node
> - >> end captured logging << -
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/tools.py", line 253, in wrapped
> f(obj)
>   File "/home/automaton/cassandra-dtest/upgrade_tests/cql_tests.py", line 
> 4294, in whole_list_conditional_test
> check_applies("l != null AND l IN (['foo', 'bar', 'foobar'])")
>   File "/home/automaton/cassandra-dtest/upgrade_tests/cql_tests.py", line 
> 4282, in check_applies
> assert_one(cursor, "SELECT * FROM %s" % (table,), [0, ['foo', 'bar', 
> 'foobar']])
>   File "/home/automaton/cassandra-dtest/assertions.py", line 50, in assert_one
> assert list_res == [expected], "Expected %s from %s, but got %s" % 
> ([expected], query, list_res)
> "Expected [[0, ['foo', 'bar', 'foobar']]] from SELECT * FROM tlist, but got 
> [[0, None]]\n >> begin captured logging << 
> \ndtest: DEBUG: cluster ccm directory: 
> /mnt/tmp/dtest-SF2dOV\ndtest: DEBUG: Custom init_config not found. Setting 
> defaults.\ndtest: DEBUG: Done setting configuration options:\n{   
> 'initial_token': None,\n'num_tokens': '32',\n'phi_convict_threshold': 
> 5,\n'range_request_timeout_in_ms': 1,\n
> 'read_request_timeout_in_ms': 1,\n'request_timeout_in_ms': 1,\n   
>  'truncate_request_timeout_in_ms': 1,\n'write_request_timeout_in_ms': 
> 1}\ndtest: DEBUG: upgrading node1 to 2.2.5\ndtest: DEBUG: Querying 
> upgraded node\n- >> end captured logging << 
> -"
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11031) Support “ALLOW FILTERING" for Partition Key

[ZhaoYang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203336#comment-15203336
 ] 

ZhaoYang commented on CASSANDRA-11031:
--

[~slebresne] Hi Sylvain, I think in C-6377, it still cannot support allow 
filtering on Partition Key. IMO, filtering on Partition Key should be faster 
than filtering on clustering key or non-key column.

> Support “ALLOW FILTERING" for Partition Key
> ---
>
> Key: CASSANDRA-11031
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11031
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: ZhaoYang
>Assignee: ZhaoYang
> Fix For: 2.1.x, 3.x
>
>
> Currently, Allow Filtering only works for secondary Index column or 
> clustering columns. And it's slow, because Cassandra will read all data from 
> SSTABLE from hard-disk to memory to filter.
> But we can support allow filtering on Partition Key, as far as I know, 
> Partition Key is in memory, so we can easily filter them, and then read 
> required data from SSTable.
> This will similar to "Select * from table" which scan through entire cluster.
> CREATE TABLE my_table (
>   pk1 text,
>   pk2 text,
>   c1 text,
>   c2 text,
>   v1 text,
>   v2 text,
>   PRIMARY KEY ((pk1,pk2),c1,c2)
> ) ;
> Select * from my_table where pk1 = "partition key part 1" allow filtering;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Change repair message for keyspaces with RF 1

[marcuse]

Repository: cassandra
Updated Branches:
  refs/heads/trunk b210bd234 -> 6a8c747c6


Change repair message for keyspaces with RF 1

Patch by Branimir Lambov; reviewed by marcuse for CASSANDRA-11203


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6a8c747c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6a8c747c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6a8c747c

Branch: refs/heads/trunk
Commit: 6a8c747c65833674057044f07c39418c5fe6302f
Parents: b210bd2
Author: Branimir Lambov 
Authored: Thu Mar 17 13:19:00 2016 +0200
Committer: Marcus Eriksson 
Committed: Thu Mar 17 13:11:11 2016 +0100

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/tools/RepairRunner.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6a8c747c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 311c1fa..df6d950 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,7 @@
 3.6
  * Store repair options in parent_repair_history (CASSANDRA-11244)
  * Print current leveling in sstableofflinerelevel (CASSANDRA-9588)
+ * Change repair message for keyspaces with RF 1 (CASSANDRA-11203)
  * Remove hard-coded SSL cipher suites and protocols (CASSANDRA-10508)
  * Improve concurrency in CompactionStrategyManager (CASSANDRA-10099)
  * (cqlsh) interpret CQL type for formatting blobs (CASSANDRA-11274)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6a8c747c/src/java/org/apache/cassandra/tools/RepairRunner.java
--
diff --git a/src/java/org/apache/cassandra/tools/RepairRunner.java 
b/src/java/org/apache/cassandra/tools/RepairRunner.java
index 0813775..8497a1b 100644
--- a/src/java/org/apache/cassandra/tools/RepairRunner.java
+++ b/src/java/org/apache/cassandra/tools/RepairRunner.java
@@ -56,7 +56,8 @@ public class RepairRunner extends 
JMXNotificationProgressListener
 cmd = ssProxy.repairAsync(keyspace, options);
 if (cmd <= 0)
 {
-String message = String.format("[%s] Nothing to repair for 
keyspace '%s'", format.format(System.currentTimeMillis()), keyspace);
+// repairAsync can only return 0 for replication factor 1.
+String message = String.format("[%s] Replication factor is 1. No 
repair is needed for keyspace '%s'", format.format(System.currentTimeMillis()), 
keyspace);
 out.println(message);
 }
 else

[jira] [Commented] (CASSANDRA-10649) Improve field-checking and error reporting in cassandra.yaml

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199652#comment-15199652
 ] 

Sylvain Lebresne commented on CASSANDRA-10649:
--

Dealing with an empty file better is certainly a nice addition, though checking 
the line of the stack in the description on 2.1.11 shows that this is related 
to the the client encryption configuration, and indeed, if you have this in the 
yaml:
{noformat}
client_encryption_options:
#enabled: false
## If enabled and optional is set to true encrypted and unencrypted 
connections are handled.
#optional: false
#keystore: conf/.keystore
#keystore_password: cassandra
{noformat}
that is, you declare {{client_encryption_options}} but everything else is 
commented, you do get a server side NPE with nothing returned to the client 
(it's in {{NativeTransportService}} on trunk but this still reproduce). We 
should try to fix that (either by complaining or by assuming {{enabled: false}} 
if it's not there).

Another nit that the comments above make me thing of regarding yaml error 
reporting is that when you add an unknown property, we exit with a stack. We, 
devs, reads stack trace as a 2nd nature so it's fine, but some of our user may 
not even be that familiar with JAVA and getting a full stack when the only 
problem is you've typoed a property in the yaml is scary and not super user 
friendly. In general, I think that if we a {{ConfigurationException}} at 
startup, we should probably just print the message of the error to stderr and 
exit, rather than letting the exception escape. Would be cool if we could use 
that ticket for that too.


> Improve field-checking and error reporting in cassandra.yaml
> 
>
> Key: CASSANDRA-10649
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10649
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration
> Environment: Linux: Fedora-16 64 bit
>Reporter: sandeep thakur
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.6, 3.0.5, 3.5
>
> Attachments: 10649-2.2.txt, 10649-3.0.txt, cassandra.yaml
>
>
> I am trying to setup cassandra single node cluster. i've downloaded below 
> build:
> apache-cassandra-2.1.11-bin.tar.gz
> I've upgraded Java to 1.8 as well, as earlier it was throwing errors related 
> to Java version.
> {code}
> [root@localhost cassandra]# java -version
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
> {code}
> I've also verified the cassandra.yaml file from "http://www.yamllint.com/; as 
> well. But while starting cassandra, I am getting vague exception as below:
> {code}
> INFO  15:52:11 Compacting 
> [SSTableReader(path='/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-18-Data.db'),
>  
> SSTableReader(path='/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-17-Data.db'),
>  
> SSTableReader(path='/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-20-Data.db'),
>  
> SSTableReader(path='/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-19-Data.db')]
> INFO  15:52:11 Node localhost/127.0.0.1 state jump to normal
> INFO  15:52:11 Netty using native Epoll event loop
> ERROR 15:52:11 Exception encountered during startup
> java.lang.NullPointerException: null
> at org.apache.cassandra.transport.Server.run(Server.java:171) 
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.transport.Server.start(Server.java:117) 
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:492) 
> [apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:575)
>  [apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:651) 
> [apache-cassandra-2.1.11.jar:2.1.11]
> java.lang.NullPointerException
> at org.apache.cassandra.transport.Server.run(Server.java:171)
> at org.apache.cassandra.transport.Server.start(Server.java:117)
> at 
> org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:492)
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:575)
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:651)
> Exception encountered during startup: null
> INFO  15:52:11 Announcing shutdown
> INFO  15:52:11 Node localhost/127.0.0.1 state jump 

[jira] [Commented] (CASSANDRA-11224) dtest failure in bootstrap_test.TestBootstrap.failed_bootstrap_wiped_node_can_join_test

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201976#comment-15201976
 ] 

Russ Hatch commented on CASSANDRA-11224:


100 repeitions pass fine on ci. must have been isolated failure.

> dtest failure in 
> bootstrap_test.TestBootstrap.failed_bootstrap_wiped_node_can_join_test
> ---
>
> Key: CASSANDRA-11224
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11224
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Russ Hatch
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/209/testReport/bootstrap_test/TestBootstrap/failed_bootstrap_wiped_node_can_join_test
> Failed on CassCI build cassandra-2.1_novnode_dtest #209
> Happening only intermittently.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10751) "Pool is shutdown" error when running Hadoop jobs on Yarn

[Cyril Scetbon (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200936#comment-15200936
 ] 

Cyril Scetbon commented on CASSANDRA-10751:
---

[~alexliu68] ?

> "Pool is shutdown" error when running Hadoop jobs on Yarn
> -
>
> Key: CASSANDRA-10751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10751
> Project: Cassandra
>  Issue Type: Bug
> Environment: Hadoop 2.7.1 (HDP 2.3.2)
> Cassandra 2.1.11
>Reporter: Cyril Scetbon
>Assignee: Cyril Scetbon
> Attachments: CASSANDRA-10751-2.2.patch, CASSANDRA-10751-3.0.patch, 
> output.log
>
>
> Trying to execute an Hadoop job on Yarn, I get errors from Cassandra's 
> internal code. It seems that connections are shutdown but we can't understand 
> why ...
> Here is a subtract of the errors. I also add a file with the complete debug 
> logs.
> {code}
> 15/11/22 20:05:54 [main]: DEBUG core.RequestHandler: Error querying 
> node006.internal.net/192.168.12.22:9042, trying next host (error is: 
> com.datastax.driver.core.ConnectionException: 
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown)
> Failed with exception java.io.IOException:java.io.IOException: 
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) 
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042 
> (com.datastax.driver.core.ConnectionException: 
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> 15/11/22 20:05:54 [main]: ERROR CliDriver: Failed with exception 
> java.io.IOException:java.io.IOException: 
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) 
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042 
> (com.datastax.driver.core.ConnectionException: 
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> java.io.IOException: java.io.IOException: 
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) 
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042 
> (com.datastax.driver.core.ConnectionException: 
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
>   at 
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508)
>   at 
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:415)
>   at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
>   at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1672)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
>   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
>   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>   at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.io.IOException: 
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) 
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042 
> (com.datastax.driver.core.ConnectionException: 
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
>   at 
> org.apache.hadoop.hive.cassandra.input.cql.HiveCqlInputFormat.getRecordReader(HiveCqlInputFormat.java:132)
>   at 
> org.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit.getRecordReader(FetchOperator.java:674)
>   at 
> org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:324)
>   at 
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:446)
>   ... 15 more
> Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All 
> host(s) tried for query failed (tried: 
> node006.internal.net/192.168.12.22:9042 
> (com.datastax.driver.core.ConnectionException: 
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
>   at 
> com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
>   at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
>   at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:214)
>   at 
> 

[jira] [Resolved] (CASSANDRA-7159) sstablemetadata command should print some more stuff

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson resolved CASSANDRA-7159.

   Resolution: Fixed
Fix Version/s: (was: 3.x)
   3.6

committed, thanks

> sstablemetadata command should print some more stuff
> 
>
> Key: CASSANDRA-7159
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7159
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
>Priority: Trivial
>  Labels: lhf
> Fix For: 3.6
>
> Attachments: 
> CASSANDRA-7159_-_sstablemetadata_command_should_print_some_more_stuff.patch
>
>
> It would be nice if the sstablemetadata command printed out some more of the 
> stuff we track.  Like the Min/Max column names and the min/max token in the 
> file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11352) Include units of metrics in the cassandra-stress tool

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197745#comment-15197745
 ] 

Joel Knighton commented on CASSANDRA-11352:
---

I wouldn't make the changes to 2.1/2.2/3.0.x.

2.1 is critical fixes only at this point; 2.2/3.0.x should mostly be bug fixes 
without a very strong case otherwise. For these types of minor improvements, 
you should work off trunk.

> Include units of metrics in the cassandra-stress tool 
> --
>
> Key: CASSANDRA-11352
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11352
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Rajath Subramanyam
>Assignee: Giampaolo
>Priority: Minor
>  Labels: lhf
> Fix For: 2.1.0
>
>
> cassandra-stress in the Results section can have units for the metrics as an 
> improvement to make the tool more usable. 
> Results:
> op rate   : 14668 [READ:7334, WRITE:7334]
> partition rate: 14668 [READ:7334, WRITE:7334]
> row rate  : 14668 [READ:7334, WRITE:7334]
> latency mean  : 0.7 [READ:0.7, WRITE:0.7]
> latency median: 0.6 [READ:0.6, WRITE:0.6]
> latency 95th percentile   : 0.8 [READ:0.8, WRITE:0.8]
> latency 99th percentile   : 1.2 [READ:1.2, WRITE:1.2]
> latency 99.9th percentile : 8.8 [READ:8.9, WRITE:9.0]
> latency max   : 448.7 [READ:162.3, WRITE:448.7]
> Total partitions  : 105612753 [READ:52805915, WRITE:52806838]
> Total errors  : 0 [READ:0, WRITE:0]
> total gc count: 0
> total gc mb   : 0
> total gc time (s) : 0
> avg gc time(ms)   : NaN
> stdev gc time(ms) : 0
> Total operation time  : 02:00:00
> END



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-7115) Column Family (Table) partitioning with column keys as partition keys (Sorta TTLed Table)

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-7115.
--
Resolution: Won't Fix

This could work, but arguably not worth the extra complexity. There are many 
ways to expire-cleanup data already, and you can model around it if necessary.

> Column Family (Table) partitioning with column keys as partition keys (Sorta 
> TTLed Table)
> -
>
> Key: CASSANDRA-7115
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7115
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Haebin Na
>Priority: Minor
>  Labels: features
>
> We need a better solution to expire columns than TTLed columns.
> If you set TTL 6 months for a column in a frequently updated(deleted, yes, 
> this is anti-pattern) wide row, it is not likely to be deleted since the row 
> would be highly fragmented.
> In order to solve the problem above, I suggest partitioning column family 
> (table) with column key (column1) as partition key.
> It is like a set of column families (tables) which share the same structure 
> and cover certain range of columns per CF. This means that a row is 
> deterministically fragmented by column key.
> If you use timestamp like column key, then you would be able to truncate 
> specific partition (a sub-table or CF with specific range) if it is older 
> than certain age easily without worrying about zombie tombstones. 
> It is not optimal to have many column families, yet even with small set like 
> by biyearly or quarterly, it could be whole lot more efficient than TTLed 
> columns.
> What do you think?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11096) Upgrade netty to >= 4.0.34

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-11096:
---
Fix Version/s: (was: 3.x)
   3.6
   Status: Patch Available  (was: Open)

> Upgrade netty to >= 4.0.34
> --
>
> Key: CASSANDRA-11096
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11096
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Brandon Williams
>Assignee: Benjamin Lerer
> Fix For: 3.6
>
>
> Amongst other things, the native protocol will not bind ipv6 easily (see 
> CASSANDRA-11047) until we upgrade.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10748) UTF8Validator.validate() wrong ??

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10748:
---
Resolution: Fixed
Status: Resolved  (was: Ready to Commit)

Committed into 2.2 at 42cd66a1f91dcd0cf9238e7915d5efe5779b9933 and merged into 
3.0, 3.5 and trunk

> UTF8Validator.validate() wrong ??
> -
>
> Key: CASSANDRA-10748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10748
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.6, 3.0.5, 3.5
>
>
> The switch-statement in {{UTF8Validator.validate()}} can never go into {{case 
> TWO_80}} as the assignment {{state = State.TWO_80;}} in line 75 is dead.
> I assume that the {{TWO_80}} case is completely superfluous - but would like 
> to have a 2nd opinion on this.
> /cc [~carlyeks] (CASSANDRA-4495)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11338) dtest failure in consistency_test.TestConsistency.quorum_available_during_failure_test

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197553#comment-15197553
 ] 

Philip Thompson commented on CASSANDRA-11338:
-

Fixed by 
https://github.com/pcmanus/ccm/pull/465

and 

https://github.com/riptano/cassandra-dtest/pull/856

> dtest failure in 
> consistency_test.TestConsistency.quorum_available_during_failure_test
> --
>
> Key: CASSANDRA-11338
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11338
> Project: Cassandra
>  Issue Type: Test
>Reporter: Jim Witschey
>Assignee: Philip Thompson
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/68/testReport/consistency_test/TestConsistency/quorum_available_during_failure_test
> Failed on CassCI build trunk_offheap_dtest #68
> This seems to be failing after merging this CCM PR:
> https://github.com/pcmanus/ccm/pull/461
> I'm not sure why it would fail with that error-checking code but not without 
> it. On this test run, it ran after these two tests:
> {code}
> test_simple_strategy_each_quorum (consistency_test.TestAvailability) ... ok
> test_simple_strategy_each_quorum_counters (consistency_test.TestAccuracy) ... 
> ok
> {code}
> so, maybe processes are hanging around after one of those tests. 
> [~philipthompson] can you have a first look here?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11318) dtest failure in replication_test.SnitchConfigurationUpdateTest.test_rf_collapse_property_file_snitch

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197612#comment-15197612
 ] 

Philip Thompson commented on CASSANDRA-11318:
-

Two minutes is already a pretty high timeout. I can see in the logs that the 
node allegedly updated its understanding of the topology. It's unclear why 
nodetool status isn't changing, though I can't compare timestamps of the status 
calls with the logs. The logs also don't say *what* it is updating the topology 
to.

I'm going to fix the annotation in dtest (It's on the wrong test), and maybe 
wait if we see this again?

> dtest failure in 
> replication_test.SnitchConfigurationUpdateTest.test_rf_collapse_property_file_snitch
> -
>
> Key: CASSANDRA-11318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11318
> Project: Cassandra
>  Issue Type: Test
>Reporter: Jim Witschey
>Assignee: Philip Thompson
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.0_dtest/592/testReport/replication_test/SnitchConfigurationUpdateTest/test_rf_collapse_property_file_snitch
> Failed on CassCI build cassandra-3.0_dtest #592
> The test failed on this line:
> https://github.com/riptano/cassandra-dtest/blob/7a331cda807c96ae107b58017854f0e57996d8c3/replication_test.py#L567
> So, a node's expected move from one rack to another doesn't happen in the 
> allotted timeout time. This is the only flap I've seen. Maybe the thing to do 
> here is increase the timeout and keep an eye on it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11361) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_round_trip_with_rate_file

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-11361:


 Summary: dtest failure in 
cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_round_trip_with_rate_file
 Key: CASSANDRA-11361
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11361
 Project: Cassandra
  Issue Type: Test
Reporter: Jim Witschey
Assignee: DS Test Eng


This test has started failing recently:

http://cassci.datastax.com/job/cassandra-2.1_offheap_dtest/318/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_round_trip_with_rate_file

In addition to failing on the offheap-memtable job, it also fails on the 
vanilla 2.1 job:

http://cassci.datastax.com/job/cassandra-2.1_dtest/lastCompletedBuild/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_round_trip_with_rate_file/

which makes me think this isn't just a weird flap. I have not yet seen this 
failure on versions higher than 2.1.

Test Eng should take the first crack at debugging this, but if you don't make 
headway, Stefania added the test in November, so she's probably the person to 
escalate this to.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10748) UTF8Validator.validate() wrong ??

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199219#comment-15199219
 ] 

Benjamin Lerer commented on CASSANDRA-10748:


{quote}In Modified UTF-8, the null character (U+) is encoded as 
{{0xC0,0x80}}.{quote}
This seems to indicate that this code is needed. I believe the problem is more 
that an else is missing.

> UTF8Validator.validate() wrong ??
> -
>
> Key: CASSANDRA-10748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10748
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Benjamin Lerer
>Priority: Minor
>
> The switch-statement in {{UTF8Validator.validate()}} can never go into {{case 
> TWO_80}} as the assignment {{state = State.TWO_80;}} in line 75 is dead.
> I assume that the {{TWO_80}} case is completely superfluous - but would like 
> to have a 2nd opinion on this.
> /cc [~carlyeks] (CASSANDRA-4495)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-10340) Stress should exit with non-zero status after failure

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania reassigned CASSANDRA-10340:


Assignee: Stefania

> Stress should exit with non-zero status after failure
> -
>
> Key: CASSANDRA-10340
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10340
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Paulo Motta
>Assignee: Stefania
>Priority: Minor
>  Labels: lhf, stress
>
> Currently, stress always exits with sucess status, even if after a failure. 
> In order to be able to rely on stress exit status during dtests it would be 
> nice if it exited with a non-zero status after failures.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-5983) Include sample CQL schema with distribution

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-5983.
--
Resolution: Not A Problem
  Assignee: (was: Patrick McFadin)

This belongs to the docs and tutorials. Arguably we don't need another place 
for it where it can and will get stale.

> Include sample CQL schema with distribution
> ---
>
> Key: CASSANDRA-5983
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5983
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Patrick McFadin
>Priority: Minor
>
> Having a sample schema will give new users a easy place to start with CQL. I 
> have been using an example schema for over a year that tries to cover many 
> data model topics. 
> I have made the examples available on my Githib account: 
> https://github.com/pmcfadin/cassandra-videodb-sample-schema
> I need to add more sample queries but would like to start getting feedback on 
> any specific CQL features to add. I would like to show as many features as 
> possible and still make it useful. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-5502) Secondary Index Storage

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-5502.
--
Resolution: Won't Fix

As we are very likely moving to making SASI (CASSANDRA-10661) the new default 
implementation (CASSANDRA-11075), and indexes are components of sstables there, 
closing the ticket as Won't Fix.

> Secondary Index Storage
> ---
>
> Key: CASSANDRA-5502
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5502
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Brooke Bryan
>Priority: Minor
>
> Currently, both the CF data, and the secondary index data are stored within 
> the same folder on disk.  Being able to split these into separate folders 
> could be a great improvement with performance.  With our data, and secondary 
> index, we will query the index a lot more, so can optimise hardware for the 
> two traffic shapes coming through.
> Something like /Keyspace/CF/indexes/indexname/*.db
> Not too sure how much would be involved to change this, but I would imagine 
> fairly cheap cost to reward.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11265) dtest failure in materialized_views_test.TestMaterializedViews.view_tombstone_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-11265:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

> dtest failure in 
> materialized_views_test.TestMaterializedViews.view_tombstone_test
> --
>
> Key: CASSANDRA-11265
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11265
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Philip Thompson
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.0_dtest/564/testReport/materialized_views_test/TestMaterializedViews/view_tombstone_test
> Failed on CassCI build cassandra-3.0_dtest #564
> intermittent failure, error:
> {noformat}
> Expected [[1, 1, 'b', 3.0]] from SELECT * FROM t_by_v WHERE v = 1, but got []
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-4986) Allow finer control of ALLOW FILTERING behavior

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-4986.
--
   Resolution: Later
Fix Version/s: (was: 3.x)

> Allow finer control of ALLOW FILTERING behavior
> ---
>
> Key: CASSANDRA-4986
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4986
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Priority: Minor
>
> CASSANDRA-4915 added {{ALLOW FILTERING}} to warn people when they do 
> potentially inefficient queries. However, as discussed in the former issue it 
> would be interesting to allow controlling that mode more precisely by 
> allowing something like:
> {noformat}
> ... ALLOW FILTERING MAX 500
> {noformat}
> whose behavior would be that the query would be short-circuited if it filters 
> (i.e. read but discard from the ResultSet) more than 500 CQL3 rows.
> There is however 2 details I'm not totally clear on:
> # what to do exactly when we reach the max filtering allowed. Do we return 
> what we have so far, but then we need to have a way to say in the result set 
> that the query was short-circuited. Or do we just throw an exception 
> TooManyFiltered (simpler but maybe a little bit less useful).
> # what about deleted records? Should we count them as 'filtered'? Imho the 
> logical thing is to not count them as filtered, since after all we "filter 
> them out" in the normal path (i.e. even when ALLOW FILTERING is not used).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-4476) Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE)

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-4476.
--
   Resolution: Duplicate
 Assignee: (was: Oded Peer)
Fix Version/s: (was: 3.x)

Closing as Dup of SASI (CASSANDRA-10661).

> Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE)
> 
>
> Key: CASSANDRA-4476
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4476
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Sylvain Lebresne
>Priority: Minor
>  Labels: cql
> Attachments: 4476-2.patch, 4476-3.patch, 4476-5.patch, 
> cassandra-trunk-4476.patch
>
>
> Currently, a query that uses 2ndary indexes must have at least one EQ clause 
> (on an indexed column). Given that indexed CFs are local (and use 
> LocalPartitioner that order the row by the type of the indexed column), we 
> should extend 2ndary indexes to allow querying indexed columns even when no 
> EQ clause is provided.
> As far as I can tell, the main problem to solve for this is to update 
> KeysSearcher.highestSelectivityPredicate(). I.e. how do we estimate the 
> selectivity of non-EQ clauses? I note however that if we can do that estimate 
> reasonably accurately, this might provide better performance even for index 
> queries that both EQ and non-EQ clauses, because some non-EQ clauses may have 
> a much better selectivity than EQ ones (say you index both the user country 
> and birth date, for SELECT * FROM users WHERE country = 'US' AND birthdate > 
> 'Jan 2009' AND birtdate < 'July 2009', you'd better use the birthdate index 
> first).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10809) Create a -D option to prevent gossip startup

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10809:
-
   Resolution: Fixed
Fix Version/s: (was: 2.1.x)
   3.5
   3.0.5
   2.2.6
   Status: Resolved  (was: Patch Available)

Alright then, committed, thanks.

> Create a -D option to prevent gossip startup
> 
>
> Key: CASSANDRA-10809
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10809
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Distributed Metadata
>Reporter: Brandon Williams
>Assignee: Sylvain Lebresne
> Fix For: 2.2.6, 3.0.5, 3.5
>
> Attachments: 10809.txt
>
>
> In CASSANDRA-6961 we changed how join_ring=false works, to great benefit.  
> However, sometimes you need to a node to come up, but not interact with other 
> nodes whatsoever - for example if you have a schema problem, it will still 
> pull the schema from another node because they still gossip even though we're 
> in a dead state.
> We can add a way to restore the previous behavior by simply adding something 
> like -Dcassandra.start_gossip=false.
> In the meantime we can workaround by setting listen_address to localhost, but 
> that's kind of a pain.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11381) Node running with join_ring=false and authentication can not serve requests

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202064#comment-15202064
 ] 

mck commented on CASSANDRA-11381:
-

@JoshuaMcKenzie, I've only run unit tests against trunk. And I'm trying to 
figure out how to write a dtest for this (since unit tests and system property 
don't mix (afaik)).

If you're happy to proceed without the dtest, say and I'll 'submit patch'.

> Node running with join_ring=false and authentication can not serve requests
> ---
>
> Key: CASSANDRA-11381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11381
> Project: Cassandra
>  Issue Type: Bug
>Reporter: mck
>Assignee: mck
> Attachments: 11381-2.0.txt, 11381-2.1.txt, 11381-2.2.txt, 
> 11381-3.0.txt, 11381-trunk.txt
>
>
> Starting up a node with {{-Dcassandra.join_ring=false}} in a cluster that has 
> authentication configured, eg PasswordAuthenticator, won't be able to serve 
> requests. This is because {{Auth.setup()}} never gets called during the 
> startup.
> Without {{Auth.setup()}} having been called in {{StorageService}} clients 
> connecting to the node fail with the node throwing
> {noformat}
> java.lang.NullPointerException
> at 
> org.apache.cassandra.auth.PasswordAuthenticator.authenticate(PasswordAuthenticator.java:119)
> at 
> org.apache.cassandra.thrift.CassandraServer.login(CassandraServer.java:1471)
> at 
> org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3505)
> at 
> org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3489)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at com.thinkaurelius.thrift.Message.invoke(Message.java:314)
> at 
> com.thinkaurelius.thrift.Message$Invocation.execute(Message.java:90)
> at 
> com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:695)
> at 
> com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:689)
> at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:112)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The exception thrown from the 
> [code|https://github.com/apache/cassandra/blob/cassandra-2.0.16/src/java/org/apache/cassandra/auth/PasswordAuthenticator.java#L119]
> {code}
> ResultMessage.Rows rows = 
> authenticateStatement.execute(QueryState.forInternalCalls(), new 
> QueryOptions(consistencyForUser(username),
>   
>Lists.newArrayList(ByteBufferUtil.bytes(username;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10091) Align JMX authentication with internal authentication

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197941#comment-15197941
 ] 

Sam Tunnicliffe commented on CASSANDRA-10091:
-

In an offline discussion, [~tjake] reminded me about CASSANDRA-2967 and the 
{{sun.rmi.dgc.server.gcInterval}} system property.
When a {{JMXConnectorServer}} is created programmatically (i.e. not by the 
default management agent), it automatically schedules a full GC to run 
periodically. This is the reason for adding the {{-XX:+DisableExplicitGC}} when 
running with {{LOCAL_JMX}} currently, as the forcing the server to bind only to 
a loopback address involves creating it programatically. It is possible to 
avoid this though, by mimicking how the management agent creates the server. 
I've pushed another commit which does this using a custom {{RMIExporter}} 
implementation in {{JMXServerUtils}}. Hopefully, the comments in that class 
should explain the risks in doing this and their mitigation (which I think 
makes it an acceptable thing to do). 


> Align JMX authentication with internal authentication
> -
>
> Key: CASSANDRA-10091
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10091
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jan Karlsson
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 3.x
>
>
> It would be useful to authenticate with JMX through Cassandra's internal 
> authentication. This would reduce the overhead of keeping passwords in files 
> on the machine and would consolidate passwords to one location. It would also 
> allow the possibility to handle JMX permissions in Cassandra.
> It could be done by creating our own JMX server and setting custom classes 
> for the authenticator and authorizer. We could then add some parameters where 
> the user could specify what authenticator and authorizer to use in case they 
> want to make their own.
> This could also be done by creating a premain method which creates a jmx 
> server. This would give us the feature without changing the Cassandra code 
> itself. However I believe this would be a good feature to have in Cassandra.
> I am currently working on a solution which creates a JMX server and uses a 
> custom authenticator and authorizer. It is currently build as a premain, 
> however it would be great if we could put this in Cassandra instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Clarify that now() function is calculated on the coordinator node in CQL documentation

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 9995521fb -> a64ad4e0f


Clarify that now() function is calculated on the coordinator node in CQL 
documentation

patch by Benjamin Lerer; reviewed by Sylvain Lebresne for CASSANDRA-10900


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6d383bed
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6d383bed
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6d383bed

Branch: refs/heads/cassandra-3.0
Commit: 6d383bedd4db29277858144b4b02048df7ec8ffb
Parents: 5182376
Author: Benjamin Lerer 
Authored: Thu Mar 17 09:47:46 2016 +0100
Committer: Benjamin Lerer 
Committed: Thu Mar 17 10:07:37 2016 +0100

--
 CHANGES.txt  | 1 +
 doc/cql3/CQL.textile | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6d383bed/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index dca4f8a..fdb5807 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.6
+ * Clarify that now() function is calculated on the coordinator node in CQL 
documentation (CASSANDRA-10900)
  * Fix bloom filter sizing with LCS (CASSANDRA-11344)
  * (cqlsh) Fix error when result is 0 rows with EXPAND ON (CASSANDRA-11092)
  * Fix intra-node serialization issue for multicolumn-restrictions 
(CASSANDRA-11196)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6d383bed/doc/cql3/CQL.textile
--
diff --git a/doc/cql3/CQL.textile b/doc/cql3/CQL.textile
index ca68509..1130479 100644
--- a/doc/cql3/CQL.textile
+++ b/doc/cql3/CQL.textile
@@ -1819,7 +1819,7 @@ h3(#timeuuidFun). Timeuuid functions
 
 h4. @now@
 
-The @now@ function takes no arguments and generates a new unique timeuuid (at 
the time where the statement using it is executed). Note that this method is 
useful for insertion but is largely non-sensical in @WHERE@ clauses. For 
instance, a query of the form
+The @now@ function takes no arguments and generates, on the coordinator node, 
a new unique timeuuid (at the time where the statement using it is executed). 
Note that this method is useful for insertion but is largely non-sensical in 
@WHERE@ clauses. For instance, a query of the form
 
 bc(sample). 
 SELECT * FROM myTable WHERE t = now()

[jira] [Commented] (CASSANDRA-11224) dtest failure in bootstrap_test.TestBootstrap.failed_bootstrap_wiped_node_can_join_test

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201899#comment-15201899
 ] 

Russ Hatch commented on CASSANDRA-11224:


going to run a 100 test trial here: 
http://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/18/

> dtest failure in 
> bootstrap_test.TestBootstrap.failed_bootstrap_wiped_node_can_join_test
> ---
>
> Key: CASSANDRA-11224
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11224
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Russ Hatch
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/209/testReport/bootstrap_test/TestBootstrap/failed_bootstrap_wiped_node_can_join_test
> Failed on CassCI build cassandra-2.1_novnode_dtest #209
> Happening only intermittently.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10971) Compressed commit log has no backpressure and can OOM

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10971:
---
Attachment: 10971-3.5.txt
10971-3.0.txt

Attached the latest patches.

> Compressed commit log has no backpressure and can OOM
> -
>
> Key: CASSANDRA-10971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10971
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 3.0.x, 3.x
>
> Attachments: 10971-3.0.txt, 10971-3.5.txt
>
>
> I validated this via a unit test that slowed the ability of the log to drain 
> to the filesystem. The compressed commit log will keep allocating buffers 
> pending compression until it OOMs.
> I have a fix that am not very happy with because the whole signal a thread to 
> allocate a segment that depends on a resource that may not be available 
> results in some obtuse usage of {{CompleatableFuture}} to rendezvous 
> available buffers with {{CommitLogSegmentManager}} thread waiting to finish 
> constructing a new segment. The {{CLSM}} thread is in turn signaled by the 
> thread(s) that actually wants to write to the next segment, but aren't able 
> to do it themselves.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9738) Migrate key-cache to be fully off-heap

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203267#comment-15203267
 ] 

Robert Stupp commented on CASSANDRA-9738:
-

With CASSANDRA-11206 we eliminate a big pain (IndexInfo GC pressure and 
serialization).

We _could_ move the key-cache into off-heap after 11206. But it's also possible 
that other optimizations (CASSANDRA-8931, CASSANDRA-9754 and CASSANDRA-9786) 
may result in a completely different structure. Having that said, I tend to 
prefer at least 8931 and 9786 before tackling this one and see what the outcome 
of these is.

> Migrate key-cache to be fully off-heap
> --
>
> Key: CASSANDRA-9738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9738
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.x
>
>
> Key cache still uses a concurrent map on-heap. This could go to off-heap and 
> feels doable now after CASSANDRA-8099.
> Evaluation should be done in advance based on a POC to prove that pure 
> off-heap counter cache buys a performance and/or gc-pressure improvement.
> In theory, elimination of on-heap management of the map should buy us some 
> benefit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-11206) Support large partitions on the 3.0 sstable format

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203258#comment-15203258
 ] 

Robert Stupp edited comment on CASSANDRA-11206 at 3/20/16 12:02 PM:


I just finished with most of the coding for this ticket - i.e. "shallow" 
RowIndexEntry without {{IndexInfo}} - and ran a poor-man's comparison of 
current trunk against 11206 using different partition sizes covering writes, a 
major compaction and reads. The results are really promising especially with 
big and huge partitions (tested up to 8G partitions).

Reads against big partitions really benefit from 11206. For example, with 11206 
it takes a couple of seconds for 5000 random reads against 8G partitions vs. 
many minutes (not a typo) on current trunk). At the same time, the heap is 
quite full and causes a lot of GC pressure.

Compactions also benefit from 11206 GC-wise - but not CPU- or I/O-wise since 
it's still the same amount of work to be done. 11206 "just" reduces GC pressure.

Flushes also benefit, since it can "forget" IndexInfo objects sooner.

This ticket will *not* raise the limit on cells.

[~doanduyhai], you're right. Having the ability to handle big partitions has a 
direct influence to data modeling. I'd not say "you are not longer limited by 
the size of your partitions". This ticket _raises_ the current limitation WRT 
GC pressure and read performance. In theory the limit went away, but as you 
say, compaction gets even more important and other operational tasks like 
replacing nodes or changing topology need to be considered.

My next steps are:
* fix some unit tests that no longer work as they relied on the old 
implementation (expected to have IndexInfo on heap)
* cleanup the code
* run some tests on cstar

I only ran a poor-man's comparison - on my laptop with small-ish 3G heap with 
default unit test settings. That's why I did not note exact numbers. But I'd 
like to show the GC pressure of the same test ran against trunk (took 3 hours) 
and 11206 (took 1 hour):

!trunk-gc.png|GC on current trunk, width=800!

!11206-gc.png|GC on 11206, width=800!



was (Author: snazy):
I just finished with most of the coding for this ticket - i.e. "shallow" 
RowIndexEntry without {{IndexInfo}} - and ran a poor-man's comparison of 
current trunk against 11206 using different partition sizes covering writes, a 
major compaction and reads. The results are really promising especially with 
big and huge partitions (tested up to 8G partitions).

Reads against big partitions really benefit from 11206. For example, with 11206 
it takes a couple of seconds for 5000 random reads against 8G partitions vs. 
many minutes (not a typo) on current trunk). At the same time, the heap is 
quite full and causes a lot of GC pressure.

Compactions also benefit from 11206 GC-wise - but not CPU- or I/O-wise since 
it's still the same amount of work to be done. 11206 "just" reduces GC pressure.

Flushes also benefit, since it can "forget" IndexInfo objects sooner.

This ticket will *not* raise the limit on cells.

[~doanduyhai], you're right. Having the ability to handle big partitions has a 
direct influence to data modeling. I'd not say "you are not longer limited by 
the size of your partitions". This ticket _raises_ the current limitation WRT 
GC pressure and read performance. In theory the limit went away, but as you 
say, compaction gets even more important and other operational tasks like 
replacing nodes or changing topology need to be considered.

My next steps are:
* fix some unit tests that no longer work as they relied on the old 
implementation (expected to have IndexInfo on heap)
* cleanup the code
* run some tests on cstar

I only ran a poor-man's comparison - on my laptop with small-ish 3G heap with 
default unit test settings. That's why I did not note exact numbers. But I'd 
like to show the GC pressure of the same test ran against trunk and 11206:

!trunk-gc.png|GC on current trunk!

!11206-gc.png|GC on 11206!


> Support large partitions on the 3.0 sstable format
> --
>
> Key: CASSANDRA-11206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11206
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
> Fix For: 3.x
>
> Attachments: 11206-gc.png, trunk-gc.png
>
>
> Cassandra saves a sample of IndexInfo objects that store the offset within 
> each partition of every 64KB (by default) range of rows.  To find a row, we 
> binary search this sample, then scan the partition of the appropriate range.
> The problem is that this scales poorly as partitions grow: on a cache miss, 
> we deserialize the entire set of IndexInfo, which both creates a lot of GC 
> overhead (as noted in CASSANDRA-9754) but is also non-negligible 

[jira] [Comment Edited] (CASSANDRA-11206) Support large partitions on the 3.0 sstable format

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203258#comment-15203258
 ] 

Robert Stupp edited comment on CASSANDRA-11206 at 3/20/16 11:59 AM:


I just finished with most of the coding for this ticket - i.e. "shallow" 
RowIndexEntry without {{IndexInfo}} - and ran a poor-man's comparison of 
current trunk against 11206 using different partition sizes covering writes, a 
major compaction and reads. The results are really promising especially with 
big and huge partitions (tested up to 8G partitions).

Reads against big partitions really benefit from 11206. For example, with 11206 
it takes a couple of seconds for 5000 random reads against 8G partitions vs. 
many minutes (not a typo) on current trunk). At the same time, the heap is 
quite full and causes a lot of GC pressure.

Compactions also benefit from 11206 GC-wise - but not CPU- or I/O-wise since 
it's still the same amount of work to be done. 11206 "just" reduces GC pressure.

Flushes also benefit, since it can "forget" IndexInfo objects sooner.

This ticket will *not* raise the limit on cells.

[~doanduyhai], you're right. Having the ability to handle big partitions has a 
direct influence to data modeling. I'd not say "you are not longer limited by 
the size of your partitions". This ticket _raises_ the current limitation WRT 
GC pressure and read performance. In theory the limit went away, but as you 
say, compaction gets even more important and other operational tasks like 
replacing nodes or changing topology need to be considered.

My next steps are:
* fix some unit tests that no longer work as they relied on the old 
implementation (expected to have IndexInfo on heap)
* cleanup the code
* run some tests on cstar

I only ran a poor-man's comparison - on my laptop with small-ish 3G heap with 
default unit test settings. That's why I did not note exact numbers. But I'd 
like to show the GC pressure of the same test ran against trunk and 11206:

!trunk-gc.png|GC on current trunk!

!11206-gc.png|GC on 11206!



was (Author: snazy):
I just finished with most of the coding for this ticket - i.e. "shallow" 
RowIndexEntry without {{IndexInfo}} - and ran a poor-man's comparison of 
current trunk against 11206 using different partition sizes covering writes, a 
major compaction and reads. The results are really promising especially with 
big and huge partitions (tested up to 8G partitions).

Reads against big partitions really benefit from 11206. For example, with 11206 
it takes a couple of seconds for 5000 random reads against 8G partitions vs. 
many minutes (not a typo) on current trunk). At the same time, the heap is 
quite full and causes a lot of GC pressure.

Compactions also benefit from 11206 GC-wise - but not CPU- or I/O-wise since 
it's still the same amount of work to be done. 11206 "just" reduces GC pressure.

Flushes also benefit, since it can "forget" IndexInfo objects sooner.

This ticket will *not* raise the limit on cells.

[~doanduyhai], you're right. Having the ability to handle big partitions has a 
direct influence to data modeling. I'd not say "you are not longer limited by 
the size of your partitions". This ticket _raises_ the current limitation WRT 
GC pressure and read performance. In theory the limit went away, but as you 
say, compaction gets even more important and other operational tasks like 
replacing nodes or changing topology need to be considered.

My next steps are:
* fix some unit tests that no longer work as they relied on the old 
implementation (expected to have IndexInfo on heap)
* cleanup the code
* run some tests on cstar

I only ran a poor-man's comparison - on my laptop with small-ish 3G heap with 
default unit test settings. That's why I did not note exact numbers. But I'd 
like to show the GC pressure of the same test ran against trunk and 11206:

!trunk-gc.png!GC on current trunk!

!11206-gc.png!GC on 11206!


> Support large partitions on the 3.0 sstable format
> --
>
> Key: CASSANDRA-11206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11206
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
> Fix For: 3.x
>
> Attachments: 11206-gc.png, trunk-gc.png
>
>
> Cassandra saves a sample of IndexInfo objects that store the offset within 
> each partition of every 64KB (by default) range of rows.  To find a row, we 
> binary search this sample, then scan the partition of the appropriate range.
> The problem is that this scales poorly as partitions grow: on a cache miss, 
> we deserialize the entire set of IndexInfo, which both creates a lot of GC 
> overhead (as noted in CASSANDRA-9754) but is also non-negligible i/o activity 
> (relative to reading a single 64KB 

[jira] [Updated] (CASSANDRA-11206) Support large partitions on the 3.0 sstable format

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-11206:
-
Attachment: trunk-gc.png
11206-gc.png

I just finished with most of the coding for this ticket - i.e. "shallow" 
RowIndexEntry without {{IndexInfo}} - and ran a poor-man's comparison of 
current trunk against 11206 using different partition sizes covering writes, a 
major compaction and reads. The results are really promising especially with 
big and huge partitions (tested up to 8G partitions).

Reads against big partitions really benefit from 11206. For example, with 11206 
it takes a couple of seconds for 5000 random reads against 8G partitions vs. 
many minutes (not a typo) on current trunk). At the same time, the heap is 
quite full and causes a lot of GC pressure.

Compactions also benefit from 11206 GC-wise - but not CPU- or I/O-wise since 
it's still the same amount of work to be done. 11206 "just" reduces GC pressure.

Flushes also benefit, since it can "forget" IndexInfo objects sooner.

This ticket will *not* raise the limit on cells.

[~doanduyhai], you're right. Having the ability to handle big partitions has a 
direct influence to data modeling. I'd not say "you are not longer limited by 
the size of your partitions". This ticket _raises_ the current limitation WRT 
GC pressure and read performance. In theory the limit went away, but as you 
say, compaction gets even more important and other operational tasks like 
replacing nodes or changing topology need to be considered.

My next steps are:
* fix some unit tests that no longer work as they relied on the old 
implementation (expected to have IndexInfo on heap)
* cleanup the code
* run some tests on cstar

I only ran a poor-man's comparison - on my laptop with small-ish 3G heap with 
default unit test settings. That's why I did not note exact numbers. But I'd 
like to show the GC pressure of the same test ran against trunk and 11206:

!trunk-gc.png!GC on current trunk!

!11206-gc.png!GC on 11206!


> Support large partitions on the 3.0 sstable format
> --
>
> Key: CASSANDRA-11206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11206
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
> Fix For: 3.x
>
> Attachments: 11206-gc.png, trunk-gc.png
>
>
> Cassandra saves a sample of IndexInfo objects that store the offset within 
> each partition of every 64KB (by default) range of rows.  To find a row, we 
> binary search this sample, then scan the partition of the appropriate range.
> The problem is that this scales poorly as partitions grow: on a cache miss, 
> we deserialize the entire set of IndexInfo, which both creates a lot of GC 
> overhead (as noted in CASSANDRA-9754) but is also non-negligible i/o activity 
> (relative to reading a single 64KB row range) as partitions get truly large.
> We introduced an "offset map" in CASSANDRA-10314 that allows us to perform 
> the IndexInfo bsearch while only deserializing IndexInfo that we need to 
> compare against, i.e. log(N) deserializations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-4763) SSTableLoader shouldn't get keyspace from path

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197521#comment-15197521
 ] 

Yuki Morishita commented on CASSANDRA-4763:
---

After CASSANDRA-6962, SSTable file does not have keyspace and table name 
anymore.
So inferring keyspace and table name from file name is not possible.

But there maybe a use case that users want to load SSTables to different 
keyspace and table(of course, this may fail because of incompatibility).
Then, specifying keyspace and table name through command line option maybe 
useful.

> SSTableLoader shouldn't get keyspace from path
> --
>
> Key: CASSANDRA-4763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4763
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Affects Versions: 1.2.0 beta 1
>Reporter: Nick Bailey
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 2.1.x
>
>
> SSTableLoader currently gets the keyspace it is going to load to from the 
> path of the directoy of sstables it is loading. This isn't really documented 
> (or I didn't see it), but also isn't really a good way of doing it in general.
> {noformat}
> this.keyspace = directory.getParentFile().getName();
> {noformat}
> We should probably just let users pass the name in. If you are loading a 
> snapshot the file names will have the keyspace which is slightly better but 
> people manually creating their own sstables might not format them the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11142) Confusing error message on schema updates when nodes are down

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-11142:

Labels: cqlsh  (was: )

> Confusing error message on schema updates when nodes are down
> -
>
> Key: CASSANDRA-11142
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11142
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: PROD
>Reporter: Anubhav Kale
>Assignee: Tyler Hobbs
>Priority: Minor
>  Labels: cqlsh
>
> Repro steps are as follows (this was tested on Windows and is a consistent 
> repro)
> . Start a two node cluster.
> . Ensure that "nodetool status" shows both nodes as UN on both nodes
> . Stop Node2
> . Ensure that "nodetool status" shows that Node2 in DN.
> . Start cqlsh on Node1
> . Create a table
> . cqlsh times out with below message (coming from .py)
> Warning: schema version mismatch detected, which might be caused by DOWN 
> nodes; if this is not the case, check the schema versions of your nodes in 
> system.local and system.peers.
> OperationTimedOut: errors={}, last_host=10.1.0.10
> . Do a select * on the table that just timed out. It works fine.
> It just seems odd that there are no errors, but the table gets created fine. 
> We should either fix the timeout exception with a real error or not throw 
> timeout. Not sure what the best approach is.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11206) Support large partitions on the 3.0 sstable format

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201135#comment-15201135
 ] 

Stefania commented on CASSANDRA-11206:
--

Thank you!

> Support large partitions on the 3.0 sstable format
> --
>
> Key: CASSANDRA-11206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11206
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
> Fix For: 3.x
>
>
> Cassandra saves a sample of IndexInfo objects that store the offset within 
> each partition of every 64KB (by default) range of rows.  To find a row, we 
> binary search this sample, then scan the partition of the appropriate range.
> The problem is that this scales poorly as partitions grow: on a cache miss, 
> we deserialize the entire set of IndexInfo, which both creates a lot of GC 
> overhead (as noted in CASSANDRA-9754) but is also non-negligible i/o activity 
> (relative to reading a single 64KB row range) as partitions get truly large.
> We introduced an "offset map" in CASSANDRA-10314 that allows us to perform 
> the IndexInfo bsearch while only deserializing IndexInfo that we need to 
> compare against, i.e. log(N) deserializations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-3590) Use multiple connection to share the OutboutTCPConnection

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-3590.
--
Resolution: Duplicate

Superseded by CASSANDRA-8457 which is now in progress.

> Use multiple connection to share the OutboutTCPConnection
> -
>
> Key: CASSANDRA-3590
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3590
> Project: Cassandra
>  Issue Type: Improvement
>Affects Versions: 1.1.0
>Reporter: Vijay
>Priority: Minor
> Attachments: TCPTest.xlsx, TCPTest.zip
>
>
> Currently there is one connection between any given host to another host in 
> the cluster, the problem with this is:
> 1) This can become a bottleneck in some cases where the latencies are higher.
> 2) When a connection is dropped we also drop the queue and recreate a new one 
> and hence the messages can be lost (Currently hints will take care of it and 
> clients also can retry)
> by making it a configurable option to configure the number of connections and 
> also making the queue common to those connections the above 2 issues can be 
> resolved.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11383) SASI index build leads to massive OOM

[DOAN DuyHai (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203243#comment-15203243
 ] 

DOAN DuyHai commented on CASSANDRA-11383:
-

bq. and numerical indexes are not required to be always marked as SPARSE

 I missed this. I thought that mode {{PREFIX}} is only reserved to {{text}} 
types



> SASI index build leads to massive OOM
> -
>
> Key: CASSANDRA-11383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11383
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: C* 3.4
>Reporter: DOAN DuyHai
> Attachments: CASSANDRA-11383.patch, 
> SASI_Index_build_LCS_1G_Max_SSTable_Size_logs.tar.gz, 
> new_system_log_CMS_8GB_OOM.log, system.log_sasi_build_oom
>
>
> 13 bare metal machines
> - 6 cores CPU (12 HT)
> - 64Gb RAM
> - 4 SSD in RAID0
>  JVM settings:
> - G1 GC
> - Xms32G, Xmx32G
> Data set:
>  - ≈ 100Gb/per node
>  - 1.3 Tb cluster-wide
>  - ≈ 20Gb for all SASI indices
> C* settings:
> - concurrent_compactors: 1
> - compaction_throughput_mb_per_sec: 256
> - memtable_heap_space_in_mb: 2048
> - memtable_offheap_space_in_mb: 2048
> I created 9 SASI indices
>  - 8 indices with text field, NonTokenizingAnalyser,  PREFIX mode, 
> case-insensitive
>  - 1 index with numeric field, SPARSE mode
>  After a while, the nodes just gone OOM.
>  I attach log files. You can see a lot of GC happening while index segments 
> are flush to disk. At some point the node OOM ...
> /cc [~xedin]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11383) SASI index build leads to massive OOM

[DOAN DuyHai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

DOAN DuyHai updated CASSANDRA-11383:

Attachment: SASI_Index_build_LCS_1G_Max_SSTable_Size_logs.tar.gz

Attached are the logs (*SASI_Index_build_LCS_1G_Max_SSTable_Size_logs.tar.gz*)  
of an idle cluster just building the {{periode_end_month}} index with a lot of 
GC

> SASI index build leads to massive OOM
> -
>
> Key: CASSANDRA-11383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11383
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: C* 3.4
>Reporter: DOAN DuyHai
> Attachments: CASSANDRA-11383.patch, 
> SASI_Index_build_LCS_1G_Max_SSTable_Size_logs.tar.gz, 
> new_system_log_CMS_8GB_OOM.log, system.log_sasi_build_oom
>
>
> 13 bare metal machines
> - 6 cores CPU (12 HT)
> - 64Gb RAM
> - 4 SSD in RAID0
>  JVM settings:
> - G1 GC
> - Xms32G, Xmx32G
> Data set:
>  - ≈ 100Gb/per node
>  - 1.3 Tb cluster-wide
>  - ≈ 20Gb for all SASI indices
> C* settings:
> - concurrent_compactors: 1
> - compaction_throughput_mb_per_sec: 256
> - memtable_heap_space_in_mb: 2048
> - memtable_offheap_space_in_mb: 2048
> I created 9 SASI indices
>  - 8 indices with text field, NonTokenizingAnalyser,  PREFIX mode, 
> case-insensitive
>  - 1 index with numeric field, SPARSE mode
>  After a while, the nodes just gone OOM.
>  I attach log files. You can see a lot of GC happening while index segments 
> are flush to disk. At some point the node OOM ...
> /cc [~xedin]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11383) SASI index build leads to massive OOM

[Pavel Yaskevich (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203226#comment-15203226
 ] 

Pavel Yaskevich commented on CASSANDRA-11383:
-

[~doanduyhai] It looks like to me that currently SPARSE creates more confusion 
than good, I'm going to remove that mode as part of this patch and will look 
into maybe having different index type for columns like timestamp if that 
proves to be a problem for people.

> SASI index build leads to massive OOM
> -
>
> Key: CASSANDRA-11383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11383
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: C* 3.4
>Reporter: DOAN DuyHai
> Attachments: CASSANDRA-11383.patch, new_system_log_CMS_8GB_OOM.log, 
> system.log_sasi_build_oom
>
>
> 13 bare metal machines
> - 6 cores CPU (12 HT)
> - 64Gb RAM
> - 4 SSD in RAID0
>  JVM settings:
> - G1 GC
> - Xms32G, Xmx32G
> Data set:
>  - ≈ 100Gb/per node
>  - 1.3 Tb cluster-wide
>  - ≈ 20Gb for all SASI indices
> C* settings:
> - concurrent_compactors: 1
> - compaction_throughput_mb_per_sec: 256
> - memtable_heap_space_in_mb: 2048
> - memtable_offheap_space_in_mb: 2048
> I created 9 SASI indices
>  - 8 indices with text field, NonTokenizingAnalyser,  PREFIX mode, 
> case-insensitive
>  - 1 index with numeric field, SPARSE mode
>  After a while, the nodes just gone OOM.
>  I attach log files. You can see a lot of GC happening while index segments 
> are flush to disk. At some point the node OOM ...
> /cc [~xedin]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11368) List of UDT can't be updated properly when using USING TIMESTAMP

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199467#comment-15199467
 ] 

Sylvain Lebresne commented on CASSANDRA-11368:
--

Yes, I can agree that behavior is surprising and it means list inserts are not 
truly idempotent. This is however a downside of the design of lists and to be 
perfectly honest, I'm not sure there is an easy fix for that (though if someone 
has an idea, please share). So I'm not saying it wouldn't be great if this was 
made idempotent, but I'd rather be upfront that unless has a very clever idea, 
this is likely to stay as a known limitation of lists for the foreseeable 
future. Fyi, this is not the only gotcha of lists and we generally advise to 
prefer sets over lists unless you absolutely absolutely need the ordering. And 
even then, make sure a frozen list (which doesn't have this problem) isn't good 
enough for you.

I'll do note that the issue is due to having both statements having the exact 
same timestamp. If you'd use a bigger timestamp for the 2nd insert for 
instance, this would work as expected. The use of UDT has also no impact on 
this. So updating the title to reflect both of those.

> List of UDT can't be updated properly when using USING TIMESTAMP
> 
>
> Key: CASSANDRA-11368
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11368
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Thanh
>
> List of UDT can't be updated properly when using USING TIMESTAMP
> Observe:
> {code}
> cqlsh:t360> CREATE TYPE fullname ( 
> ... fname text, 
> ... lname text 
> ... );
> cqlsh:t360> CREATE TABLE users ( 
> ... id text PRIMARY KEY, 
> ... names list>, 
> ... phone text 
> ... ); 
> cqlsh:t360> UPDATE users USING TIMESTAMP 1458019725701 SET names = [{ fname: 
> 'fname1', lname: 'lname1'},{ fname: 'fname2', lname: 'lname2'},{ fname: 
> 'fname3', lname: 'lname3'}] WHERE id='a'; 
> cqlsh:t360> select * from users;
> id | names | phone 
> +--+---
>  
> a | [{lname: 'lname1', fname: 'fname1'}, {lname: 'lname2', fname: 'fname2'}, 
> {lname: 'lname3', fname: 'fname3'}] | null
> (1 rows) 
> cqlsh:t360> UPDATE users USING TIMESTAMP 1458019725701 SET names = [{ fname: 
> 'fname1', lname: 'lname1'},{ fname: 'fname2', lname: 'lname2'},{ fname: 
> 'fname3', lname: 'lname3'}] WHERE id='a'; 
> cqlsh:t360> select * from users;
> id | names | phone 
> +--+---
>  
> a | [{lname: 'lname1', fname: 'fname1'}, {lname: 'lname2', fname: 'fname2'}, 
> {lname: 'lname3', fname: 'fname3'}, {lname: 'lname1', fname: 'fname1'}, 
> {lname: 'lname2', fname: 'fname2'}, {lname: 'lname3', fname: 'fname3'}] | null
> (1 rows)
> {code}
> => the list doesn't get replaced, it gets appended, which is not the 
> expected/desired result



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11378) dtest failure in topology_test.TestTopology.decommissioned_node_cant_rejoin_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-11378:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

> dtest failure in 
> topology_test.TestTopology.decommissioned_node_cant_rejoin_test
> 
>
> Key: CASSANDRA-11378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11378
> Project: Cassandra
>  Issue Type: Test
>Reporter: Jim Witschey
>Assignee: Philip Thompson
>  Labels: dtest
>
> This fails hard on 3.0:
> http://cassci.datastax.com/job/cassandra-3.0_novnode_dtest/186/testReport/topology_test/TestTopology/decommissioned_node_cant_rejoin_test
> Here's the changeset from the first failure on that job:
> http://cassci.datastax.com/job/cassandra-3.0_novnode_dtest/186/changes
> (just this commit: 
> https://github.com/apache/cassandra/commit/719caa67649bf6f27cdd99dd7d6055d2aa8546ae)
> It has failed on trunk as well:
> http://cassci.datastax.com/view/trunk/job/trunk_novnode_dtest/lastCompletedBuild/testReport/topology_test/TestTopology/decommissioned_node_cant_rejoin_test/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[9/9] cassandra git commit: Merge branch 'cassandra-3.5' into trunk

[slebresne]

Merge branch 'cassandra-3.5' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/700dd0e5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/700dd0e5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/700dd0e5

Branch: refs/heads/trunk
Commit: 700dd0e59f98433e7e1242cff745c56979ce1629
Parents: 68e6a83 ef86d91
Author: Sylvain Lebresne 
Authored: Fri Mar 18 10:52:26 2016 +0100
Committer: Sylvain Lebresne 
Committed: Fri Mar 18 10:52:26 2016 +0100

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 7 ++-
 2 files changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/700dd0e5/CHANGES.txt
--

[jira] [Commented] (CASSANDRA-11383) SASI index build leads to massive OOM

[Pavel Yaskevich (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203216#comment-15203216
 ] 

Pavel Yaskevich commented on CASSANDRA-11383:
-

[~doanduyhai] Yes I think I will be able to at least minimize memory 
requirement for the stitching stage that should be able to solve most of it and 
once again - "period_end_month_int" is *not* a SPARSE index and numerical 
indexes are not required to be always marked as SPARSE only in cases where each 
(or most) of the keys have unique value for such columns e.g. timestamp where 
each key is going to have milli-/micro-second value which is almost guaranteed 
to be unique for every given row.

> SASI index build leads to massive OOM
> -
>
> Key: CASSANDRA-11383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11383
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: C* 3.4
>Reporter: DOAN DuyHai
> Attachments: CASSANDRA-11383.patch, new_system_log_CMS_8GB_OOM.log, 
> system.log_sasi_build_oom
>
>
> 13 bare metal machines
> - 6 cores CPU (12 HT)
> - 64Gb RAM
> - 4 SSD in RAID0
>  JVM settings:
> - G1 GC
> - Xms32G, Xmx32G
> Data set:
>  - ≈ 100Gb/per node
>  - 1.3 Tb cluster-wide
>  - ≈ 20Gb for all SASI indices
> C* settings:
> - concurrent_compactors: 1
> - compaction_throughput_mb_per_sec: 256
> - memtable_heap_space_in_mb: 2048
> - memtable_offheap_space_in_mb: 2048
> I created 9 SASI indices
>  - 8 indices with text field, NonTokenizingAnalyser,  PREFIX mode, 
> case-insensitive
>  - 1 index with numeric field, SPARSE mode
>  After a while, the nodes just gone OOM.
>  I attach log files. You can see a lot of GC happening while index segments 
> are flush to disk. At some point the node OOM ...
> /cc [~xedin]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11206) Support large partitions on the 3.0 sstable format

[DOAN DuyHai (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203187#comment-15203187
 ] 

DOAN DuyHai commented on CASSANDRA-11206:
-

I have some questions related to the outcome of this JIRA.

 Since 2.1 incremental repair only repairs *chunks* of a partition (e.g. the 
chunks that are in the un-repaired SSTables set) so even in case of mismatch we 
no longer stream the *entire* partition. And using paging we can read through 
very wide partitions. With the improvement brought by this JIRA, does it mean 
that now we can handle *virtually* unbounded or partitions exceeding 2.10^9 
physical columns ?

 I'm asking because it will impact greatly the way we model data. There are 
still some points that can cause trouble with ultra-wide partitions:

 - bootstrapping/adding new nodes to the cluster --> streaming of an ultra-wide 
partitions. What happens if the streaming fails in the middle ? Do we restart 
the streaming of the whole partition or can we *resume* at the last clustering ?
 - compaction. With LCS, ultra wide partitions can create overly huge SSTables. 
In general, how compaction ultra wide partitions will impact node stability ? 
 - read path with STCS --> more SSTables to touch on disk

> Support large partitions on the 3.0 sstable format
> --
>
> Key: CASSANDRA-11206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11206
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
> Fix For: 3.x
>
>
> Cassandra saves a sample of IndexInfo objects that store the offset within 
> each partition of every 64KB (by default) range of rows.  To find a row, we 
> binary search this sample, then scan the partition of the appropriate range.
> The problem is that this scales poorly as partitions grow: on a cache miss, 
> we deserialize the entire set of IndexInfo, which both creates a lot of GC 
> overhead (as noted in CASSANDRA-9754) but is also non-negligible i/o activity 
> (relative to reading a single 64KB row range) as partitions get truly large.
> We introduced an "offset map" in CASSANDRA-10314 that allows us to perform 
> the IndexInfo bsearch while only deserializing IndexInfo that we need to 
> compare against, i.e. log(N) deserializations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10649) Improve field-checking and error reporting in cassandra.yaml

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10649:
---
Attachment: 10649-3.0.txt
10649-2.2.txt

I could not reproduce the problem in {{2.2}} with the corrupted file. The only 
way I managed to create a NPE was by providing an empty configuration file in 
which case {{yaml.loadAs}} returned a null {{Config}} which caused a NPE later 
on when the {{Config}} was accessed.
If the returned file is empty, the patch creates a new {{Config}} instance. The 
validation will then work normally and mandatory parameters will be requested 
as expected.

> Improve field-checking and error reporting in cassandra.yaml
> 
>
> Key: CASSANDRA-10649
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10649
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration
> Environment: Linux: Fedora-16 64 bit
>Reporter: sandeep thakur
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.1.x
>
> Attachments: 10649-2.2.txt, 10649-3.0.txt, cassandra.yaml
>
>
> I am trying to setup cassandra single node cluster. i've downloaded below 
> build:
> apache-cassandra-2.1.11-bin.tar.gz
> I've upgraded Java to 1.8 as well, as earlier it was throwing errors related 
> to Java version.
> {code}
> [root@localhost cassandra]# java -version
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
> {code}
> I've also verified the cassandra.yaml file from "http://www.yamllint.com/; as 
> well. But while starting cassandra, I am getting vague exception as below:
> {code}
> INFO  15:52:11 Compacting 
> [SSTableReader(path='/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-18-Data.db'),
>  
> SSTableReader(path='/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-17-Data.db'),
>  
> SSTableReader(path='/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-20-Data.db'),
>  
> SSTableReader(path='/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-19-Data.db')]
> INFO  15:52:11 Node localhost/127.0.0.1 state jump to normal
> INFO  15:52:11 Netty using native Epoll event loop
> ERROR 15:52:11 Exception encountered during startup
> java.lang.NullPointerException: null
> at org.apache.cassandra.transport.Server.run(Server.java:171) 
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.transport.Server.start(Server.java:117) 
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:492) 
> [apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:575)
>  [apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:651) 
> [apache-cassandra-2.1.11.jar:2.1.11]
> java.lang.NullPointerException
> at org.apache.cassandra.transport.Server.run(Server.java:171)
> at org.apache.cassandra.transport.Server.start(Server.java:117)
> at 
> org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:492)
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:575)
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:651)
> Exception encountered during startup: null
> INFO  15:52:11 Announcing shutdown
> INFO  15:52:11 Node localhost/127.0.0.1 state jump to normal
> INFO  15:52:11 Compacted 4 sstables to 
> [/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-21,].
>   11,427 bytes to 5,749 (~50% of original) in 199ms = 0.027551MB/s.  4 total 
> partitions merged to 1.  Partition merge counts were {4:1, }
> INFO  15:52:13 Waiting for messaging service to quiesce
> INFO  15:52:13 MessagingService has terminated the accept() thread
> [root@localhost server]#
> {code}
> I've also posted the issue on stack overflow as well:
> http://stackoverflow.com/questions/33514745/cassandra-startup-failed-with-exception-exception-encountered-during-startup
> Request some one to assist on this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-5437) multiget specific key filtering

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-5437.
--
Resolution: Won't Fix

With Thrift deprecated now, closing the ticket.

> multiget specific key filtering
> ---
>
> Key: CASSANDRA-5437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5437
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Brooke Bryan
>Priority: Minor
>
> Currently, you are able to specify a list of columns, or a slice when pulling 
> back keys from a multiget, however, we find ourselves pulling back a lot more 
> data than we actually need when columns overlap.  We currently have 2 options 
> (as far as im aware).
> 1. perform multiple get calls, and get the required data back, but connect to 
> thrift more
> 2. perform a multiget, and get more data due to column crossovers, but in a 
> single request.
> Similar to a batch mutation, a batch read would solve this problem perfectly.
> e.g.
> read = [
>   'columnFamily' => [
> 'key' => ColumnPath,
> 'key2' => SlicePredicate,
> 'key3' => ColumnPath
>   ]
> ];
> result = batch_read(read);
> //result
> 'columnFamily' => [
> 'key' => [1 => 'a',2 => 'b',3 => 'c'],
> 'key2' => [4 => 'd',5 => 'e'],
> 'key3' => [1 => 'a',3 => 'c']
>   ]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-11383) SASI index build leads to massive OOM

[DOAN DuyHai (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203172#comment-15203172
 ] 

DOAN DuyHai edited comment on CASSANDRA-11383 at 3/20/16 8:58 AM:
--

[~xedin]

Last update from the testing. I put the cluster in *ideal* conditions as you 
recommended.

JVM settings:
 - CMS
 - Xmx8g, Xms8G
 
C* settings:
 - concurrent_compactors: 6

Test conditions:
 - cluster *idle* (no write, no read)
 - LCS with *sstable_size_in_mb* = 1024 (1Gb)
 - *no compaction ongoing* (took a whole night to compact for LCS)
 - {{CREATE CUSTOM INDEX resource_period_end_month_int_idx ON 
sharon.resource_bench (period_end_month_int) USING 
'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = \{'mode': 
'SPARSE'\};}}

Observations:
 - I/O idle, CPU not exceeding 20% on average 
(http://postimg.org/image/f664wm8dp/)
 - {{nodetool compactionstats}} only show 1 index rebuild ongoing per node  
{noformat}
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b4f4b0-ee6a-11e5-81f5-bd5584064785 Secondary index build sharon   
resource_bench 9535985318 18920482745 bytes 50.40%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b65440-ee6a-11e5-b44b-4deeb5ac98a3 Secondary index build sharon   
resource_bench 9464081317 20988668046 bytes 45.09%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b3bc30-ee6a-11e5-a152-db40f4fbe6b8 Secondary index build sharon   
resource_bench 9471325678 17061191471 bytes 55.51%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b45870-ee6a-11e5-b26b-53ed13e9667e Secondary index build sharon   
resource_bench 9120598050 18921737677 bytes 48.20%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b45870-ee6a-11e5-b2a3-331c04173c53 Secondary index build sharon   
resource_bench 8943568835 20591008789 bytes 43.43%
id   compaction type   keyspace table   
   completed   total   unit  progress
d8b47f80-ee6a-11e5-9fc8-0597212274c1 Secondary index build sharon   
resource_bench 10172038156 21422242706 bytes 47.48%
id   compaction type   keyspace table   
   completed   total   unit  progress
d8b34700-ee6a-11e5-a642-6dee841e75e5 Secondary index build sharon   
resource_bench 10161205385 18730171060 bytes 54.25%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b6f080-ee6a-11e5-8da4-bd70732fdab1 Secondary index build sharon   
resource_bench 9961529350 21294352899 bytes 46.78%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b43160-ee6a-11e5-8ac9-f59d626eedfa Secondary index build sharon   
resource_bench 9160286080 22153527929 bytes 41.35%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b51bc0-ee6a-11e5-8aa0-b9e611280aba Secondary index build sharon   
resource_bench 9397690505 22791700212 bytes 41.23%
id   compaction type   keyspace table   
   completed   total   unit  progress
d8b542d0-ee6a-11e5-8521-fbd14b018db6 Secondary index build sharon   
resource_bench 10029096174 18910334578 bytes 53.04%
id   compaction type   keyspace table   
   completed   total   unit  progress
d8b40a50-ee6a-11e5-a7b2-4b114ced0935 Secondary index build sharon   
resource_bench 10118551269 16938426769 bytes 59.74%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b039c0-ee6a-11e5-9a98-ff9a6f2af762 Secondary index build sharon   
resource_bench 9003236945 18252472495 bytes 49.33%
{noformat}
 - *there are still A LOT of GC* (with some ConcurrentMarkSweep lasting up to 
10 secs!)
{noformat}
INFO  [Service Thread] 2016-03-20 09:46:44,695 GCInspector.java:284 - ParNew GC 
in 455ms.  CMS Old Gen: 2964960608 -> 3487392640; Par Eden Space: 1006632960 -> 
0;
INFO  [Service Thread] 2016-03-20 09:46:47,250 GCInspector.java:284 - ParNew GC 
in 460ms.  CMS Old Gen: 3487392640 -> 3990379824; Par Eden Space: 1006632960 -> 
0; Par Survivor Space: 125829120 -> 125828160
INFO  [Service Thread] 2016-03-20 09:46:49,452 GCInspector.java:284 - ParNew GC 
in 414ms.  CMS Old Gen: 3990379824 -> 4445691424; Par Eden Space: 1006632960 -> 
0; Par Survivor Space: 125828160 -> 125827840
INFO  [Service Thread] 2016-03-20 09:46:52,328 GCInspector.java:284 - ParNew GC 
in 484ms.  CMS Old Gen: 4445691424 

[jira] [Commented] (CASSANDRA-11383) SASI index build leads to massive OOM

[DOAN DuyHai (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203172#comment-15203172
 ] 

DOAN DuyHai commented on CASSANDRA-11383:
-

[~xedin]

Last update from the testing. I put the cluster in *ideal* conditions as you 
recommended.

JVM settings:
 - CMS
 - Xmx8g, Xms8G
 
C* settings:
 - concurrent_compactors: 6

Test conditions:
 - cluster *idle* (no write, no read)
 - LCS with *sstable_size_in_mb* = 1024 (1Gb)
 - *no compaction ongoing* (took a whole night to compact for LCS)
 - {{CREATE CUSTOM INDEX resource_period_end_month_int_idx ON 
sharon.resource_bench (period_end_month_int) USING 
'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = \{'mode': 
'SPARSE'\};}}

Observations:
 - I/O idle, CPU not exceeding 20% on average 
(http://postimg.org/image/f664wm8dp/)
 - {{nodetool compactionstats}} only show 1 index rebuild ongoing per node  
{noformat}
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b4f4b0-ee6a-11e5-81f5-bd5584064785 Secondary index build sharon   
resource_bench 9535985318 18920482745 bytes 50.40%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b65440-ee6a-11e5-b44b-4deeb5ac98a3 Secondary index build sharon   
resource_bench 9464081317 20988668046 bytes 45.09%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b3bc30-ee6a-11e5-a152-db40f4fbe6b8 Secondary index build sharon   
resource_bench 9471325678 17061191471 bytes 55.51%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b45870-ee6a-11e5-b26b-53ed13e9667e Secondary index build sharon   
resource_bench 9120598050 18921737677 bytes 48.20%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b45870-ee6a-11e5-b2a3-331c04173c53 Secondary index build sharon   
resource_bench 8943568835 20591008789 bytes 43.43%
id   compaction type   keyspace table   
   completed   total   unit  progress
d8b47f80-ee6a-11e5-9fc8-0597212274c1 Secondary index build sharon   
resource_bench 10172038156 21422242706 bytes 47.48%
id   compaction type   keyspace table   
   completed   total   unit  progress
d8b34700-ee6a-11e5-a642-6dee841e75e5 Secondary index build sharon   
resource_bench 10161205385 18730171060 bytes 54.25%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b6f080-ee6a-11e5-8da4-bd70732fdab1 Secondary index build sharon   
resource_bench 9961529350 21294352899 bytes 46.78%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b43160-ee6a-11e5-8ac9-f59d626eedfa Secondary index build sharon   
resource_bench 9160286080 22153527929 bytes 41.35%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b51bc0-ee6a-11e5-8aa0-b9e611280aba Secondary index build sharon   
resource_bench 9397690505 22791700212 bytes 41.23%
id   compaction type   keyspace table   
   completed   total   unit  progress
d8b542d0-ee6a-11e5-8521-fbd14b018db6 Secondary index build sharon   
resource_bench 10029096174 18910334578 bytes 53.04%
id   compaction type   keyspace table   
   completed   total   unit  progress
d8b40a50-ee6a-11e5-a7b2-4b114ced0935 Secondary index build sharon   
resource_bench 10118551269 16938426769 bytes 59.74%
id   compaction type   keyspace table   
   completed  total   unit  progress
d8b039c0-ee6a-11e5-9a98-ff9a6f2af762 Secondary index build sharon   
resource_bench 9003236945 18252472495 bytes 49.33%
{noformat}
 - *there are still A LOT of GC*
{noformat}
INFO  [Service Thread] 2016-03-20 09:46:44,695 GCInspector.java:284 - ParNew GC 
in 455ms.  CMS Old Gen: 2964960608 -> 3487392640; Par Eden Space: 1006632960 -> 
0;
INFO  [Service Thread] 2016-03-20 09:46:47,250 GCInspector.java:284 - ParNew GC 
in 460ms.  CMS Old Gen: 3487392640 -> 3990379824; Par Eden Space: 1006632960 -> 
0; Par Survivor Space: 125829120 -> 125828160
INFO  [Service Thread] 2016-03-20 09:46:49,452 GCInspector.java:284 - ParNew GC 
in 414ms.  CMS Old Gen: 3990379824 -> 4445691424; Par Eden Space: 1006632960 -> 
0; Par Survivor Space: 125828160 -> 125827840
INFO  [Service Thread] 2016-03-20 09:46:52,328 GCInspector.java:284 - ParNew GC 
in 484ms.  CMS Old Gen: 4445691424 -> 4968532112; Par Eden Space: 1006632960 -> 
0; Par Survivor Space: 125827840 -> 125827440
INFO  [Service 

[jira] [Comment Edited] (CASSANDRA-11381) Node running with join_ring=false and authentication can not serve requests

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202064#comment-15202064
 ] 

mck edited comment on CASSANDRA-11381 at 3/20/16 8:29 AM:
--

[~JoshuaMcKenzie], I've only run unit tests against trunk. And I'm trying to 
figure out how to write a dtest for this (since unit tests and system property 
don't mix (afaik)).

If you're happy to proceed without the dtest, say and I'll 'submit patch'. And 
I do want to double-check that it's not possible to create auth tables without 
even the system tables having been created (eg bootstrap & join_ring=false).


was (Author: michaelsembwever):
@JoshuaMcKenzie, I've only run unit tests against trunk. And I'm trying to 
figure out how to write a dtest for this (since unit tests and system property 
don't mix (afaik)).

If you're happy to proceed without the dtest, say and I'll 'submit patch'. And 
I do want to double-check that it's not possible to create auth tables without 
even the system tables having been created (eg bootstrap & join_ring=false).

> Node running with join_ring=false and authentication can not serve requests
> ---
>
> Key: CASSANDRA-11381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11381
> Project: Cassandra
>  Issue Type: Bug
>Reporter: mck
>Assignee: mck
> Attachments: 11381-2.0.txt, 11381-2.1.txt, 11381-2.2.txt, 
> 11381-3.0.txt, 11381-trunk.txt
>
>
> Starting up a node with {{-Dcassandra.join_ring=false}} in a cluster that has 
> authentication configured, eg PasswordAuthenticator, won't be able to serve 
> requests. This is because {{Auth.setup()}} never gets called during the 
> startup.
> Without {{Auth.setup()}} having been called in {{StorageService}} clients 
> connecting to the node fail with the node throwing
> {noformat}
> java.lang.NullPointerException
> at 
> org.apache.cassandra.auth.PasswordAuthenticator.authenticate(PasswordAuthenticator.java:119)
> at 
> org.apache.cassandra.thrift.CassandraServer.login(CassandraServer.java:1471)
> at 
> org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3505)
> at 
> org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3489)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at com.thinkaurelius.thrift.Message.invoke(Message.java:314)
> at 
> com.thinkaurelius.thrift.Message$Invocation.execute(Message.java:90)
> at 
> com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:695)
> at 
> com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:689)
> at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:112)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The exception thrown from the 
> [code|https://github.com/apache/cassandra/blob/cassandra-2.0.16/src/java/org/apache/cassandra/auth/PasswordAuthenticator.java#L119]
> {code}
> ResultMessage.Rows rows = 
> authenticateStatement.execute(QueryState.forInternalCalls(), new 
> QueryOptions(consistencyForUser(username),
>   
>Lists.newArrayList(ByteBufferUtil.bytes(username;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10748) UTF8Validator.validate() wrong ??

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10748:
---
Status: Ready to Commit  (was: Patch Available)

> UTF8Validator.validate() wrong ??
> -
>
> Key: CASSANDRA-10748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10748
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.6, 3.0.5, 3.5
>
>
> The switch-statement in {{UTF8Validator.validate()}} can never go into {{case 
> TWO_80}} as the assignment {{state = State.TWO_80;}} in line 75 is dead.
> I assume that the {{TWO_80}} case is completely superfluous - but would like 
> to have a 2nd opinion on this.
> /cc [~carlyeks] (CASSANDRA-4495)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11370) Display sstable count per level according to repair status on nodetool tablestats

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-11370:

Reviewer: Marcus Eriksson

> Display sstable count per level according to repair status on nodetool 
> tablestats 
> --
>
> Key: CASSANDRA-11370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11370
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Paulo Motta
>Priority: Minor
>  Labels: lhf
>
> After CASSANDRA-8004 we still display sstables in each level on nodetool 
> tablestats as if we had a single compaction strategy, while we have one 
> strategy for repaired and another for unrepaired data. 
> We should split display into repaired and unrepaired set, so this:
> SSTables in each level: [2, 20/10, 15, 0, 0, 0, 0, 0, 0]
> Would become:
> SSTables in each level (repaired): [1, 10, 0, 0, 0, 0, 0, 0, 0]
> SSTables in each level (unrepaired): [1, 10, 15, 0, 0, 0, 0, 0, 0]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9935) Repair fails with RuntimeException

[Ruoran Wang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198847#comment-15198847
 ] 

Ruoran Wang commented on CASSANDRA-9935:


I cleaned the data in the cluseter, then generated some data and the error is 
reproduced again.

Here are part of those out of order errors. The first part of DecoratedKey is 
the Toke. In org.apache.cassandra.repair.Validator#add, there is an assertion 
{noformat}  assert lastKey == null || lastKey.compareTo(row.key) < 0 : "row " + 
row.key + " received out of order wrt " + lastKey; {noformat}, which is trying 
to make sure if the lastKey is not null lastKey should be smaller than 
currentkey. The compare method will try to compare token (if tokens of those 
two keys are equal, compare the byte), and all those following failures 
(including other failures posted above) are caused by lastKey.token >= 
currentKey.token. Not sure why that's happening.

{noformat}
java.lang.AssertionError: row DecoratedKey(-8369102073622366180, 
000400010801538239650100) received out of order wrt 
DecoratedKey(-8357216522748296009, 000408015382acbe8d00) column 
statsnull
java.lang.AssertionError: row DecoratedKey(-6257362846602517264, 
000408015382949a0500) received out of order wrt 
DecoratedKey(-6236290075537674781, 0004000108015382a27e9600) column 
statsnull
java.lang.AssertionError: row DecoratedKey(2478458424628257677, 
000400010801538271539a00) received out of order wrt 
DecoratedKey(2490779404447159202, 0004000108015382662a5000) column 
statsnull
java.lang.AssertionError: row DecoratedKey(8880802316577320376, 
0004000108015382821cf300) received out of order wrt 
DecoratedKey(8881355423529151128, 00040001080153829533b900) column 
statsnull
java.lang.AssertionError: row DecoratedKey(-1344138391208803679, 
00040001080153828d23dc00) received out of order wrt 
DecoratedKey(-1339348872117800450, 00040001080153829e30ea00) column 
statsnull
java.lang.AssertionError: row DecoratedKey(-3057182277536415874, 
00040801538286c44600) received out of order wrt 
DecoratedKey(-3053575924537508805, 00040801538294cb5a00) column 
statsnull
java.lang.AssertionError: row DecoratedKey(5646261254263909895, 
0004080153824e3a4f00) received out of order wrt 
DecoratedKey(5658365860829244661, 0004080153827dd3a600) column 
statsnull
{noformat}

> Repair fails with RuntimeException
> --
>
> Key: CASSANDRA-9935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9935
> Project: Cassandra
>  Issue Type: Bug
> Environment: C* 2.1.8, Debian Wheezy
>Reporter: mlowicki
>Assignee: Yuki Morishita
> Fix For: 2.1.x
>
> Attachments: db1.sync.lati.osa.cassandra.log, 
> db5.sync.lati.osa.cassandra.log, system.log.10.210.3.117, 
> system.log.10.210.3.221, system.log.10.210.3.230
>
>
> We had problems with slow repair in 2.1.7 (CASSANDRA-9702) but after upgrade 
> to 2.1.8 it started to work faster but now it fails with:
> {code}
> ...
> [2015-07-29 20:44:03,956] Repair session 23a811b0-3632-11e5-a93e-4963524a8bde 
> for range (-5474076923322749342,-5468600594078911162] finished
> [2015-07-29 20:44:03,957] Repair session 336f8740-3632-11e5-a93e-4963524a8bde 
> for range (-8631877858109464676,-8624040066373718932] finished
> [2015-07-29 20:44:03,957] Repair session 4ccd8430-3632-11e5-a93e-4963524a8bde 
> for range (-5372806541854279315,-5369354119480076785] finished
> [2015-07-29 20:44:03,957] Repair session 59f129f0-3632-11e5-a93e-4963524a8bde 
> for range (8166489034383821955,8168408930184216281] finished
> [2015-07-29 20:44:03,957] Repair session 6ae7a9a0-3632-11e5-a93e-4963524a8bde 
> for range (6084602890817326921,6088328703025510057] finished
> [2015-07-29 20:44:03,957] Repair session 8938e4a0-3632-11e5-a93e-4963524a8bde 
> for range (-781874602493000830,-781745173070807746] finished
> [2015-07-29 20:44:03,957] Repair command #4 finished
> error: nodetool failed, check server logs
> -- StackTrace --
> java.lang.RuntimeException: nodetool failed, check server logs
> at 
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:290)
> at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:202)
> {code}
> After running:
> {code}
> nodetool repair --partitioner-range --parallel --in-local-dc sync
> {code}
> Last records in logs regarding repair are:
> {code}
> INFO  [Thread-173887] 2015-07-29 20:44:03,956 StorageService.java:2952 - 
> Repair session 09ff9e40-3632-11e5-a93e-4963524a8bde for range 
> (-7695808664784761779,-7693529816291585568] finished
> INFO  [Thread-173887] 2015-07-29 20:44:03,956 StorageService.java:2952 - 
> Repair session 

[jira] [Updated] (CASSANDRA-4763) SSTableLoader shouldn't get keyspace from path

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-4763:
---
Assignee: Yuki Morishita

> SSTableLoader shouldn't get keyspace from path
> --
>
> Key: CASSANDRA-4763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4763
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Affects Versions: 1.2.0 beta 1
>Reporter: Nick Bailey
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 2.1.x
>
>
> SSTableLoader currently gets the keyspace it is going to load to from the 
> path of the directoy of sstables it is loading. This isn't really documented 
> (or I didn't see it), but also isn't really a good way of doing it in general.
> {noformat}
> this.keyspace = directory.getParentFile().getName();
> {noformat}
> We should probably just let users pass the name in. If you are loading a 
> snapshot the file names will have the keyspace which is slightly better but 
> people manually creating their own sstables might not format them the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[07/12] cassandra git commit: Support streaming of older version sstables in 3.0

[yukim]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e8651b66/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_simple_counter_compact/ma-1-big-TOC.txt
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_simple_counter_compact/ma-1-big-TOC.txt
 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_simple_counter_compact/ma-1-big-TOC.txt
new file mode 100644
index 000..a29a600
--- /dev/null
+++ 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_simple_counter_compact/ma-1-big-TOC.txt
@@ -0,0 +1,8 @@
+Index.db
+Statistics.db
+Digest.crc32
+Summary.db
+Data.db
+TOC.txt
+Filter.db
+CompressionInfo.db

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e8651b66/test/unit/org/apache/cassandra/AbstractSerializationsTester.java
--
diff --git a/test/unit/org/apache/cassandra/AbstractSerializationsTester.java 
b/test/unit/org/apache/cassandra/AbstractSerializationsTester.java
index 501f4ae..3a1f348 100644
--- a/test/unit/org/apache/cassandra/AbstractSerializationsTester.java
+++ b/test/unit/org/apache/cassandra/AbstractSerializationsTester.java
@@ -25,7 +25,6 @@ import 
org.apache.cassandra.io.util.DataInputPlus.DataInputStreamPlus;
 import org.apache.cassandra.io.util.DataOutputBuffer;
 import org.apache.cassandra.io.util.DataOutputStreamPlus;
 import org.apache.cassandra.io.util.BufferedDataOutputStreamPlus;
-import org.apache.cassandra.io.util.NIODataInputStream;
 import org.apache.cassandra.net.MessagingService;
 
 import java.io.File;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e8651b66/test/unit/org/apache/cassandra/db/DirectoriesTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/DirectoriesTest.java 
b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
index 8732881..f864bbc 100644
--- a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
+++ b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
@@ -243,46 +243,76 @@ public class DirectoriesTest
 for (CFMetaData cfm : CFM)
 {
 Directories directories = new Directories(cfm);
-Directories.SSTableLister lister;
-Set listed;
+checkFiles(cfm, directories);
+}
+}
 
-// List all but no snapshot, backup
-lister = directories.sstableLister(Directories.OnTxnErr.THROW);
-listed = new HashSet<>(lister.listFiles());
-for (File f : files.get(cfm.cfName))
-{
-if (f.getPath().contains(Directories.SNAPSHOT_SUBDIR) || 
f.getPath().contains(Directories.BACKUPS_SUBDIR))
-assert !listed.contains(f) : f + " should not be listed";
-else
-assert listed.contains(f) : f + " is missing";
-}
+private void checkFiles(CFMetaData cfm, Directories directories)
+{
+Directories.SSTableLister lister;
+Set listed;// List all but no snapshot, backup
+lister = directories.sstableLister(Directories.OnTxnErr.THROW);
+listed = new HashSet<>(lister.listFiles());
+for (File f : files.get(cfm.cfName))
+{
+if (f.getPath().contains(Directories.SNAPSHOT_SUBDIR) || 
f.getPath().contains(Directories.BACKUPS_SUBDIR))
+assertFalse(f + " should not be listed", listed.contains(f));
+else
+assertTrue(f + " is missing", listed.contains(f));
+}
 
-// List all but including backup (but no snapshot)
-lister = 
directories.sstableLister(Directories.OnTxnErr.THROW).includeBackups(true);
-listed = new HashSet<>(lister.listFiles());
-for (File f : files.get(cfm.cfName))
-{
-if (f.getPath().contains(Directories.SNAPSHOT_SUBDIR))
-assert !listed.contains(f) : f + " should not be listed";
-else
-assert listed.contains(f) : f + " is missing";
-}
+// List all but including backup (but no snapshot)
+lister = 
directories.sstableLister(Directories.OnTxnErr.THROW).includeBackups(true);
+listed = new HashSet<>(lister.listFiles());
+for (File f : files.get(cfm.cfName))
+{
+if (f.getPath().contains(Directories.SNAPSHOT_SUBDIR))
+assertFalse(f + " should not be listed", listed.contains(f));
+else
+assertTrue(f + " is missing", listed.contains(f));
+}
 
-// Skip temporary and compacted
-lister = 
directories.sstableLister(Directories.OnTxnErr.THROW).skipTemporary(true);
-listed = new HashSet<>(lister.listFiles());
-for (File f : files.get(cfm.cfName))
-{
-if (f.getPath().contains(Directories.SNAPSHOT_SUBDIR) || 

[jira] [Commented] (CASSANDRA-11383) SASI index build leads to massive OOM

[DOAN DuyHai (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202223#comment-15202223
 ] 

DOAN DuyHai commented on CASSANDRA-11383:
-

Ok

1. drop all indices
2. clean all *idx* files on all machines
3. {{CREATE CUSTOM INDEX resource_period_end_month_int_idx ON 
sharon.resource_bench (period_end_month_int) USING 
'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = \{'mode': 'SPARSE', 
'max_compaction_flush_memory_in_mb': '128'\};}}

Now let it just run for the night and let's see tomorrow morning

> SASI index build leads to massive OOM
> -
>
> Key: CASSANDRA-11383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11383
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: C* 3.4
>Reporter: DOAN DuyHai
> Attachments: CASSANDRA-11383.patch, new_system_log_CMS_8GB_OOM.log, 
> system.log_sasi_build_oom
>
>
> 13 bare metal machines
> - 6 cores CPU (12 HT)
> - 64Gb RAM
> - 4 SSD in RAID0
>  JVM settings:
> - G1 GC
> - Xms32G, Xmx32G
> Data set:
>  - ≈ 100Gb/per node
>  - 1.3 Tb cluster-wide
>  - ≈ 20Gb for all SASI indices
> C* settings:
> - concurrent_compactors: 1
> - compaction_throughput_mb_per_sec: 256
> - memtable_heap_space_in_mb: 2048
> - memtable_offheap_space_in_mb: 2048
> I created 9 SASI indices
>  - 8 indices with text field, NonTokenizingAnalyser,  PREFIX mode, 
> case-insensitive
>  - 1 index with numeric field, SPARSE mode
>  After a while, the nodes just gone OOM.
>  I attach log files. You can see a lot of GC happening while index segments 
> are flush to disk. At some point the node OOM ...
> /cc [~xedin]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-11354) PrimaryKeyRestrictionSet should be refactored

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197572#comment-15197572
 ] 

Alex Petrov edited comment on CASSANDRA-11354 at 3/16/16 4:16 PM:
--

Is {{AbstractSingleRestriction}} still required as an abstract class? It's also 
possible to implement the {{false}} returning methods on the interface 
directly. 

Another minor thing. There are two places where clustering column restrictions 
are "validated": 

One is in {{ClusteringColumnRestrictions}} constructor:
https://github.com/apache/cassandra/compare/trunk...blerer:11354-trunk#diff-fb9315f3449e9fcc3b947ccdd0c1dfdfR63

And the second one 
{{StatementRestrictions::processClusteringColumnsRestrictions}}:
https://github.com/apache/cassandra/compare/trunk...blerer:11354-trunk#diff-559d2a170c681fa6e692c8321afbfea3R468

Probably here also {{usesSecondaryIndexing}} comes into play


was (Author: ifesdjeen):
Is {{AbstractSingleRestriction}} still required as an abstract class? It's also 
possible to implement the {{false}} returning methods on the interface 
directly. 

> PrimaryKeyRestrictionSet should be refactored
> -
>
> Key: CASSANDRA-11354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11354
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>
> While reviewing CASSANDRA-11310 I realized that the code of 
> {{PrimaryKeyRestrictionSet}} was really confusing.
> The main 2 issues are:
> * the fact that it is used for partition keys and clustering columns 
> restrictions whereas those types of column required different processing
> * the {{isEQ}}, {{isSlice}}, {{isIN}} and {{isContains}} methods should not 
> be there as the set of restrictions might not match any of those categories 
> when secondary indexes are used.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10971) Compressed commit log has no backpressure and can OOM

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10971:
---
   Resolution: Fixed
Fix Version/s: (was: 3.0.x)
   (was: 3.x)
   3.5
   3.0.5
   Status: Resolved  (was: Ready to Commit)

Committed into 3.0 at 9995521fb9b3f510ee9c7012d75e6970ec7d5fb7 and into 3.5 at 
ee40e3b4529aa77d4d83fc3e7073902402cb3753.
Merged into trunk

> Compressed commit log has no backpressure and can OOM
> -
>
> Key: CASSANDRA-10971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10971
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 3.0.5, 3.5
>
> Attachments: 10971-3.0.txt, 10971-3.5.txt
>
>
> I validated this via a unit test that slowed the ability of the log to drain 
> to the filesystem. The compressed commit log will keep allocating buffers 
> pending compression until it OOMs.
> I have a fix that am not very happy with because the whole signal a thread to 
> allocate a segment that depends on a resource that may not be available 
> results in some obtuse usage of {{CompleatableFuture}} to rendezvous 
> available buffers with {{CommitLogSegmentManager}} thread waiting to finish 
> constructing a new segment. The {{CLSM}} thread is in turn signaled by the 
> thread(s) that actually wants to write to the next segment, but aren't able 
> to do it themselves.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11381) Node running with join_ring=false and authentication can not serve requests

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201666#comment-15201666
 ] 

Joshua McKenzie commented on CASSANDRA-11381:
-

Is this ticket Patch Available?

> Node running with join_ring=false and authentication can not serve requests
> ---
>
> Key: CASSANDRA-11381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11381
> Project: Cassandra
>  Issue Type: Bug
>Reporter: mck
>Assignee: mck
> Attachments: 11381-2.0.txt, 11381-2.1.txt, 11381-2.2.txt, 
> 11381-3.0.txt, 11381-trunk.txt
>
>
> Starting up a node with {{-Dcassandra.join_ring=false}} in a cluster that has 
> authentication configured, eg PasswordAuthenticator, won't be able to serve 
> requests. This is because {{Auth.setup()}} never gets called during the 
> startup.
> Without {{Auth.setup()}} having been called in {{StorageService}} clients 
> connecting to the node fail with the node throwing
> {noformat}
> java.lang.NullPointerException
> at 
> org.apache.cassandra.auth.PasswordAuthenticator.authenticate(PasswordAuthenticator.java:119)
> at 
> org.apache.cassandra.thrift.CassandraServer.login(CassandraServer.java:1471)
> at 
> org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3505)
> at 
> org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3489)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at com.thinkaurelius.thrift.Message.invoke(Message.java:314)
> at 
> com.thinkaurelius.thrift.Message$Invocation.execute(Message.java:90)
> at 
> com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:695)
> at 
> com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:689)
> at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:112)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The exception thrown from the 
> [code|https://github.com/apache/cassandra/blob/cassandra-2.0.16/src/java/org/apache/cassandra/auth/PasswordAuthenticator.java#L119]
> {code}
> ResultMessage.Rows rows = 
> authenticateStatement.execute(QueryState.forInternalCalls(), new 
> QueryOptions(consistencyForUser(username),
>   
>Lists.newArrayList(ByteBufferUtil.bytes(username;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11374) LEAK DETECTED during repair

[Jean-Francois Gosselin (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199874#comment-15199874
 ] 

Jean-Francois Gosselin commented on CASSANDRA-11374:


Same issue as CASSANDRA-9117 but not fixed in 2.1.x ?

> LEAK DETECTED during repair
> ---
>
> Key: CASSANDRA-11374
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11374
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jean-Francois Gosselin
>
> When running a range repair we are seeing the following LEAK DETECTED errors:
> {noformat}
> ERROR [Reference-Reaper:1] 2016-03-17 06:58:52,261 Ref.java:179 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@5ee90b43) to class 
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@367168611:[[OffHeapBitSet]]
>  was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2016-03-17 06:58:52,262 Ref.java:179 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@4ea9d4a7) to class 
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1875396681:Memory@[7f34b905fd10..7f34b9060b7a)
>  was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2016-03-17 06:58:52,262 Ref.java:179 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@27a6b614) to class 
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@838594402:Memory@[7f34bae11ce0..7f34bae11d84)
>  was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2016-03-17 06:58:52,263 Ref.java:179 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@64e7b566) to class 
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@674656075:Memory@[7f342deab4e0..7f342deb7ce0)
>  was not released before the reference was garbage collected
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11363) Blocked NTR When Connecting Causing Excessive Load

[Russell Bradberry (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199757#comment-15199757
 ] 

Russell Bradberry commented on CASSANDRA-11363:
---

After some digging, we found the issue to be introduce in Cassandra 2.1.12.  We 
are running DSE so the issue manifested in DSE 4.8.4

> Blocked NTR When Connecting Causing Excessive Load
> --
>
> Key: CASSANDRA-11363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11363
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Russell Bradberry
> Attachments: cassandra-102-cms.stack, cassandra-102-g1gc.stack
>
>
> When upgrading from 2.1.9 to 2.1.13, we are witnessing an issue where the 
> machine load increases to very high levels (> 120 on an 8 core machine) and 
> native transport requests get blocked in tpstats.
> I was able to reproduce this in both CMS and G1GC as well as on JVM 7 and 8.
> The issue does not seem to affect the nodes running 2.1.9.
> The issue seems to coincide with the number of connections OR the number of 
> total requests being processed at a given time (as the latter increases with 
> the former in our system)
> Currently there is between 600 and 800 client connections on each machine and 
> each machine is handling roughly 2000-3000 client requests per second.
> Disabling the binary protocol fixes the issue for this node but isn't a 
> viable option cluster-wide.
> Here is the output from tpstats:
> {code}
> Pool NameActive   Pending  Completed   Blocked  All 
> time blocked
> MutationStage 0 88387821 0
>  0
> ReadStage 0 0 355860 0
>  0
> RequestResponseStage  0 72532457 0
>  0
> ReadRepairStage   0 0150 0
>  0
> CounterMutationStage 32   104 897560 0
>  0
> MiscStage 0 0  0 0
>  0
> HintedHandoff 0 0 65 0
>  0
> GossipStage   0 0   2338 0
>  0
> CacheCleanupExecutor  0 0  0 0
>  0
> InternalResponseStage 0 0  0 0
>  0
> CommitLogArchiver 0 0  0 0
>  0
> CompactionExecutor2   190474 0
>  0
> ValidationExecutor0 0  0 0
>  0
> MigrationStage0 0 10 0
>  0
> AntiEntropyStage  0 0  0 0
>  0
> PendingRangeCalculator0 0310 0
>  0
> Sampler   0 0  0 0
>  0
> MemtableFlushWriter   110 94 0
>  0
> MemtablePostFlush 134257 0
>  0
> MemtableReclaimMemory 0 0 94 0
>  0
> Native-Transport-Requests   128   156 38795716
> 278451
> Message type   Dropped
> READ 0
> RANGE_SLICE  0
> _TRACE   0
> MUTATION 0
> COUNTER_MUTATION 0
> BINARY   0
> REQUEST_RESPONSE 0
> PAGED_RANGE  0
> READ_REPAIR  0
> {code}
> Attached is the jstack output for both CMS and G1GC.
> Flight recordings are here:
> https://s3.amazonaws.com/simple-logs/cassandra-102-cms.jfr
> https://s3.amazonaws.com/simple-logs/cassandra-102-g1gc.jfr
> It is interesting to note that while the flight recording was taking place, 
> the load on the machine went back to healthy, and when the flight recording 
> finished the load went back to > 100.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-11360) dtest failure in pushed_notifications_test.TestPushedNotifications.restart_node_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson reassigned CASSANDRA-11360:
---

Assignee: Philip Thompson  (was: DS Test Eng)

> dtest failure in 
> pushed_notifications_test.TestPushedNotifications.restart_node_test
> 
>
> Key: CASSANDRA-11360
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11360
> Project: Cassandra
>  Issue Type: Test
>Reporter: Jim Witschey
>Assignee: Philip Thompson
>  Labels: dtest
>
> This has flaked on the 2.1 offheap memtables job several times. This is the 
> most recent failure:
> http://cassci.datastax.com/job/cassandra-2.1_offheap_dtest/318/testReport/pushed_notifications_test/TestPushedNotifications/restart_node_test
> But it's different from the other recent failures:
> http://cassci.datastax.com/job/cassandra-2.1_offheap_dtest/303/testReport/junit/pushed_notifications_test/TestPushedNotifications/restart_node_test/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11375) COPY FROM fails when importing blob

[Philippe Thibaudeau (JIRA)]

Philippe Thibaudeau created CASSANDRA-11375:
---

 Summary: COPY FROM fails when importing blob
 Key: CASSANDRA-11375
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11375
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
Reporter: Philippe Thibaudeau
 Fix For: 2.1.x


When we try to COPY TO a table containing a blob, we get this error 
COPY test.blobTable FROM '/tmp/test1.csv' WITH NULL='null' AND DELIMITER=',' 
AND QUOTE='"';

/opt/apache-cassandra-2.1.13.4/bin/../pylib/cqlshlib/copyutil.py:1602: 
DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
/opt/apache-cassandra-2.1.13.4/bin/../pylib/cqlshlib/copyutil.py:1850: 
DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
Failed to import 5 rows: ParseError - fromhex() argument 1 must be unicode, not 
str -  given up without retries
Failed to process 5 rows; failed rows written to import_test_blobTable.err

Same COPY TO function worked fine with 2.1.9
The csv is generated by doing a COPY FROM on the same table.
Is there any work around this issue?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)