[jira] [Updated] (CASSANDRA-11542) Create a benchmark to compare HDFS and Cassandra bulk read times

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-11542:
-
Description: 
I propose creating a benchmark for comparing Cassandra and HDFS bulk reading 
performance. Simple Spark queries will be performed on data stored in HDFS or 
Cassandra, and the entire duration will be measured. An example query would be 
the max or min of a column or a count\(*\).

This benchmark should allow determining the impact of:
* partition size
* number of clustering columns
* number of value columns (cells)


  was:
I propose creating a benchmark for comparing Cassandra and HDFS bulk reading 
performance. Data will be imported into Spark to perform very simple queries 
and the entire duration will be measured. An example query would be the max or 
min of a column or a count\(*\).

This benchmark should allow determining the impact of:
* partition size
* number of clustering columns
* number of value columns (cells)



> Create a benchmark to compare HDFS and Cassandra bulk read times
> 
>
> Key: CASSANDRA-11542
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11542
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Testing
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
>
> I propose creating a benchmark for comparing Cassandra and HDFS bulk reading 
> performance. Simple Spark queries will be performed on data stored in HDFS or 
> Cassandra, and the entire duration will be measured. An example query would 
> be the max or min of a column or a count\(*\).
> This benchmark should allow determining the impact of:
> * partition size
> * number of clustering columns
> * number of value columns (cells)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11358) Making Cassandra reloadable: allow for schema reloading

[Emmanuel Hugonnet (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234501#comment-15234501
 ] 

Emmanuel Hugonnet commented on CASSANDRA-11358:
---

Yes, but it is one step in the restartable direction. 
This patch hides enough issues that I don't encounter them :). I'm using 
currently a single node (so that's may be the cause for no more issue 
appearing) and I can update the C* configuration and restart the node without 
restarting the JVM.
https://github.com/ehsavoie/wildfly-cassandra/tree/wf-10 for the whole code.

> Making Cassandra reloadable: allow for schema reloading
> ---
>
> Key: CASSANDRA-11358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11358
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Emmanuel Hugonnet
>Assignee: Emmanuel Hugonnet
>Priority: Minor
> Fix For: 3.x
>
> Attachments: ticket-CASSANDRA-11358.txt
>
>
> When embedding a Cassandra node, if I want to reload it without shutting down 
> the JVM I have an error since the Schema are in some incorrect state. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11543) Cluster Nodes Token Range

[Dinesh Kumar (JIRA)]

Dinesh Kumar created CASSANDRA-11543:


 Summary: Cluster Nodes Token Range
 Key: CASSANDRA-11543
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11543
 Project: Cassandra
  Issue Type: New Feature
  Components: Tools
Reporter: Dinesh Kumar
Priority: Minor


Hello,

Is there any way to check the token ranges(like below) of nodes in a cluster?

|*Node*|*StartToken*|*EndToken*|
|node1| 1  | 20|
|node2| 21 | 40|
|node3| 41 | 60|

Thanks,
Dinesh



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11529) Checking if an unlogged batch is local is inefficient

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234470#comment-15234470
 ] 

Stefania commented on CASSANDRA-11529:
--

CI should be fine. The failing dtests on trunk do not seem related and all pass 
locally.

Commit information:

||2.1||2.2||3.0||trunk||
|[patch|https://github.com/stef1927/cassandra/commits/11529-2.1]|[patch|https://github.com/stef1927/cassandra/commits/11529-2.2]|[patch|https://github.com/stef1927/cassandra/commits/11529-3.0]|[patch|https://github.com/stef1927/cassandra/commits/11529]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11529-2.1-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11529-2.2-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11529-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11529-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11529-2.1-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11529-2.2-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11529-3.0-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11529-dtest/]|

No single patch up-merges cleanly, there are conflicts for all branches and the 
trunk patch is slightly different in cassandra.yaml and has 2 authors.

There is also a dtest [pull 
request|https://github.com/riptano/cassandra-dtest/pull/919].

> Checking if an unlogged batch is local is inefficient
> -
>
> Key: CASSANDRA-11529
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11529
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Paulo Motta
>Assignee: Stefania
>Priority: Critical
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> Based on CASSANDRA-11363 report I noticed that on CASSANDRA-9303 we 
> introduced the following check to avoid printing a {{WARN}} in case an 
> unlogged batch statement is local:
> {noformat}
>  for (IMutation im : mutations)
>  {
>  keySet.add(im.key());
>  for (ColumnFamily cf : im.getColumnFamilies())
>  ksCfPairs.add(String.format("%s.%s", 
> cf.metadata().ksName, cf.metadata().cfName));
> +
> +if (localMutationsOnly)
> +localMutationsOnly &= isMutationLocal(localTokensByKs, 
> im);
>  }
>  
> +// CASSANDRA-9303: If we only have local mutations we do not warn
> +if (localMutationsOnly)
> +return;
> +
>  NoSpamLogger.log(logger, NoSpamLogger.Level.WARN, 1, 
> TimeUnit.MINUTES, unloggedBatchWarning,
>   keySet.size(), keySet.size() == 1 ? "" : "s",
>   ksCfPairs.size() == 1 ? "" : "s", ksCfPairs);
> {noformat}
> The {{isMutationLocal}} check uses 
> {{StorageService.instance.getLocalRanges(mutation.getKeyspaceName())}}, which 
> underneaths uses {{AbstractReplication.getAddressRanges}} to calculate local 
> ranges. 
> Recalculating this at every unlogged batch can be pretty inefficient, so we 
> should at the very least cache it every time the ring changes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11529) Checking if an unlogged batch is local is inefficient

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-11529:
-
Status: Ready to Commit  (was: Patch Available)

> Checking if an unlogged batch is local is inefficient
> -
>
> Key: CASSANDRA-11529
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11529
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Paulo Motta
>Assignee: Stefania
>Priority: Critical
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> Based on CASSANDRA-11363 report I noticed that on CASSANDRA-9303 we 
> introduced the following check to avoid printing a {{WARN}} in case an 
> unlogged batch statement is local:
> {noformat}
>  for (IMutation im : mutations)
>  {
>  keySet.add(im.key());
>  for (ColumnFamily cf : im.getColumnFamilies())
>  ksCfPairs.add(String.format("%s.%s", 
> cf.metadata().ksName, cf.metadata().cfName));
> +
> +if (localMutationsOnly)
> +localMutationsOnly &= isMutationLocal(localTokensByKs, 
> im);
>  }
>  
> +// CASSANDRA-9303: If we only have local mutations we do not warn
> +if (localMutationsOnly)
> +return;
> +
>  NoSpamLogger.log(logger, NoSpamLogger.Level.WARN, 1, 
> TimeUnit.MINUTES, unloggedBatchWarning,
>   keySet.size(), keySet.size() == 1 ? "" : "s",
>   ksCfPairs.size() == 1 ? "" : "s", ksCfPairs);
> {noformat}
> The {{isMutationLocal}} check uses 
> {{StorageService.instance.getLocalRanges(mutation.getKeyspaceName())}}, which 
> underneaths uses {{AbstractReplication.getAddressRanges}} to calculate local 
> ranges. 
> Recalculating this at every unlogged batch can be pretty inefficient, so we 
> should at the very least cache it every time the ring changes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11542) Create a benchmark to compare HDFS and Cassandra bulk read times

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-11542:
-
Description: 
I propose creating a benchmark for comparing Cassandra and HDFS bulk reading 
performance. Data will be imported into Spark to perform very simple queries 
and the entire duration will be measured. An example query would be the max or 
min of a column or a count\(*\).

This benchmark should allow determining the impact of:
* partition size
* number of clustering columns
* number of value columns (cells)


  was:
I propose creating a benchmark for comparing Cassandra and HDFS bulk reading 
performance. Data will be imported into Spark to perform very simple queries 
and the entire duration will be measured. An example query would be the max or 
min of a column or a count(*).

This benchmark should allow determining the impact of:
* partition size
* number of clustering columns
* number of value columns (cells)



> Create a benchmark to compare HDFS and Cassandra bulk read times
> 
>
> Key: CASSANDRA-11542
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11542
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Testing
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
>
> I propose creating a benchmark for comparing Cassandra and HDFS bulk reading 
> performance. Data will be imported into Spark to perform very simple queries 
> and the entire duration will be measured. An example query would be the max 
> or min of a column or a count\(*\).
> This benchmark should allow determining the impact of:
> * partition size
> * number of clustering columns
> * number of value columns (cells)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11521) Implement streaming for bulk read requests

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234461#comment-15234461
 ] 

Stefania commented on CASSANDRA-11521:
--

I like this suggested approach; it has many advantages, as already pointed out 
above. I will prototype it and then compare performance with the existing 
"streaming" prototype. 

I have created CASSANDRA-11542, so we can actually compare with HDFS 
performance as well. Once this benchmark is available, and the new approach has 
been prototyped, we will then have 4 measurements: 

* HDFS
* Cassandra trunk
* Cassandra with current "streaming" approach
* Cassandra with this new approach.

> Implement streaming for bulk read requests
> --
>
> Key: CASSANDRA-11521
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11521
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Local Write-Read Paths
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
>
> Allow clients to stream data from a C* host, bypassing the coordination layer 
> and eliminating the need to query individual pages one by one.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11542) Create a benchmark to compare HDFS and Cassandra bulk read times

[Stefania (JIRA)]

Stefania created CASSANDRA-11542:


 Summary: Create a benchmark to compare HDFS and Cassandra bulk 
read times
 Key: CASSANDRA-11542
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11542
 Project: Cassandra
  Issue Type: Sub-task
  Components: Testing
Reporter: Stefania
Assignee: Stefania
 Fix For: 3.x


I propose creating a benchmark for comparing Cassandra and HDFS bulk reading 
performance. Data will be imported into Spark to perform very simple queries 
and the entire duration will be measured. An example query would be the max or 
min of a column or a count(*).

This benchmark should allow determining the impact of:
* partition size
* number of clustering columns
* number of value columns (cells)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11206) Support large partitions on the 3.0 sstable format

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234447#comment-15234447
 ] 

T Jake Luciani commented on CASSANDRA-11206:


I haven't dug into this much but on the surface this effectively breaks 
CASSANDRA-7443 since you removed all generics from the IndexEntry. 
I don't see any reason you can't support a serializer implementaion per format.

> Support large partitions on the 3.0 sstable format
> --
>
> Key: CASSANDRA-11206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11206
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
> Fix For: 3.x
>
> Attachments: 11206-gc.png, trunk-gc.png
>
>
> Cassandra saves a sample of IndexInfo objects that store the offset within 
> each partition of every 64KB (by default) range of rows.  To find a row, we 
> binary search this sample, then scan the partition of the appropriate range.
> The problem is that this scales poorly as partitions grow: on a cache miss, 
> we deserialize the entire set of IndexInfo, which both creates a lot of GC 
> overhead (as noted in CASSANDRA-9754) but is also non-negligible i/o activity 
> (relative to reading a single 64KB row range) as partitions get truly large.
> We introduced an "offset map" in CASSANDRA-10314 that allows us to perform 
> the IndexInfo bsearch while only deserializing IndexInfo that we need to 
> compare against, i.e. log(N) deserializations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11505) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_max_parse_errors

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234430#comment-15234430
 ] 

Stefania commented on CASSANDRA-11505:
--

This is a partial back-port of CASSANDRA-11320:

||2.2||
|[patch|https://github.com/stef1927/cassandra/commits/11505-2.2]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11505-2.2-dtest/]|

It introduces a daemon thread for sending messages in {{OneWayChannel}}, which 
should prevent the parent process from hanging on exit if it cannot send a 
poison pill to the child processes. It is currently under test in 
[CSTAR-479|https://datastax.jira.com/browse/CSTAR-479].


> dtest failure in 
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_max_parse_errors
> -
>
> Key: CASSANDRA-11505
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11505
> Project: Cassandra
>  Issue Type: Test
>Reporter: Michael Shuler
>Assignee: Stefania
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.0_novnode_dtest/197/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_reading_max_parse_errors
> Failed on CassCI build cassandra-3.0_novnode_dtest #197
> {noformat}
> Error Message
> False is not true
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: /mnt/tmp/dtest-c2AJlu
> dtest: DEBUG: Custom init_config not found. Setting defaults.
> dtest: DEBUG: Done setting configuration options:
> {   'num_tokens': None,
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> dtest: DEBUG: Importing csv file /mnt/tmp/tmp2O43PH with 10 max parse errors
> - >> end captured logging << -
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", 
> line 943, in test_reading_max_parse_errors
> self.assertTrue(num_rows_imported < (num_rows / 2))  # less than the 
> maximum number of valid rows in the csv
>   File "/usr/lib/python2.7/unittest/case.py", line 422, in assertTrue
> raise self.failureException(msg)
> "False is not true\n >> begin captured logging << 
> \ndtest: DEBUG: cluster ccm directory: 
> /mnt/tmp/dtest-c2AJlu\ndtest: DEBUG: Custom init_config not found. Setting 
> defaults.\ndtest: DEBUG: Done setting configuration options:\n{   
> 'num_tokens': None,\n'phi_convict_threshold': 5,\n
> 'range_request_timeout_in_ms': 1,\n'read_request_timeout_in_ms': 
> 1,\n'request_timeout_in_ms': 1,\n
> 'truncate_request_timeout_in_ms': 1,\n'write_request_timeout_in_ms': 
> 1}\ndtest: DEBUG: Importing csv file /mnt/tmp/tmp2O43PH with 10 max parse 
> errors\n- >> end captured logging << 
> -"
> Standard Output
> (EE)  Using CQL driver:  '/home/automaton/cassandra/bin/../lib/cassandra-driver-internal-only-3.0.0-6af642d.zip/cassandra-driver-3.0.0-6af642d/cassandra/__init__.py'>(EE)
>   Using connect timeout: 5 seconds(EE)  Using 'utf-8' encoding(EE)  
> :2:Failed to import 2500 rows: ParseError - could not convert string 
> to float: abc,  given up without retries(EE)  :2:Exceeded maximum 
> number of parse errors 10(EE)  :2:Failed to process 2500 rows; failed 
> rows written to import_ks_testmaxparseerrors.err(EE)  :2:Exceeded 
> maximum number of parse errors 10(EE)  
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10433) Reduce contention in CompositeType instance interning

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234421#comment-15234421
 ] 

Paulo Motta commented on CASSANDRA-10433:
-

This contention showed up in the JFR files provided on CASSANDRA-11363 for C* 
2.1. Given this might affect the critical path of Thrift composite tables, and 
it merges cleanly I think we should backport it to 2.1.

Backported patch and CI tests below:
||2.1||
|[branch|https://github.com/apache/cassandra/compare/cassandra-2.1...pauloricardomg:2.1-10433]|
|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.1-10433-testall/lastCompletedBuild/testReport/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.1-10433-dtest/lastCompletedBuild/testReport/]|

> Reduce contention in CompositeType instance interning
> -
>
> Key: CASSANDRA-10433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10433
> Project: Cassandra
>  Issue Type: Improvement
> Environment: Cassandra 2.2.1 running on 6 AWS c3.4xlarge nodes, 
> CentOS 6.6
>Reporter: David Schlosnagle
>Assignee: David Schlosnagle
>Priority: Minor
> Fix For: 2.2.4
>
> Attachments: 
> 0001-Avoid-contention-in-CompositeType-instance-interning.patch
>
>
> While running some workload tests on Cassandra 2.2.1 and profiling with 
> flight recorder in a test environment, we have noticed significant contention 
> on the static synchronized 
> org.apache.cassandra.db.marshal.CompositeType.getInstance(List) method.
> We are seeing threads blocked for 22.828 seconds from a 60 second snapshot 
> while under a mix of reads and writes from a Thrift based client.
> I would propose to reduce contention in 
> org.apache.cassandra.db.marshal.CompositeType.getInstance(List) by using a 
> ConcurrentHashMap for the instances cache.
> {code}
> Contention Back Trace
> org.apache.cassandra.db.marshal.CompositeType.getInstance(List)
>   
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.asAbstractType()
> org.apache.cassandra.db.SuperColumns.getComparatorFor(CFMetaData, boolean)
>   org.apache.cassandra.db.SuperColumns.getComparatorFor(CFMetaData, 
> ByteBuffer)
> 
> org.apache.cassandra.thrift.ThriftValidation.validateColumnNames(CFMetaData, 
> ByteBuffer, Iterable)
>   
> org.apache.cassandra.thrift.ThriftValidation.validateColumnPath(CFMetaData, 
> ColumnPath)
> 
> org.apache.cassandra.thrift.ThriftValidation.validateColumnOrSuperColumn(CFMetaData,
>  ByteBuffer, ColumnOrSuperColumn)
>   
> org.apache.cassandra.thrift.ThriftValidation.validateMutation(CFMetaData, 
> ByteBuffer, Mutation)
> 
> org.apache.cassandra.thrift.CassandraServer.createMutationList(ConsistencyLevel,
>  Map, boolean)
>   
> org.apache.cassandra.thrift.CassandraServer.batch_mutate(Map, 
> ConsistencyLevel)
> 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra$Iface,
>  Cassandra$batch_mutate_args)
> 
> org.apache.cassandra.thrift.ThriftValidation.validateRange(CFMetaData, 
> ColumnParent, SliceRange)
>   
> org.apache.cassandra.thrift.ThriftValidation.validatePredicate(CFMetaData, 
> ColumnParent, SlicePredicate)
> 
> org.apache.cassandra.thrift.CassandraServer.get_range_slices(ColumnParent, 
> SlicePredicate, KeyRange, ConsistencyLevel)
>   
> org.apache.cassandra.thrift.Cassandra$Processor$get_range_slices.getResult(Cassandra$Iface,
>  Cassandra$get_range_slices_args)
> 
> org.apache.cassandra.thrift.Cassandra$Processor$get_range_slices.getResult(Object,
>  TBase)
>   org.apache.thrift.ProcessFunction.process(int, TProtocol, 
> TProtocol, Object)
> org.apache.thrift.TBaseProcessor.process(TProtocol, 
> TProtocol)
>   
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run()
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker)
>   java.util.concurrent.ThreadPoolExecutor$Worker.run()
> 
> org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(String, 
> List, ColumnParent, long, SlicePredicate, ConsistencyLevel, ClientState)
>   
> org.apache.cassandra.thrift.CassandraServer.multiget_slice(List, 
> ColumnParent, SlicePredicate, ConsistencyLevel)
> 
> org.apache.cassandra.thrift.Cassandra$Processor$multiget_slice.getResult(Cassandra$Iface,
>  Cassandra$multiget_slice_args)
>   
> org.apache.cassandra.thrift.Cassandra$Processor$multiget_slice.getResult(Object,
>  TBase)
> {code}



--
This message was sent by Atlassian JIRA

[jira] [Assigned] (CASSANDRA-11505) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_max_parse_errors

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania reassigned CASSANDRA-11505:


Assignee: Stefania  (was: DS Test Eng)

> dtest failure in 
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_max_parse_errors
> -
>
> Key: CASSANDRA-11505
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11505
> Project: Cassandra
>  Issue Type: Test
>Reporter: Michael Shuler
>Assignee: Stefania
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.0_novnode_dtest/197/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_reading_max_parse_errors
> Failed on CassCI build cassandra-3.0_novnode_dtest #197
> {noformat}
> Error Message
> False is not true
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: /mnt/tmp/dtest-c2AJlu
> dtest: DEBUG: Custom init_config not found. Setting defaults.
> dtest: DEBUG: Done setting configuration options:
> {   'num_tokens': None,
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> dtest: DEBUG: Importing csv file /mnt/tmp/tmp2O43PH with 10 max parse errors
> - >> end captured logging << -
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", 
> line 943, in test_reading_max_parse_errors
> self.assertTrue(num_rows_imported < (num_rows / 2))  # less than the 
> maximum number of valid rows in the csv
>   File "/usr/lib/python2.7/unittest/case.py", line 422, in assertTrue
> raise self.failureException(msg)
> "False is not true\n >> begin captured logging << 
> \ndtest: DEBUG: cluster ccm directory: 
> /mnt/tmp/dtest-c2AJlu\ndtest: DEBUG: Custom init_config not found. Setting 
> defaults.\ndtest: DEBUG: Done setting configuration options:\n{   
> 'num_tokens': None,\n'phi_convict_threshold': 5,\n
> 'range_request_timeout_in_ms': 1,\n'read_request_timeout_in_ms': 
> 1,\n'request_timeout_in_ms': 1,\n
> 'truncate_request_timeout_in_ms': 1,\n'write_request_timeout_in_ms': 
> 1}\ndtest: DEBUG: Importing csv file /mnt/tmp/tmp2O43PH with 10 max parse 
> errors\n- >> end captured logging << 
> -"
> Standard Output
> (EE)  Using CQL driver:  '/home/automaton/cassandra/bin/../lib/cassandra-driver-internal-only-3.0.0-6af642d.zip/cassandra-driver-3.0.0-6af642d/cassandra/__init__.py'>(EE)
>   Using connect timeout: 5 seconds(EE)  Using 'utf-8' encoding(EE)  
> :2:Failed to import 2500 rows: ParseError - could not convert string 
> to float: abc,  given up without retries(EE)  :2:Exceeded maximum 
> number of parse errors 10(EE)  :2:Failed to process 2500 rows; failed 
> rows written to import_ks_testmaxparseerrors.err(EE)  :2:Exceeded 
> maximum number of parse errors 10(EE)  
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11470) dtest failure in materialized_views_test.TestMaterializedViews.base_replica_repair_test

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-11470:
-
Status: Patch Available  (was: In Progress)

> dtest failure in 
> materialized_views_test.TestMaterializedViews.base_replica_repair_test
> ---
>
> Key: CASSANDRA-11470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11470
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node2.log, node2_debug.log, node3.log, 
> node3_debug.log
>
>
> base_replica_repair_test has failed on trunk with the following exception in 
> the log of node2:
> {code}
> ERROR [main] 2016-03-31 08:48:46,949 CassandraDaemon.java:708 - Exception 
> encountered during startup
> java.lang.RuntimeException: Failed to list files in 
> /mnt/tmp/dtest-du964e/test/node2/data0/system_schema/views-9786ac1cdd583201a7cdad556410c985
> at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:53)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:547)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.Directories$SSTableLister.filter(Directories.java:725)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.Directories$SSTableLister.list(Directories.java:690) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:567)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:555)
>  ~[main/:na]
> at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:383) 
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.(Keyspace.java:320) 
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:130) 
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:107) 
> ~[main/:na]
> at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.(StatementRestrictions.java:139)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepareRestrictions(SelectStatement.java:864)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:811)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:799)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:505)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.parseStatement(QueryProcessor.java:242)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.prepareInternal(QueryProcessor.java:286)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:294)
>  ~[main/:na]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.query(SchemaKeyspace.java:1246) 
> ~[main/:na]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:875)
>  ~[main/:na]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:867)
>  ~[main/:na]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:134) 
> ~[main/:na]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:124) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229) 
> [main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:562)
>  [main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691) 
> [main/:na]
> Caused by: java.lang.RuntimeException: Failed to list directory files in 
> /mnt/tmp/dtest-du964e/test/node2/data0/system_schema/views-9786ac1cdd583201a7cdad556410c985,
>  inconsistent disk state for transaction 
> [ma_txn_flush_58db56b0-f71d-11e5-bf68-03a01adb9f11.log in 
> /mnt/tmp/dtest-du964e/test/node2/data0/system_schema/views-9786ac1cdd583201a7cdad556410c985]
> at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.classifyFiles(LogAwareFileLister.java:149)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.classifyFiles(LogAwareFileLister.java:103)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister$$Lambda$48/35984028.accept(Unknown
>  Source) ~[na:na]
> at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) 
> ~[na:1.8.0_45]
> at 
> 

[jira] [Commented] (CASSANDRA-11470) dtest failure in materialized_views_test.TestMaterializedViews.base_replica_repair_test

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234394#comment-15234394
 ] 

Stefania commented on CASSANDRA-11470:
--

Thanks for rerunning. We can see several failures and they are caused by 
missing NEW files, so looking at OLD files definitely fixes the problem.

I don't know why it only happens on trunk, IMO the problem exists in 3.0 as 
well and so I've produced a patch for both branches:

||3.0||trunk||
|[patch|https://github.com/stef1927/cassandra/commits/11470-3.0]|[patch|https://github.com/stef1927/cassandra/commits/11470]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11470-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11470-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11470-3.0-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11470-dtest/]|

The two patches are slightly different because on trunk we have a method in 
LogTransaction that prints the txn contents, whilst on 3.0 we need to access 
the records in order to print them.

[~krummas] would you be OK reviewing this?


> dtest failure in 
> materialized_views_test.TestMaterializedViews.base_replica_repair_test
> ---
>
> Key: CASSANDRA-11470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11470
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node2.log, node2_debug.log, node3.log, 
> node3_debug.log
>
>
> base_replica_repair_test has failed on trunk with the following exception in 
> the log of node2:
> {code}
> ERROR [main] 2016-03-31 08:48:46,949 CassandraDaemon.java:708 - Exception 
> encountered during startup
> java.lang.RuntimeException: Failed to list files in 
> /mnt/tmp/dtest-du964e/test/node2/data0/system_schema/views-9786ac1cdd583201a7cdad556410c985
> at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:53)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:547)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.Directories$SSTableLister.filter(Directories.java:725)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.Directories$SSTableLister.list(Directories.java:690) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:567)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:555)
>  ~[main/:na]
> at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:383) 
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.(Keyspace.java:320) 
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:130) 
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:107) 
> ~[main/:na]
> at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.(StatementRestrictions.java:139)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepareRestrictions(SelectStatement.java:864)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:811)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:799)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:505)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.parseStatement(QueryProcessor.java:242)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.prepareInternal(QueryProcessor.java:286)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:294)
>  ~[main/:na]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.query(SchemaKeyspace.java:1246) 
> ~[main/:na]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:875)
>  ~[main/:na]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:867)
>  ~[main/:na]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:134) 
> ~[main/:na]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:124) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229) 
> [main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:562)
>  [main/:na]
> at 

svn commit: r13112 - in /release/cassandra: 3.0.5/ debian/dists/30x/ debian/dists/30x/main/binary-amd64/ debian/dists/30x/main/binary-i386/ debian/dists/30x/main/source/ debian/pool/main/c/cassandra/

[jake]

Author: jake
Date: Mon Apr 11 01:21:44 2016
New Revision: 13112

Log:
3.0.5 rel

Added:
release/cassandra/3.0.5/
release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz   (with props)
release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.asc
release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.asc.md5
release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.asc.sha1
release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.md5
release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.sha1
release/cassandra/3.0.5/apache-cassandra-3.0.5-src.tar.gz   (with props)
release/cassandra/3.0.5/apache-cassandra-3.0.5-src.tar.gz.asc
release/cassandra/3.0.5/apache-cassandra-3.0.5-src.tar.gz.asc.md5
release/cassandra/3.0.5/apache-cassandra-3.0.5-src.tar.gz.asc.sha1
release/cassandra/3.0.5/apache-cassandra-3.0.5-src.tar.gz.md5
release/cassandra/3.0.5/apache-cassandra-3.0.5-src.tar.gz.sha1

release/cassandra/debian/pool/main/c/cassandra/cassandra-tools_3.0.5_all.deb   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_3.0.5.diff.gz   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_3.0.5.dsc
release/cassandra/debian/pool/main/c/cassandra/cassandra_3.0.5.orig.tar.gz  
 (with props)

release/cassandra/debian/pool/main/c/cassandra/cassandra_3.0.5.orig.tar.gz.asc
release/cassandra/debian/pool/main/c/cassandra/cassandra_3.0.5_all.deb   
(with props)
Modified:
release/cassandra/debian/dists/30x/InRelease
release/cassandra/debian/dists/30x/Release
release/cassandra/debian/dists/30x/Release.gpg
release/cassandra/debian/dists/30x/main/binary-amd64/Packages
release/cassandra/debian/dists/30x/main/binary-amd64/Packages.gz
release/cassandra/debian/dists/30x/main/binary-i386/Packages
release/cassandra/debian/dists/30x/main/binary-i386/Packages.gz
release/cassandra/debian/dists/30x/main/source/Sources.gz

Added: release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz
==
Binary file - no diff available.

Propchange: release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz
--
svn:mime-type = application/octet-stream

Added: release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.asc
==
--- release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.asc (added)
+++ release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.asc Mon Apr 11 
01:21:44 2016
@@ -0,0 +1,17 @@
+-BEGIN PGP SIGNATURE-
+Version: GnuPG v1
+
+iQIcBAABAgAGBQJW/7YuAAoJEHSdbuwDU7EsIfYP/RkF7zDhGbLyz/OJkebEwMqD
+OZKxicV6woDWAcrqHdd9PtXpt1xL461PObolt4XqNFSGz0ZmHFuteyzCP1BHquFF
+o5B7bi86hSSYWnPka5UW4hDUOD+H6+R6cErLiyUkCGvPWLBh62JxOnpIlnT5ceXc
+qDlmDSYbuEm/SSH2WVXU9uD7sAP1UjKJicMNxhGXh4bG+NkltmZkryhKRNLIfKP7
+KhnzpaloiHREokRpfwufgTWli+CaN51qjC6fi3QE751MdQv9Ajsqj3TOWLs1YBYt
+QJOMHdC1HKGJds1BiEvU8WZl6woaE3ABNOd9WATRha/ElAd+Ps41NVs1dX+6ERdH
+cSID2riZ6Eexw/pivQuiss6GoA8cdHQAT9n+tPlbvrna9tMrN0fbJJCaA6VwaKyX
+PWgXd1VG8Wd46IexdSBlIcIysl1UN4l6UQEleTJiNkVNL2AnvyMXe/LkixscIAa7
+0N4SnADj0VdfRN953Cvp7UfMQgGiFcbFhySerC2AVUclKHgPDP9MalVDqC6kUj+9
+JFe878IdKTiGJA3onYgBmFZ8e2eFf6RfYOJd48NVJJLSkLGOM4mb3X7lb+t1MiWF
+hkHr81Hb9gHP1aNFtyhD31aBpxp0pSymxP8WpSSYW8Tq4tesULEliqKpsw5y8MfT
+ObyYTNgSVIH+CFYvh13a
+=rviR
+-END PGP SIGNATURE-

Added: release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.asc.md5
==
--- release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.asc.md5 (added)
+++ release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.asc.md5 Mon Apr 
11 01:21:44 2016
@@ -0,0 +1 @@
+34d6cbbd4bcd01949c04f6f5e0902460
\ No newline at end of file

Added: release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.asc.sha1
==
--- release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.asc.sha1 (added)
+++ release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.asc.sha1 Mon Apr 
11 01:21:44 2016
@@ -0,0 +1 @@
+1e8b27cbfcc39cbb5a970e8e9aee6c5f7fce5b26
\ No newline at end of file

Added: release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.md5
==
--- release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.md5 (added)
+++ release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.md5 Mon Apr 11 
01:21:44 2016
@@ -0,0 +1 @@
+1c4cf9f4f8f939c30902053942d95b16
\ No newline at end of file

Added: release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.sha1
==
--- release/cassandra/3.0.5/apache-cassandra-3.0.5-bin.tar.gz.sha1 (added)
+++ 

[cassandra] Git Push Summary

[jake]

Repository: cassandra
Updated Tags:  refs/tags/cassandra-3.0.5 [created] 5dab8843a

[cassandra] Git Push Summary

[jake]

Repository: cassandra
Updated Tags:  refs/tags/3.0.5-tentative [deleted] c6e6fa94d

[jira] [Commented] (CASSANDRA-11529) Checking if an unlogged batch is local is inefficient

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234378#comment-15234378
 ] 

Stefania commented on CASSANDRA-11529:
--

Thanks for the suggestion, it LGTM.

I've incorporated your commit into my trunk patch, rebased and squashed with 
correct authors. 

I've restarted CI on trunk because there was a problem with the cqlsh version. 
If CI on trunk is OK, I will mark as ready to commit.

> Checking if an unlogged batch is local is inefficient
> -
>
> Key: CASSANDRA-11529
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11529
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Paulo Motta
>Assignee: Stefania
>Priority: Critical
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> Based on CASSANDRA-11363 report I noticed that on CASSANDRA-9303 we 
> introduced the following check to avoid printing a {{WARN}} in case an 
> unlogged batch statement is local:
> {noformat}
>  for (IMutation im : mutations)
>  {
>  keySet.add(im.key());
>  for (ColumnFamily cf : im.getColumnFamilies())
>  ksCfPairs.add(String.format("%s.%s", 
> cf.metadata().ksName, cf.metadata().cfName));
> +
> +if (localMutationsOnly)
> +localMutationsOnly &= isMutationLocal(localTokensByKs, 
> im);
>  }
>  
> +// CASSANDRA-9303: If we only have local mutations we do not warn
> +if (localMutationsOnly)
> +return;
> +
>  NoSpamLogger.log(logger, NoSpamLogger.Level.WARN, 1, 
> TimeUnit.MINUTES, unloggedBatchWarning,
>   keySet.size(), keySet.size() == 1 ? "" : "s",
>   ksCfPairs.size() == 1 ? "" : "s", ksCfPairs);
> {noformat}
> The {{isMutationLocal}} check uses 
> {{StorageService.instance.getLocalRanges(mutation.getKeyspaceName())}}, which 
> underneaths uses {{AbstractReplication.getAddressRanges}} to calculate local 
> ranges. 
> Recalculating this at every unlogged batch can be pretty inefficient, so we 
> should at the very least cache it every time the ring changes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10855) Use Caffeine (W-TinyLFU) for on-heap caches

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234302#comment-15234302
 ] 

Benedict commented on CASSANDRA-10855:
--

Yes, I agree it would not be difficult, and the rest of my statement holds for 
the post-TPC world - I would still think Tiny-LFU likely the best choice.  That 
aspect of my comment was only meant to serve as an extra point in favour of a 
near-zero-cost solution in the meantime, saving any development expenditure for 
the more long-term solution.

> Use Caffeine (W-TinyLFU) for on-heap caches
> ---
>
> Key: CASSANDRA-10855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10855
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Ben Manes
>  Labels: performance
>
> Cassandra currently uses 
> [ConcurrentLinkedHashMap|https://code.google.com/p/concurrentlinkedhashmap] 
> for performance critical caches (key, counter) and Guava's cache for 
> non-critical (auth, metrics, security). All of these usages have been 
> replaced by [Caffeine|https://github.com/ben-manes/caffeine], written by the 
> author of the previously mentioned libraries.
> The primary incentive is to switch from LRU policy to W-TinyLFU, which 
> provides [near optimal|https://github.com/ben-manes/caffeine/wiki/Efficiency] 
> hit rates. It performs particularly well in database and search traces, is 
> scan resistant, and as adds a very small time/space overhead to LRU.
> Secondarily, Guava's caches never obtained similar 
> [performance|https://github.com/ben-manes/caffeine/wiki/Benchmarks] to CLHM 
> due to some optimizations not being ported over. This change results in 
> faster reads and not creating garbage as a side-effect.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10855) Use Caffeine (W-TinyLFU) for on-heap caches

[Ben Manes (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234296#comment-15234296
 ] 

Ben Manes commented on CASSANDRA-10855:
---

When Cassandra moves to TPC, it should take less than a day to write a TinyLFU 
cache if you borrow my [4-bit CountMin 
sketch|https://github.com/ben-manes/caffeine/blob/master/caffeine/src/main/java/com/github/benmanes/caffeine/cache/FrequencySketch.java].
 I might evolve that to include the doorkeeper and other tricks, but it hasn't 
been a priority yet. The simulator includes a variant using incremental reset 
(which would be my preference for a TPC) and shows a negligible difference in 
hit rates. A Go developer read the paper and wrote an implementation for fun in 
a few days, and I'm happy to review anyone's version.

Caffeine tackled the research and concurrency side, but no reason to adopt it 
once TPC. Best to take ideas, leverage its simulator to fine tune, and share 
insights.

> Use Caffeine (W-TinyLFU) for on-heap caches
> ---
>
> Key: CASSANDRA-10855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10855
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Ben Manes
>  Labels: performance
>
> Cassandra currently uses 
> [ConcurrentLinkedHashMap|https://code.google.com/p/concurrentlinkedhashmap] 
> for performance critical caches (key, counter) and Guava's cache for 
> non-critical (auth, metrics, security). All of these usages have been 
> replaced by [Caffeine|https://github.com/ben-manes/caffeine], written by the 
> author of the previously mentioned libraries.
> The primary incentive is to switch from LRU policy to W-TinyLFU, which 
> provides [near optimal|https://github.com/ben-manes/caffeine/wiki/Efficiency] 
> hit rates. It performs particularly well in database and search traces, is 
> scan resistant, and as adds a very small time/space overhead to LRU.
> Secondarily, Guava's caches never obtained similar 
> [performance|https://github.com/ben-manes/caffeine/wiki/Benchmarks] to CLHM 
> due to some optimizations not being ported over. This change results in 
> faster reads and not creating garbage as a side-effect.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10855) Use Caffeine (W-TinyLFU) for on-heap caches

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234287#comment-15234287
 ] 

Benedict commented on CASSANDRA-10855:
--

Since I may be partially to blame for that, having made a comment suggesting 
LIRS quite some time ago, I'd like to chip in my two cents:

The employment of probabilistic data structures for a cache is the obvious next 
step in their evolution (something I had a hope to explore before Tiny-LFU 
showed up), and Tiny-LFU being the first such scheme should absolutely be 
considered *strongly*.  It is simply natural that a probabilistic approach 
should be capable of yielding better returns for a probabilistic application 
(hit rate), and if anything I'm surprised the world has been so slow to exploit 
this idea.  Were I making the comment again today, I would have suggested that 
Tiny-LFU be considered as the most sensible choice.

That goes doubly for the fact that any given _implementation_ is a stop-gap 
until the transition to Thread-Per-Core completes and we can make all of our 
algorithms more memory (and execution) efficient.  Since Tiny-LFU already 
exists, it does seem an obvious choice on that front.


> Use Caffeine (W-TinyLFU) for on-heap caches
> ---
>
> Key: CASSANDRA-10855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10855
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Ben Manes
>  Labels: performance
>
> Cassandra currently uses 
> [ConcurrentLinkedHashMap|https://code.google.com/p/concurrentlinkedhashmap] 
> for performance critical caches (key, counter) and Guava's cache for 
> non-critical (auth, metrics, security). All of these usages have been 
> replaced by [Caffeine|https://github.com/ben-manes/caffeine], written by the 
> author of the previously mentioned libraries.
> The primary incentive is to switch from LRU policy to W-TinyLFU, which 
> provides [near optimal|https://github.com/ben-manes/caffeine/wiki/Efficiency] 
> hit rates. It performs particularly well in database and search traces, is 
> scan resistant, and as adds a very small time/space overhead to LRU.
> Secondarily, Guava's caches never obtained similar 
> [performance|https://github.com/ben-manes/caffeine/wiki/Benchmarks] to CLHM 
> due to some optimizations not being ported over. This change results in 
> faster reads and not creating garbage as a side-effect.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10855) Use Caffeine (W-TinyLFU) for on-heap caches

[Ben Manes (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234277#comment-15234277
 ] 

Ben Manes commented on CASSANDRA-10855:
---

Some unfortunate duplication of work on the DataStax side, where a custom LIRS 
page cache was implemented for evaluation.

HBase (and Accumulo) are evaluating a migration from their custom SLRU on-heap 
caches. That might add some insight that could aid in Cassandra's decision.

> Use Caffeine (W-TinyLFU) for on-heap caches
> ---
>
> Key: CASSANDRA-10855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10855
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Ben Manes
>  Labels: performance
>
> Cassandra currently uses 
> [ConcurrentLinkedHashMap|https://code.google.com/p/concurrentlinkedhashmap] 
> for performance critical caches (key, counter) and Guava's cache for 
> non-critical (auth, metrics, security). All of these usages have been 
> replaced by [Caffeine|https://github.com/ben-manes/caffeine], written by the 
> author of the previously mentioned libraries.
> The primary incentive is to switch from LRU policy to W-TinyLFU, which 
> provides [near optimal|https://github.com/ben-manes/caffeine/wiki/Efficiency] 
> hit rates. It performs particularly well in database and search traces, is 
> scan resistant, and as adds a very small time/space overhead to LRU.
> Secondarily, Guava's caches never obtained similar 
> [performance|https://github.com/ben-manes/caffeine/wiki/Benchmarks] to CLHM 
> due to some optimizations not being ported over. This change results in 
> faster reads and not creating garbage as a side-effect.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11452) Cache implementation using LIRS eviction for in-process page cache

[Ben Manes (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234269#comment-15234269
 ] 

Ben Manes commented on CASSANDRA-11452:
---

[CASSANDRA-10855|https://issues.apache.org/jira/browse/CASSANDRA-10855] 
introduces [TinyLFU|https://github.com/ben-manes/caffeine/wiki/Efficiency] 
which matches or beats LIRS at much lower space usage and complexity. This is 
because it uses a frequency sketch instead of retaining a large number of ghost 
entries. See the [HighScalability 
article|http://highscalability.com/blog/2016/1/25/design-of-a-modern-cache.html]
 for details. The patch uses [Caffeine|https://github.com/ben-manes/caffeine], 
the successor to Guava's cache and CLHM (used by Cassandra) which are LRU.

Can you evaluate against that?

> Cache implementation using LIRS eviction for in-process page cache
> --
>
> Key: CASSANDRA-11452
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11452
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
>
> Following up from CASSANDRA-5863, to make best use of caching and to avoid 
> having to explicitly marking compaction accesses as non-cacheable, we need a 
> cache implementation that uses an eviction algorithm that can better handle 
> non-recurring accesses.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11513) Result set is not unique on primary key (cql)

[Joel Knighton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-11513:
--
Reproduced In: 3.4
   Status: Patch Available  (was: In Progress)

I've uploaded a small patch that adds a NoRowsReader (analogous to the 
noRowsIterator used in the memtable case) as well as tests covering when slices 
is empty. This is the most elegant and least invasive solution I could find, 
but I'm pretty unfamiliar with this part of the codebase, so any alternatives 
suggestion would be appreciated.

CI looks clean relative to upstream.
||branch||testall||dtest||
|[11513-trunk|https://github.com/jkni/cassandra/tree/11513-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/jkni/job/jkni-11513-trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/jkni/job/jkni-11513-trunk-dtest]|


> Result set is not unique on primary key (cql) 
> --
>
> Key: CASSANDRA-11513
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11513
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tianshi Wang
>Assignee: Joel Knighton
>
> [cqlsh 5.0.1 | Cassandra 3.4 | CQL spec 3.4.0 | Native protocol v4]
> Run followings,
> {code}
> drop table if exists test0;
> CREATE TABLE test0 (
> pk int,
> a int,
> b text,
> s text static,
> PRIMARY KEY (pk, a)
> );
> insert into test0 (pk,a,b,s) values (0,1,'b1','hello b1');
> insert into test0 (pk,a,b,s) values (0,2,'b2','hello b2');
> insert into test0 (pk,a,b,s) values (0,3,'b3','hello b3');
> create index on test0 (b);
> insert into test0 (pk,a,b,s) values (0,2,'b2 again','b2 again');
> {code}
> Now select one record based on primary key, we got all three records.
> {code}
> cqlsh:ops> select * from test0 where pk=0 and a=2;
>  pk | a | s| b
> +---+--+--
>   0 | 1 | b2 again |   b1
>   0 | 2 | b2 again | b2 again
>   0 | 3 | b2 again |   b3
> {code}
> {code}
> cqlsh:ops> desc test0;
> CREATE TABLE ops.test0 (
> pk int,
> a int,
> b text,
> s text static,
> PRIMARY KEY (pk, a)
> ) WITH CLUSTERING ORDER BY (a ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> CREATE INDEX test0_b_idx ON ops.test0 (b);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Ninja-add test/conf to IntelliJ IDE JUnit default config

[snazy]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 94b234313 -> 2db8bd560


Ninja-add test/conf to IntelliJ IDE JUnit default config


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2db8bd56
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2db8bd56
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2db8bd56

Branch: refs/heads/trunk
Commit: 2db8bd560a41653af9838a8ae2a04d847fc8f277
Parents: 94b2343
Author: Robert Stupp 
Authored: Sun Apr 10 18:00:19 2016 +0200
Committer: Robert Stupp 
Committed: Sun Apr 10 18:00:19 2016 +0200

--
 ide/idea-iml-file.xml | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2db8bd56/ide/idea-iml-file.xml
--
diff --git a/ide/idea-iml-file.xml b/ide/idea-iml-file.xml
index e6c4aee..f14fe2e 100644
--- a/ide/idea-iml-file.xml
+++ b/ide/idea-iml-file.xml
@@ -33,6 +33,7 @@
 
 
 
+
 
 
 

[jira] [Commented] (CASSANDRA-8720) Provide tools for finding wide row/partition keys

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234152#comment-15234152
 ] 

Robert Stupp commented on CASSANDRA-8720:
-

I like the patch, but it has two issues:
# the size of the last partition is sometimes negative. Example from 
compaction_history
{code}
95a04430-fef1-11e5-b65b-6b58859e2c86 442
ea30b2c0-fef9-11e5-b65b-6b58859e2c86 -68
{code}
# it somehow collides with reference counting in 2.2 (but works fine in 2.1):
{code}
Exception in thread "main" java.lang.NullPointerException
at 
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.setup(SSTableReader.java:2043)
at 
org.apache.cassandra.io.sstable.format.SSTableReader.setup(SSTableReader.java:1986)
at 
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:461)
at 
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:363)
at 
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:358)
at 
org.apache.cassandra.tools.SSTableExport.enumeratekeys(SSTableExport.java:240)
at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:461)
ERROR 15:16:40 Error when closing class 
org.apache.cassandra.io.sstable.format.SSTableReader$DescriptorTypeTidy@1234160209:/Users/snazy/devel/cassandra/2.2/data/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/snapshots/1460131305667/la-10-big
java.lang.NullPointerException: null
at 
org.apache.cassandra.io.sstable.format.SSTableReader$DescriptorTypeTidy.tidy(SSTableReader.java:2148)
 ~[main/:na]
at 
org.apache.cassandra.utils.concurrent.Ref$GlobalState.release(Ref.java:283) 
~[main/:na]
at 
org.apache.cassandra.utils.concurrent.Ref$State.release(Ref.java:183) 
~[main/:na]
at org.apache.cassandra.utils.concurrent.Ref.release(Ref.java:77) 
[main/:na]
at 
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier$1.run(SSTableReader.java:2086)
 [main/:na]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_77]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
[na:1.8.0_77]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
 [na:1.8.0_77]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
 [na:1.8.0_77]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[na:1.8.0_77]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_77]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
{code}


> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: J.B. Langston
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 8720.txt
>
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11358) Making Cassandra reloadable: allow for schema reloading

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234149#comment-15234149
 ] 

Robert Stupp commented on CASSANDRA-11358:
--

I doubt this is really enough to make C* reloadable/restartable. Please also 
note that C* is not designed to be restartable (some of the reasons are 
mentioned in 
[CASSANDRA-11154|https://issues.apache.org/jira/browse/CASSANDRA-11154?focusedCommentId=15224916=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15224916]


> Making Cassandra reloadable: allow for schema reloading
> ---
>
> Key: CASSANDRA-11358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11358
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Emmanuel Hugonnet
>Assignee: Emmanuel Hugonnet
>Priority: Minor
> Fix For: 3.x
>
> Attachments: ticket-CASSANDRA-11358.txt
>
>
> When embedding a Cassandra node, if I want to reload it without shutting down 
> the JVM I have an error since the Schema are in some incorrect state. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11541) correct the java documentation for SlabAllocator and NativeAllocator

[ZhaoYang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-11541:
-
Labels: document lhf  (was: )
Status: Patch Available  (was: Open)

> correct the java documentation for SlabAllocator and NativeAllocator
> 
>
> Key: CASSANDRA-11541
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11541
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Trivial
>  Labels: lhf, document
> Fix For: 2.1.14, 3.5
>
> Attachments: CASSANDRA-11541.patch
>
>
> I heard a lot that Cassandra uses 2MB slab allocation strategy for memtable 
> to improve its memory efficiency. But in fact, it has been changed from 2MB 
> to 1 MB. 
> And in NativeAllocator, it's logarithmically from 8kb to 1mb.
> large number of tables may not cause too much trouble in term of memtable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11541) correct the java documentation for SlabAllocator and NativeAllocator

[ZhaoYang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-11541:
-
Attachment: CASSANDRA-11541.patch

> correct the java documentation for SlabAllocator and NativeAllocator
> 
>
> Key: CASSANDRA-11541
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11541
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Trivial
>  Labels: document, lhf
> Fix For: 2.1.14, 3.5
>
> Attachments: CASSANDRA-11541.patch
>
>
> I heard a lot that Cassandra uses 2MB slab allocation strategy for memtable 
> to improve its memory efficiency. But in fact, it has been changed from 2MB 
> to 1 MB. 
> And in NativeAllocator, it's logarithmically from 8kb to 1mb.
> large number of tables may not cause too much trouble in term of memtable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11541) correct the java documentation for SlabAllocator and NativeAllocator

[ZhaoYang (JIRA)]

ZhaoYang created CASSANDRA-11541:


 Summary: correct the java documentation for SlabAllocator and 
NativeAllocator
 Key: CASSANDRA-11541
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11541
 Project: Cassandra
  Issue Type: Improvement
Reporter: ZhaoYang
Assignee: ZhaoYang
Priority: Trivial
 Fix For: 2.1.14, 3.5


I heard a lot that Cassandra uses 2MB slab allocation strategy for memtable to 
improve its memory efficiency. But in fact, it has been changed from 2MB to 1 
MB. 

And in NativeAllocator, it's logarithmically from 8kb to 1mb.

large number of tables may not cause too much trouble in term of memtable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7396) Allow selecting Map key, List index

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-7396:

Labels: cql docs-impacting  (was: cql)
Status: Patch Available  (was: Open)

[Branch|https://github.com/apache/cassandra/compare/trunk...snazy:7396-coll-slice]
[utests|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-7396-coll-slice-testall/lastSuccessfulBuild/]
[dtests|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-7396-coll-slice-dtest/lastSuccessfulBuild/]

{{serializeForNativeProtocol}} is now back in {{CollectionType}} 
implementations.

Unlike described in my earlier comment, the return type of single element 
selections is just the set/list element resp. map value type.

The updated patch also contains select/slice support for LWTs and deletions.

Unfortunately it requires a read-before-write for slice deletions for sets and 
maps - I could not find a functionality to add a “slice tombstone” and 
RangeTombstone does only seem to support deletion of a range of (CQL) rows but 
not for a range of a single column’s CellPaths.


> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
>  Labels: cql, docs-impacting
> Fix For: 3.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]."  (Selecting a UDT subfield 
> is already supported.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11460) memory leak

[stone (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234049#comment-15234049
 ] 

stone commented on CASSANDRA-11460:
---

this issue is not related to bugCASSANDRA-9549
Summary:
out-of-memory caused by ConcurrentLinkedQueue,i think it is duplicate with 
bugCASSANDRA-9549,now I understand I make a mistake.
Cassandra9549 caused by Ref.class,and resolved.
my issue caused by SEPExecutor.class
protected final ConcurrentLinkedQueue tasks = new 
ConcurrentLinkedQueue<>();

I just see tasks.add method,but not remove operation,so is this one of the 
reason?

ConcurrentLinkedQueue is non-blocking,and due to my poor cassandra cluster 
environment.
if it cannot consume as fast as produce,it will also increase and then 
Out-of-Memory.is that right

> memory leak
> ---
>
> Key: CASSANDRA-11460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11460
> Project: Cassandra
>  Issue Type: Bug
>Reporter: stone
>Priority: Critical
> Attachments: aaa.jpg
>
>
> env:
> cassandra3.3
> jdk8
> 8G Ram
> so set
> MAX_HEAP_SIZE="2G"
> HEAP_NEWSIZE="400M"
> 1.met same problem about this:
> https://issues.apache.org/jira/browse/CASSANDRA-9549
> I confuse about that this was fixed in release 3.3 according this page:
> https://github.com/apache/cassandra/blob/trunk/CHANGES.txt
> so I change to 3.4,and also have  found this problem again 
> I think this fix should be included in 3.3.3.4
> can you explain about this?
> 2.our write rate exceed the value that our cassandra env can support,
> but i think it should descrese the write rate,or block.consumer the writed 
> data,keep the memory down,then go on writing,not cause out-of-memory instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)