[jira] [Created] (CASSANDRA-8636) Inconsistencies between two tables if BATCH used
mlowicki created CASSANDRA-8636:
---
Summary: Inconsistencies between two tables if BATCH used
Key: CASSANDRA-8636
URL: https://issues.apache.org/jira/browse/CASSANDRA-8636
Project: Cassandra
Issue Type: Bug
Environment: Cassandra 2.1.2, cqlengine 0.20.0, Debian Wheezy
Reporter: mlowicki
Two tables:
* First one *entity* has log-like structure - whenever entity is modified we
create new version of it and put into the table with new mtime which is part of
compound key. Old one is removed.
* Second one called *entity_by_id* is manually managed index for *entity*. By
having only id you can get basic entity attributes from *entity_by_id*.
While adding entity we do two inserts - to *entity* and *entity_by_id* (in this
order)
While deleting we do the same using the same order so first we remove record
from entity table.
It turned out that these two tables were inconsistent. We had ~260 records in
*entity_by_id* for which there is no corresponding record in *entity*. In
*entity* table it's much worse because ~7000 records in *entity_by_id* are
missing and it was growing much faster.
We were using LOCAL_QUROUM. Two datacenters. We didn't get any exceptions while
inserts or deletes. BatchQuery from cqlengine has been used.
if BatchQuery is not used:
{code}
with BatchQuery() as b:
-entity.batch(b).save()
-entity_by_id = EntityById.copy_fields_from(entity)
-entity_by_id.batch(b).save()
+entity.save()
+entity_by_id = EntityById.copy_fields_from(entity)
+entity_by_id.save()
{code}
Everything is fine. We don't have more inconsistencies. I've check what
cqlengine generates and seems that works as expected:
{code}
('BEGIN BATCH\n UPDATE sync.entity SET name = %(4)s WHERE user_id = %(0)s
AND data_type_id = %(1)s AND version = %(2)s AND id = %(3)s\n INSERT
INTO sync.entity_by_id (user_id, id, parent_id, deleted, folder,
data_type_id, version) VALUES (%(5)s, %(6)s, %(7)s, %(8)s, %(9)s, %(10)s,
%(11)s)\nAPPLY BATCH;',)
{code}
Schemas:
{code}
CREATE TABLE entity (
user_id text,
data_type_id int,
version bigint,
id text,
cache_guid text,
client_defined_unique_tag text,
ctime timestamp,
deleted boolean,
folder boolean,
mtime timestamp,
name text,
originator_client_item_id text,
parent_id text,
position blob,
server_defined_unique_tag text,
specifics blob,
PRIMARY KEY (user_id, data_type_id, version, id)
) WITH CLUSTERING ORDER BY (data_type_id ASC, version ASC, id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
CREATE INDEX index_entity_parent_id ON sync.entity (parent_id);
CREATE TABLE entity_by_id (
user_id text,
id text,
cache_guid text,
data_type_id int,
deleted boolean,
folder boolean,
originator_client_item_id text,
parent_id text,
version bigint,
PRIMARY KEY (user_id, id)
) WITH CLUSTERING ORDER BY (id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
CREATE INDEX index_entity_by_id_parent_id ON entity_by_id (parent_id);
{code}
Previously we had many batch size exceeded warnings but limit is increased
now as sometimes we put many KB of data into *specifics* blob field
(*batch_size_warn_threshold_in_kb* is set now to 20).
We had similar data model in other project where we have the same issue. There
are not missing records in *entity* but couple of thousands in *entity_by_id*.
I'll send more details on this soon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8502) Static columns returning null for pages after first
[
https://issues.apache.org/jira/browse/CASSANDRA-8502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279974#comment-14279974
]
Sylvain Lebresne commented on CASSANDRA-8502:
-
Right, and this is not limited to paging: any reversed query with a {{LIMIT}}
might miss the static parts (paging just make it worst in that even queries
without a limit are problematic).
I also agree that doing 2 reads seems like the only solution outside of
specializing statics internally. The somewhat good news is that CASSANDRA-8099
actually does special case statics (for other reasons initially, but this issue
adds a reason why it's a good idea), so we should be able to do it easily (and
much more transparently) with a single read in 3.0.
Now, while we can do 2 reads for 2.0/2.1, it bugs me a bit because:
# It'll probably be hacky code-wise, especially in the paging case since the
pager doesn't have a very good way to know if we're querying statics or not
(we'd either have to pass that information down as a boolean to the ctor, or
manually inspect the slices to detect it, both of which are ugly).
# It'll be surprising performance wise. If it's going to be a I'm doing this
and it's slow, how come? Oh, yes, don't do that, it does 2 reads internally,
I'd almost rather not support it.
So the other option we could consider would be to simply refuse reversed slice
when statics are involved on the ground that we currently don't have a good way
to do it efficiently. It's certainly not perfect, but users can do 2 queries if
they really want to as a work-around, and in doing so the performance impact
would at least be clear upfront.
Static columns returning null for pages after first
---
Key: CASSANDRA-8502
URL: https://issues.apache.org/jira/browse/CASSANDRA-8502
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Flavien Charlon
Assignee: Tyler Hobbs
Fix For: 2.1.3, 2.0.13
Attachments: null-static-column.txt
When paging is used for a query containing a static column, the first page
contains the right value for the static column, but subsequent pages have
null null for the static column instead of the expected value.
Repro steps:
- Create a table with a static column
- Create a partition with 500 cells
- Using cqlsh, query that partition
Actual result:
- You will see that first, the static column appears as expected, but if you
press a key after ---MORE---, the static columns will appear as null.
See the attached file for a repro of the output.
I am using a single node cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[
https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280003#comment-14280003
]
Sylvain Lebresne commented on CASSANDRA-8603:
-
bq. there is an additional byte that is creating a difference and thus the code
is not catching.
This is what is called the {{EOC}} in the code and that is what is
differenciate the beginning of the given prefix (the start of the range
tombstone) from the end of the given prefix (the stop of the range
tombstone). It's both expected and very much needed.
As I said earlier, the start and stop of a RangeTombstone are not expected to
be equal and in that sense this ticket is invalid. The one known exception
being an empty composite (the first of the log lines you pasted) but that is
special cased anyway to reuse the same singleton object (so the patch is not
helpful in that case).
Hence the log lines you pasted where {{start}} is actually equal to {{stop}}
are indeed weird and it would be interesting to track what produces them. But I
can't reproduce them and quickly checking at the places where we create new RT,
nothing jumped out as potential culprit. As this happens during compaction,
could you check which table is compacting?
Cut tombstone memory footprint in half for cql deletes
--
Key: CASSANDRA-8603
URL: https://issues.apache.org/jira/browse/CASSANDRA-8603
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Dominic Letz
Labels: tombstone
Attachments: cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt,
system.log
As CQL does not yet support range deletes every delete from CQL results in a
Semi-RangeTombstone which actually has the same start and end values - but
until today they are copies. Effectively doubling the required heap memory to
store the RangeTombstone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8636) Inconsistencies between two tables if BATCH used
[
https://issues.apache.org/jira/browse/CASSANDRA-8636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280105#comment-14280105
]
mlowicki commented on CASSANDRA-8636:
-
Second project use a bit simpler schema:
{code}
CREATE TABLE entity (
folder_id text,
mtime timestamp,
id text,
access_token text static,
ctime timestamp,
deleted boolean,
expire timestamp static,
favicon_url text,
folder boolean,
image_url text,
name text,
parent_id text,
position int,
specifics blob,
url text,
version bigint,
PRIMARY KEY (folder_id, mtime, id)
) WITH CLUSTERING ORDER BY (mtime DESC, id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
CREATE TABLE entity_by_id (
folder_id text,
id text,
mtime timestamp,
PRIMARY KEY (folder_id, id)
) WITH CLUSTERING ORDER BY (id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
{code}
We don't use *specifics* field in this project so we don't have any problem
with batch size.
Inconsistencies between two tables if BATCH used
Key: CASSANDRA-8636
URL: https://issues.apache.org/jira/browse/CASSANDRA-8636
Project: Cassandra
Issue Type: Bug
Environment: Cassandra 2.1.2, cqlengine 0.20.0, Debian Wheezy
Reporter: mlowicki
Two tables:
* First one *entity* has log-like structure - whenever entity is modified we
create new version of it and put into the table with new mtime which is part
of compound key. Old one is removed.
* Second one called *entity_by_id* is manually managed index for *entity*. By
having only id you can get basic entity attributes from *entity_by_id*.
While adding entity we do two inserts - to *entity* and *entity_by_id* (in
this order)
While deleting we do the same using the same order so first we remove record
from entity table.
It turned out that these two tables were inconsistent. We had ~260 records in
*entity_by_id* for which there is no corresponding record in *entity*. In
*entity* table it's much worse because ~7000 records in *entity_by_id* are
missing and it was growing much faster.
We were using LOCAL_QUROUM. Two datacenters. We didn't get any exceptions
while inserts or deletes. BatchQuery from cqlengine has been used.
if BatchQuery is not used:
{code}
with BatchQuery() as b:
-entity.batch(b).save()
-entity_by_id = EntityById.copy_fields_from(entity)
-entity_by_id.batch(b).save()
+entity.save()
+entity_by_id = EntityById.copy_fields_from(entity)
+entity_by_id.save()
{code}
Everything is fine. We don't have more inconsistencies. I've check what
cqlengine generates and seems that works as expected:
{code}
('BEGIN BATCH\n UPDATE sync.entity SET name = %(4)s WHERE user_id =
%(0)s AND data_type_id = %(1)s AND version = %(2)s AND id = %(3)s\n
INSERT INTO sync.entity_by_id (user_id, id, parent_id, deleted,
folder, data_type_id, version) VALUES (%(5)s, %(6)s, %(7)s, %(8)s,
%(9)s, %(10)s, %(11)s)\nAPPLY BATCH;',)
{code}
Schemas:
{code}
CREATE TABLE entity (
user_id text,
data_type_id int,
version bigint,
id text,
cache_guid text,
client_defined_unique_tag text,
ctime timestamp,
deleted boolean,
folder boolean,
mtime timestamp,
name text,
originator_client_item_id text,
parent_id text,
position blob,
server_defined_unique_tag text,
specifics blob,
PRIMARY KEY (user_id, data_type_id, version, id)
) WITH CLUSTERING ORDER BY (data_type_id ASC, version ASC, id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching
[jira] [Commented] (CASSANDRA-8493) nodetool options parsing doesn't allow -h hostname to be at the end anymore
[
https://issues.apache.org/jira/browse/CASSANDRA-8493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280119#comment-14280119
]
Anuja Mandlecha commented on CASSANDRA-8493:
I fired simple $ ./nodetool command using cassandra 2.1.2 and it gave me
following console output:
user@ubuntu-vm:~/apache-cassandra-2.1.2/bin$ ./nodetool
usage: nodetool [(-u username | --username username)]
[(-pw password | --password password)] [(-h host | --host host)]
[(-p port | --port port)]
[(-pwf passwordFilePath | --password-file passwordFilePath)]
command
[args]
The most commonly used nodetool commands are:
So according to the usage, the command (ring in your example) should always
come at the end as:
$ ./nodetool -h 127.0.0.1 ring
nodetool options parsing doesn't allow -h hostname to be at the end anymore
--
Key: CASSANDRA-8493
URL: https://issues.apache.org/jira/browse/CASSANDRA-8493
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Jeremiah Jordan
Priority: Minor
This used to work:
{noformat}
$ ./nodetool ring -h 127.0.0.1
Error: The keyspace 127.0.0.1, does not exist
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8627) Support Total/Recent latency histogram metrics for range slices
[ https://issues.apache.org/jira/browse/CASSANDRA-8627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-8627: --- Attachment: 8627.txt Support Total/Recent latency histogram metrics for range slices --- Key: CASSANDRA-8627 URL: https://issues.apache.org/jira/browse/CASSANDRA-8627 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Lohfink Assignee: Sam Tunnicliffe Labels: metrics Fix For: 2.1.3 Attachments: 8627.txt The Metrics histogram is pretty bad at non-normal data like latencies and (empirically tested and theoretically) is untrustworthy at 99th percentile. For applications that care about the percentiles having the more statistically accurate version is beneficial. Adding the deprecated methods like other latency histograms for CASSANDRA-7338 temporarily would help. This is just for 2.1 branch. CASSANDRA-5657 solves everything in 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError
[
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280184#comment-14280184
]
Andrei Ivanov commented on CASSANDRA-8548:
--
Could someone help me solve the problem with compacting large rows? Sorry if
this shouldn't be asked here, but I get a similar assertion error.
Nodetool Cleanup - java.lang.AssertionError
---
Key: CASSANDRA-8548
URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
Project: Cassandra
Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
Fix For: 2.0.12
Attachments: 0001-make-sure-we-unmark-compacting.patch
Needed to free up some space on a node but getting the dump below when
running nodetool cleanup.
Tried turning on debug to try to obtain additional details in the logs but
nothing gets added to the logs when running cleanup. Added:
log4j.logger.org.apache.cassandra.db=DEBUG
in log4j-server.properties
See the stack trace below:
root@cassandra-019:~# nodetool cleanup
{code}Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
at
org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
at
org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
at
org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.IllegalArgumentException
at
[jira] [Comment Edited] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError
[
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280184#comment-14280184
]
Andrei Ivanov edited comment on CASSANDRA-8548 at 1/16/15 1:18 PM:
---
Could someone help me solve the problem with compacting large rows? Sorry if
this shouldn't be asked here, but I get a similar assertion error.
I was able to cancel all the compactions, but scrubbing still throws the same
error.
was (Author: aivanov93):
Could someone help me solve the problem with compacting large rows? Sorry if
this shouldn't be asked here, but I get a similar assertion error.
Nodetool Cleanup - java.lang.AssertionError
---
Key: CASSANDRA-8548
URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
Project: Cassandra
Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
Fix For: 2.0.12
Attachments: 0001-make-sure-we-unmark-compacting.patch
Needed to free up some space on a node but getting the dump below when
running nodetool cleanup.
Tried turning on debug to try to obtain additional details in the logs but
nothing gets added to the logs when running cleanup. Added:
log4j.logger.org.apache.cassandra.db=DEBUG
in log4j-server.properties
See the stack trace below:
root@cassandra-019:~# nodetool cleanup
{code}Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
at
org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
at
org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
at
org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at
[jira] [Commented] (CASSANDRA-8493) nodetool options parsing doesn't allow -h hostname to be at the end anymore
[
https://issues.apache.org/jira/browse/CASSANDRA-8493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280204#comment-14280204
]
Jeremiah Jordan commented on CASSANDRA-8493:
Correct. But in Cassandra 2.0 and earlier it was not required to be before the
command. This broke all our scripts after upgrading to 2.1. So we might want
to have it act like it did previously.
nodetool options parsing doesn't allow -h hostname to be at the end anymore
--
Key: CASSANDRA-8493
URL: https://issues.apache.org/jira/browse/CASSANDRA-8493
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Jeremiah Jordan
Priority: Minor
This used to work:
{noformat}
$ ./nodetool ring -h 127.0.0.1
Error: The keyspace 127.0.0.1, does not exist
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: DirectByteBuffer compatible LZ4 methods
Repository: cassandra
Updated Branches:
refs/heads/trunk a6ae07f16 - 3dd9c38a1
DirectByteBuffer compatible LZ4 methods
Patch by Branimir Lambov; reviewed by tjake for CASSANDRA-7039
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3dd9c38a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3dd9c38a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3dd9c38a
Branch: refs/heads/trunk
Commit: 3dd9c38a14ae593591d46279e2f99a032288bd65
Parents: a6ae07f
Author: T Jake Luciani j...@apache.org
Authored: Mon Dec 8 15:18:07 2014 -0500
Committer: T Jake Luciani j...@apache.org
Committed: Fri Jan 16 09:34:13 2015 -0500
--
CHANGES.txt| 1 +
build.xml | 2 +-
lib/licenses/lz4-1.2.0.txt | 235
lib/licenses/lz4-1.3.0.txt | 235
lib/lz4-1.2.0.jar | Bin 165505 - 0 bytes
lib/lz4-1.3.0.jar | Bin 0 - 277311 bytes
6 files changed, 237 insertions(+), 236 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3dd9c38a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 37630e6..27b511a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0
+ * DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039)
* Add role based access control (CASSANDRA-7653)
* Group sstables for anticompaction correctly (CASSANDRA-8578)
* Add ReadFailureException to native protocol, respond
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3dd9c38a/build.xml
--
diff --git a/build.xml b/build.xml
index 06c79e0..baf6a77 100644
--- a/build.xml
+++ b/build.xml
@@ -317,7 +317,7 @@
scm connection=${scm.connection}
developerConnection=${scm.developerConnection} url=${scm.url}/
dependencyManagement
dependency groupId=org.xerial.snappy artifactId=snappy-java
version=1.0.5/
- dependency groupId=net.jpountz.lz4 artifactId=lz4
version=1.2.0/
+ dependency groupId=net.jpountz.lz4 artifactId=lz4
version=1.3.0/
dependency groupId=com.ning artifactId=compress-lzf
version=0.8.4/
dependency groupId=com.google.guava artifactId=guava
version=16.0/
dependency groupId=commons-cli artifactId=commons-cli
version=1.1/
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3dd9c38a/lib/licenses/lz4-1.2.0.txt
--
diff --git a/lib/licenses/lz4-1.2.0.txt b/lib/licenses/lz4-1.2.0.txt
deleted file mode 100644
index 7f3ef36..000
--- a/lib/licenses/lz4-1.2.0.txt
+++ /dev/null
@@ -1,235 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
-http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- License shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- Licensor shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- Legal Entity shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- control means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- You (or Your) shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- Source form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- Object form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- Work shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- Derivative Works shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial
[jira] [Resolved] (CASSANDRA-7039) DirectByteBuffer compatible LZ4 methods
[ https://issues.apache.org/jira/browse/CASSANDRA-7039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani resolved CASSANDRA-7039. --- Resolution: Fixed Reviewer: T Jake Luciani committed DirectByteBuffer compatible LZ4 methods --- Key: CASSANDRA-7039 URL: https://issues.apache.org/jira/browse/CASSANDRA-7039 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Branimir Lambov Priority: Minor Labels: performance Fix For: 3.0 Attachments: 7039.patch As we move more things off-heap, it's becoming more and more essential to be able to use DirectByteBuffer (or native pointers) in various places. Unfortunately LZ4 doesn't currently support this operation, despite being JNI based - this means we both have to perform unnecessary copies to de/compress data from DBB, but also we can stall GC as any JNI method operating over a java array using the GetPrimitiveArrayCritical enters a critical section that prevents GC for its duration. This means STWs will be at least as long any running compression/decompression (and no GC will happen until they complete, so it's additive). We should temporarily fork (and then resubmit upstream) jpountz-lz4 to support operating over a native pointer, so that we can pass a DBB or a raw pointer we have allocated ourselves. This will help improve performance when flushing the new offheap memtables, as well as enable us to implement CASSANDRA-6726 and finish CASSANDRA-4338. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Support direct buffer decompression for reads
Repository: cassandra
Updated Branches:
refs/heads/trunk 3dd9c38a1 - 895ec3ea1
Support direct buffer decompression for reads
Patch by tjake; reviewed by Branimir Lambov for CASSANDRA-8464
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/895ec3ea
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/895ec3ea
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/895ec3ea
Branch: refs/heads/trunk
Commit: 895ec3ea1e68b74ee025317f57a09c1ef0b512dd
Parents: 3dd9c38
Author: T Jake Luciani j...@apache.org
Authored: Fri Jan 16 09:37:06 2015 -0500
Committer: T Jake Luciani j...@apache.org
Committed: Fri Jan 16 09:38:42 2015 -0500
--
CHANGES.txt | 1 +
conf/cassandra-env.sh | 1 +
.../compress/CompressedRandomAccessReader.java | 179 +--
.../io/compress/CompressionMetadata.java| 11 +-
.../io/compress/DeflateCompressor.java | 17 ++
.../cassandra/io/compress/ICompressor.java | 12 ++
.../cassandra/io/compress/LZ4Compressor.java| 43 -
.../cassandra/io/compress/SnappyCompressor.java | 13 ++
.../cassandra/io/sstable/format/Version.java| 2 -
.../io/sstable/format/big/BigFormat.java| 20 +--
.../org/apache/cassandra/io/util/FileUtils.java | 5 +-
.../cassandra/io/util/MmappedSegmentedFile.java | 1 +
.../cassandra/io/util/RandomAccessReader.java | 16 +-
.../compress/CompressedInputStream.java | 15 +-
.../compress/CompressedStreamReader.java| 2 +-
.../org/apache/cassandra/utils/FBUtilities.java | 67 +++
.../CompressedRandomAccessReaderTest.java | 6 +-
.../cassandra/io/compress/CompressorTest.java | 133 ++
.../io/compress/LZ4CompressorTest.java | 84 -
.../compress/CompressedInputStreamTest.java | 2 +-
20 files changed, 474 insertions(+), 156 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/895ec3ea/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 27b511a..79181e1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0
+ * Support direct buffer decompression for reads (CASSANDRA-8464)
* DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039)
* Add role based access control (CASSANDRA-7653)
* Group sstables for anticompaction correctly (CASSANDRA-8578)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/895ec3ea/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index 58605ca..f9641ed 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -251,6 +251,7 @@ fi
# uncomment to have Cassandra JVM log internal method compilation (developers
only)
# JVM_OPTS=$JVM_OPTS -XX:+UnlockDiagnosticVMOptions -XX:+LogCompilation
+# JVM_OPTS=$JVM_OPTS -XX:+UnlockCommercialFeatures -XX:+FlightRecorder
# Prefer binding to IPv4 network intefaces (when net.ipv6.bindv6only=1). See
# http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6342561 (short version:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/895ec3ea/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
index dca5ade..57abba9 100644
---
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
+++
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
@@ -19,14 +19,22 @@ package org.apache.cassandra.io.compress;
import java.io.*;
import java.nio.ByteBuffer;
+import java.nio.MappedByteBuffer;
+import java.nio.channels.FileChannel;
+import java.util.Map;
+import java.util.TreeMap;
import java.util.concurrent.ThreadLocalRandom;
import java.util.zip.Adler32;
-import java.util.zip.CRC32;
-import java.util.zip.Checksum;
+
+import com.google.common.primitives.Ints;
+
+import org.apache.cassandra.config.Config;
+import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.io.FSReadError;
import org.apache.cassandra.io.sstable.CorruptSSTableException;
import org.apache.cassandra.io.util.CompressedPoolingSegmentedFile;
+import org.apache.cassandra.io.util.FileUtils;
import org.apache.cassandra.io.util.PoolingSegmentedFile;
import org.apache.cassandra.io.util.RandomAccessReader;
import org.apache.cassandra.utils.FBUtilities;
@@ -37,6 +45,8 @@ import org.apache.cassandra.utils.FBUtilities;
*/
public class CompressedRandomAccessReader extends RandomAccessReader
{
+
[jira] [Updated] (CASSANDRA-8560) Make CassandraException be an unchecked exception
[
https://issues.apache.org/jira/browse/CASSANDRA-8560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-8560:
Assignee: Robert Stupp
Make CassandraException be an unchecked exception
-
Key: CASSANDRA-8560
URL: https://issues.apache.org/jira/browse/CASSANDRA-8560
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Robert Stupp
Priority: Minor
Fix For: 3.0
{{CassandraException}} (which is the base class of our query validation and
execution exception, including {{InvalidRequestException}},
{{UnavailableException}}, ...) is a checked exception. Those exceptions are
pervasive and are rarely meant to be caught within Cassandra since they are
meant for reporting problems to the end user and so I'm not convinced the
benefit of checked exceptions outweight the cost of having to put throws
everywhere.
Concretely, the fact that these are checked exception is currently a pain for
2 outstanding tickets:
* CASSANDRA-8528: as Robert put it, it forces to touch half of the source
files just to add a throws/catch even in code that can never use UDFs
* CASSANDRA-8099: the ticket transform some code (in StorageProxy for
instance) to iterators, but an iterator can't throw checked exception. In
fact, the current WIP patch for that ticket already switch
{{CassandraException}} to extend {{RuntimeException}} for that very reason.
I understand that checked vs unchecked exceptions is an old debate with
proponent of both camp, but I'm pretty sure the costs of checked outweight
the cons in that case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8421) Cassandra 2.1.1 Cassandra 2.1.2 UDT not returning value for LIST type as UDT
[
https://issues.apache.org/jira/browse/CASSANDRA-8421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280339#comment-14280339
]
Benjamin Lerer commented on CASSANDRA-8421:
---
It seems that there are 2 issues:
* When retrieving the partition keys from the index they appear to not be in
the correct order. If they have been inserted with a unit tests they are
returned in the correct order but not if they have been inserted manually. This
could means that the problem has occured during the index compaction.
This problem is probably responsible for the fact that we had some trouble to
reproduce the issue with some unit tests.
* The second problem is that if a record is filtered out within
{{CompositesSearcher}} for a collection value index all the following records
with the same prefix will be rejected.
The second part of the problem can be reproduced with the following modified
version of the {{testOneIndex}} unit test:
{code}
@Test
public void testOneIndex() throws Throwable
{
String fieldmap = createType(CREATE TYPE %s (key text, value text));
createTable(CREATE TABLE %s (e int PRIMARY KEY, b int, f listfrozen
+ fieldmap + , s int));
createIndex(CREATE INDEX ON %s(f));
for(int i = 0; i 17; i++)
{
execute(INSERT INTO %s (e, b, f, s) VALUES (?, ?, [{key: 'state',
value: 'Dubai'}], ?), i, i+1, 4);
}
for(int i = 17; i 20; i++)
{
execute(INSERT INTO %s (e, b, f, s) VALUES (?, ?, [{key: 'state',
value: 'Dubai'}], ?), i, i+1, 3);
}
UntypedResultSet resultSet = execute(SELECT * FROM %s WHERE f CONTAINS
{key: 'state', value: 'Dubai'} AND s=3 allow filtering);
Assert.assertEquals(3, resultSet.size());
}
{code}
Cassandra 2.1.1 Cassandra 2.1.2 UDT not returning value for LIST type as UDT
--
Key: CASSANDRA-8421
URL: https://issues.apache.org/jira/browse/CASSANDRA-8421
Project: Cassandra
Issue Type: Bug
Components: API
Environment: single node cassandra
Reporter: madheswaran
Assignee: Benjamin Lerer
Fix For: 3.0, 2.1.3
Attachments: 8421-unittest.txt, entity_data.csv
I using List and its data type is UDT.
UDT:
{code}
CREATE TYPE
fieldmap (
key text,
value text
);
{code}
TABLE:
{code}
CREATE TABLE entity (
entity_id uuid PRIMARY KEY,
begining int,
domain text,
domain_type text,
entity_template_name text,
field_values listfieldmap,
global_entity_type text,
revision_time timeuuid,
status_key int,
status_name text,
uuid timeuuid
) {code}
INDEX:
{code}
CREATE INDEX entity_domain_idx_1 ON galaxy_dev.entity (domain);
CREATE INDEX entity_field_values_idx_1 ON galaxy_dev.entity (field_values);
CREATE INDEX entity_global_entity_type_idx_1 ON galaxy_dev.entity (gen_type );
{code}
QUERY
{code}
SELECT * FROM entity WHERE status_key 3 and field_values contains {key:
'userName', value: 'Sprint5_22'} and gen_type = 'USER' and domain =
'S4_1017.abc.com' allow filtering;
{code}
The above query return value for some row and not for many rows but those
rows and data's are exist.
Observation:
If I execute query with other than field_maps, then it returns value. I
suspect the problem with LIST with UDT.
I have single node cassadra DB. Please let me know why this strange behavior
from cassandra.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8631) long-test MeteredFlusherTest fails with heap OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-8631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-8631:
--
Assignee: Benedict
long-test MeteredFlusherTest fails with heap OOM
Key: CASSANDRA-8631
URL: https://issues.apache.org/jira/browse/CASSANDRA-8631
Project: Cassandra
Issue Type: Test
Components: Tests
Environment: cassandra-2.0 commit 90780b5
Reporter: Michael Shuler
Assignee: Benedict
Priority: Minor
Labels: unit-test
I attempted to find a spot where this test actually passes, and I was unable
to do so. I tried back to 1.1 HEAD and various release tags under jdk7 and
jdk6 for = 1.2 tags. In 1.2.0 with jdk6, the test times out with
java.lang.OutOfMemoryError: GC overhead limit exceeded, so it's similar.
{noformat}
[junit] Testcase:
testManyMemtables(org.apache.cassandra.db.MeteredFlusherTest):Caused an
ERROR
[junit] Java heap space
[junit] java.lang.OutOfMemoryError: Java heap space
[junit] at java.nio.HeapByteBuffer.init(HeapByteBuffer.java:57)
[junit] at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
[junit] at
org.apache.cassandra.utils.SlabAllocator$Region.init(SlabAllocator.java:157)
[junit] at
org.apache.cassandra.utils.SlabAllocator$Region.init(SlabAllocator.java:131)
[junit] at
org.apache.cassandra.utils.SlabAllocator.getRegion(SlabAllocator.java:101)
[junit] at
org.apache.cassandra.utils.SlabAllocator.allocate(SlabAllocator.java:73)
[junit] at
org.apache.cassandra.utils.Allocator.clone(Allocator.java:30)
[junit] at org.apache.cassandra.db.Column.localCopy(Column.java:277)
[junit] at org.apache.cassandra.db.Memtable$1.apply(Memtable.java:114)
[junit] at org.apache.cassandra.db.Memtable$1.apply(Memtable.java:111)
[junit] at
org.apache.cassandra.db.AtomicSortedColumns.addAllWithSizeDelta(AtomicSortedColumns.java:194)
[junit] at org.apache.cassandra.db.Memtable.resolve(Memtable.java:218)
[junit] at org.apache.cassandra.db.Memtable.put(Memtable.java:165)
[junit] at
org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:901)
[junit] at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:374)
[junit] at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339)
[junit] at
org.apache.cassandra.db.RowMutation.applyUnsafe(RowMutation.java:216)
[junit] at
org.apache.cassandra.db.MeteredFlusherTest.testManyMemtables(MeteredFlusherTest.java:59)
[junit]
[junit]
[junit] Test org.apache.cassandra.db.MeteredFlusherTest FAILED
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8636) Inconsistencies between two tables if BATCH used
[
https://issues.apache.org/jira/browse/CASSANDRA-8636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-8636:
---
Reproduced In: 2.1.2
Fix Version/s: 2.1.3
Inconsistencies between two tables if BATCH used
Key: CASSANDRA-8636
URL: https://issues.apache.org/jira/browse/CASSANDRA-8636
Project: Cassandra
Issue Type: Bug
Environment: Cassandra 2.1.2, cqlengine 0.20.0, Debian Wheezy
Reporter: mlowicki
Fix For: 2.1.3
Two tables:
* First one *entity* has log-like structure - whenever entity is modified we
create new version of it and put into the table with new mtime which is part
of compound key. Old one is removed.
* Second one called *entity_by_id* is manually managed index for *entity*. By
having only id you can get basic entity attributes from *entity_by_id*.
While adding entity we do two inserts - to *entity* and *entity_by_id* (in
this order)
While deleting we do the same using the same order so first we remove record
from entity table.
It turned out that these two tables were inconsistent. We had ~260 records in
*entity_by_id* for which there is no corresponding record in *entity*. In
*entity* table it's much worse because ~7000 records in *entity_by_id* are
missing and it was growing much faster.
We were using LOCAL_QUROUM. Two datacenters. We didn't get any exceptions
while inserts or deletes. BatchQuery from cqlengine has been used.
if BatchQuery is not used:
{code}
with BatchQuery() as b:
-entity.batch(b).save()
-entity_by_id = EntityById.copy_fields_from(entity)
-entity_by_id.batch(b).save()
+entity.save()
+entity_by_id = EntityById.copy_fields_from(entity)
+entity_by_id.save()
{code}
Everything is fine. We don't have more inconsistencies. I've check what
cqlengine generates and seems that works as expected:
{code}
('BEGIN BATCH\n UPDATE sync.entity SET name = %(4)s WHERE user_id =
%(0)s AND data_type_id = %(1)s AND version = %(2)s AND id = %(3)s\n
INSERT INTO sync.entity_by_id (user_id, id, parent_id, deleted,
folder, data_type_id, version) VALUES (%(5)s, %(6)s, %(7)s, %(8)s,
%(9)s, %(10)s, %(11)s)\nAPPLY BATCH;',)
{code}
Schemas:
{code}
CREATE TABLE entity (
user_id text,
data_type_id int,
version bigint,
id text,
cache_guid text,
client_defined_unique_tag text,
ctime timestamp,
deleted boolean,
folder boolean,
mtime timestamp,
name text,
originator_client_item_id text,
parent_id text,
position blob,
server_defined_unique_tag text,
specifics blob,
PRIMARY KEY (user_id, data_type_id, version, id)
) WITH CLUSTERING ORDER BY (data_type_id ASC, version ASC, id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
CREATE INDEX index_entity_parent_id ON sync.entity (parent_id);
CREATE TABLE entity_by_id (
user_id text,
id text,
cache_guid text,
data_type_id int,
deleted boolean,
folder boolean,
originator_client_item_id text,
parent_id text,
version bigint,
PRIMARY KEY (user_id, id)
) WITH CLUSTERING ORDER BY (id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
CREATE INDEX index_entity_by_id_parent_id ON entity_by_id (parent_id);
{code}
Previously we had many batch size exceeded warnings but limit is increased
now as sometimes we put many KB of data into *specifics* blob field
(*batch_size_warn_threshold_in_kb* is set now to 20).
We had similar data model in other project where we have the same issue.
[jira] [Commented] (CASSANDRA-8608) Fix cassandra-stress bug introduced by 7964, and cleanup related code a little
[ https://issues.apache.org/jira/browse/CASSANDRA-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280438#comment-14280438 ] T Jake Luciani commented on CASSANDRA-8608: --- Well it seems to now use more than a single node but it isn't respecting the limit per batch. You can test it easily with the profile from CASSANDRA-8632 Fix cassandra-stress bug introduced by 7964, and cleanup related code a little -- Key: CASSANDRA-8608 URL: https://issues.apache.org/jira/browse/CASSANDRA-8608 Project: Cassandra Issue Type: Bug Reporter: Benedict Assignee: Benedict 7964 had a very basic mistake present when wiring up writes - the limit was not reset, so only the first row would be returned (the fact _ANY_ row was returned was itself sort-of a bug in this scenario, and I've changed this also). At the same time I've changed the definition of limit to lastRow so that the related functionality is clearer and more obviously correct. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8631) long-test MeteredFlusherTest fails with heap OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-8631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280444#comment-14280444
]
Benedict commented on CASSANDRA-8631:
-
Do we need this test, or want to fix it? The problem is almost certainly that
1s is not frequent enough to keep anywhere near our goal memory limit when
running on a local only spam test. Since we only enforce memory constraints in
2.0 by the flush queue length (by default, iirc, this is 4, so 4s have to
elapse before we actually start _blocking_ writes) this happily allows us to go
well over limit. It's generally a known problem in 2.0. To make it work we need
to artificially run MeteredFlusher (MUCH) more frequently, at which point we
aren't testing C* as it actually runs. Alternatively, we could fix a rate of
work production that should guarantee 1s is often enough to keep us within our
limits, but this doesn't really perform much of use that a simple unit test
couldn't - it just checks metered flusher runs, not that it enforces anything
much.
Personally I think we should just drop this test from 2.0, since we know 2.0's
metered flusher approach is a bit broken. The test functions as it was
originally designed in 2.1, because memory limits are directly enforced,
whereas designing a _useful_ test that also works with 2.0's semantics seems
nontrivial. However making either of the two suggested modifications so it at
least _appears_ to do something useful _is_ pretty trivial. So if somebody has
a strong opinion on this (I don't really), please chime in!
long-test MeteredFlusherTest fails with heap OOM
Key: CASSANDRA-8631
URL: https://issues.apache.org/jira/browse/CASSANDRA-8631
Project: Cassandra
Issue Type: Test
Components: Tests
Environment: cassandra-2.0 commit 90780b5
Reporter: Michael Shuler
Assignee: Benedict
Priority: Minor
Labels: unit-test
I attempted to find a spot where this test actually passes, and I was unable
to do so. I tried back to 1.1 HEAD and various release tags under jdk7 and
jdk6 for = 1.2 tags. In 1.2.0 with jdk6, the test times out with
java.lang.OutOfMemoryError: GC overhead limit exceeded, so it's similar.
{noformat}
[junit] Testcase:
testManyMemtables(org.apache.cassandra.db.MeteredFlusherTest):Caused an
ERROR
[junit] Java heap space
[junit] java.lang.OutOfMemoryError: Java heap space
[junit] at java.nio.HeapByteBuffer.init(HeapByteBuffer.java:57)
[junit] at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
[junit] at
org.apache.cassandra.utils.SlabAllocator$Region.init(SlabAllocator.java:157)
[junit] at
org.apache.cassandra.utils.SlabAllocator$Region.init(SlabAllocator.java:131)
[junit] at
org.apache.cassandra.utils.SlabAllocator.getRegion(SlabAllocator.java:101)
[junit] at
org.apache.cassandra.utils.SlabAllocator.allocate(SlabAllocator.java:73)
[junit] at
org.apache.cassandra.utils.Allocator.clone(Allocator.java:30)
[junit] at org.apache.cassandra.db.Column.localCopy(Column.java:277)
[junit] at org.apache.cassandra.db.Memtable$1.apply(Memtable.java:114)
[junit] at org.apache.cassandra.db.Memtable$1.apply(Memtable.java:111)
[junit] at
org.apache.cassandra.db.AtomicSortedColumns.addAllWithSizeDelta(AtomicSortedColumns.java:194)
[junit] at org.apache.cassandra.db.Memtable.resolve(Memtable.java:218)
[junit] at org.apache.cassandra.db.Memtable.put(Memtable.java:165)
[junit] at
org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:901)
[junit] at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:374)
[junit] at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339)
[junit] at
org.apache.cassandra.db.RowMutation.applyUnsafe(RowMutation.java:216)
[junit] at
org.apache.cassandra.db.MeteredFlusherTest.testManyMemtables(MeteredFlusherTest.java:59)
[junit]
[junit]
[junit] Test org.apache.cassandra.db.MeteredFlusherTest FAILED
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8608) Fix cassandra-stress bug introduced by 7964, and cleanup related code a little
[ https://issues.apache.org/jira/browse/CASSANDRA-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280445#comment-14280445 ] Benedict commented on CASSANDRA-8608: - See CASSANDRA-8633. That's a different problem. Fix cassandra-stress bug introduced by 7964, and cleanup related code a little -- Key: CASSANDRA-8608 URL: https://issues.apache.org/jira/browse/CASSANDRA-8608 Project: Cassandra Issue Type: Bug Reporter: Benedict Assignee: Benedict 7964 had a very basic mistake present when wiring up writes - the limit was not reset, so only the first row would be returned (the fact _ANY_ row was returned was itself sort-of a bug in this scenario, and I've changed this also). At the same time I've changed the definition of limit to lastRow so that the related functionality is clearer and more obviously correct. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8608) Fix cassandra-stress bug introduced by 7964, and cleanup related code a little
[
https://issues.apache.org/jira/browse/CASSANDRA-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280451#comment-14280451
]
T Jake Luciani commented on CASSANDRA-8608:
---
ok.
If I raise the limit 10x I start getting this error...
{code}
Exception in thread Thread-64 java.lang.AssertionError
at
org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.advance(PartitionIterator.java:446)
at
org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.seek(PartitionIterator.java:351)
at
org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.reset(PartitionIterator.java:259)
at
org.apache.cassandra.stress.generate.PartitionIterator.reset(PartitionIterator.java:101)
at org.apache.cassandra.stress.Operation.ready(Operation.java:110)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:288)
{code}
Fix cassandra-stress bug introduced by 7964, and cleanup related code a little
--
Key: CASSANDRA-8608
URL: https://issues.apache.org/jira/browse/CASSANDRA-8608
Project: Cassandra
Issue Type: Bug
Reporter: Benedict
Assignee: Benedict
7964 had a very basic mistake present when wiring up writes - the limit was
not reset, so only the first row would be returned (the fact _ANY_ row was
returned was itself sort-of a bug in this scenario, and I've changed this
also). At the same time I've changed the definition of limit to lastRow so
that the related functionality is clearer and more obviously correct.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8637) Update LZ4 library to 1.3.0
Luke Gordon created CASSANDRA-8637: -- Summary: Update LZ4 library to 1.3.0 Key: CASSANDRA-8637 URL: https://issues.apache.org/jira/browse/CASSANDRA-8637 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Luke Gordon Priority: Minor Fix For: 2.1.3 This is a follow up of https://issues.apache.org/jira/browse/CASSANDRA-7573. A fix for https://github.com/jpountz/lz4-java/pull/46 has been released in LZ4 1.3.0. Apache Cassandra is still using 1.2.0. Can this library be upgraded to correct the issue outlined in 7573? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8608) Fix cassandra-stress bug introduced by 7964, and cleanup related code a little
[ https://issues.apache.org/jira/browse/CASSANDRA-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280453#comment-14280453 ] Benedict commented on CASSANDRA-8608: - _raise_ the limit? Could you paste the exact yaml/settings? Fix cassandra-stress bug introduced by 7964, and cleanup related code a little -- Key: CASSANDRA-8608 URL: https://issues.apache.org/jira/browse/CASSANDRA-8608 Project: Cassandra Issue Type: Bug Reporter: Benedict Assignee: Benedict 7964 had a very basic mistake present when wiring up writes - the limit was not reset, so only the first row would be returned (the fact _ANY_ row was returned was itself sort-of a bug in this scenario, and I've changed this also). At the same time I've changed the definition of limit to lastRow so that the related functionality is clearer and more obviously correct. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8608) Fix cassandra-stress bug introduced by 7964, and cleanup related code a little
[
https://issues.apache.org/jira/browse/CASSANDRA-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280458#comment-14280458
]
T Jake Luciani commented on CASSANDRA-8608:
---
{code}
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 24bab09..0dbc6b0 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -547,7 +547,7 @@ column_index_size_in_kb: 64
batch_size_warn_threshold_in_kb: 5
# Fail any batch exceeding this value. 50kb (10x warn threshold) by default.
-batch_size_fail_threshold_in_kb: 50
+batch_size_fail_threshold_in_kb: 500
{code}
The stress cmd line is:
{code}
./tools/bin/cassandra-stress user profile=partition.yaml ops\(insert=1\)
n=100 -rate threads=50
{code}
Fix cassandra-stress bug introduced by 7964, and cleanup related code a little
--
Key: CASSANDRA-8608
URL: https://issues.apache.org/jira/browse/CASSANDRA-8608
Project: Cassandra
Issue Type: Bug
Reporter: Benedict
Assignee: Benedict
7964 had a very basic mistake present when wiring up writes - the limit was
not reset, so only the first row would be returned (the fact _ANY_ row was
returned was itself sort-of a bug in this scenario, and I've changed this
also). At the same time I've changed the definition of limit to lastRow so
that the related functionality is clearer and more obviously correct.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8608) Fix cassandra-stress bug introduced by 7964, and cleanup related code a little
[ https://issues.apache.org/jira/browse/CASSANDRA-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280592#comment-14280592 ] T Jake Luciani commented on CASSANDRA-8608: --- +1 with the fix you pushed for the above issue Fix cassandra-stress bug introduced by 7964, and cleanup related code a little -- Key: CASSANDRA-8608 URL: https://issues.apache.org/jira/browse/CASSANDRA-8608 Project: Cassandra Issue Type: Bug Reporter: Benedict Assignee: Benedict 7964 had a very basic mistake present when wiring up writes - the limit was not reset, so only the first row would be returned (the fact _ANY_ row was returned was itself sort-of a bug in this scenario, and I've changed this also). At the same time I've changed the definition of limit to lastRow so that the related functionality is clearer and more obviously correct. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8638) CQLSH -f option should ignore BOM in files
Sotirios Delimanolis created CASSANDRA-8638:
---
Summary: CQLSH -f option should ignore BOM in files
Key: CASSANDRA-8638
URL: https://issues.apache.org/jira/browse/CASSANDRA-8638
Project: Cassandra
Issue Type: Improvement
Components: API, Drivers (now out of tree)
Environment: Red Hat linux
Reporter: Sotirios Delimanolis
Priority: Trivial
I fell in byte order mark trap trying to execute a CQL script through CQLSH.
The file contained the simple (plus BOM)
{noformat}
CREATE KEYSPACE IF NOT EXISTS xobni WITH replication = {'class':
'SimpleStrategy', 'replication_factor': '3'} AND durable_writes = true;
-- and another CREATE TABLE bucket_flags query
{noformat}
I executed the script
{noformat}
[~]$ cqlsh --file /home/selimanolis/Schema/patches/setup.cql
/home/selimanolis/Schema/patches/setup.cql:2:Invalid syntax at char 1
/home/selimanolis/Schema/patches/setup.cql:2: CREATE KEYSPACE IF NOT EXISTS
test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '3'}
AND durable_writes = true;
/home/selimanolis/Schema/patches/setup.cql:2: ^
/home/selimanolis/Schema/patches/setup.cql:22:ConfigurationException:
ErrorMessage code=2300 [Query invalid because of configuration issue]
message=Cannot add column family 'bucket_flags' to non existing keyspace
'test'.
{noformat}
I realized much later that the file had a BOM which was seemingly screwing with
how CQLSH parsed the file.
It would be nice to have CQLSH ignore the BOM when processing files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8638) CQLSH -f option should ignore BOM in files
[
https://issues.apache.org/jira/browse/CASSANDRA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sotirios Delimanolis updated CASSANDRA-8638:
Description:
I fell in byte order mark trap trying to execute a CQL script through CQLSH.
The file contained the simple (plus BOM)
{noformat}
CREATE KEYSPACE IF NOT EXISTS xobni WITH replication = {'class':
'SimpleStrategy', 'replication_factor': '3'} AND durable_writes = true;
-- and another CREATE TABLE bucket_flags query
{noformat}
I executed the script
{noformat}
[~]$ cqlsh --file /home/selimanolis/Schema/patches/setup.cql
/home/selimanolis/Schema/patches/setup.cql:2:Invalid syntax at char 1
/home/selimanolis/Schema/patches/setup.cql:2: CREATE KEYSPACE IF NOT EXISTS
test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '3'}
AND durable_writes = true;
/home/selimanolis/Schema/patches/setup.cql:2: ^
/home/selimanolis/Schema/patches/setup.cql:22:ConfigurationException:
ErrorMessage code=2300 [Query invalid because of configuration issue]
message=Cannot add column family 'bucket_flags' to non existing keyspace
'test'.
{noformat}
I realized much later that the file had a BOM which was seemingly screwing with
how CQLSH parsed the file.
It would be nice to have CQLSH ignore the BOM when processing files.
(The C# driver also failed when executing the content of the script
{noformat}
var session = cluster.Connect ();
string script = File.ReadAllText (schemaLocation);
session.Execute (script);
{noformat}
but this can be avoided by ignoring the BOM application-side.)
was:
I fell in byte order mark trap trying to execute a CQL script through CQLSH.
The file contained the simple (plus BOM)
{noformat}
CREATE KEYSPACE IF NOT EXISTS xobni WITH replication = {'class':
'SimpleStrategy', 'replication_factor': '3'} AND durable_writes = true;
-- and another CREATE TABLE bucket_flags query
{noformat}
I executed the script
{noformat}
[~]$ cqlsh --file /home/selimanolis/Schema/patches/setup.cql
/home/selimanolis/Schema/patches/setup.cql:2:Invalid syntax at char 1
/home/selimanolis/Schema/patches/setup.cql:2: CREATE KEYSPACE IF NOT EXISTS
test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '3'}
AND durable_writes = true;
/home/selimanolis/Schema/patches/setup.cql:2: ^
/home/selimanolis/Schema/patches/setup.cql:22:ConfigurationException:
ErrorMessage code=2300 [Query invalid because of configuration issue]
message=Cannot add column family 'bucket_flags' to non existing keyspace
'test'.
{noformat}
I realized much later that the file had a BOM which was seemingly screwing with
how CQLSH parsed the file.
It would be nice to have CQLSH ignore the BOM when processing files.
CQLSH -f option should ignore BOM in files
--
Key: CASSANDRA-8638
URL: https://issues.apache.org/jira/browse/CASSANDRA-8638
Project: Cassandra
Issue Type: Improvement
Components: API, Drivers (now out of tree)
Environment: Red Hat linux
Reporter: Sotirios Delimanolis
Priority: Trivial
I fell in byte order mark trap trying to execute a CQL script through CQLSH.
The file contained the simple (plus BOM)
{noformat}
CREATE KEYSPACE IF NOT EXISTS xobni WITH replication = {'class':
'SimpleStrategy', 'replication_factor': '3'} AND durable_writes = true;
-- and another CREATE TABLE bucket_flags query
{noformat}
I executed the script
{noformat}
[~]$ cqlsh --file /home/selimanolis/Schema/patches/setup.cql
/home/selimanolis/Schema/patches/setup.cql:2:Invalid syntax at char 1
/home/selimanolis/Schema/patches/setup.cql:2: CREATE KEYSPACE IF NOT EXISTS
test WITH replication = {'class': 'SimpleStrategy', 'replication_factor':
'3'} AND durable_writes = true;
/home/selimanolis/Schema/patches/setup.cql:2: ^
/home/selimanolis/Schema/patches/setup.cql:22:ConfigurationException:
ErrorMessage code=2300 [Query invalid because of configuration issue]
message=Cannot add column family 'bucket_flags' to non existing keyspace
'test'.
{noformat}
I realized much later that the file had a BOM which was seemingly screwing
with how CQLSH parsed the file.
It would be nice to have CQLSH ignore the BOM when processing files.
(The C# driver also failed when executing the content of the script
{noformat}
var session = cluster.Connect ();
string script = File.ReadAllText (schemaLocation);
session.Execute (script);
{noformat}
but this can be avoided by ignoring the BOM application-side.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8614) Select optimal CRC32 implementation at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-8614: Assignee: Ariel Weisberg (was: Benedict) Select optimal CRC32 implementation at runtime -- Key: CASSANDRA-8614 URL: https://issues.apache.org/jira/browse/CASSANDRA-8614 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Ariel Weisberg Assignee: Ariel Weisberg Labels: performance Attachments: 8614.patch, Sample.java JDK 8 has support for an intrinsic for CRC32 that runs at 12-13 gigabytes/sec per core in my quick and dirty test. PureJavaCRC32 is 800 megabytes/sec if I recall and it has a lookup table that evicts random cache lines every time it runs. In order to capture the benefit of that when it is available we can select a CRC32 implementation at startup in a static block. If JDK 8 is not what is running we can fall back to the existing PureJavaCRC32 implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8614) Select optimal CRC32 implementation at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280634#comment-14280634 ] Benedict commented on CASSANDRA-8614: - Not sure what I keep pressing to accidentally adopt other people's tickets Select optimal CRC32 implementation at runtime -- Key: CASSANDRA-8614 URL: https://issues.apache.org/jira/browse/CASSANDRA-8614 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Ariel Weisberg Assignee: Ariel Weisberg Labels: performance Attachments: 8614.patch, Sample.java JDK 8 has support for an intrinsic for CRC32 that runs at 12-13 gigabytes/sec per core in my quick and dirty test. PureJavaCRC32 is 800 megabytes/sec if I recall and it has a lookup table that evicts random cache lines every time it runs. In order to capture the benefit of that when it is available we can select a CRC32 implementation at startup in a static block. If JDK 8 is not what is running we can fall back to the existing PureJavaCRC32 implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7032) Improve vnode allocation
[ https://issues.apache.org/jira/browse/CASSANDRA-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Branimir Lambov updated CASSANDRA-7032: --- Attachment: TestVNodeAllocation.java New version with acceptable performance (adding a node to 1000 node 256 vnode cluster takes ~0.1s). No changes in the algorithm in this one, just restructures the way things are stored to replace a lot of previously complex operations with O(1) steps. Improve vnode allocation Key: CASSANDRA-7032 URL: https://issues.apache.org/jira/browse/CASSANDRA-7032 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Branimir Lambov Labels: performance, vnodes Fix For: 3.0 Attachments: TestVNodeAllocation.java, TestVNodeAllocation.java, TestVNodeAllocation.java, TestVNodeAllocation.java, TestVNodeAllocation.java It's been known for a little while that random vnode allocation causes hotspots of ownership. It should be possible to improve dramatically on this with deterministic allocation. I have quickly thrown together a simple greedy algorithm that allocates vnodes efficiently, and will repair hotspots in a randomly allocated cluster gradually as more nodes are added, and also ensures that token ranges are fairly evenly spread between nodes (somewhat tunably so). The allocation still permits slight discrepancies in ownership, but it is bound by the inverse of the size of the cluster (as opposed to random allocation, which strangely gets worse as the cluster size increases). I'm sure there is a decent dynamic programming solution to this that would be even better. If on joining the ring a new node were to CAS a shared table where a canonical allocation of token ranges lives after running this (or a similar) algorithm, we could then get guaranteed bounds on the ownership distribution in a cluster. This will also help for CASSANDRA-6696. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8638) CQLSH -f option should ignore BOM in files
[
https://issues.apache.org/jira/browse/CASSANDRA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-8638:
-
Labels: cqlsh lhf (was: lhf)
CQLSH -f option should ignore BOM in files
--
Key: CASSANDRA-8638
URL: https://issues.apache.org/jira/browse/CASSANDRA-8638
Project: Cassandra
Issue Type: Improvement
Components: API, Drivers (now out of tree)
Environment: Red Hat linux
Reporter: Sotirios Delimanolis
Priority: Trivial
Labels: cqlsh, lhf
I fell in byte order mark trap trying to execute a CQL script through CQLSH.
The file contained the simple (plus BOM)
{noformat}
CREATE KEYSPACE IF NOT EXISTS xobni WITH replication = {'class':
'SimpleStrategy', 'replication_factor': '3'} AND durable_writes = true;
-- and another CREATE TABLE bucket_flags query
{noformat}
I executed the script
{noformat}
[~]$ cqlsh --file /home/selimanolis/Schema/patches/setup.cql
/home/selimanolis/Schema/patches/setup.cql:2:Invalid syntax at char 1
/home/selimanolis/Schema/patches/setup.cql:2: CREATE KEYSPACE IF NOT EXISTS
test WITH replication = {'class': 'SimpleStrategy', 'replication_factor':
'3'} AND durable_writes = true;
/home/selimanolis/Schema/patches/setup.cql:2: ^
/home/selimanolis/Schema/patches/setup.cql:22:ConfigurationException:
ErrorMessage code=2300 [Query invalid because of configuration issue]
message=Cannot add column family 'bucket_flags' to non existing keyspace
'test'.
{noformat}
I realized much later that the file had a BOM which was seemingly screwing
with how CQLSH parsed the file.
It would be nice to have CQLSH ignore the BOM when processing files.
(The C# driver also failed when executing the content of the script
{noformat}
var session = cluster.Connect ();
string script = File.ReadAllText (schemaLocation);
session.Execute (script);
{noformat}
but this can be avoided by ignoring the BOM application-side.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8638) CQLSH -f option should ignore BOM in files
[
https://issues.apache.org/jira/browse/CASSANDRA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-8638:
-
Labels: lhf (was: )
CQLSH -f option should ignore BOM in files
--
Key: CASSANDRA-8638
URL: https://issues.apache.org/jira/browse/CASSANDRA-8638
Project: Cassandra
Issue Type: Improvement
Components: API, Drivers (now out of tree)
Environment: Red Hat linux
Reporter: Sotirios Delimanolis
Priority: Trivial
Labels: cqlsh, lhf
I fell in byte order mark trap trying to execute a CQL script through CQLSH.
The file contained the simple (plus BOM)
{noformat}
CREATE KEYSPACE IF NOT EXISTS xobni WITH replication = {'class':
'SimpleStrategy', 'replication_factor': '3'} AND durable_writes = true;
-- and another CREATE TABLE bucket_flags query
{noformat}
I executed the script
{noformat}
[~]$ cqlsh --file /home/selimanolis/Schema/patches/setup.cql
/home/selimanolis/Schema/patches/setup.cql:2:Invalid syntax at char 1
/home/selimanolis/Schema/patches/setup.cql:2: CREATE KEYSPACE IF NOT EXISTS
test WITH replication = {'class': 'SimpleStrategy', 'replication_factor':
'3'} AND durable_writes = true;
/home/selimanolis/Schema/patches/setup.cql:2: ^
/home/selimanolis/Schema/patches/setup.cql:22:ConfigurationException:
ErrorMessage code=2300 [Query invalid because of configuration issue]
message=Cannot add column family 'bucket_flags' to non existing keyspace
'test'.
{noformat}
I realized much later that the file had a BOM which was seemingly screwing
with how CQLSH parsed the file.
It would be nice to have CQLSH ignore the BOM when processing files.
(The C# driver also failed when executing the content of the script
{noformat}
var session = cluster.Connect ();
string script = File.ReadAllText (schemaLocation);
session.Execute (script);
{noformat}
but this can be avoided by ignoring the BOM application-side.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8638) CQLSH -f option should ignore BOM in files
[
https://issues.apache.org/jira/browse/CASSANDRA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sotirios Delimanolis updated CASSANDRA-8638:
Description:
I fell in byte order mark trap trying to execute a CQL script through CQLSH.
The file contained the simple (plus BOM)
{noformat}
CREATE KEYSPACE IF NOT EXISTS xobni WITH replication = {'class':
'SimpleStrategy', 'replication_factor': '3'} AND durable_writes = true;
-- and another CREATE TABLE bucket_flags query
{noformat}
I executed the script
{noformat}
[~]$ cqlsh --file /home/selimanolis/Schema/patches/setup.cql
/home/selimanolis/Schema/patches/setup.cql:2:Invalid syntax at char 1
/home/selimanolis/Schema/patches/setup.cql:2: CREATE KEYSPACE IF NOT EXISTS
test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '3'}
AND durable_writes = true;
/home/selimanolis/Schema/patches/setup.cql:2: ^
/home/selimanolis/Schema/patches/setup.cql:22:ConfigurationException:
ErrorMessage code=2300 [Query invalid because of configuration issue]
message=Cannot add column family 'bucket_flags' to non existing keyspace
'test'.
{noformat}
I realized much later that the file had a BOM which was seemingly screwing with
how CQLSH parsed the file.
It would be nice to have CQLSH ignore the BOM when processing files.
was:
I fell in byte order mark trap trying to execute a CQL script through CQLSH.
The file contained the simple (plus BOM)
{noformat}
CREATE KEYSPACE IF NOT EXISTS xobni WITH replication = {'class':
'SimpleStrategy', 'replication_factor': '3'} AND durable_writes = true;
-- and another CREATE TABLE bucket_flags query
{noformat}
I executed the script
{noformat}
[~]$ cqlsh --file /home/selimanolis/Schema/patches/setup.cql
/home/selimanolis/Schema/patches/setup.cql:2:Invalid syntax at char 1
/home/selimanolis/Schema/patches/setup.cql:2: CREATE KEYSPACE IF NOT EXISTS
test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '3'}
AND durable_writes = true;
/home/selimanolis/Schema/patches/setup.cql:2: ^
/home/selimanolis/Schema/patches/setup.cql:22:ConfigurationException:
ErrorMessage code=2300 [Query invalid because of configuration issue]
message=Cannot add column family 'bucket_flags' to non existing keyspace
'test'.
{noformat}
I realized much later that the file had a BOM which was seemingly screwing with
how CQLSH parsed the file.
It would be nice to have CQLSH ignore the BOM when processing files.
(The C# driver also failed when executing the content of the script
{noformat}
var session = cluster.Connect ();
string script = File.ReadAllText (schemaLocation);
session.Execute (script);
{noformat}
but this can be avoided by ignoring the BOM application-side.)
CQLSH -f option should ignore BOM in files
--
Key: CASSANDRA-8638
URL: https://issues.apache.org/jira/browse/CASSANDRA-8638
Project: Cassandra
Issue Type: Improvement
Components: API, Drivers (now out of tree)
Environment: Red Hat linux
Reporter: Sotirios Delimanolis
Priority: Trivial
Labels: cqlsh, lhf
I fell in byte order mark trap trying to execute a CQL script through CQLSH.
The file contained the simple (plus BOM)
{noformat}
CREATE KEYSPACE IF NOT EXISTS xobni WITH replication = {'class':
'SimpleStrategy', 'replication_factor': '3'} AND durable_writes = true;
-- and another CREATE TABLE bucket_flags query
{noformat}
I executed the script
{noformat}
[~]$ cqlsh --file /home/selimanolis/Schema/patches/setup.cql
/home/selimanolis/Schema/patches/setup.cql:2:Invalid syntax at char 1
/home/selimanolis/Schema/patches/setup.cql:2: CREATE KEYSPACE IF NOT EXISTS
test WITH replication = {'class': 'SimpleStrategy', 'replication_factor':
'3'} AND durable_writes = true;
/home/selimanolis/Schema/patches/setup.cql:2: ^
/home/selimanolis/Schema/patches/setup.cql:22:ConfigurationException:
ErrorMessage code=2300 [Query invalid because of configuration issue]
message=Cannot add column family 'bucket_flags' to non existing keyspace
'test'.
{noformat}
I realized much later that the file had a BOM which was seemingly screwing
with how CQLSH parsed the file.
It would be nice to have CQLSH ignore the BOM when processing files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8639) Can OOM on CL replay with dense mutations
T Jake Luciani created CASSANDRA-8639: - Summary: Can OOM on CL replay with dense mutations Key: CASSANDRA-8639 URL: https://issues.apache.org/jira/browse/CASSANDRA-8639 Project: Cassandra Issue Type: Bug Reporter: T Jake Luciani Priority: Minor Fix For: 2.1.3 If you write dense mutations with many clustering keys, the replay of the CL can quickly overwhelm a node on startup. This looks to be caused by the fact we only ensure there are 1000 mutations in flight at a time. but those mutations could have thousands of cells in them. A better approach would be to limit the CL replay to the amount of memory in flight using cell.unsharedHeapSize() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8638) CQLSH -f option should ignore BOM in files
[
https://issues.apache.org/jira/browse/CASSANDRA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-8638:
---
Component/s: (was: Drivers (now out of tree))
(was: API)
Tools
Fix Version/s: 2.1.3
CQLSH -f option should ignore BOM in files
--
Key: CASSANDRA-8638
URL: https://issues.apache.org/jira/browse/CASSANDRA-8638
Project: Cassandra
Issue Type: Improvement
Components: Tools
Environment: Red Hat linux
Reporter: Sotirios Delimanolis
Priority: Trivial
Labels: cqlsh, lhf
Fix For: 2.1.3
I fell in byte order mark trap trying to execute a CQL script through CQLSH.
The file contained the simple (plus BOM)
{noformat}
CREATE KEYSPACE IF NOT EXISTS xobni WITH replication = {'class':
'SimpleStrategy', 'replication_factor': '3'} AND durable_writes = true;
-- and another CREATE TABLE bucket_flags query
{noformat}
I executed the script
{noformat}
[~]$ cqlsh --file /home/selimanolis/Schema/patches/setup.cql
/home/selimanolis/Schema/patches/setup.cql:2:Invalid syntax at char 1
/home/selimanolis/Schema/patches/setup.cql:2: CREATE KEYSPACE IF NOT EXISTS
test WITH replication = {'class': 'SimpleStrategy', 'replication_factor':
'3'} AND durable_writes = true;
/home/selimanolis/Schema/patches/setup.cql:2: ^
/home/selimanolis/Schema/patches/setup.cql:22:ConfigurationException:
ErrorMessage code=2300 [Query invalid because of configuration issue]
message=Cannot add column family 'bucket_flags' to non existing keyspace
'test'.
{noformat}
I realized much later that the file had a BOM which was seemingly screwing
with how CQLSH parsed the file.
It would be nice to have CQLSH ignore the BOM when processing files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8637) Update LZ4 library to 1.3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-8637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280838#comment-14280838 ] Aleksey Yeschenko commented on CASSANDRA-8637: -- We just upgraded it in 3.0 (see CASSANDRA-7039), but we tend to not upgrade our bundled libraries in minor releases, for risk reasons. Update LZ4 library to 1.3.0 --- Key: CASSANDRA-8637 URL: https://issues.apache.org/jira/browse/CASSANDRA-8637 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Luke Gordon Priority: Minor Fix For: 2.1.3 This is a follow up of https://issues.apache.org/jira/browse/CASSANDRA-7573. A fix for https://github.com/jpountz/lz4-java/pull/46 has been released in LZ4 1.3.0. Apache Cassandra is still using 1.2.0. Can this library be upgraded to correct the issue outlined in 7573? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8528) Add an ExecutionException to the protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-8528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8528: - Reviewer: Aleksey Yeschenko (was: Tyler Hobbs) Add an ExecutionException to the protocol - Key: CASSANDRA-8528 URL: https://issues.apache.org/jira/browse/CASSANDRA-8528 Project: Cassandra Issue Type: Sub-task Reporter: Sylvain Lebresne Assignee: Robert Stupp Labels: client-impacting, protocolv4 Fix For: 3.0 Attachments: 8528-001.txt With the introduction of UDF, we should add an ExecutionException (or FunctionExecutionException or something like that) to the exceptions that can be sent back to client. We can't guarantee that UDFs won't throw and none of our existing exception is terribly adapted to report such event to the client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280945#comment-14280945
]
Ariel Weisberg commented on CASSANDRA-7438:
---
I ran the benchmark on the develop branch today using a c3.8xlarge and profiled
with flight recorder. There is definitely some contention on the lock in JNA. I
also see a little in AbstractQueuedSynchronizer from locking the segments.
along with some park/unpark activity.
I built jemalloc (-march=native --disable-fill --disable-stats). The Ubuntu
package compiles at o2 instead of o3. I am getting full utilization across 30
threads if I increase the number of segments to 256 otherwise it hovers around
2600% (with 30 threads). It cuts in half the number of instances of contention
in the profiler.
The workload settings you ran with resulted in a lot of cache (ohcache, not CPU
cache) misses. I think a real workload where the cache is useful will have more
hits.
One note about the benchmark, building the histogram of buckets is not a
lightweight operation. I think that should be off by default. I removed it for
my testing. Otherwise it looks ok. Using the Timer as shared state in a
micro-benchmarks is probably not the way to go. I would have a timer per driver
thread and then aggregate.
I am running 1-30 threads and it will take a few hours to finish. I am going to
look into benchmarking inside C* and comparing the existing cache
implementation to OHC now.
I used this which gave me mostly cache hits and filled up quite a bit of RAM.
It takes a minute or two to fill the cache.
{noformat}
#!/bin/sh
LD_PRELOAD=~/jemalloc-3.6.0/lib/libjemalloc.so.1 \
java -Xmx8g -XX:+UnlockCommercialFeatures -XX:+FlightRecorder \
-DDISABLE_JEMALLOC=true \
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=7091
-Dcom.sun.management.jmxremote.local.only=false \
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false \
-Djava.rmi.server.hostname=ec2-54-172-234-230.compute-1.amazonaws.com \
-jar ohc-benchmark/target/ohc-benchmark-0.3-SNAPSHOT.jar \
-rkd 'gaussian(1..1500,2)' -wkd 'gaussian(1..1500,2)' -vs
'gaussian(1024..4096,2)' -r .9 -cap 320 \
-d 120 -t 30 \
-sc 256
{noformat}
256 segments, jemalloc LD_PRELOAD, -DDISABLE_JEMALLOC=true
{noformat}
Reads : one/five/fifteen/mean: 2503894/2143858/2036336/2459949
count: 295258886
min/max/mean/stddev: 0.00047/ 0.76172/ 0.00652/ 0.03865
75/95/98/99/999/median: 0.00439/ 0.00697/ 0.01147/ 0.03458/
0.75864/ 0.00342
Writes: one/five/fifteen/mean: 278134/238242/226326/273275
count:32800525
min/max/mean/stddev: 0.00176/ 0.89665/ 0.00945/ 0.03986
75/95/98/99/999/median: 0.00719/ 0.01180/ 0.01816/ 0.11640/
0.89006/ 0.00556
{noformat}
256 segments, jemalloc via jna
{noformat}
Reads : one/five/fifteen/mean: 2343872/1458688/1159829/2387622
count: 286635526
min/max/mean/stddev: 0.00054/ 0.97114/ 0.00756/ 0.04664
75/95/98/99/999/median: 0.00435/ 0.00675/ 0.00985/ 0.05139/
0.95959/ 0.00341
Writes: one/five/fifteen/mean: 260376/162076/128883/265250
count:31843705
min/max/mean/stddev: 0.00267/ 0.70586/ 0.01502/ 0.05161
75/95/98/99/999/median: 0.01049/ 0.01695/ 0.04193/ 0.36639/
0.70331/ 0.00859
{noformat}
default segments, jemalloc LD_PRELOAD, -DDISABLE_JEMALLOC=true
{noformat}
Reads : one/five/fifteen/mean: 2148677/1630379/1448226/2202878
count: 264549288
min/max/mean/stddev: 0.00035/ 0.66081/ 0.00820/ 0.03519
75/95/98/99/999/median: 0.00435/ 0.01247/ 0.05423/ 0.20834/
0.65286/ 0.00323
Writes: one/five/fifteen/mean: 238699/180945/160641/244767
count:29395103
min/max/mean/stddev: 0.00172/ 0.39821/ 0.01120/ 0.03079
75/95/98/99/999/median: 0.00805/ 0.02124/ 0.08665/ 0.18473/
0.39776/ 0.00574
{noformat}
Serializing Row cache alternative (Fully off heap)
--
Key: CASSANDRA-7438
URL: https://issues.apache.org/jira/browse/CASSANDRA-7438
Project: Cassandra
Issue Type: Improvement
Components: Core
Environment: Linux
Reporter: Vijay
Assignee: Robert Stupp
Labels: performance
Fix For: 3.0
Attachments: 0001-CASSANDRA-7438.patch, tests.zip
Currently SerializingCache is partially off heap, keys are still stored in
JVM heap as BB,
* There is a higher GC costs
[jira] [Created] (CASSANDRA-8640) Paxos requires all nodes for CAS
Anthony Cozzie created CASSANDRA-8640:
-
Summary: Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Sylvain Lebresne
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Fails owing to the order of operations. I am not sure just adding parentheses
will fix the problem, though, as the original code differentiated between
pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anthony Cozzie updated CASSANDRA-8640:
--
Description:
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not sure
just adding parentheses will fix the problem, though, as the original code
differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
was:
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Fails owing to the order of operations. I am not sure just adding parentheses
will fix the problem, though, as the original code differentiated between
pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Sylvain Lebresne
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anthony Cozzie updated CASSANDRA-8640:
--
Description:
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not sure
just adding parentheses will fix the problem, though, as the original code
differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
was:
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not sure
just adding parentheses will fix the problem, though, as the original code
differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Sylvain Lebresne
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-8640:
-
Fix Version/s: 2.0.12
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Sylvain Lebresne
Fix For: 2.0.12
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-8640:
-
Assignee: Anthony Cozzie (was: Sylvain Lebresne)
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Anthony Cozzie
Fix For: 2.0.12
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-8640:
-
Reviewer: Aleksey Yeschenko
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Anthony Cozzie
Fix For: 2.0.12
Attachments: 0001-Fix-parentheses-bug-in-Paxos.patch
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anthony Cozzie updated CASSANDRA-8640:
--
Attachment: 0001-Fix-parentheses-bug-in-Paxos.patch
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Anthony Cozzie
Fix For: 2.0.12
Attachments: 0001-Fix-parentheses-bug-in-Paxos.patch
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-8640:
-
Fix Version/s: 2.1.3
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Anthony Cozzie
Fix For: 2.0.12, 2.1.3
Attachments: 0001-Fix-parentheses-bug-in-Paxos.patch
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8502) Static columns returning null for pages after first
[ https://issues.apache.org/jira/browse/CASSANDRA-8502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-8502: --- Attachment: 8502-2.0.txt Static columns returning null for pages after first --- Key: CASSANDRA-8502 URL: https://issues.apache.org/jira/browse/CASSANDRA-8502 Project: Cassandra Issue Type: Bug Components: Core Reporter: Flavien Charlon Assignee: Tyler Hobbs Fix For: 2.1.3, 2.0.13 Attachments: 8502-2.0.txt, null-static-column.txt When paging is used for a query containing a static column, the first page contains the right value for the static column, but subsequent pages have null null for the static column instead of the expected value. Repro steps: - Create a table with a static column - Create a partition with 500 cells - Using cqlsh, query that partition Actual result: - You will see that first, the static column appears as expected, but if you press a key after ---MORE---, the static columns will appear as null. See the attached file for a repro of the output. I am using a single node cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Fix LWT requiring all nodes for CAS
Repository: cassandra Updated Branches: refs/heads/cassandra-2.0 90780b550 - 61bd68885 Fix LWT requiring all nodes for CAS patch by Anthony Cozzie; reviewed by Aleksey Yeschenko for CASSANDRA-8640 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/61bd6888 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/61bd6888 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/61bd6888 Branch: refs/heads/cassandra-2.0 Commit: 61bd68885e1a40a8c40522a73bf9d269b58868e9 Parents: 90780b5 Author: Anthony Cozzie acoz...@datastax.com Authored: Sat Jan 17 03:38:46 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sat Jan 17 03:38:46 2015 +0300 -- CHANGES.txt | 2 +- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/61bd6888/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 45b2b9c..54a6096 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -28,7 +28,7 @@ * Fix NPE after dropping a keyspace (CASSANDRA-8332) * Fix error message on read repair timeouts (CASSANDRA-7947) * Default DTCS base_time_seconds changed to 60 (CASSANDRA-8417) - * Refuse Paxos operation with more than one pending endpoint (CASSANDRA-8346) + * Refuse Paxos operation with more than one pending endpoint (CASSANDRA-8346, 8640) * Throw correct exception when trying to bind a keyspace or table name (CASSANDRA-6952) * Make HHOM.compact synchronized (CASSANDRA-8416) http://git-wip-us.apache.org/repos/asf/cassandra/blob/61bd6888/src/java/org/apache/cassandra/service/StorageProxy.java -- diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java b/src/java/org/apache/cassandra/service/StorageProxy.java index 45af1c8..1fd4079 100644 --- a/src/java/org/apache/cassandra/service/StorageProxy.java +++ b/src/java/org/apache/cassandra/service/StorageProxy.java @@ -314,7 +314,7 @@ public class StorageProxy implements StorageProxyMBean pendingEndpoints = ImmutableList.copyOf(Iterables.filter(pendingEndpoints, isLocalDc)); } int participants = pendingEndpoints.size() + naturalEndpoints.size(); -int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833 +int requiredParticipants = (participants + 1) / 2; // See CASSANDRA-8346, CASSANDRA-833 ListInetAddress liveEndpoints = ImmutableList.copyOf(Iterables.filter(Iterables.concat(naturalEndpoints, pendingEndpoints), IAsyncCallback.isAlive)); if (liveEndpoints.size() requiredParticipants) throw new UnavailableException(consistencyForPaxos, requiredParticipants, liveEndpoints.size());
[1/2] cassandra git commit: Fix LWT requiring all nodes for CAS
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 a3234e191 - e4e3fd91c Fix LWT requiring all nodes for CAS patch by Anthony Cozzie; reviewed by Aleksey Yeschenko for CASSANDRA-8640 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/61bd6888 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/61bd6888 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/61bd6888 Branch: refs/heads/cassandra-2.1 Commit: 61bd68885e1a40a8c40522a73bf9d269b58868e9 Parents: 90780b5 Author: Anthony Cozzie acoz...@datastax.com Authored: Sat Jan 17 03:38:46 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sat Jan 17 03:38:46 2015 +0300 -- CHANGES.txt | 2 +- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/61bd6888/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 45b2b9c..54a6096 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -28,7 +28,7 @@ * Fix NPE after dropping a keyspace (CASSANDRA-8332) * Fix error message on read repair timeouts (CASSANDRA-7947) * Default DTCS base_time_seconds changed to 60 (CASSANDRA-8417) - * Refuse Paxos operation with more than one pending endpoint (CASSANDRA-8346) + * Refuse Paxos operation with more than one pending endpoint (CASSANDRA-8346, 8640) * Throw correct exception when trying to bind a keyspace or table name (CASSANDRA-6952) * Make HHOM.compact synchronized (CASSANDRA-8416) http://git-wip-us.apache.org/repos/asf/cassandra/blob/61bd6888/src/java/org/apache/cassandra/service/StorageProxy.java -- diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java b/src/java/org/apache/cassandra/service/StorageProxy.java index 45af1c8..1fd4079 100644 --- a/src/java/org/apache/cassandra/service/StorageProxy.java +++ b/src/java/org/apache/cassandra/service/StorageProxy.java @@ -314,7 +314,7 @@ public class StorageProxy implements StorageProxyMBean pendingEndpoints = ImmutableList.copyOf(Iterables.filter(pendingEndpoints, isLocalDc)); } int participants = pendingEndpoints.size() + naturalEndpoints.size(); -int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833 +int requiredParticipants = (participants + 1) / 2; // See CASSANDRA-8346, CASSANDRA-833 ListInetAddress liveEndpoints = ImmutableList.copyOf(Iterables.filter(Iterables.concat(naturalEndpoints, pendingEndpoints), IAsyncCallback.isAlive)); if (liveEndpoints.size() requiredParticipants) throw new UnavailableException(consistencyForPaxos, requiredParticipants, liveEndpoints.size());
[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e4e3fd91 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e4e3fd91 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e4e3fd91 Branch: refs/heads/cassandra-2.1 Commit: e4e3fd91cdcb6cdad9c72bdb3214f4105971e964 Parents: a3234e1 61bd688 Author: Aleksey Yeschenko alek...@apache.org Authored: Sat Jan 17 03:41:34 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sat Jan 17 03:41:34 2015 +0300 -- CHANGES.txt | 2 +- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e3fd91/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e3fd91/src/java/org/apache/cassandra/service/StorageProxy.java --
[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e4e3fd91 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e4e3fd91 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e4e3fd91 Branch: refs/heads/trunk Commit: e4e3fd91cdcb6cdad9c72bdb3214f4105971e964 Parents: a3234e1 61bd688 Author: Aleksey Yeschenko alek...@apache.org Authored: Sat Jan 17 03:41:34 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sat Jan 17 03:41:34 2015 +0300 -- CHANGES.txt | 2 +- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e3fd91/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e3fd91/src/java/org/apache/cassandra/service/StorageProxy.java --
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8c4db1b0 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8c4db1b0 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8c4db1b0 Branch: refs/heads/trunk Commit: 8c4db1b0e8907d6d5c532051775a09f6541c7449 Parents: 895ec3e e4e3fd9 Author: Aleksey Yeschenko alek...@apache.org Authored: Sat Jan 17 03:43:03 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sat Jan 17 03:43:03 2015 +0300 -- CHANGES.txt | 2 +- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/8c4db1b0/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/8c4db1b0/src/java/org/apache/cassandra/service/StorageProxy.java --
[1/3] cassandra git commit: Fix LWT requiring all nodes for CAS
Repository: cassandra Updated Branches: refs/heads/trunk 895ec3ea1 - 8c4db1b0e Fix LWT requiring all nodes for CAS patch by Anthony Cozzie; reviewed by Aleksey Yeschenko for CASSANDRA-8640 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/61bd6888 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/61bd6888 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/61bd6888 Branch: refs/heads/trunk Commit: 61bd68885e1a40a8c40522a73bf9d269b58868e9 Parents: 90780b5 Author: Anthony Cozzie acoz...@datastax.com Authored: Sat Jan 17 03:38:46 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sat Jan 17 03:38:46 2015 +0300 -- CHANGES.txt | 2 +- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/61bd6888/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 45b2b9c..54a6096 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -28,7 +28,7 @@ * Fix NPE after dropping a keyspace (CASSANDRA-8332) * Fix error message on read repair timeouts (CASSANDRA-7947) * Default DTCS base_time_seconds changed to 60 (CASSANDRA-8417) - * Refuse Paxos operation with more than one pending endpoint (CASSANDRA-8346) + * Refuse Paxos operation with more than one pending endpoint (CASSANDRA-8346, 8640) * Throw correct exception when trying to bind a keyspace or table name (CASSANDRA-6952) * Make HHOM.compact synchronized (CASSANDRA-8416) http://git-wip-us.apache.org/repos/asf/cassandra/blob/61bd6888/src/java/org/apache/cassandra/service/StorageProxy.java -- diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java b/src/java/org/apache/cassandra/service/StorageProxy.java index 45af1c8..1fd4079 100644 --- a/src/java/org/apache/cassandra/service/StorageProxy.java +++ b/src/java/org/apache/cassandra/service/StorageProxy.java @@ -314,7 +314,7 @@ public class StorageProxy implements StorageProxyMBean pendingEndpoints = ImmutableList.copyOf(Iterables.filter(pendingEndpoints, isLocalDc)); } int participants = pendingEndpoints.size() + naturalEndpoints.size(); -int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833 +int requiredParticipants = (participants + 1) / 2; // See CASSANDRA-8346, CASSANDRA-833 ListInetAddress liveEndpoints = ImmutableList.copyOf(Iterables.filter(Iterables.concat(naturalEndpoints, pendingEndpoints), IAsyncCallback.isAlive)); if (liveEndpoints.size() requiredParticipants) throw new UnavailableException(consistencyForPaxos, requiredParticipants, liveEndpoints.size());
[jira] [Commented] (CASSANDRA-8502) Static columns returning null for pages after first
[
https://issues.apache.org/jira/browse/CASSANDRA-8502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281070#comment-14281070
]
Tyler Hobbs commented on CASSANDRA-8502:
Since I basically had a patch ready to go before this comment, I went ahead and
attached a 2.0 version of the patch to discuss it. There are some new
[dtests|https://github.com/thobbs/cassandra-dtest/tree/CASSANDRA-8502] that
exercise this pretty well, and the existing unit/dtests seem to be passing.
bq. It'll probably be hacky code-wise, especially in the paging case since the
pager doesn't have a very good way to know if we're querying statics or not
(we'd either have to pass that information down as a boolean to the ctor, or
manually inspect the slices to detect it, both of which are ugly).
Yeah, the patch is a little ugly, but it's not terrible. In the reversed case,
the split happens at {{SliceFromReadCommand.getRow()}} instead of at the paging
level. Checking for statics basically involves seeing if the table has static
columns and seeing if the slice start/finish (depending on reversal) is
EMPTY_BYTE_BUFFER. IMO, the related changes to trimming the results (to avoid
trimming or counting statics) are uglier.
bq. It'll be surprising performance wise. If it's going to be a I'm doing this
and it's slow, how come? Oh, yes, don't do that, it does 2 reads internally,
I'd almost rather not support it.
A ~2x penalty isn't _too_ bad, and at least it only significantly affects
reversed queries on tables with static columns, which should be somewhat rare.
The fact that we have plans to improve this in 3.0 also helps.
Static columns returning null for pages after first
---
Key: CASSANDRA-8502
URL: https://issues.apache.org/jira/browse/CASSANDRA-8502
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Flavien Charlon
Assignee: Tyler Hobbs
Fix For: 2.1.3, 2.0.13
Attachments: 8502-2.0.txt, null-static-column.txt
When paging is used for a query containing a static column, the first page
contains the right value for the static column, but subsequent pages have
null null for the static column instead of the expected value.
Repro steps:
- Create a table with a static column
- Create a partition with 500 cells
- Using cqlsh, query that partition
Actual result:
- You will see that first, the static column appears as expected, but if you
press a key after ---MORE---, the static columns will appear as null.
See the attached file for a repro of the output.
I am using a single node cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[1/2] cassandra git commit: Ninja-fix CASSANDRA-8640
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 e4e3fd91c - f88864cc6 Ninja-fix CASSANDRA-8640 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ce207cba Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ce207cba Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ce207cba Branch: refs/heads/cassandra-2.1 Commit: ce207cba413381986324f2c8edad9e090e5be5b1 Parents: 61bd688 Author: Aleksey Yeschenko alek...@apache.org Authored: Sat Jan 17 04:06:21 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sat Jan 17 04:06:21 2015 +0300 -- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce207cba/src/java/org/apache/cassandra/service/StorageProxy.java -- diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java b/src/java/org/apache/cassandra/service/StorageProxy.java index 1fd4079..fcc9665 100644 --- a/src/java/org/apache/cassandra/service/StorageProxy.java +++ b/src/java/org/apache/cassandra/service/StorageProxy.java @@ -314,7 +314,7 @@ public class StorageProxy implements StorageProxyMBean pendingEndpoints = ImmutableList.copyOf(Iterables.filter(pendingEndpoints, isLocalDc)); } int participants = pendingEndpoints.size() + naturalEndpoints.size(); -int requiredParticipants = (participants + 1) / 2; // See CASSANDRA-8346, CASSANDRA-833 +int requiredParticipants = participants / 2 + 1; // See CASSANDRA-8346, CASSANDRA-833 ListInetAddress liveEndpoints = ImmutableList.copyOf(Iterables.filter(Iterables.concat(naturalEndpoints, pendingEndpoints), IAsyncCallback.isAlive)); if (liveEndpoints.size() requiredParticipants) throw new UnavailableException(consistencyForPaxos, requiredParticipants, liveEndpoints.size());
cassandra git commit: Ninja-fix CASSANDRA-8640
Repository: cassandra Updated Branches: refs/heads/cassandra-2.0 61bd68885 - ce207cba4 Ninja-fix CASSANDRA-8640 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ce207cba Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ce207cba Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ce207cba Branch: refs/heads/cassandra-2.0 Commit: ce207cba413381986324f2c8edad9e090e5be5b1 Parents: 61bd688 Author: Aleksey Yeschenko alek...@apache.org Authored: Sat Jan 17 04:06:21 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sat Jan 17 04:06:21 2015 +0300 -- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce207cba/src/java/org/apache/cassandra/service/StorageProxy.java -- diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java b/src/java/org/apache/cassandra/service/StorageProxy.java index 1fd4079..fcc9665 100644 --- a/src/java/org/apache/cassandra/service/StorageProxy.java +++ b/src/java/org/apache/cassandra/service/StorageProxy.java @@ -314,7 +314,7 @@ public class StorageProxy implements StorageProxyMBean pendingEndpoints = ImmutableList.copyOf(Iterables.filter(pendingEndpoints, isLocalDc)); } int participants = pendingEndpoints.size() + naturalEndpoints.size(); -int requiredParticipants = (participants + 1) / 2; // See CASSANDRA-8346, CASSANDRA-833 +int requiredParticipants = participants / 2 + 1; // See CASSANDRA-8346, CASSANDRA-833 ListInetAddress liveEndpoints = ImmutableList.copyOf(Iterables.filter(Iterables.concat(naturalEndpoints, pendingEndpoints), IAsyncCallback.isAlive)); if (liveEndpoints.size() requiredParticipants) throw new UnavailableException(consistencyForPaxos, requiredParticipants, liveEndpoints.size());
[jira] [Updated] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anthony Cozzie updated CASSANDRA-8640:
--
Attachment: 0001-Fix-quorum-required-nodes-calculation-for-Paxos.patch
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Anthony Cozzie
Fix For: 2.0.12, 2.1.3
Attachments: 0001-Fix-parentheses-bug-in-Paxos.patch,
0001-Fix-quorum-required-nodes-calculation-for-Paxos.patch
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f88864cc Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f88864cc Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f88864cc Branch: refs/heads/cassandra-2.1 Commit: f88864cc6b56c95008896b6cb3e70cb2b8303a9a Parents: e4e3fd9 ce207cb Author: Aleksey Yeschenko alek...@apache.org Authored: Sat Jan 17 04:07:15 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sat Jan 17 04:07:15 2015 +0300 -- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f88864cc/src/java/org/apache/cassandra/service/StorageProxy.java --
[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f88864cc Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f88864cc Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f88864cc Branch: refs/heads/trunk Commit: f88864cc6b56c95008896b6cb3e70cb2b8303a9a Parents: e4e3fd9 ce207cb Author: Aleksey Yeschenko alek...@apache.org Authored: Sat Jan 17 04:07:15 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sat Jan 17 04:07:15 2015 +0300 -- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f88864cc/src/java/org/apache/cassandra/service/StorageProxy.java --
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6ba09994 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6ba09994 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6ba09994 Branch: refs/heads/trunk Commit: 6ba099940d71aee8aaea54ed898a1f4eaebf9296 Parents: 8c4db1b f88864c Author: Aleksey Yeschenko alek...@apache.org Authored: Sat Jan 17 04:08:48 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sat Jan 17 04:08:48 2015 +0300 -- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6ba09994/src/java/org/apache/cassandra/service/StorageProxy.java --
[1/3] cassandra git commit: Ninja-fix CASSANDRA-8640
Repository: cassandra Updated Branches: refs/heads/trunk 8c4db1b0e - 6ba099940 Ninja-fix CASSANDRA-8640 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ce207cba Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ce207cba Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ce207cba Branch: refs/heads/trunk Commit: ce207cba413381986324f2c8edad9e090e5be5b1 Parents: 61bd688 Author: Aleksey Yeschenko alek...@apache.org Authored: Sat Jan 17 04:06:21 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sat Jan 17 04:06:21 2015 +0300 -- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce207cba/src/java/org/apache/cassandra/service/StorageProxy.java -- diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java b/src/java/org/apache/cassandra/service/StorageProxy.java index 1fd4079..fcc9665 100644 --- a/src/java/org/apache/cassandra/service/StorageProxy.java +++ b/src/java/org/apache/cassandra/service/StorageProxy.java @@ -314,7 +314,7 @@ public class StorageProxy implements StorageProxyMBean pendingEndpoints = ImmutableList.copyOf(Iterables.filter(pendingEndpoints, isLocalDc)); } int participants = pendingEndpoints.size() + naturalEndpoints.size(); -int requiredParticipants = (participants + 1) / 2; // See CASSANDRA-8346, CASSANDRA-833 +int requiredParticipants = participants / 2 + 1; // See CASSANDRA-8346, CASSANDRA-833 ListInetAddress liveEndpoints = ImmutableList.copyOf(Iterables.filter(Iterables.concat(naturalEndpoints, pendingEndpoints), IAsyncCallback.isAlive)); if (liveEndpoints.size() requiredParticipants) throw new UnavailableException(consistencyForPaxos, requiredParticipants, liveEndpoints.size());
[jira] [Commented] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281088#comment-14281088
]
Anthony Cozzie commented on CASSANDRA-8640:
---
On second thought, just adding the parentheses won't work; the +1 needs to come
after the division not before.
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Anthony Cozzie
Fix For: 2.0.12, 2.1.3
Attachments: 0001-Fix-parentheses-bug-in-Paxos.patch,
0001-Fix-quorum-required-nodes-calculation-for-Paxos.patch
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281091#comment-14281091
]
Aleksey Yeschenko commented on CASSANDRA-8640:
--
Already ninja-fixed in a follow-up commit.
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Anthony Cozzie
Fix For: 2.0.12, 2.1.3
Attachments: 0001-Fix-parentheses-bug-in-Paxos.patch,
0001-Fix-quorum-required-nodes-calculation-for-Paxos.patch
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko resolved CASSANDRA-8640.
--
Resolution: Fixed
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Anthony Cozzie
Fix For: 2.0.12, 2.1.3
Attachments: 0001-Fix-parentheses-bug-in-Paxos.patch,
0001-Fix-quorum-required-nodes-calculation-for-Paxos.patch
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281107#comment-14281107
]
Jeremiah Jordan commented on CASSANDRA-8640:
Pretty sure this is wrong now from a range movement consistency standpoint. Why
did we change the old formula?
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Anthony Cozzie
Fix For: 2.0.12, 2.1.3
Attachments: 0001-Fix-parentheses-bug-in-Paxos.patch,
0001-Fix-quorum-required-nodes-calculation-for-Paxos.patch
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281113#comment-14281113
]
Aleksey Yeschenko commented on CASSANDRA-8640:
--
We throw if pending 2, so it shouldn't be an issue there
(https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/service/StorageProxy.java#L325)
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Anthony Cozzie
Fix For: 2.0.12, 2.1.3
Attachments: 0001-Fix-parentheses-bug-in-Paxos.patch,
0001-Fix-quorum-required-nodes-calculation-for-Paxos.patch
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281114#comment-14281114
]
Jeremiah Jordan commented on CASSANDRA-8640:
Yeah the only issue was pending 1
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Anthony Cozzie
Fix For: 2.0.12, 2.1.3
Attachments: 0001-Fix-parentheses-bug-in-Paxos.patch,
0001-Fix-quorum-required-nodes-calculation-for-Paxos.patch
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8641) Repair causes a large number of tiny SSTables
Flavien Charlon created CASSANDRA-8641:
--
Summary: Repair causes a large number of tiny SSTables
Key: CASSANDRA-8641
URL: https://issues.apache.org/jira/browse/CASSANDRA-8641
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Ubuntu 14.04
Reporter: Flavien Charlon
I have a 3 nodes cluster with RF = 3, quad core and 32 GB or RAM. I am running
2.1.2 with all the default settings. I'm seeing some strange behaviors during
incremental repair (under write load).
Taking the example of one particular column family, before running an
incremental repair, I have about 13 SSTables. After finishing the incremental
repair, I have over 114000 SSTables.
{noformat}
Table: customers
SSTable count: 114688
Space used (live): 97203707290
Space used (total): 99175455072
Space used by snapshots (total): 0
SSTable Compression Ratio: 0.28281112416526505
Memtable cell count: 0
Memtable data size: 0
Memtable switch count: 1069
Local read count: 0
Local read latency: NaN ms
Local write count: 11548705
Local write latency: 0.030 ms
Pending flushes: 0
Bloom filter false positives: 0
Bloom filter false ratio: 0.0
Bloom filter space used: 144145152
Compacted partition minimum bytes: 311
Compacted partition maximum bytes: 1996099046
Compacted partition mean bytes: 3419
Average live cells per slice (last five minutes): 0.0
Maximum live cells per slice (last five minutes): 0.0
Average tombstones per slice (last five minutes): 0.0
Maximum tombstones per slice (last five minutes): 0.0
{noformat}
Looking at the logs during the repair, it seems Cassandra is struggling to
compact minuscule memtables (often just a few kilobytes):
{noformat}
INFO [CompactionExecutor:337] 2015-01-17 01:44:27,011 CompactionTask.java:251
- Compacted 32 sstables to
[/mnt/data/cassandra/data/business/customers-d9d42d209ccc11e48ca54553c90a9d45/business-customers-ka-228341,].
8,332 bytes to 6,547 (~78% of original) in 80,476ms = 0.78MB/s. 32 total
partitions merged to 32. Partition merge counts were {1:32, }
INFO [CompactionExecutor:337] 2015-01-17 01:45:35,519 CompactionTask.java:251
- Compacted 32 sstables to
[/mnt/data/cassandra/data/business/customers-d9d42d209ccc11e48ca54553c90a9d45/business-customers-ka-229348,].
8,384 bytes to 6,563 (~78% of original) in 6,880ms = 0.000910MB/s. 32 total
partitions merged to 32. Partition merge counts were {1:32, }
INFO [CompactionExecutor:339] 2015-01-17 01:47:46,475 CompactionTask.java:251
- Compacted 32 sstables to
[/mnt/data/cassandra/data/business/customers-d9d42d209ccc11e48ca54553c90a9d45/business-customers-ka-229351,].
8,423 bytes to 6,401 (~75% of original) in 10,416ms = 0.000586MB/s. 32 total
partitions merged to 32. Partition merge counts were {1:32, }
{noformat}
At the end of the repair, the cluster has become unusable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8641) Repair causes a large number of tiny SSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-8641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flavien Charlon updated CASSANDRA-8641:
---
Since Version: (was: 2.1.2)
Repair causes a large number of tiny SSTables
-
Key: CASSANDRA-8641
URL: https://issues.apache.org/jira/browse/CASSANDRA-8641
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Ubuntu 14.04
Reporter: Flavien Charlon
I have a 3 nodes cluster with RF = 3, quad core and 32 GB or RAM. I am
running 2.1.2 with all the default settings. I'm seeing some strange
behaviors during incremental repair (under write load).
Taking the example of one particular column family, before running an
incremental repair, I have about 13 SSTables. After finishing the incremental
repair, I have over 114000 SSTables.
{noformat}
Table: customers
SSTable count: 114688
Space used (live): 97203707290
Space used (total): 99175455072
Space used by snapshots (total): 0
SSTable Compression Ratio: 0.28281112416526505
Memtable cell count: 0
Memtable data size: 0
Memtable switch count: 1069
Local read count: 0
Local read latency: NaN ms
Local write count: 11548705
Local write latency: 0.030 ms
Pending flushes: 0
Bloom filter false positives: 0
Bloom filter false ratio: 0.0
Bloom filter space used: 144145152
Compacted partition minimum bytes: 311
Compacted partition maximum bytes: 1996099046
Compacted partition mean bytes: 3419
Average live cells per slice (last five minutes): 0.0
Maximum live cells per slice (last five minutes): 0.0
Average tombstones per slice (last five minutes): 0.0
Maximum tombstones per slice (last five minutes): 0.0
{noformat}
Looking at the logs during the repair, it seems Cassandra is struggling to
compact minuscule memtables (often just a few kilobytes):
{noformat}
INFO [CompactionExecutor:337] 2015-01-17 01:44:27,011
CompactionTask.java:251 - Compacted 32 sstables to
[/mnt/data/cassandra/data/business/customers-d9d42d209ccc11e48ca54553c90a9d45/business-customers-ka-228341,].
8,332 bytes to 6,547 (~78% of original) in 80,476ms = 0.78MB/s. 32
total partitions merged to 32. Partition merge counts were {1:32, }
INFO [CompactionExecutor:337] 2015-01-17 01:45:35,519
CompactionTask.java:251 - Compacted 32 sstables to
[/mnt/data/cassandra/data/business/customers-d9d42d209ccc11e48ca54553c90a9d45/business-customers-ka-229348,].
8,384 bytes to 6,563 (~78% of original) in 6,880ms = 0.000910MB/s. 32
total partitions merged to 32. Partition merge counts were {1:32, }
INFO [CompactionExecutor:339] 2015-01-17 01:47:46,475
CompactionTask.java:251 - Compacted 32 sstables to
[/mnt/data/cassandra/data/business/customers-d9d42d209ccc11e48ca54553c90a9d45/business-customers-ka-229351,].
8,423 bytes to 6,401 (~75% of original) in 10,416ms = 0.000586MB/s. 32
total partitions merged to 32. Partition merge counts were {1:32, }
{noformat}
At the end of the repair, the cluster has become unusable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8641) Repair causes a large number of tiny SSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-8641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flavien Charlon updated CASSANDRA-8641:
---
Fix Version/s: 2.1.3
Repair causes a large number of tiny SSTables
-
Key: CASSANDRA-8641
URL: https://issues.apache.org/jira/browse/CASSANDRA-8641
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Ubuntu 14.04
Reporter: Flavien Charlon
Fix For: 2.1.3
I have a 3 nodes cluster with RF = 3, quad core and 32 GB or RAM. I am
running 2.1.2 with all the default settings. I'm seeing some strange
behaviors during incremental repair (under write load).
Taking the example of one particular column family, before running an
incremental repair, I have about 13 SSTables. After finishing the incremental
repair, I have over 114000 SSTables.
{noformat}
Table: customers
SSTable count: 114688
Space used (live): 97203707290
Space used (total): 99175455072
Space used by snapshots (total): 0
SSTable Compression Ratio: 0.28281112416526505
Memtable cell count: 0
Memtable data size: 0
Memtable switch count: 1069
Local read count: 0
Local read latency: NaN ms
Local write count: 11548705
Local write latency: 0.030 ms
Pending flushes: 0
Bloom filter false positives: 0
Bloom filter false ratio: 0.0
Bloom filter space used: 144145152
Compacted partition minimum bytes: 311
Compacted partition maximum bytes: 1996099046
Compacted partition mean bytes: 3419
Average live cells per slice (last five minutes): 0.0
Maximum live cells per slice (last five minutes): 0.0
Average tombstones per slice (last five minutes): 0.0
Maximum tombstones per slice (last five minutes): 0.0
{noformat}
Looking at the logs during the repair, it seems Cassandra is struggling to
compact minuscule memtables (often just a few kilobytes):
{noformat}
INFO [CompactionExecutor:337] 2015-01-17 01:44:27,011
CompactionTask.java:251 - Compacted 32 sstables to
[/mnt/data/cassandra/data/business/customers-d9d42d209ccc11e48ca54553c90a9d45/business-customers-ka-228341,].
8,332 bytes to 6,547 (~78% of original) in 80,476ms = 0.78MB/s. 32
total partitions merged to 32. Partition merge counts were {1:32, }
INFO [CompactionExecutor:337] 2015-01-17 01:45:35,519
CompactionTask.java:251 - Compacted 32 sstables to
[/mnt/data/cassandra/data/business/customers-d9d42d209ccc11e48ca54553c90a9d45/business-customers-ka-229348,].
8,384 bytes to 6,563 (~78% of original) in 6,880ms = 0.000910MB/s. 32
total partitions merged to 32. Partition merge counts were {1:32, }
INFO [CompactionExecutor:339] 2015-01-17 01:47:46,475
CompactionTask.java:251 - Compacted 32 sstables to
[/mnt/data/cassandra/data/business/customers-d9d42d209ccc11e48ca54553c90a9d45/business-customers-ka-229351,].
8,423 bytes to 6,401 (~75% of original) in 10,416ms = 0.000586MB/s. 32
total partitions merged to 32. Partition merge counts were {1:32, }
{noformat}
At the end of the repair, the cluster has become unusable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8625) LIST USERS and LIST PERMISSIONS command in cqlsh return Keyspace None not found.
[ https://issues.apache.org/jira/browse/CASSANDRA-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281130#comment-14281130 ] ZhongYu commented on CASSANDRA-8625: Are this bug fixed in 2.1.2? LIST USERS and LIST PERMISSIONS command in cqlsh return Keyspace None not found. -- Key: CASSANDRA-8625 URL: https://issues.apache.org/jira/browse/CASSANDRA-8625 Project: Cassandra Issue Type: Bug Components: Tools Environment: cqlsh 5.0.1 | Cassandra 2.1.2 | CQL spec 3.2.0 | Native protocol v3 Reporter: ZhongYu When open Cassandra authorization and authentication, LIST USERS and LIST PERMISSIONS command in cqlsh not work and always return Keyspace None not found. When I login as super user cassandra and create some users. cassandra@cqlsh list users; Keyspace None not found. cassandra@cqlsh list all permissions; Keyspace None not found. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8625) LIST USERS and LIST PERMISSIONS command in cqlsh return Keyspace None not found.
[ https://issues.apache.org/jira/browse/CASSANDRA-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281193#comment-14281193 ] Aleksey Yeschenko commented on CASSANDRA-8625: -- In the unreleased 2.1.3. LIST USERS and LIST PERMISSIONS command in cqlsh return Keyspace None not found. -- Key: CASSANDRA-8625 URL: https://issues.apache.org/jira/browse/CASSANDRA-8625 Project: Cassandra Issue Type: Bug Components: Tools Environment: cqlsh 5.0.1 | Cassandra 2.1.2 | CQL spec 3.2.0 | Native protocol v3 Reporter: ZhongYu When open Cassandra authorization and authentication, LIST USERS and LIST PERMISSIONS command in cqlsh not work and always return Keyspace None not found. When I login as super user cassandra and create some users. cassandra@cqlsh list users; Keyspace None not found. cassandra@cqlsh list all permissions; Keyspace None not found. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Git Push Summary
Repository: cassandra Updated Tags: refs/tags/2.0.12-tentative [deleted] 90780b550
Git Push Summary
Repository: cassandra Updated Tags: refs/tags/2.0.12-tentative [created] ce207cba4
[jira] [Commented] (CASSANDRA-8637) Update LZ4 library to 1.3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-8637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281244#comment-14281244 ] Luke Gordon commented on CASSANDRA-8637: I can understand wanting to mitigate risk, but this doesn't seem to be uncharacteristic or unprecedented in Apache Cassandra's history. The LZ4 version bump is on a minor release, but in past minor version bumps to Apache Cassandra, they've upgrade full versions of 3rd party libraries. For example: Apache Cassandra 2.0.9 Guava - 15.0 Netty - 3.6.6.Final Apache Cassandra 2.1.2 Guava - 16.0 Netty - 4.0.23.Final commons-math3 (this was added and not in 2.0.9) I think the bump from 1.2.0 to 1.3.0 is pretty low risk. Asking for people to do a full major bump for this 1 library's minor release seems a bit extreme. Update LZ4 library to 1.3.0 --- Key: CASSANDRA-8637 URL: https://issues.apache.org/jira/browse/CASSANDRA-8637 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Luke Gordon Priority: Minor Fix For: 2.1.3 This is a follow up of https://issues.apache.org/jira/browse/CASSANDRA-7573. A fix for https://github.com/jpountz/lz4-java/pull/46 has been released in LZ4 1.3.0. Apache Cassandra is still using 1.2.0. Can this library be upgraded to correct the issue outlined in 7573? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
