Git Push Summary
Updated Tags: refs/tags/2.0.5-tentative [deleted] b71372146
Git Push Summary
Updated Tags: refs/tags/2.0.5-tentative [created] 29670eb66
[jira] [Commented] (CASSANDRA-6648) Race condition during node bootstrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-6648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891958#comment-13891958
]
Sergio Bossa commented on CASSANDRA-6648:
-
Works for me.
Regarding the Schema.instance.updateVersionAndAnnounce() call inside
StorageService#joinTokenRing, it's definitely race-prone with regard to
Gossiper notifications causing a major state change and a schema update (which
I reproduced by using breakpoints on the two concurrent threads): but, even if
a lost update can occur and re-establish an empty version, I verified the
correct schema version is later recomputed by another schema pull, hence this
shouldn't cause any actual problems.
Race condition during node bootstrapping
Key: CASSANDRA-6648
URL: https://issues.apache.org/jira/browse/CASSANDRA-6648
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Sergio Bossa
Assignee: Sergio Bossa
Priority: Critical
Fix For: 1.2.15, 2.0.6
Attachments: 6648-v2.txt, 6648-v3-1.2.txt, 6648-v3.txt,
CASSANDRA-6648.patch
When bootstrapping a new node, data is missing as if the new node didn't
actually bootstrap, which I tracked down to the following scenario:
1) New node joins token ring and waits for schema to be settled before
actually bootstrapping.
2) The schema scheck somewhat passes and it starts bootstrapping.
3) Bootstrapping doesn't find the ks/cf that should have received from the
other node.
4) Queries at this point cause NPEs, until when later they recover but data
is missed.
The problem seems to be caused by a race condition between the migration
manager and the bootstrapper, with the former running after the latter.
I think this is supposed to protect against such scenarios:
{noformat}
while (!MigrationManager.isReadyForBootstrap())
{
setMode(Mode.JOINING, waiting for schema information to
complete, true);
Uninterruptibles.sleepUninterruptibly(1, TimeUnit.SECONDS);
}
{noformat}
But MigrationManager.isReadyForBootstrap() implementation is quite fragile
and doesn't take into account slow schema propagation.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (CASSANDRA-6652) Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer
Benedict created CASSANDRA-6652: --- Summary: Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer Key: CASSANDRA-6652 URL: https://issues.apache.org/jira/browse/CASSANDRA-6652 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor What it says on the tin. Currently CLS.close() calls sync() before cleaning the buffer, which may result in unnecessary IO, but also in the case of a failure on the commit disk may prevent CLS from being discarded, or in the case of a faulty file may prevent any progress on reclaiming CLS. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6652) Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer
[ https://issues.apache.org/jira/browse/CASSANDRA-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6652: Attachment: tmp-2.0.patch Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer Key: CASSANDRA-6652 URL: https://issues.apache.org/jira/browse/CASSANDRA-6652 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor Attachments: tmp-2.0.patch What it says on the tin. Currently CLS.close() calls sync() before cleaning the buffer, which may result in unnecessary IO, but also in the case of a failure on the commit disk may prevent CLS from being discarded, or in the case of a faulty file may prevent any progress on reclaiming CLS. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6652) Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer
[ https://issues.apache.org/jira/browse/CASSANDRA-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6652: Attachment: (was: tmp-2.0.patch) Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer Key: CASSANDRA-6652 URL: https://issues.apache.org/jira/browse/CASSANDRA-6652 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor Attachments: tmp-2.0.patch What it says on the tin. Currently CLS.close() calls sync() before cleaning the buffer, which may result in unnecessary IO, but also in the case of a failure on the commit disk may prevent CLS from being discarded, or in the case of a faulty file may prevent any progress on reclaiming CLS. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6652) Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer
[ https://issues.apache.org/jira/browse/CASSANDRA-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6652: Attachment: tmp-2.0.patch Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer Key: CASSANDRA-6652 URL: https://issues.apache.org/jira/browse/CASSANDRA-6652 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor Attachments: tmp-2.0.patch What it says on the tin. Currently CLS.close() calls sync() before cleaning the buffer, which may result in unnecessary IO, but also in the case of a failure on the commit disk may prevent CLS from being discarded, or in the case of a faulty file may prevent any progress on reclaiming CLS. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6652) Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer
[ https://issues.apache.org/jira/browse/CASSANDRA-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6652: Attachment: (was: tmp-2.0.patch) Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer Key: CASSANDRA-6652 URL: https://issues.apache.org/jira/browse/CASSANDRA-6652 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor Attachments: tmp-2.0.patch What it says on the tin. Currently CLS.close() calls sync() before cleaning the buffer, which may result in unnecessary IO, but also in the case of a failure on the commit disk may prevent CLS from being discarded, or in the case of a faulty file may prevent any progress on reclaiming CLS. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6652) Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer
[ https://issues.apache.org/jira/browse/CASSANDRA-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6652: Attachment: tmp-2.0.patch Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer Key: CASSANDRA-6652 URL: https://issues.apache.org/jira/browse/CASSANDRA-6652 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor Attachments: tmp-2.0.patch What it says on the tin. Currently CLS.close() calls sync() before cleaning the buffer, which may result in unnecessary IO, but also in the case of a failure on the commit disk may prevent CLS from being discarded, or in the case of a faulty file may prevent any progress on reclaiming CLS. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6652) Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer
[ https://issues.apache.org/jira/browse/CASSANDRA-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891980#comment-13891980 ] Benedict commented on CASSANDRA-6652: - Attaching a patch that solves this problem for 2.0.4, and also tries particularly hard to make sure the file is zeroed/unreadable by replay in the event that we fail to delete it. Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer Key: CASSANDRA-6652 URL: https://issues.apache.org/jira/browse/CASSANDRA-6652 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor Attachments: tmp-2.0.patch What it says on the tin. Currently CLS.close() calls sync() before cleaning the buffer, which may result in unnecessary IO, but also in the case of a failure on the commit disk may prevent CLS from being discarded, or in the case of a faulty file may prevent any progress on reclaiming CLS. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6652) Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer
[ https://issues.apache.org/jira/browse/CASSANDRA-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6652: Attachment: tmp2-2.0.patch New version with messages attempting remedial action after failed delete logging messages at error level. Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer Key: CASSANDRA-6652 URL: https://issues.apache.org/jira/browse/CASSANDRA-6652 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor Attachments: tmp-2.0.patch, tmp2-2.0.patch What it says on the tin. Currently CLS.close() calls sync() before cleaning the buffer, which may result in unnecessary IO, but also in the case of a failure on the commit disk may prevent CLS from being discarded, or in the case of a faulty file may prevent any progress on reclaiming CLS. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6648) Race condition during node bootstrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-6648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-6648:
Reproduced In: 1.2.14 (was: 1.2.14, 2.0.4)
Race condition during node bootstrapping
Key: CASSANDRA-6648
URL: https://issues.apache.org/jira/browse/CASSANDRA-6648
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Sergio Bossa
Assignee: Sergio Bossa
Priority: Critical
Fix For: 1.2.15, 2.0.6
Attachments: 6648-v2.txt, 6648-v3-1.2.txt, 6648-v3.txt,
CASSANDRA-6648.patch
When bootstrapping a new node, data is missing as if the new node didn't
actually bootstrap, which I tracked down to the following scenario:
1) New node joins token ring and waits for schema to be settled before
actually bootstrapping.
2) The schema scheck somewhat passes and it starts bootstrapping.
3) Bootstrapping doesn't find the ks/cf that should have received from the
other node.
4) Queries at this point cause NPEs, until when later they recover but data
is missed.
The problem seems to be caused by a race condition between the migration
manager and the bootstrapper, with the former running after the latter.
I think this is supposed to protect against such scenarios:
{noformat}
while (!MigrationManager.isReadyForBootstrap())
{
setMode(Mode.JOINING, waiting for schema information to
complete, true);
Uninterruptibles.sleepUninterruptibly(1, TimeUnit.SECONDS);
}
{noformat}
But MigrationManager.isReadyForBootstrap() implementation is quite fragile
and doesn't take into account slow schema propagation.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6648) Race condition during node bootstrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-6648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-6648:
Fix Version/s: (was: 2.0.6)
2.0.5
Race condition during node bootstrapping
Key: CASSANDRA-6648
URL: https://issues.apache.org/jira/browse/CASSANDRA-6648
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Sergio Bossa
Assignee: Sergio Bossa
Priority: Critical
Fix For: 1.2.15, 2.0.5
Attachments: 6648-v2.txt, 6648-v3-1.2.txt, 6648-v3.txt,
CASSANDRA-6648.patch
When bootstrapping a new node, data is missing as if the new node didn't
actually bootstrap, which I tracked down to the following scenario:
1) New node joins token ring and waits for schema to be settled before
actually bootstrapping.
2) The schema scheck somewhat passes and it starts bootstrapping.
3) Bootstrapping doesn't find the ks/cf that should have received from the
other node.
4) Queries at this point cause NPEs, until when later they recover but data
is missed.
The problem seems to be caused by a race condition between the migration
manager and the bootstrapper, with the former running after the latter.
I think this is supposed to protect against such scenarios:
{noformat}
while (!MigrationManager.isReadyForBootstrap())
{
setMode(Mode.JOINING, waiting for schema information to
complete, true);
Uninterruptibles.sleepUninterruptibly(1, TimeUnit.SECONDS);
}
{noformat}
But MigrationManager.isReadyForBootstrap() implementation is quite fragile
and doesn't take into account slow schema propagation.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5323) Revisit disabled dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-5323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892136#comment-13892136 ] Michael Shuler commented on CASSANDRA-5323: --- 17:32 driftx I feel like notification about simple bootstrap test would've caught 6648 but was masked by all the other crap 17:33 exlt ok - we'll look at those Revisit disabled dtests --- Key: CASSANDRA-5323 URL: https://issues.apache.org/jira/browse/CASSANDRA-5323 Project: Cassandra Issue Type: Test Reporter: Ryan McGuire Assignee: Michael Shuler The following dtests are disabled in buildbot, if they can be re-enabled great, if they can't can they be fixed? upgrade|decommission|sstable_gen|global_row|putget_2dc|cql3_insert -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (CASSANDRA-6653) Attempting to bootstrap causes nodes to lock up in GC
Keith Wright created CASSANDRA-6653:
---
Summary: Attempting to bootstrap causes nodes to lock up in GC
Key: CASSANDRA-6653
URL: https://issues.apache.org/jira/browse/CASSANDRA-6653
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: VNodes using Murmur3
Reporter: Keith Wright
We have been struggling with the inability to bootstrap nodes into our 1.2.13
environment with Vnodes using centos 6.4 with Java 7. We have an 8 node
cluster (32 GB RAM, dual hex core, SSDs, 8 GB heap with 1200 MB eden space,
RF3) with around 1 TB per node using murmur3. When we go to bootstrap a new
node this is what we see:
Bootstrapping node assigns tokens and requests data from cluster
5-6 nodes within the cluster begin to stream data
Around 2 minutes after bootstrap start, between 1 and 4 nodes (sometimes the
bootstrapping node and sometimes not) become unresponsive in par new GCs
IF no nodes go down during the first 5 minutes of bootstrap, then the bootstrap
will succeed without issue
GC mired nodes tend to recover after a minute or two but the receiving node
stops attempting to get more data from the nodes
Bootstrap eventually fails (after streaming all the data from nodes that did
not go down) with Unable to Fetch Ranges
We have tried the following and it appears that sometimes a bootstrap will
succeed (perhaps 1 in 10) but with no discernible pattern:
Increase phi_convict to 16
Restart all nodes prior to bootstrap (to ensure heap is as “clean” as possible)
Stop production load against the cluster (to reduce par new churn); after 5
minutes we know if the bootstrap will succeed so we then re-enable load
Distribute soft interrupts across all CPUs
Below is an output from the GC log of the bootstrapping node when it was stuck
in GC.
This is our production cluster and our inability to grow is a BLOCKING issue
for us. Any ideas would be VERY helpful.
Thanks
Relevant portion of GC log from bootstrapping node:
{Heap before GC invocations=109 (full 0):
par new generation total 1105920K, used 1021140K [0x0005fae0,
0x000645e0, 0x000645e0)
eden space 983040K, 100% used [0x0005fae0, 0x000636e0,
0x000636e0)
from space 122880K, 31% used [0x00063e60, 0x000640b350f0,
0x000645e0)
to space 122880K, 0% used [0x000636e0, 0x000636e0,
0x00063e60)
concurrent mark-sweep generation total 7159808K, used 3826815K
[0x000645e0, 0x0007fae0, 0x0007fae0)
concurrent-mark-sweep perm gen total 24512K, used 24368K [0x0007fae0,
0x0007fc5f, 0x0008)
2014-02-05T13:27:49.621+: 210.242: [GC 210.242: [ParNew:
1021140K-122880K(1105920K), 0.2963210 secs] 4847955K-4024095K(8265728K),
0.2965270 secs] [Times: user=4.97 sys=0.00, real=0.30 secs]
Heap after GC invocations=110 (full 0):
par new generation total 1105920K, used 122880K [0x0005fae0,
0x000645e0, 0x000645e0)
eden space 983040K, 0% used [0x0005fae0, 0x0005fae0,
0x000636e0)
from space 122880K, 100% used [0x000636e0, 0x00063e60,
0x00063e60)
to space 122880K, 0% used [0x00063e60, 0x00063e60,
0x000645e0)
concurrent mark-sweep generation total 7159808K, used 3901215K
[0x000645e0, 0x0007fae0, 0x0007fae0)
concurrent-mark-sweep perm gen total 24512K, used 24368K [0x0007fae0,
0x0007fc5f, 0x0008)
}
Total time for which application threads were stopped: 0.2968550 seconds
Application time: 1.5953840 seconds
Total time for which application threads were stopped: 0.0002040 seconds
Application time: 0.510 seconds
Relevant portion of GC log from non-bootstrapping node:
{Heap before GC invocations=518 (full 1):
par new generation total 1105920K, used 17921K [0x0005fae0,
0x000645e0, 0x000645e0)
eden space 983040K, 1% used [0x0005fae0, 0x0005fbf29db8,
0x000636e0)
from space 122880K, 0% used [0x000636e0, 0x000636e56938,
0x00063e60)
to space 122880K, 0% used [0x00063e60, 0x00063e60,
0x000645e0)
concurrent mark-sweep generation total 7159808K, used 6367511K
[0x000645e0, 0x0007fae0, 0x0007fae0)
concurrent-mark-sweep perm gen total 29888K, used 29784K [0x0007fae0,
0x0007fcb3, 0x0008)
2014-02-04T16:16:44.471+: 945.646: [GC 945.646: [ParNew:
17921K-364K(1105920K), 0.0090810 secs]945.655:
[CMS2014-02-04T16:16:48.373+: 949.548: [CMS-concurrent-sweep: 3.938/4.362
secs] [Times: user=9.10 sys=0.19, real=4.36 secs]
(concurrent mode failure): 6367540K-4453666K(7159808K), 16.4971830 secs]
6385433K-4453666K(8265728K), [CMS Perm : 29784K-29740K(29888K)],
[jira] [Updated] (CASSANDRA-6654) Droppable tombstones are not being removed from LCS table despite being above 20%
[
https://issues.apache.org/jira/browse/CASSANDRA-6654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Keith Wright updated CASSANDRA-6654:
Attachment: Screen Shot 2014-02-05 at 9.38.20 AM.png
Droppable tombstones are not being removed from LCS table despite being above
20%
-
Key: CASSANDRA-6654
URL: https://issues.apache.org/jira/browse/CASSANDRA-6654
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: 1.2.13 VNodes with murmur3
Reporter: Keith Wright
Attachments: Screen Shot 2014-02-05 at 9.38.20 AM.png
JMX is showing that one of our CQL3 LCS tables has a droppable tombstone
ratio above 20% and increasing (currently at 28%). Compactions are not
falling behind and we are using the OOTB setting for this feature so I would
expect not to go above 20% (will attach screen shot from JMX). Table
description:
CREATE TABLE global_user (
user_id timeuuid,
app_id int,
type text,
name text,
extra_param maptext, text,
last timestamp,
paid boolean,
sku_time maptext, timestamp,
values maptimestamp, float,
PRIMARY KEY (user_id, app_id, type, name)
) WITH
bloom_filter_fp_chance=0.10 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=86400 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'sstable_size_in_mb': '160', 'class':
'LeveledCompactionStrategy'} AND
compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1',
'sstable_compression': 'LZ4Compressor'};
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (CASSANDRA-6654) Droppable tombstones are not being removed from LCS table despite being above 20%
Keith Wright created CASSANDRA-6654:
---
Summary: Droppable tombstones are not being removed from LCS table
despite being above 20%
Key: CASSANDRA-6654
URL: https://issues.apache.org/jira/browse/CASSANDRA-6654
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: 1.2.13 VNodes with murmur3
Reporter: Keith Wright
JMX is showing that one of our CQL3 LCS tables has a droppable tombstone ratio
above 20% and increasing (currently at 28%). Compactions are not falling
behind and we are using the OOTB setting for this feature so I would expect not
to go above 20% (will attach screen shot from JMX). Table description:
CREATE TABLE global_user (
user_id timeuuid,
app_id int,
type text,
name text,
extra_param maptext, text,
last timestamp,
paid boolean,
sku_time maptext, timestamp,
values maptimestamp, float,
PRIMARY KEY (user_id, app_id, type, name)
) WITH
bloom_filter_fp_chance=0.10 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=86400 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'sstable_size_in_mb': '160', 'class':
'LeveledCompactionStrategy'} AND
compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1',
'sstable_compression': 'LZ4Compressor'};
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5357) Query cache / partition head cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892159#comment-13892159 ] Sylvain Lebresne commented on CASSANDRA-5357: - I've pushed an additional commit on top of the branch above at https://github.com/pcmanus/cassandra/commits/5357-2 that adds a bunch of comments (there is a few subtlety going on here :)), a few minor nits and: * refactor getThroughCache() a bit so we can maximize the cases where we can use a cached partition (typically, if a slice query bounds are fully included in cache cf, we know we're good). * improve a bit the handling of expiring columns: when comparing the number of rows in a cached CF to rowsPerPartitionToCache, we should indeed not expire columns as this could lead to think we're caching the whole partition when we don't, but when comparing checking if the cached CF has enough rows for the query filter, we must expire with the query TTL or we might end up return less rows than we should (that last part wasn't done by the previous patch). * make sure we test for !reversed in isHeadFilter() * I've reverted to CFS.getRawCachedRow and instead have move the decision of whether the cached row can be used to RowIteratorFactory. It didn't felt particularly logical to do that in getRawCachedRow given nothing in the name of the method suggests it, and it allows more easily to maximize the usage of the cache for range queries. * doesn't increment metric.rowCacheHit when the cached value can't be used due to the filter, only increment metric.rowCacheHitOutOfRange, as that feels more natural to me. Provided that additional commit looks reasonable, I believe I'm good with this. bq. I'll to do the row cache - partition cache renaming in a separate ticket Right. Though now that I think about it, row cache is not all that horrible, it does cache rows, it just cache only some number per partition :). Maybe it's not worth the pain of a rename. Anyway, I'm good leaving that discussion for another time/ticket. Query cache / partition head cache -- Key: CASSANDRA-5357 URL: https://issues.apache.org/jira/browse/CASSANDRA-5357 Project: Cassandra Issue Type: New Feature Reporter: Jonathan Ellis Assignee: Marcus Eriksson Fix For: 2.1 Attachments: 0001-Cache-a-configurable-amount-of-columns.patch I think that most people expect the row cache to act like a query cache, because that's a reasonable model. Caching the entire partition is, in retrospect, not really reasonable, so it's not surprising that it catches people off guard, especially given the confusion we've inflicted on ourselves as to what a row constitutes. I propose replacing it with a true query cache. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6628) Cassandra crashes on Solaris sparcv9 using java 64bit
[ https://issues.apache.org/jira/browse/CASSANDRA-6628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-6628: Fix Version/s: (was: 2.0.6) 2.0.5 Cassandra crashes on Solaris sparcv9 using java 64bit - Key: CASSANDRA-6628 URL: https://issues.apache.org/jira/browse/CASSANDRA-6628 Project: Cassandra Issue Type: Bug Components: Core Environment: checked 1.2.x line and 2.0.x Reporter: Dmitry Shohov Assignee: Dmitry Shohov Fix For: 2.0.5 Attachments: solaris_unsafe_fix.patch, tmp.patch When running cassandra 2.0.4 (and other versions) on Solaris and java 64 bit, JVM crashes. Issue is described once in CASSANDRA-4646 but closed as invalid. The reason for this crash is some memory allignment related problems and incorrect sun.misc.Unsafe usage. If you look into DirectByteBuffer in jdk, you will see that it checks os.arch before using getLong methods. I have a patch, which check for the os.arch and if it is not one of the known, it reads longs and ints byte by byte. Although patch fixes the problem in cassandra, it will still crash without similar fixes in the lz4 library. I already provided the patch for Unsafe usage in lz4. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-5930) Offline scrubs can choke on broken files
[
https://issues.apache.org/jira/browse/CASSANDRA-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-5930:
Fix Version/s: (was: 2.0.6)
2.0.5
Offline scrubs can choke on broken files
Key: CASSANDRA-5930
URL: https://issues.apache.org/jira/browse/CASSANDRA-5930
Project: Cassandra
Issue Type: Bug
Reporter: Jeremiah Jordan
Assignee: Tyler Hobbs
Priority: Minor
Fix For: 2.0.5
Attachments: 5930-v1.patch, 5930-v2.patch
There are cases where offline scrub can hit an exception and die, like:
{noformat}
WARNING: Non-fatal error reading row (stacktrace follows)
Exception in thread main java.io.IOError: java.io.IOError:
java.io.EOFException
at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:242)
at
org.apache.cassandra.tools.StandaloneScrubber.main(StandaloneScrubber.java:121)
Caused by: java.io.IOError: java.io.EOFException
at
org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:116)
at
org.apache.cassandra.db.compaction.PrecompactedRow.init(PrecompactedRow.java:99)
at
org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:176)
at
org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:182)
at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:171)
... 1 more
Caused by: java.io.EOFException
at java.io.RandomAccessFile.readFully(RandomAccessFile.java:399)
at java.io.RandomAccessFile.readFully(RandomAccessFile.java:377)
at
org.apache.cassandra.utils.BytesReadTracker.readFully(BytesReadTracker.java:95)
at
org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:401)
at
org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:363)
at
org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:120)
at
org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:37)
at
org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:144)
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:234)
at
org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:112)
... 5 more
{noformat}
Since the purpose of offline scrub is to fix broken stuff, it should be more
resilient to broken stuff...
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5351) Avoid repairing already-repaired data by default
[ https://issues.apache.org/jira/browse/CASSANDRA-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892202#comment-13892202 ] Marcus Eriksson commented on CASSANDRA-5351: Fixed the todos and a few bugs: https://github.com/krummas/cassandra/commits/marcuse/5351 Avoid repairing already-repaired data by default Key: CASSANDRA-5351 URL: https://issues.apache.org/jira/browse/CASSANDRA-5351 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Lyuben Todorov Labels: repair Fix For: 2.1 Attachments: 5351_node1.log, 5351_node2.log, 5351_node3.log, 5351_nodetool.log Repair has always built its merkle tree from all the data in a columnfamily, which is guaranteed to work but is inefficient. We can improve this by remembering which sstables have already been successfully repaired, and only repairing sstables new since the last repair. (This automatically makes CASSANDRA-3362 much less of a problem too.) The tricky part is, compaction will (if not taught otherwise) mix repaired data together with non-repaired. So we should segregate unrepaired sstables from the repaired ones. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6648) Race condition during node bootstrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-6648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892214#comment-13892214
]
Ryan McGuire commented on CASSANDRA-6648:
-
{quote}
Ryan McGuire would be great if you could push that test of yours above as a
dtest.
{quote}
[Done|https://github.com/riptano/cassandra-dtest/commit/17e8f3f5680f6d78755577d8817b1eeb64b642d8].
We already have a few different tests to catch this, but all were being masked
for one reason or another :(
Race condition during node bootstrapping
Key: CASSANDRA-6648
URL: https://issues.apache.org/jira/browse/CASSANDRA-6648
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Sergio Bossa
Assignee: Sergio Bossa
Priority: Critical
Fix For: 1.2.15, 2.0.5
Attachments: 6648-v2.txt, 6648-v3-1.2.txt, 6648-v3.txt,
CASSANDRA-6648.patch
When bootstrapping a new node, data is missing as if the new node didn't
actually bootstrap, which I tracked down to the following scenario:
1) New node joins token ring and waits for schema to be settled before
actually bootstrapping.
2) The schema scheck somewhat passes and it starts bootstrapping.
3) Bootstrapping doesn't find the ks/cf that should have received from the
other node.
4) Queries at this point cause NPEs, until when later they recover but data
is missed.
The problem seems to be caused by a race condition between the migration
manager and the bootstrapper, with the former running after the latter.
I think this is supposed to protect against such scenarios:
{noformat}
while (!MigrationManager.isReadyForBootstrap())
{
setMode(Mode.JOINING, waiting for schema information to
complete, true);
Uninterruptibles.sleepUninterruptibly(1, TimeUnit.SECONDS);
}
{noformat}
But MigrationManager.isReadyForBootstrap() implementation is quite fragile
and doesn't take into account slow schema propagation.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (CASSANDRA-6655) Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush
Benedict created CASSANDRA-6655: --- Summary: Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush Key: CASSANDRA-6655 URL: https://issues.apache.org/jira/browse/CASSANDRA-6655 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor Fix For: 2.0.5 In the extreme case of only deletes the memtable will never flush, and we will OOM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (CASSANDRA-6656) Exception logging
Ding Yuan created CASSANDRA-6656:
Summary: Exception logging
Key: CASSANDRA-6656
URL: https://issues.apache.org/jira/browse/CASSANDRA-6656
Project: Cassandra
Issue Type: Improvement
Components: Core, Tools
Reporter: Ding Yuan
Fix For: 2.0.4
Reporting a few cases where informative exceptions can be silently swallowed.
Attaching a proposed patch.
=
Case 1
Line: 95, File: org/apache/cassandra/utils/Hex.java
An actual failure in the underlying constructor will be lost.
Propose to log it.
try
{
s = stringConstructor.newInstance(0, c.length, c);
+ }
+ catch (InvocationTargetException ite) {
+ // The underlying constructor failed. Unwrapping the exception.
+ logger.info(Underlying constructor throws exception: ,
ite.getCause());
}
catch (Exception e)
{
// Swallowing as we'll just use a copying constructor
}
return s == null ? new String(c) : s;
==
=
Case 2
Line: 192, File: org/apache/cassandra/db/marshal/DynamicCompositeType.java
The actual cause of comparator error can be lost as it can fail in multiple
locations.
AbstractType? comparator = null;
int header = getShortLength(bb);
if ((header 0x8000) == 0)
{
ByteBuffer value = getBytes(bb, header);
try
{
comparator = TypeParser.parse(ByteBufferUtil.string(value));
}
catch (Exception e)
{
--- can fail here
// we'll deal with this below since comparator == null
}
}
else
{
comparator = aliases.get((byte)(header 0xFF));
--- can fail here
}
if (comparator == null)
throw new MarshalException(Cannot find comparator for component
+ i);
Propose to log the exception.
==
=
Case 3
Line: 239, File: org/apache/cassandra/tools/NodeProbe.java
Exception ignored in finally. Propose log them with debug or trace.
232: finally
233: {
234: try
235: {
236: ssProxy.removeNotificationListener(runner);
236: ssProxy.removeNotificationListener(runner);
237: jmxc.removeConnectionNotificationListener(runner);
238: }
239: catch (Throwable ignored) {}
240: }
Similar case is at line 264 in the same file.
==
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6656) Exception logging
[
https://issues.apache.org/jira/browse/CASSANDRA-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ding Yuan updated CASSANDRA-6656:
-
Attachment: trunk-6656.txt
Exception logging
-
Key: CASSANDRA-6656
URL: https://issues.apache.org/jira/browse/CASSANDRA-6656
Project: Cassandra
Issue Type: Improvement
Components: Core, Tools
Reporter: Ding Yuan
Fix For: 2.0.4
Attachments: trunk-6656.txt
Reporting a few cases where informative exceptions can be silently swallowed.
Attaching a proposed patch.
=
Case 1
Line: 95, File: org/apache/cassandra/utils/Hex.java
An actual failure in the underlying constructor will be lost.
Propose to log it.
try
{
s = stringConstructor.newInstance(0, c.length, c);
+ }
+ catch (InvocationTargetException ite) {
+ // The underlying constructor failed. Unwrapping the
exception.
+ logger.info(Underlying constructor throws exception: ,
ite.getCause());
}
catch (Exception e)
{
// Swallowing as we'll just use a copying constructor
}
return s == null ? new String(c) : s;
==
=
Case 2
Line: 192, File: org/apache/cassandra/db/marshal/DynamicCompositeType.java
The actual cause of comparator error can be lost as it can fail in multiple
locations.
AbstractType? comparator = null;
int header = getShortLength(bb);
if ((header 0x8000) == 0)
{
ByteBuffer value = getBytes(bb, header);
try
{
comparator = TypeParser.parse(ByteBufferUtil.string(value));
}
catch (Exception e)
{
--- can fail here
// we'll deal with this below since comparator == null
}
}
else
{
comparator = aliases.get((byte)(header 0xFF));
--- can fail here
}
if (comparator == null)
throw new MarshalException(Cannot find comparator for component
+ i);
Propose to log the exception.
==
=
Case 3
Line: 239, File: org/apache/cassandra/tools/NodeProbe.java
Exception ignored in finally. Propose log them with debug or trace.
232: finally
233: {
234: try
235: {
236: ssProxy.removeNotificationListener(runner);
236: ssProxy.removeNotificationListener(runner);
237: jmxc.removeConnectionNotificationListener(runner);
238: }
239: catch (Throwable ignored) {}
240: }
Similar case is at line 264 in the same file.
==
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
git commit: Remove/update invalid sentences in CQL doc
Updated Branches:
refs/heads/cassandra-1.2 178e086f1 - 16efdf4a0
Remove/update invalid sentences in CQL doc
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/16efdf4a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/16efdf4a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/16efdf4a
Branch: refs/heads/cassandra-1.2
Commit: 16efdf4a0300b2499346156fd4e2a8e0785d2690
Parents: 178e086
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Feb 5 16:32:24 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Feb 5 16:32:24 2014 +0100
--
doc/cql3/CQL.textile | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/16efdf4a/doc/cql3/CQL.textile
--
diff --git a/doc/cql3/CQL.textile b/doc/cql3/CQL.textile
index 8b6cb08..b18ce22 100644
--- a/doc/cql3/CQL.textile
+++ b/doc/cql3/CQL.textile
@@ -275,8 +275,6 @@ CREATE TABLE t (
PRIMARY KEY (k)
)
-Moreover, a table must define at least one column that is not part of the
PRIMARY KEY as a row exists in Cassandra only if it contains at least one value
for one such column.
-
h4(#createTablepartitionClustering). Partition key and clustering columns
In CQL, the order in which columns are defined for the @PRIMARY KEY@ matters.
The first column of the key is called the __partition key__. It has the
property that all the rows sharing the same partition key (even across table in
fact) are stored on the same physical node. Also, insertion/update/deletion on
rows sharing the same partition key for a given table are performed
__atomically__ and in __isolation__. Note that it is possible to have a
composite partition key, i.e. a partition key formed of multiple columns, using
an extra set of parentheses to define which columns forms the partition key.
@@ -445,7 +443,7 @@ INSERT INTO NerdMovies (movie, director, main_actor, year)
VALUES ('Serenity', 'Joss Whedon', 'Nathan Fillion', 2005)
USING TTL 86400;
-The @INSERT@ statement writes one or more columns for a given row in a table.
Note that since a row is identified by its @PRIMARY KEY@, the columns that
compose it must be specified. Also, since a row only exists when it contains
one value for a column not part of the @PRIMARY KEY@, one such value must be
specified too.
+The @INSERT@ statement writes one or more columns for a given row in a table.
Note that since a row is identified by its @PRIMARY KEY@, at least the columns
composing it must be specified.
Note that unlike in SQL, @INSERT@ does not check the prior existence of the
row: the row is created if none existed before, and updated otherwise.
Furthermore, there is no mean to know which of creation or update happened. In
fact, the semantic of @INSERT@ and @UPDATE@ are identical.
[4/4] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0
Conflicts:
CHANGES.txt
NEWS.txt
build.xml
debian/changelog
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49bb972c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49bb972c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49bb972c
Branch: refs/heads/cassandra-2.0
Commit: 49bb972c6f79c6717b4750488bdcb035bb770784
Parents: 29670eb 16efdf4
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Feb 5 16:36:50 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Feb 5 16:36:50 2014 +0100
--
doc/cql3/CQL.textile | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/49bb972c/doc/cql3/CQL.textile
--
diff --cc doc/cql3/CQL.textile
index cf73708,b18ce22..f82fc19
--- a/doc/cql3/CQL.textile
+++ b/doc/cql3/CQL.textile
@@@ -459,11 -443,9 +457,11 @@@ INSERT INTO NerdMovies (movie, director
VALUES ('Serenity', 'Joss Whedon', 'Nathan Fillion', 2005)
USING TTL 86400;
- The @INSERT@ statement writes one or more columns for a given row in a table.
Note that since a row is identified by its @PRIMARY KEY@, the columns that
compose it must be specified. Also, since a row only exists when it contains
one value for a column not part of the @PRIMARY KEY@, one such value must be
specified too.
+ The @INSERT@ statement writes one or more columns for a given row in a table.
Note that since a row is identified by its @PRIMARY KEY@, at least the columns
composing it must be specified.
-Note that unlike in SQL, @INSERT@ does not check the prior existence of the
row: the row is created if none existed before, and updated otherwise.
Furthermore, there is no mean to know which of creation or update happened. In
fact, the semantic of @INSERT@ and @UPDATE@ are identical.
+Note that unlike in SQL, @INSERT@ does not check the prior existence of the
row by default: the row is created if none existed before, and updated
otherwise. Furthermore, there is no mean to know which of creation or update
happened.
+
+It is however possible to use the @IF NOT EXISTS@ condition to only insert if
the row does not exist prior to the insertion. But please note that using @IF
NOT EXISTS@ will incur a non negligible performance cost (internally, Paxos
will be used) so this should be used sparingly.
All updates for an @INSERT@ are applied atomically and in isolation.
[5/5] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/58d1a4f8 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/58d1a4f8 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/58d1a4f8 Branch: refs/heads/trunk Commit: 58d1a4f816251a889f3eb4eed801dc8b0ccfc42d Parents: c004f9f 49bb972 Author: Sylvain Lebresne sylv...@datastax.com Authored: Wed Feb 5 16:38:18 2014 +0100 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Wed Feb 5 16:38:18 2014 +0100 -- CHANGES.txt | 5 + NEWS.txt | 13 +++-- doc/cql3/CQL.textile | 4 +--- 3 files changed, 9 insertions(+), 13 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/58d1a4f8/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/58d1a4f8/NEWS.txt -- diff --cc NEWS.txt index 185f60c,b18150f..9dd2ad6 --- a/NEWS.txt +++ b/NEWS.txt @@@ -13,47 -13,7 +13,37 @@@ restore snapshots created with the prev 'sstableloader' tool. You can upgrade the file format of your snapshots using the provided 'sstableupgrade' tool. +2.1 +=== + +New features + + - SSTable data directory name is slightly changed. Each directory will + have hex string appended after CF name, e.g. + ks/cf-5be396077b811e3a3ab9dc4b9ac088d/ + This hex string part represents unique ColumnFamily ID. + Note that existing directories are used as is, so only newly created + directories after upgrade have new directory name format. + - Saved key cache files also have ColumnFamily ID in their file name. + +Upgrading +- + - Rolling upgrades from anything pre-2.0.5 is not supported. + - For leveled compaction users, 2.0 must be atleast started before + upgrading to 2.1 due to the fact that the old JSON leveled + manifest is migrated into the sstable metadata files on startup + in 2.0 and this code is gone from 2.1. + - For size-tiered compaction users, Cassandra now defaults to ignoring + the coldest 5% of sstables. This can be customized with the + cold_reads_to_omit compaction option; 0.0 omits nothing (the old + behavior) and 1.0 omits everything. + - Multithreaded compaction has been removed. + - Counters implementation has been changed, replaced by a safer one with + less caveats, but different performance characteristics. You might have + to change your data model to accomodate the new implementation. + (See https://issues.apache.org/jira/browse/CASSANDRA-6504 and the dev + blog post at http://www.datastax.com/dev/blog/PLACEHOLDER for details). - 2.0.6 - = - - New features - - - Scrub can now optionally skip corrupt counter partitions. Please note - that this will lead to the loss of all the counter updates in the skipped - partition. See the --skip-corrupted option. - - 2.0.5 = http://git-wip-us.apache.org/repos/asf/cassandra/blob/58d1a4f8/doc/cql3/CQL.textile --
[2/5] git commit: Update changes and news file for 2.0.5 re-roll
Update changes and news file for 2.0.5 re-roll Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/29670eb6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/29670eb6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/29670eb6 Branch: refs/heads/trunk Commit: 29670eb6692f239a3e9b0db05f2d5a1b5d4eb8b0 Parents: 41de0bd Author: Sylvain Lebresne sylv...@datastax.com Authored: Wed Feb 5 09:22:13 2014 +0100 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Wed Feb 5 09:22:13 2014 +0100 -- CHANGES.txt | 14 +- NEWS.txt| 13 +++-- 2 files changed, 8 insertions(+), 19 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/29670eb6/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 7e0a89e..9599e56 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,12 +1,3 @@ -2.0.6 - * Fix direct Memory on architectures that do not support unaligned long access - (CASSANDRA-6628) - * Let scrub optionally skip broken counter partitions (CASSANDRA-5930) -Merged from 1.2: - * Move handling of migration event source to solve bootstrap race. (CASSANDRA-6648) - * Make sure compaction throughput value doesn't overflow with int math (CASSANDRA-6647) - - 2.0.5 * Reduce garbage generated by bloom filter lookups (CASSANDRA-6609) * Add ks.cf names to tombstone logging (CASSANDRA-6597) @@ -29,6 +20,9 @@ Merged from 1.2: * Switch stress to use ITransportFactory (CASSANDRA-6641) * Fix IllegalArgumentException during prepare (CASSANDRA-6592) * Fix possible loss of 2ndary index entries during compaction (CASSANDRA-6517) + * Fix direct Memory on architectures that do not support unaligned long access + (CASSANDRA-6628) + * Let scrub optionally skip broken counter partitions (CASSANDRA-5930) Merged from 1.2: * fsync compression metadata (CASSANDRA-6531) * Validate CF existence on execution for prepared statement (CASSANDRA-6535) @@ -43,6 +37,8 @@ Merged from 1.2: * Fix preparing with batch and delete from collection (CASSANDRA-6607) * Fix ABSC reverse iterator's remove() method (CASSANDRA-6629) * Handle host ID conflicts properly (CASSANDRA-6615) + * Move handling of migration event source to solve bootstrap race. (CASSANDRA-6648) + * Make sure compaction throughput value doesn't overflow with int math (CASSANDRA-6647) 2.0.4 http://git-wip-us.apache.org/repos/asf/cassandra/blob/29670eb6/NEWS.txt -- diff --git a/NEWS.txt b/NEWS.txt index b21fbaa..b18150f 100644 --- a/NEWS.txt +++ b/NEWS.txt @@ -14,16 +14,6 @@ restore snapshots created with the previous major version using the using the provided 'sstableupgrade' tool. -2.0.6 -= - -New features - -- Scrub can now optionally skip corrupt counter partitions. Please note - that this will lead to the loss of all the counter updates in the skipped - partition. See the --skip-corrupted option. - - 2.0.5 = @@ -31,6 +21,9 @@ New features - Batchlog replay can be, and is throttled by default now. See batchlog_replay_throttle_in_kb setting in cassandra.yaml. +- Scrub can now optionally skip corrupt counter partitions. Please note + that this will lead to the loss of all the counter updates in the skipped + partition. See the --skip-corrupted option. Upgrading -
[3/4] git commit: Remove/update invalid sentences in CQL doc
Remove/update invalid sentences in CQL doc
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/16efdf4a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/16efdf4a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/16efdf4a
Branch: refs/heads/cassandra-2.0
Commit: 16efdf4a0300b2499346156fd4e2a8e0785d2690
Parents: 178e086
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Feb 5 16:32:24 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Feb 5 16:32:24 2014 +0100
--
doc/cql3/CQL.textile | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/16efdf4a/doc/cql3/CQL.textile
--
diff --git a/doc/cql3/CQL.textile b/doc/cql3/CQL.textile
index 8b6cb08..b18ce22 100644
--- a/doc/cql3/CQL.textile
+++ b/doc/cql3/CQL.textile
@@ -275,8 +275,6 @@ CREATE TABLE t (
PRIMARY KEY (k)
)
-Moreover, a table must define at least one column that is not part of the
PRIMARY KEY as a row exists in Cassandra only if it contains at least one value
for one such column.
-
h4(#createTablepartitionClustering). Partition key and clustering columns
In CQL, the order in which columns are defined for the @PRIMARY KEY@ matters.
The first column of the key is called the __partition key__. It has the
property that all the rows sharing the same partition key (even across table in
fact) are stored on the same physical node. Also, insertion/update/deletion on
rows sharing the same partition key for a given table are performed
__atomically__ and in __isolation__. Note that it is possible to have a
composite partition key, i.e. a partition key formed of multiple columns, using
an extra set of parentheses to define which columns forms the partition key.
@@ -445,7 +443,7 @@ INSERT INTO NerdMovies (movie, director, main_actor, year)
VALUES ('Serenity', 'Joss Whedon', 'Nathan Fillion', 2005)
USING TTL 86400;
-The @INSERT@ statement writes one or more columns for a given row in a table.
Note that since a row is identified by its @PRIMARY KEY@, the columns that
compose it must be specified. Also, since a row only exists when it contains
one value for a column not part of the @PRIMARY KEY@, one such value must be
specified too.
+The @INSERT@ statement writes one or more columns for a given row in a table.
Note that since a row is identified by its @PRIMARY KEY@, at least the columns
composing it must be specified.
Note that unlike in SQL, @INSERT@ does not check the prior existence of the
row: the row is created if none existed before, and updated otherwise.
Furthermore, there is no mean to know which of creation or update happened. In
fact, the semantic of @INSERT@ and @UPDATE@ are identical.
[2/4] git commit: Update changes and news file for 2.0.5 re-roll
Update changes and news file for 2.0.5 re-roll Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/29670eb6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/29670eb6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/29670eb6 Branch: refs/heads/cassandra-2.0 Commit: 29670eb6692f239a3e9b0db05f2d5a1b5d4eb8b0 Parents: 41de0bd Author: Sylvain Lebresne sylv...@datastax.com Authored: Wed Feb 5 09:22:13 2014 +0100 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Wed Feb 5 09:22:13 2014 +0100 -- CHANGES.txt | 14 +- NEWS.txt| 13 +++-- 2 files changed, 8 insertions(+), 19 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/29670eb6/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 7e0a89e..9599e56 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,12 +1,3 @@ -2.0.6 - * Fix direct Memory on architectures that do not support unaligned long access - (CASSANDRA-6628) - * Let scrub optionally skip broken counter partitions (CASSANDRA-5930) -Merged from 1.2: - * Move handling of migration event source to solve bootstrap race. (CASSANDRA-6648) - * Make sure compaction throughput value doesn't overflow with int math (CASSANDRA-6647) - - 2.0.5 * Reduce garbage generated by bloom filter lookups (CASSANDRA-6609) * Add ks.cf names to tombstone logging (CASSANDRA-6597) @@ -29,6 +20,9 @@ Merged from 1.2: * Switch stress to use ITransportFactory (CASSANDRA-6641) * Fix IllegalArgumentException during prepare (CASSANDRA-6592) * Fix possible loss of 2ndary index entries during compaction (CASSANDRA-6517) + * Fix direct Memory on architectures that do not support unaligned long access + (CASSANDRA-6628) + * Let scrub optionally skip broken counter partitions (CASSANDRA-5930) Merged from 1.2: * fsync compression metadata (CASSANDRA-6531) * Validate CF existence on execution for prepared statement (CASSANDRA-6535) @@ -43,6 +37,8 @@ Merged from 1.2: * Fix preparing with batch and delete from collection (CASSANDRA-6607) * Fix ABSC reverse iterator's remove() method (CASSANDRA-6629) * Handle host ID conflicts properly (CASSANDRA-6615) + * Move handling of migration event source to solve bootstrap race. (CASSANDRA-6648) + * Make sure compaction throughput value doesn't overflow with int math (CASSANDRA-6647) 2.0.4 http://git-wip-us.apache.org/repos/asf/cassandra/blob/29670eb6/NEWS.txt -- diff --git a/NEWS.txt b/NEWS.txt index b21fbaa..b18150f 100644 --- a/NEWS.txt +++ b/NEWS.txt @@ -14,16 +14,6 @@ restore snapshots created with the previous major version using the using the provided 'sstableupgrade' tool. -2.0.6 -= - -New features - -- Scrub can now optionally skip corrupt counter partitions. Please note - that this will lead to the loss of all the counter updates in the skipped - partition. See the --skip-corrupted option. - - 2.0.5 = @@ -31,6 +21,9 @@ New features - Batchlog replay can be, and is throttled by default now. See batchlog_replay_throttle_in_kb setting in cassandra.yaml. +- Scrub can now optionally skip corrupt counter partitions. Please note + that this will lead to the loss of all the counter updates in the skipped + partition. See the --skip-corrupted option. Upgrading -
[3/5] git commit: Remove/update invalid sentences in CQL doc
Remove/update invalid sentences in CQL doc
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/16efdf4a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/16efdf4a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/16efdf4a
Branch: refs/heads/trunk
Commit: 16efdf4a0300b2499346156fd4e2a8e0785d2690
Parents: 178e086
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Feb 5 16:32:24 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Feb 5 16:32:24 2014 +0100
--
doc/cql3/CQL.textile | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/16efdf4a/doc/cql3/CQL.textile
--
diff --git a/doc/cql3/CQL.textile b/doc/cql3/CQL.textile
index 8b6cb08..b18ce22 100644
--- a/doc/cql3/CQL.textile
+++ b/doc/cql3/CQL.textile
@@ -275,8 +275,6 @@ CREATE TABLE t (
PRIMARY KEY (k)
)
-Moreover, a table must define at least one column that is not part of the
PRIMARY KEY as a row exists in Cassandra only if it contains at least one value
for one such column.
-
h4(#createTablepartitionClustering). Partition key and clustering columns
In CQL, the order in which columns are defined for the @PRIMARY KEY@ matters.
The first column of the key is called the __partition key__. It has the
property that all the rows sharing the same partition key (even across table in
fact) are stored on the same physical node. Also, insertion/update/deletion on
rows sharing the same partition key for a given table are performed
__atomically__ and in __isolation__. Note that it is possible to have a
composite partition key, i.e. a partition key formed of multiple columns, using
an extra set of parentheses to define which columns forms the partition key.
@@ -445,7 +443,7 @@ INSERT INTO NerdMovies (movie, director, main_actor, year)
VALUES ('Serenity', 'Joss Whedon', 'Nathan Fillion', 2005)
USING TTL 86400;
-The @INSERT@ statement writes one or more columns for a given row in a table.
Note that since a row is identified by its @PRIMARY KEY@, the columns that
compose it must be specified. Also, since a row only exists when it contains
one value for a column not part of the @PRIMARY KEY@, one such value must be
specified too.
+The @INSERT@ statement writes one or more columns for a given row in a table.
Note that since a row is identified by its @PRIMARY KEY@, at least the columns
composing it must be specified.
Note that unlike in SQL, @INSERT@ does not check the prior existence of the
row: the row is created if none existed before, and updated otherwise.
Furthermore, there is no mean to know which of creation or update happened. In
fact, the semantic of @INSERT@ and @UPDATE@ are identical.
[1/5] git commit: Update news, version in preparation for 1.2.15
Updated Branches: refs/heads/trunk c004f9f83 - 58d1a4f81 Update news, version in preparation for 1.2.15 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/178e086f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/178e086f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/178e086f Branch: refs/heads/trunk Commit: 178e086f1aaa2744dfc8046c9abb0c62e8a65895 Parents: 511787d Author: Sylvain Lebresne sylv...@datastax.com Authored: Wed Feb 5 08:47:20 2014 +0100 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Wed Feb 5 08:47:20 2014 +0100 -- CHANGES.txt | 1 + NEWS.txt | 10 ++ build.xml| 2 +- debian/changelog | 6 ++ 4 files changed, 18 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/178e086f/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 75c7104..0989dc4 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -2,6 +2,7 @@ * Move handling of migration event source to solve bootstrap race (CASSANDRA-6648) * Make sure compaction throughput value doesn't overflow with int math (CASSANDRA-6647) + 1.2.14 * Reverted code to limit CQL prepared statement cache by size (CASSANDRA-6592) * add cassandra.default_messaging_version property to allow easier http://git-wip-us.apache.org/repos/asf/cassandra/blob/178e086f/NEWS.txt -- diff --git a/NEWS.txt b/NEWS.txt index 53cb7ca..771536d 100644 --- a/NEWS.txt +++ b/NEWS.txt @@ -14,6 +14,16 @@ restore snapshots created with the previous major version using the using the provided 'sstableupgrade' tool. +1.2.15 +== + +Upgrading +- +- This release mainly fix a regression of 1.2.14, namely + https://issues.apache.org/jira/browse/CASSANDRA-6648. Users of 1.2.14 are + strongly encouraged to upgrade. + + 1.2.14 == http://git-wip-us.apache.org/repos/asf/cassandra/blob/178e086f/build.xml -- diff --git a/build.xml b/build.xml index 150e2fe..ce87a2a 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ property name=debuglevel value=source,lines,vars/ !-- default version and SCM information -- -property name=base.version value=1.2.14/ +property name=base.version value=1.2.15/ property name=scm.connection value=scm:git://git.apache.org/cassandra.git/ property name=scm.developerConnection value=scm:git://git.apache.org/cassandra.git/ property name=scm.url value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/ http://git-wip-us.apache.org/repos/asf/cassandra/blob/178e086f/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index 472063e..bb8ecf2 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (1.2.15) unstable; urgency=low + + * New release + + -- Sylvain Lebresne slebre...@apache.org Wed, 05 Feb 2014 08:42:29 +0100 + cassandra (1.2.14) unstable; urgency=low * New release
[4/5] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0
Conflicts:
CHANGES.txt
NEWS.txt
build.xml
debian/changelog
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49bb972c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49bb972c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49bb972c
Branch: refs/heads/trunk
Commit: 49bb972c6f79c6717b4750488bdcb035bb770784
Parents: 29670eb 16efdf4
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Feb 5 16:36:50 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Feb 5 16:36:50 2014 +0100
--
doc/cql3/CQL.textile | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/49bb972c/doc/cql3/CQL.textile
--
diff --cc doc/cql3/CQL.textile
index cf73708,b18ce22..f82fc19
--- a/doc/cql3/CQL.textile
+++ b/doc/cql3/CQL.textile
@@@ -459,11 -443,9 +457,11 @@@ INSERT INTO NerdMovies (movie, director
VALUES ('Serenity', 'Joss Whedon', 'Nathan Fillion', 2005)
USING TTL 86400;
- The @INSERT@ statement writes one or more columns for a given row in a table.
Note that since a row is identified by its @PRIMARY KEY@, the columns that
compose it must be specified. Also, since a row only exists when it contains
one value for a column not part of the @PRIMARY KEY@, one such value must be
specified too.
+ The @INSERT@ statement writes one or more columns for a given row in a table.
Note that since a row is identified by its @PRIMARY KEY@, at least the columns
composing it must be specified.
-Note that unlike in SQL, @INSERT@ does not check the prior existence of the
row: the row is created if none existed before, and updated otherwise.
Furthermore, there is no mean to know which of creation or update happened. In
fact, the semantic of @INSERT@ and @UPDATE@ are identical.
+Note that unlike in SQL, @INSERT@ does not check the prior existence of the
row by default: the row is created if none existed before, and updated
otherwise. Furthermore, there is no mean to know which of creation or update
happened.
+
+It is however possible to use the @IF NOT EXISTS@ condition to only insert if
the row does not exist prior to the insertion. But please note that using @IF
NOT EXISTS@ will incur a non negligible performance cost (internally, Paxos
will be used) so this should be used sparingly.
All updates for an @INSERT@ are applied atomically and in isolation.
[1/4] git commit: Update news, version in preparation for 1.2.15
Updated Branches: refs/heads/cassandra-2.0 41de0bd4e - 49bb972c6 Update news, version in preparation for 1.2.15 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/178e086f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/178e086f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/178e086f Branch: refs/heads/cassandra-2.0 Commit: 178e086f1aaa2744dfc8046c9abb0c62e8a65895 Parents: 511787d Author: Sylvain Lebresne sylv...@datastax.com Authored: Wed Feb 5 08:47:20 2014 +0100 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Wed Feb 5 08:47:20 2014 +0100 -- CHANGES.txt | 1 + NEWS.txt | 10 ++ build.xml| 2 +- debian/changelog | 6 ++ 4 files changed, 18 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/178e086f/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 75c7104..0989dc4 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -2,6 +2,7 @@ * Move handling of migration event source to solve bootstrap race (CASSANDRA-6648) * Make sure compaction throughput value doesn't overflow with int math (CASSANDRA-6647) + 1.2.14 * Reverted code to limit CQL prepared statement cache by size (CASSANDRA-6592) * add cassandra.default_messaging_version property to allow easier http://git-wip-us.apache.org/repos/asf/cassandra/blob/178e086f/NEWS.txt -- diff --git a/NEWS.txt b/NEWS.txt index 53cb7ca..771536d 100644 --- a/NEWS.txt +++ b/NEWS.txt @@ -14,6 +14,16 @@ restore snapshots created with the previous major version using the using the provided 'sstableupgrade' tool. +1.2.15 +== + +Upgrading +- +- This release mainly fix a regression of 1.2.14, namely + https://issues.apache.org/jira/browse/CASSANDRA-6648. Users of 1.2.14 are + strongly encouraged to upgrade. + + 1.2.14 == http://git-wip-us.apache.org/repos/asf/cassandra/blob/178e086f/build.xml -- diff --git a/build.xml b/build.xml index 150e2fe..ce87a2a 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ property name=debuglevel value=source,lines,vars/ !-- default version and SCM information -- -property name=base.version value=1.2.14/ +property name=base.version value=1.2.15/ property name=scm.connection value=scm:git://git.apache.org/cassandra.git/ property name=scm.developerConnection value=scm:git://git.apache.org/cassandra.git/ property name=scm.url value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/ http://git-wip-us.apache.org/repos/asf/cassandra/blob/178e086f/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index 472063e..bb8ecf2 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (1.2.15) unstable; urgency=low + + * New release + + -- Sylvain Lebresne slebre...@apache.org Wed, 05 Feb 2014 08:42:29 +0100 + cassandra (1.2.14) unstable; urgency=low * New release
[jira] [Updated] (CASSANDRA-6656) Exception logging
[
https://issues.apache.org/jira/browse/CASSANDRA-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ding Yuan updated CASSANDRA-6656:
-
Description:
Reporting a few cases where informative exceptions can be silently swallowed.
Attaching a proposed patch.
=
Case 1
Line: 95, File: org/apache/cassandra/utils/Hex.java
An actual failure in the underlying constructor will be lost.
Propose to log it.
{noformat}
try
{
s = stringConstructor.newInstance(0, c.length, c);
+ }
+ catch (InvocationTargetException ite) {
+ // The underlying constructor failed. Unwrapping the exception.
+ logger.info(Underlying constructor throws exception: ,
ite.getCause());
}
catch (Exception e)
{
// Swallowing as we'll just use a copying constructor
}
return s == null ? new String(c) : s;
{noformat}
==
=
Case 2
Line: 192, File: org/apache/cassandra/db/marshal/DynamicCompositeType.java
The actual cause of comparator error can be lost as it can fail in multiple
locations.
{noformat}
AbstractType? comparator = null;
int header = getShortLength(bb);
if ((header 0x8000) == 0)
{
ByteBuffer value = getBytes(bb, header);
try
{
comparator = TypeParser.parse(ByteBufferUtil.string(value));
}
catch (Exception e)
{
--- can fail here
// we'll deal with this below since comparator == null
}
}
else
{
comparator = aliases.get((byte)(header 0xFF));
--- can fail here
}
if (comparator == null)
throw new MarshalException(Cannot find comparator for component
+ i);
{noformat}
Propose to log the exception.
==
=
Case 3
Line: 239, File: org/apache/cassandra/tools/NodeProbe.java
Exception ignored in finally. Propose log them with debug or trace.
{noformat}
232: finally
233: {
234: try
235: {
236: ssProxy.removeNotificationListener(runner);
236: ssProxy.removeNotificationListener(runner);
237: jmxc.removeConnectionNotificationListener(runner);
238: }
239: catch (Throwable ignored) {}
240: }
{noformat}
Similar case is at line 264 in the same file.
==
was:
Reporting a few cases where informative exceptions can be silently swallowed.
Attaching a proposed patch.
=
Case 1
Line: 95, File: org/apache/cassandra/utils/Hex.java
An actual failure in the underlying constructor will be lost.
Propose to log it.
try
{
s = stringConstructor.newInstance(0, c.length, c);
+ }
+ catch (InvocationTargetException ite) {
+ // The underlying constructor failed. Unwrapping the exception.
+ logger.info(Underlying constructor throws exception: ,
ite.getCause());
}
catch (Exception e)
{
// Swallowing as we'll just use a copying constructor
}
return s == null ? new String(c) : s;
==
=
Case 2
Line: 192, File: org/apache/cassandra/db/marshal/DynamicCompositeType.java
The actual cause of comparator error can be lost as it can fail in multiple
locations.
AbstractType? comparator = null;
int header = getShortLength(bb);
if ((header 0x8000) == 0)
{
ByteBuffer value = getBytes(bb, header);
try
{
comparator = TypeParser.parse(ByteBufferUtil.string(value));
}
catch (Exception e)
{
--- can fail here
// we'll deal with this below since comparator == null
}
}
else
{
comparator = aliases.get((byte)(header 0xFF));
--- can fail here
}
if (comparator == null)
throw new MarshalException(Cannot find comparator for component
+ i);
Propose to log the exception.
==
=
Case 3
Line: 239, File: org/apache/cassandra/tools/NodeProbe.java
Exception ignored in finally. Propose log them with debug or trace.
232: finally
233: {
234: try
235: {
236: ssProxy.removeNotificationListener(runner);
236: ssProxy.removeNotificationListener(runner);
237:
[jira] [Resolved] (CASSANDRA-6642) line 297 of CQL.textile needs updated
[
https://issues.apache.org/jira/browse/CASSANDRA-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne resolved CASSANDRA-6642.
-
Resolution: Fixed
That whole paragraph about COMPACT STORAGE is indeed a tad whack. I've pushed a
hopefully better version of that paragraph that fixes the imprecision you
mentioned but also extend a bit on the limitations. Feel free to complete if
you think it's still not all that clear :)
line 297 of CQL.textile needs updated
-
Key: CASSANDRA-6642
URL: https://issues.apache.org/jira/browse/CASSANDRA-6642
Project: Cassandra
Issue Type: Bug
Components: Documentation website
Environment: [CQL 3.1.4 doc,
cassandra-trunk|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#partition-key-and-clustering]
Reporter: Kristine Hahn
Priority: Minor
[the
doc|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#partition-key-and-clustering]
says:
{quote}
The restriction for table with COMPACT STORAGE is that they support one and
only one column outside of the ones part of the PRIMARY KEY.
{quote}
Shouldn't it say, ... outside of the ones that are part of a compound primary
key?
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6655) Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush
[ https://issues.apache.org/jira/browse/CASSANDRA-6655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6655: Attachment: tmp.patch Fairly simple patch that adds the dataSize of any RangeTombstoneList to the memtable currentSize. Whilst this defect doesn't affect the 2.1 branch, this patch also reduces garbage generated when updating a memtable partition with pre-existing tombstones, which is probably worth forward-porting. Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush --- Key: CASSANDRA-6655 URL: https://issues.apache.org/jira/browse/CASSANDRA-6655 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor Fix For: 2.0.5 Attachments: tmp.patch In the extreme case of only deletes the memtable will never flush, and we will OOM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892271#comment-13892271 ] Nick Bailey commented on CASSANDRA-4757: Yeah that would probably be fine. Maybe an option to either print out the plan id or to print progress information. For the jmx case, a bulkLoadAsync call would probably be fine and is consistent with other jmx methods we have. My only other question would be if the building indexes step of bulk loading would be included in that progress as well. Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Tyler Hobbs Priority: Minor Labels: lhf Fix For: 2.0.6 Attachments: CASSANDRA-4757.txt, CASSANDRA-4757v2.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6655) Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush
[ https://issues.apache.org/jira/browse/CASSANDRA-6655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6655: Attachment: tmp-2.1.patch Added a patch for 2.1/trunk Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush --- Key: CASSANDRA-6655 URL: https://issues.apache.org/jira/browse/CASSANDRA-6655 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor Fix For: 2.0.5, 2.0.6 Attachments: tmp-2.1.patch, tmp.patch In the extreme case of only deletes the memtable will never flush, and we will OOM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Assigned] (CASSANDRA-6655) Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush
[ https://issues.apache.org/jira/browse/CASSANDRA-6655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict reassigned CASSANDRA-6655: --- Assignee: Benedict Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush --- Key: CASSANDRA-6655 URL: https://issues.apache.org/jira/browse/CASSANDRA-6655 Project: Cassandra Issue Type: Improvement Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 2.0.5, 2.0.6 Attachments: tmp-2.1.patch, tmp.patch In the extreme case of only deletes the memtable will never flush, and we will OOM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6655) Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush
[ https://issues.apache.org/jira/browse/CASSANDRA-6655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892291#comment-13892291 ] Sylvain Lebresne commented on CASSANDRA-6655: - While we're at it, we could also fix the update to currentOperations to take range tombstones into account, it doesn't particularly make sense that it's not taken into account there too (and it's currently somewhat breaking DataTacker.isEmpty()). Nit: I'd change 'isModifiedBy' to something like 'mayModify' (which imply reversing receiver and argument) to avoid future misuse since 'isModifiedBy' only tell if it may be modified but doesn't guarantee it will. The rest lgtm. Also probably worth committing to 1.2 (it's a proper bug and the fix is relatively simple). For those following along at home, this only affects deletes through range tombstones (though full partition deletions were not added to the memtable size, they were accounted in currentOperations and wouldn't end up not flushing). Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush --- Key: CASSANDRA-6655 URL: https://issues.apache.org/jira/browse/CASSANDRA-6655 Project: Cassandra Issue Type: Improvement Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 2.0.5, 2.0.6 Attachments: tmp-2.1.patch, tmp.patch In the extreme case of only deletes the memtable will never flush, and we will OOM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6645) upgradesstables causes NPE for secondary indexes without an underlying column family
[ https://issues.apache.org/jira/browse/CASSANDRA-6645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Bossa updated CASSANDRA-6645: Reproduced In: 2.0.4, 1.2.14 (was: 2.0.4) upgradesstables causes NPE for secondary indexes without an underlying column family Key: CASSANDRA-6645 URL: https://issues.apache.org/jira/browse/CASSANDRA-6645 Project: Cassandra Issue Type: Bug Components: Core Reporter: Sergio Bossa Assignee: Sergio Bossa Fix For: 2.0.6 Attachments: CASSANDRA-6645.patch SecondaryIndex#getIndexCfs is allowed to return null by contract, if the index is not backed by a column family, but this causes an NPE as StorageService#getValidColumnFamilies and StorageService#upgradeSSTables do not check for null values. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6655) Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush
[ https://issues.apache.org/jira/browse/CASSANDRA-6655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892294#comment-13892294 ] Sylvain Lebresne commented on CASSANDRA-6655: - bq. though full partition deletions were not added to the memtable size, they were accounted in currentOperations and wouldn't end up not flushing Scratch that, it seems we don't take currentOperations in account to flush anymore, so full partition deletions are also affected. Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush --- Key: CASSANDRA-6655 URL: https://issues.apache.org/jira/browse/CASSANDRA-6655 Project: Cassandra Issue Type: Improvement Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 2.0.5, 2.0.6 Attachments: tmp-2.1.patch, tmp.patch In the extreme case of only deletes the memtable will never flush, and we will OOM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6645) upgradesstables causes NPE for secondary indexes without an underlying column family
[ https://issues.apache.org/jira/browse/CASSANDRA-6645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Bossa updated CASSANDRA-6645: Attachment: CASSANDRA-6645-1_2.patch Attached patch for 1.2 too. upgradesstables causes NPE for secondary indexes without an underlying column family Key: CASSANDRA-6645 URL: https://issues.apache.org/jira/browse/CASSANDRA-6645 Project: Cassandra Issue Type: Bug Components: Core Reporter: Sergio Bossa Assignee: Sergio Bossa Fix For: 2.0.6 Attachments: CASSANDRA-6645-1_2.patch, CASSANDRA-6645.patch SecondaryIndex#getIndexCfs is allowed to return null by contract, if the index is not backed by a column family, but this causes an NPE as StorageService#getValidColumnFamilies and StorageService#upgradeSSTables do not check for null values. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6655) Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush
[ https://issues.apache.org/jira/browse/CASSANDRA-6655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892298#comment-13892298 ] Benedict commented on CASSANDRA-6655: - bq. and it's currently somewhat breaking DataTacker.isEmpty()). Good spot. bq. I'd change 'isModifiedBy' to something like 'mayModify' Sounds reasonable to me. I wanted to keep the logic the same as add() (to make sure it was super trivial and I didn't screw up) which required something of the form testXBy, and maybeModifiedBy sounded ugly at the time. But now it sounds no worse. Want me to repatch, or do you want to ninja it in? bq. For those following along at home, this only affects deletes through range tombstones (though full partition deletions were not added to the memtable size, they were accounted in currentOperations and wouldn't end up not flushing). Regrettably this would affect them too, if not quite as severely, as it's not just DataTracker.isEmpty() that's affected: the MeteredFlusher that would never select it for flushing, so it would only flush on its normal cycle or when the CL ran out of room. Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush --- Key: CASSANDRA-6655 URL: https://issues.apache.org/jira/browse/CASSANDRA-6655 Project: Cassandra Issue Type: Improvement Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 2.0.5, 2.0.6 Attachments: tmp-2.1.patch, tmp.patch In the extreme case of only deletes the memtable will never flush, and we will OOM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6655) Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush
[ https://issues.apache.org/jira/browse/CASSANDRA-6655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892301#comment-13892301 ] Benedict commented on CASSANDRA-6655: - bq. Scratch that, it seems we don't take currentOperations in account to flush anymore Okay, I take back my Good spot :-) Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush --- Key: CASSANDRA-6655 URL: https://issues.apache.org/jira/browse/CASSANDRA-6655 Project: Cassandra Issue Type: Improvement Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 2.0.5, 2.0.6 Attachments: tmp-2.1.patch, tmp.patch In the extreme case of only deletes the memtable will never flush, and we will OOM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (CASSANDRA-6657) Log the newsize value alongside the heap size at startup
Jeremy Hanna created CASSANDRA-6657: --- Summary: Log the newsize value alongside the heap size at startup Key: CASSANDRA-6657 URL: https://issues.apache.org/jira/browse/CASSANDRA-6657 Project: Cassandra Issue Type: Wish Components: Core Reporter: Jeremy Hanna It would be nice to have the newsize value logged alongside the heap size at startup to more easily track down problems. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6657) Log the newsize value alongside the heap size at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna updated CASSANDRA-6657: Priority: Trivial (was: Major) Log the newsize value alongside the heap size at startup Key: CASSANDRA-6657 URL: https://issues.apache.org/jira/browse/CASSANDRA-6657 Project: Cassandra Issue Type: Wish Components: Core Reporter: Jeremy Hanna Priority: Trivial It would be nice to have the newsize value logged alongside the heap size at startup to more easily track down problems. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6528) TombstoneOverwhelmingException is thrown while populating data in recently truncated CF
[
https://issues.apache.org/jira/browse/CASSANDRA-6528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892315#comment-13892315
]
Machiel Groeneveld commented on CASSANDRA-6528:
---
I have the same issue, after inserting 216258 records (in one row) in a new
database (I removed all files in the data directory files before starting) I
couldn't run a select query (something like 'select * from partition_key = x')
TombstoneOverwhelmingException is thrown while populating data in recently
truncated CF
---
Key: CASSANDRA-6528
URL: https://issues.apache.org/jira/browse/CASSANDRA-6528
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassadra 2.0.3, Linux, 6 nodes
Reporter: Nikolai Grigoriev
Priority: Minor
I am running some performance tests and recently I had to flush the data from
one of the tables and repopulate it. I have about 30M rows with a few columns
in each, about 5kb per row in in total. In order to repopulate the data I do
truncate table from CQLSH and then relaunch the test. The test simply
inserts the data in the table, does not read anything. Shortly after
restarting the data generator I see this on one of the nodes:
{code}
INFO [HintedHandoff:655] 2013-12-26 16:45:42,185 HintedHandOffManager.java
(line 323) Started hinted handoff f
or host: 985c8a08-3d92-4fad-a1d1-7135b2b9774a with IP: /10.5.45.158
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 SliceQueryFilter.java (line
200) Scanned ove
r 10 tombstones; query aborted (see tombstone_fail_threshold)
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 CassandraDaemon.java (line
187) Exception in thread Thread[HintedHandoff:655,1,main]
org.apache.cassandra.db.filter.TombstoneOverwhelmingException
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:201)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:56)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:351)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:309)
at
org.apache.cassandra.db.HintedHandOffManager.access$4(HintedHandOffManager.java:281)
at
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:530)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
INFO [OptionalTasks:1] 2013-12-26 16:45:53,946 MeteredFlusher.java (line 63)
flushing high-traffic column family CFS(Keyspace='test_jmeter',
ColumnFamily='test_profiles') (estimated 192717267 bytes)
{code}
I am inserting the data with CL=1.
It seems to be happening every time I do it. But I do not see any errors on
the client side and the node seems to continue operating, this is why I think
it is not a major issue. Maybe not an issue at all, but the message is logged
as ERROR.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6655) Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush
[ https://issues.apache.org/jira/browse/CASSANDRA-6655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892316#comment-13892316 ] Sylvain Lebresne commented on CASSANDRA-6655: - bq. Want me to repatch, or do you want to ninja it in? I can change while committing I guess. Writing mostly deletes to a Memtable results in undercounting the table's occupancy so it may not flush --- Key: CASSANDRA-6655 URL: https://issues.apache.org/jira/browse/CASSANDRA-6655 Project: Cassandra Issue Type: Improvement Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 2.0.5, 2.0.6 Attachments: tmp-2.1.patch, tmp.patch In the extreme case of only deletes the memtable will never flush, and we will OOM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Comment Edited] (CASSANDRA-6528) TombstoneOverwhelmingException is thrown while populating data in recently truncated CF
[
https://issues.apache.org/jira/browse/CASSANDRA-6528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892315#comment-13892315
]
Machiel Groeneveld edited comment on CASSANDRA-6528 at 2/5/14 5:14 PM:
---
I have the same issue, after inserting 216258 records (in one row) in a new
database (I removed all files in the data directory files before starting) I
couldn't run a select query (something like 'select * from partition_key = x')
In the log I get org.apache.cassandra.db.filter.TombstoneOverwhelmingException
was (Author: machielg):
I have the same issue, after inserting 216258 records (in one row) in a new
database (I removed all files in the data directory files before starting) I
couldn't run a select query (something like 'select * from partition_key = x')
TombstoneOverwhelmingException is thrown while populating data in recently
truncated CF
---
Key: CASSANDRA-6528
URL: https://issues.apache.org/jira/browse/CASSANDRA-6528
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassadra 2.0.3, Linux, 6 nodes
Reporter: Nikolai Grigoriev
Priority: Minor
I am running some performance tests and recently I had to flush the data from
one of the tables and repopulate it. I have about 30M rows with a few columns
in each, about 5kb per row in in total. In order to repopulate the data I do
truncate table from CQLSH and then relaunch the test. The test simply
inserts the data in the table, does not read anything. Shortly after
restarting the data generator I see this on one of the nodes:
{code}
INFO [HintedHandoff:655] 2013-12-26 16:45:42,185 HintedHandOffManager.java
(line 323) Started hinted handoff f
or host: 985c8a08-3d92-4fad-a1d1-7135b2b9774a with IP: /10.5.45.158
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 SliceQueryFilter.java (line
200) Scanned ove
r 10 tombstones; query aborted (see tombstone_fail_threshold)
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 CassandraDaemon.java (line
187) Exception in thread Thread[HintedHandoff:655,1,main]
org.apache.cassandra.db.filter.TombstoneOverwhelmingException
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:201)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:56)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:351)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:309)
at
org.apache.cassandra.db.HintedHandOffManager.access$4(HintedHandOffManager.java:281)
at
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:530)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
INFO [OptionalTasks:1] 2013-12-26 16:45:53,946 MeteredFlusher.java (line 63)
flushing high-traffic column family CFS(Keyspace='test_jmeter',
ColumnFamily='test_profiles') (estimated 192717267 bytes)
{code}
I am inserting the data with CL=1.
It seems to be happening every time I do it. But I do not see any errors on
the client side and the node seems to continue operating, this is why I think
it is not a major issue. Maybe not an issue at all, but the message is logged
as ERROR.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6656) Exception logging
[
https://issues.apache.org/jira/browse/CASSANDRA-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Stepura updated CASSANDRA-6656:
---
Fix Version/s: (was: 2.0.4)
Exception logging
-
Key: CASSANDRA-6656
URL: https://issues.apache.org/jira/browse/CASSANDRA-6656
Project: Cassandra
Issue Type: Improvement
Components: Core, Tools
Reporter: Ding Yuan
Attachments: trunk-6656.txt
Reporting a few cases where informative exceptions can be silently swallowed.
Attaching a proposed patch.
=
Case 1
Line: 95, File: org/apache/cassandra/utils/Hex.java
An actual failure in the underlying constructor will be lost.
Propose to log it.
{noformat}
try
{
s = stringConstructor.newInstance(0, c.length, c);
+ }
+ catch (InvocationTargetException ite) {
+ // The underlying constructor failed. Unwrapping the
exception.
+ logger.info(Underlying constructor throws exception: ,
ite.getCause());
}
catch (Exception e)
{
// Swallowing as we'll just use a copying constructor
}
return s == null ? new String(c) : s;
{noformat}
==
=
Case 2
Line: 192, File: org/apache/cassandra/db/marshal/DynamicCompositeType.java
The actual cause of comparator error can be lost as it can fail in multiple
locations.
{noformat}
AbstractType? comparator = null;
int header = getShortLength(bb);
if ((header 0x8000) == 0)
{
ByteBuffer value = getBytes(bb, header);
try
{
comparator = TypeParser.parse(ByteBufferUtil.string(value));
}
catch (Exception e)
{
--- can fail here
// we'll deal with this below since comparator == null
}
}
else
{
comparator = aliases.get((byte)(header 0xFF));
--- can fail here
}
if (comparator == null)
throw new MarshalException(Cannot find comparator for component
+ i);
{noformat}
Propose to log the exception.
==
=
Case 3
Line: 239, File: org/apache/cassandra/tools/NodeProbe.java
Exception ignored in finally. Propose log them with debug or trace.
{noformat}
232: finally
233: {
234: try
235: {
236: ssProxy.removeNotificationListener(runner);
236: ssProxy.removeNotificationListener(runner);
237: jmxc.removeConnectionNotificationListener(runner);
238: }
239: catch (Throwable ignored) {}
240: }
{noformat}
Similar case is at line 264 in the same file.
==
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6658) Nodes flap once at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-6658: Summary: Nodes flap once at startup (was: Nodes flap once at starup) Nodes flap once at startup -- Key: CASSANDRA-6658 URL: https://issues.apache.org/jira/browse/CASSANDRA-6658 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Priority: Minor Fix For: 2.0.4 Upon initially seeing each other, a node will mark another UP, then DOWN, then UP again. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (CASSANDRA-6658) Nodes flap once at starup
Brandon Williams created CASSANDRA-6658: --- Summary: Nodes flap once at starup Key: CASSANDRA-6658 URL: https://issues.apache.org/jira/browse/CASSANDRA-6658 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Priority: Minor Fix For: 2.0.4 Upon initially seeing each other, a node will mark another UP, then DOWN, then UP again. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6658) Nodes flap once at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-6658: Since Version: 2.0.4 Fix Version/s: (was: 2.0.4) Nodes flap once at startup -- Key: CASSANDRA-6658 URL: https://issues.apache.org/jira/browse/CASSANDRA-6658 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Priority: Minor Upon initially seeing each other, a node will mark another UP, then DOWN, then UP again. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
git commit: Fix partition and range deletes not triggering flush
Updated Branches:
refs/heads/cassandra-1.2 16efdf4a0 - adcb713d5
Fix partition and range deletes not triggering flush
patch by benedict; reviewed by slebresne for CASSANDRA-6655
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/adcb713d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/adcb713d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/adcb713d
Branch: refs/heads/cassandra-1.2
Commit: adcb713d597302a868b6224a87ea6ce38e718e5d
Parents: 16efdf4
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Feb 5 18:34:37 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Feb 5 18:34:37 2014 +0100
--
CHANGES.txt | 3 +++
.../org/apache/cassandra/db/AtomicSortedColumns.java | 7 ++-
src/java/org/apache/cassandra/db/DeletionInfo.java| 14 ++
src/java/org/apache/cassandra/db/Memtable.java| 10 ++
4 files changed, 29 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/adcb713d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0989dc4..cfdd148 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+1.2.16
+ * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
+
1.2.15
* Move handling of migration event source to solve bootstrap race
(CASSANDRA-6648)
* Make sure compaction throughput value doesn't overflow with int math
(CASSANDRA-6647)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/adcb713d/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
b/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
index 9803544..d6c861b 100644
--- a/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
@@ -194,7 +194,12 @@ public class AtomicSortedColumns implements ISortedColumns
{
sizeDelta = 0;
current = ref.get();
-DeletionInfo newDelInfo =
current.deletionInfo.copy().add(cm.getDeletionInfo());
+DeletionInfo newDelInfo = current.deletionInfo;
+if (cm.getDeletionInfo().mayModify(newDelInfo))
+{
+newDelInfo =
current.deletionInfo.copy().add(cm.getDeletionInfo());
+sizeDelta += newDelInfo.dataSize() -
current.deletionInfo.dataSize();
+}
modified = new Holder(current.map.clone(), newDelInfo);
for (IColumn column : cm.getSortedColumns())
http://git-wip-us.apache.org/repos/asf/cassandra/blob/adcb713d/src/java/org/apache/cassandra/db/DeletionInfo.java
--
diff --git a/src/java/org/apache/cassandra/db/DeletionInfo.java
b/src/java/org/apache/cassandra/db/DeletionInfo.java
index e486eeb..91af9fd 100644
--- a/src/java/org/apache/cassandra/db/DeletionInfo.java
+++ b/src/java/org/apache/cassandra/db/DeletionInfo.java
@@ -216,6 +216,20 @@ public class DeletionInfo
return size + (ranges == null ? 0 : ranges.dataSize());
}
+public int rangeCount()
+{
+return ranges == null ? 0 : ranges.size();
+}
+
+/**
+ * Whether this deletion info may modify the provided one if added to it.
+ */
+public boolean mayModify(DeletionInfo delInfo)
+{
+return topLevel.markedForDeleteAt delInfo.topLevel.markedForDeleteAt
+|| ranges == null;
+}
+
@Override
public String toString()
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/adcb713d/src/java/org/apache/cassandra/db/Memtable.java
--
diff --git a/src/java/org/apache/cassandra/db/Memtable.java
b/src/java/org/apache/cassandra/db/Memtable.java
index 817561b..b229060 100644
--- a/src/java/org/apache/cassandra/db/Memtable.java
+++ b/src/java/org/apache/cassandra/db/Memtable.java
@@ -192,6 +192,7 @@ public class Memtable
{
ColumnFamily previous = columnFamilies.get(key);
+long sizeDelta = 0;
if (previous == null)
{
// AtomicSortedColumns doesn't work for super columns (see #3821)
@@ -199,14 +200,15 @@ public class Memtable
// We'll add the columns later. This avoids wasting works if we
get beaten in the putIfAbsent
previous = columnFamilies.putIfAbsent(new DecoratedKey(key.token,
allocator.clone(key.key)), empty);
if (previous == null)
+{
previous = empty;
+sizeDelta +=
[jira] [Commented] (CASSANDRA-6528) TombstoneOverwhelmingException is thrown while populating data in recently truncated CF
[
https://issues.apache.org/jira/browse/CASSANDRA-6528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892335#comment-13892335
]
Aleksey Yeschenko commented on CASSANDRA-6528:
--
bq. I have the same issue, after inserting 216258 records (in one row) in a new
database
Your schema and an example query?
TombstoneOverwhelmingException is thrown while populating data in recently
truncated CF
---
Key: CASSANDRA-6528
URL: https://issues.apache.org/jira/browse/CASSANDRA-6528
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassadra 2.0.3, Linux, 6 nodes
Reporter: Nikolai Grigoriev
Priority: Minor
I am running some performance tests and recently I had to flush the data from
one of the tables and repopulate it. I have about 30M rows with a few columns
in each, about 5kb per row in in total. In order to repopulate the data I do
truncate table from CQLSH and then relaunch the test. The test simply
inserts the data in the table, does not read anything. Shortly after
restarting the data generator I see this on one of the nodes:
{code}
INFO [HintedHandoff:655] 2013-12-26 16:45:42,185 HintedHandOffManager.java
(line 323) Started hinted handoff f
or host: 985c8a08-3d92-4fad-a1d1-7135b2b9774a with IP: /10.5.45.158
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 SliceQueryFilter.java (line
200) Scanned ove
r 10 tombstones; query aborted (see tombstone_fail_threshold)
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 CassandraDaemon.java (line
187) Exception in thread Thread[HintedHandoff:655,1,main]
org.apache.cassandra.db.filter.TombstoneOverwhelmingException
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:201)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:56)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:351)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:309)
at
org.apache.cassandra.db.HintedHandOffManager.access$4(HintedHandOffManager.java:281)
at
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:530)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
INFO [OptionalTasks:1] 2013-12-26 16:45:53,946 MeteredFlusher.java (line 63)
flushing high-traffic column family CFS(Keyspace='test_jmeter',
ColumnFamily='test_profiles') (estimated 192717267 bytes)
{code}
I am inserting the data with CL=1.
It seems to be happening every time I do it. But I do not see any errors on
the client side and the node seems to continue operating, this is why I think
it is not a major issue. Maybe not an issue at all, but the message is logged
as ERROR.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6651) Repair hanging
[ https://issues.apache.org/jira/browse/CASSANDRA-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892345#comment-13892345 ] Thunder Stumpges commented on CASSANDRA-6651: - FWIW we have this exact same issue. We are running 2.0.3 on a 3 node cluster. It has happened multiple times, and happens more times than not when running nodetool repair. There is nearly always one or more AntiEntropySessions remaining according to tpstats. One strange thing about the behavior I see is that the output of nodetool compactionstats returns 0 active compactions, yet when restarting, we get the exception about Unfinished compactions reference missing sstables. It does seem like these two issues are related. Another thing I see sometimes in the ouput from nodetool repair is the following message: [2014-02-04 14:07:30,858] Starting repair command #7, repairing 768 ranges for keyspace thunder_test [2014-02-04 14:08:30,862] Lost notification. You should check server log for repair status of keyspace thunder_test [2014-02-04 14:08:30,870] Starting repair command #8, repairing 768 ranges for keyspace doan_synset [2014-02-04 14:09:30,874] Lost notification. You should check server log for repair status of keyspace doan_synset When this happens, it starts the next repair session immediately rather than waiting for the current one to finish. This doesn't however seem to always correlate to a hung session. My logs don't look much/any different from the OP, but I'd be glad to provide any more details that might be helpful. We will be upgrading to 2.0.4 in the next couple days and I will report back if we see any difference in behavior. Repair hanging -- Key: CASSANDRA-6651 URL: https://issues.apache.org/jira/browse/CASSANDRA-6651 Project: Cassandra Issue Type: Bug Components: Core Reporter: Eitan Eibschutz Fix For: 2.0.3 Hi, We have a 12 node cluster in PROD environment and we've noticed that repairs are never finishing. The behavior that we've observed is that a repair process will run until at some point it hangs and no other processing is happening. For example, at the moment, I have a repair process that has been running for two days and not finishing: nodetool tpstats is showing 2 active and 2 pending AntiEntropySessions nodetool compactionstats is showing: pending tasks: 0 Active compaction remaining time :n/a nodetools netstats is showing: Mode: NORMAL Not sending any streams. Read Repair Statistics: Attempted: 0 Mismatch (Blocking): 142110 Mismatch (Background): 0 Pool NameActive Pending Completed Commandsn/a 0 107589657 Responses n/a 0 116430785 The last entry that I see in the log is: INFO [AntiEntropySessions:18] 2014-02-03 04:01:39,145 RepairJob.java (line 116) [repair #ae78c6c0-8c2b-11e3-b950-c3b81a36bc9b] requesting merkle trees for MyCF (to [/x.x.x.x, /y.y.y.y, /z.z.z.z]) The repair started at 4am so it stopped after 1:40 minute. On node y.y.y.y I can see this in the log: INFO [MiscStage:1] 2014-02-03 04:01:38,985 ColumnFamilyStore.java (line 740) Enqueuing flush of Memtable-MyCF@1290890489(2176/5931 serialized/live bytes, 32 ops) INFO [FlushWriter:411] 2014-02-03 04:01:38,986 Memtable.java (line 333) Writing Memtable-MyCF@1290890489(2176/5931 serialized/live bytes, 32 ops) INFO [FlushWriter:411] 2014-02-03 04:01:39,048 Memtable.java (line 373) Completed flushing /var/lib/cassandra/main-db/data/MyKS/MyCF/MyKS-MyCF-jb-518-Data.db (1789 bytes) for commitlog position ReplayPosition(segmentId=1390437013339, position=21868792) INFO [ScheduledTasks:1] 2014-02-03 05:00:04,794 ColumnFamilyStore.java (line 740) Enqueuing flush of Memtable-compaction_history@1649414699(1635/17360 serialized/live bytes, 42 ops) So for some reason the merkle tree for this CF is never sent back to the node being repaired and it's hanging. I've also noticed that sometimes, restarting node y.y.y.y will cause the repair to resume. Another observation is that sometimes when restarting y.y.y.y it will not start with these errors: ERROR 16:34:18,485 Exception encountered during startup java.lang.IllegalStateException: Unfinished compactions reference missing sstables. This should never happen since compactions are marked finished before we start removing the old sstables. at org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:495) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:264) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:461) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:504)
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892350#comment-13892350 ] Tyler Hobbs commented on CASSANDRA-4757: bq. My only other question would be if the building indexes step of bulk loading would be included in that progress as well. In 2.0 index building is the last step before marking a streaming task as complete, so it's included. Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Tyler Hobbs Priority: Minor Labels: lhf Fix For: 2.0.6 Attachments: CASSANDRA-4757.txt, CASSANDRA-4757v2.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[3/3] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/AtomicSortedColumns.java src/java/org/apache/cassandra/db/Memtable.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fe4247e5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fe4247e5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fe4247e5 Branch: refs/heads/trunk Commit: fe4247e589714d9ea183187c0538b6446f16ffca Parents: 58d1a4f 58e9481 Author: Sylvain Lebresne sylv...@datastax.com Authored: Wed Feb 5 18:51:40 2014 +0100 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Wed Feb 5 18:51:40 2014 +0100 -- CHANGES.txt | 8 ++- .../apache/cassandra/db/AtomicBTreeColumns.java | 22 .../org/apache/cassandra/db/DeletionInfo.java | 14 + src/java/org/apache/cassandra/db/Memtable.java | 4 +--- 4 files changed, 30 insertions(+), 18 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/fe4247e5/CHANGES.txt -- diff --cc CHANGES.txt index 0690c38,bba5f20..7d628b5 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,40 -1,7 +1,36 @@@ +2.1 + * add listsnapshots command to nodetool (CASSANDRA-5742) + * Introduce AtomicBTreeColumns (CASSANDRA-6271) + * Multithreaded commitlog (CASSANDRA-3578) + * allocate fixed index summary memory pool and resample cold index summaries + to use less memory (CASSANDRA-5519) + * Removed multithreaded compaction (CASSANDRA-6142) + * Parallelize fetching rows for low-cardinality indexes (CASSANDRA-1337) + * change logging from log4j to logback (CASSANDRA-5883) + * switch to LZ4 compression for internode communication (CASSANDRA-5887) + * Stop using Thrift-generated Index* classes internally (CASSANDRA-5971) + * Remove 1.2 network compatibility code (CASSANDRA-5960) + * Remove leveled json manifest migration code (CASSANDRA-5996) + * Remove CFDefinition (CASSANDRA-6253) + * Use AtomicIntegerFieldUpdater in RefCountedMemory (CASSANDRA-6278) + * User-defined types for CQL3 (CASSANDRA-5590) + * Use of o.a.c.metrics in nodetool (CASSANDRA-5871, 6406) + * Batch read from OTC's queue and cleanup (CASSANDRA-1632) + * Secondary index support for collections (CASSANDRA-4511, 6383) + * SSTable metadata(Stats.db) format change (CASSANDRA-6356) + * Push composites support in the storage engine + (CASSANDRA-5417, CASSANDRA-6520) + * Add snapshot space used to cfstats (CASSANDRA-6231) + * Add cardinality estimator for key count estimation (CASSANDRA-5906) + * CF id is changed to be non-deterministic. Data dir/key cache are created + uniquely for CF id (CASSANDRA-5202) + * New counters implementation (CASSANDRA-6504) + + 2.0.6 - * Fix direct Memory on architectures that do not support unaligned long access -(CASSANDRA-6628) - * Let scrub optionally skip broken counter partitions (CASSANDRA-5930) Merged from 1.2: - * Move handling of migration event source to solve bootstrap race. (CASSANDRA-6648) - * Make sure compaction throughput value doesn't overflow with int math (CASSANDRA-6647) - + * Fix partition and range deletes not triggering flush (CASSANDRA-6655) + 2.0.5 * Reduce garbage generated by bloom filter lookups (CASSANDRA-6609) http://git-wip-us.apache.org/repos/asf/cassandra/blob/fe4247e5/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java -- diff --cc src/java/org/apache/cassandra/db/AtomicBTreeColumns.java index 238bb7c,000..fd7d4bc mode 100644,00..100644 --- a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java +++ b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java @@@ -1,457 -1,0 +1,461 @@@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * License); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an AS IS BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.db; + +import java.util.AbstractCollection; +import java.util.ArrayList;
[1/3] git commit: Fix partition and range deletes not triggering flush
Updated Branches:
refs/heads/trunk 58d1a4f81 - fe4247e58
Fix partition and range deletes not triggering flush
patch by benedict; reviewed by slebresne for CASSANDRA-6655
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/adcb713d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/adcb713d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/adcb713d
Branch: refs/heads/trunk
Commit: adcb713d597302a868b6224a87ea6ce38e718e5d
Parents: 16efdf4
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Feb 5 18:34:37 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Feb 5 18:34:37 2014 +0100
--
CHANGES.txt | 3 +++
.../org/apache/cassandra/db/AtomicSortedColumns.java | 7 ++-
src/java/org/apache/cassandra/db/DeletionInfo.java| 14 ++
src/java/org/apache/cassandra/db/Memtable.java| 10 ++
4 files changed, 29 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/adcb713d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0989dc4..cfdd148 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+1.2.16
+ * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
+
1.2.15
* Move handling of migration event source to solve bootstrap race
(CASSANDRA-6648)
* Make sure compaction throughput value doesn't overflow with int math
(CASSANDRA-6647)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/adcb713d/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
b/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
index 9803544..d6c861b 100644
--- a/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
@@ -194,7 +194,12 @@ public class AtomicSortedColumns implements ISortedColumns
{
sizeDelta = 0;
current = ref.get();
-DeletionInfo newDelInfo =
current.deletionInfo.copy().add(cm.getDeletionInfo());
+DeletionInfo newDelInfo = current.deletionInfo;
+if (cm.getDeletionInfo().mayModify(newDelInfo))
+{
+newDelInfo =
current.deletionInfo.copy().add(cm.getDeletionInfo());
+sizeDelta += newDelInfo.dataSize() -
current.deletionInfo.dataSize();
+}
modified = new Holder(current.map.clone(), newDelInfo);
for (IColumn column : cm.getSortedColumns())
http://git-wip-us.apache.org/repos/asf/cassandra/blob/adcb713d/src/java/org/apache/cassandra/db/DeletionInfo.java
--
diff --git a/src/java/org/apache/cassandra/db/DeletionInfo.java
b/src/java/org/apache/cassandra/db/DeletionInfo.java
index e486eeb..91af9fd 100644
--- a/src/java/org/apache/cassandra/db/DeletionInfo.java
+++ b/src/java/org/apache/cassandra/db/DeletionInfo.java
@@ -216,6 +216,20 @@ public class DeletionInfo
return size + (ranges == null ? 0 : ranges.dataSize());
}
+public int rangeCount()
+{
+return ranges == null ? 0 : ranges.size();
+}
+
+/**
+ * Whether this deletion info may modify the provided one if added to it.
+ */
+public boolean mayModify(DeletionInfo delInfo)
+{
+return topLevel.markedForDeleteAt delInfo.topLevel.markedForDeleteAt
+|| ranges == null;
+}
+
@Override
public String toString()
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/adcb713d/src/java/org/apache/cassandra/db/Memtable.java
--
diff --git a/src/java/org/apache/cassandra/db/Memtable.java
b/src/java/org/apache/cassandra/db/Memtable.java
index 817561b..b229060 100644
--- a/src/java/org/apache/cassandra/db/Memtable.java
+++ b/src/java/org/apache/cassandra/db/Memtable.java
@@ -192,6 +192,7 @@ public class Memtable
{
ColumnFamily previous = columnFamilies.get(key);
+long sizeDelta = 0;
if (previous == null)
{
// AtomicSortedColumns doesn't work for super columns (see #3821)
@@ -199,14 +200,15 @@ public class Memtable
// We'll add the columns later. This avoids wasting works if we
get beaten in the putIfAbsent
previous = columnFamilies.putIfAbsent(new DecoratedKey(key.token,
allocator.clone(key.key)), empty);
if (previous == null)
+{
previous = empty;
+sizeDelta +=
[2/2] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0
Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/AtomicSortedColumns.java
src/java/org/apache/cassandra/db/DeletionInfo.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/58e94818
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/58e94818
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/58e94818
Branch: refs/heads/cassandra-2.0
Commit: 58e948185e214dbdc68e4ce533edb4dfa5430b51
Parents: 49bb972 adcb713
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Feb 5 18:42:00 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Feb 5 18:42:00 2014 +0100
--
CHANGES.txt | 5 +
.../org/apache/cassandra/db/AtomicSortedColumns.java | 7 ++-
src/java/org/apache/cassandra/db/DeletionInfo.java| 14 ++
src/java/org/apache/cassandra/db/Memtable.java| 10 ++
4 files changed, 31 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/58e94818/CHANGES.txt
--
diff --cc CHANGES.txt
index 9599e56,cfdd148..bba5f20
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,29 -1,21 +1,34 @@@
-1.2.16
++2.0.6
++Merged from 1.2:
+ * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
+
-1.2.15
- * Move handling of migration event source to solve bootstrap race
(CASSANDRA-6648)
- * Make sure compaction throughput value doesn't overflow with int math
(CASSANDRA-6647)
-
+
-1.2.14
- * Reverted code to limit CQL prepared statement cache by size
(CASSANDRA-6592)
- * add cassandra.default_messaging_version property to allow easier
- upgrading from 1.1 (CASSANDRA-6619)
- * Allow executing CREATE statements multiple times (CASSANDRA-6471)
- * Don't send confusing info with timeouts (CASSANDRA-6491)
- * Don't resubmit counter mutation runnables internally (CASSANDRA-6427)
- * Don't drop local mutations without a hint (CASSANDRA-6510)
- * Don't allow null max_hint_window_in_ms (CASSANDRA-6419)
- * Validate SliceRange start and finish lengths (CASSANDRA-6521)
+2.0.5
+ * Reduce garbage generated by bloom filter lookups (CASSANDRA-6609)
+ * Add ks.cf names to tombstone logging (CASSANDRA-6597)
+ * Use LOCAL_QUORUM for LWT operations at LOCAL_SERIAL (CASSANDRA-6495)
+ * Wait for gossip to settle before accepting client connections
(CASSANDRA-4288)
+ * Delete unfinished compaction incrementally (CASSANDRA-6086)
+ * Allow specifying custom secondary index options in CQL3 (CASSANDRA-6480)
+ * Improve replica pinning for cache efficiency in DES (CASSANDRA-6485)
+ * Fix LOCAL_SERIAL from thrift (CASSANDRA-6584)
+ * Don't special case received counts in CAS timeout exceptions
(CASSANDRA-6595)
+ * Add support for 2.1 global counter shards (CASSANDRA-6505)
+ * Fix NPE when streaming connection is not yet established (CASSANDRA-6210)
+ * Avoid rare duplicate read repair triggering (CASSANDRA-6606)
+ * Fix paging discardFirst (CASSANDRA-6555)
+ * Fix ArrayIndexOutOfBoundsException in 2ndary index query (CASSANDRA-6470)
+ * Release sstables upon rebuilding 2i (CASSANDRA-6635)
+ * Add AbstractCompactionStrategy.startup() method (CASSANDRA-6637)
+ * SSTableScanner may skip rows during cleanup (CASSANDRA-6638)
+ * sstables from stalled repair sessions can resurrect deleted data
(CASSANDRA-6503)
+ * Switch stress to use ITransportFactory (CASSANDRA-6641)
+ * Fix IllegalArgumentException during prepare (CASSANDRA-6592)
+ * Fix possible loss of 2ndary index entries during compaction
(CASSANDRA-6517)
+ * Fix direct Memory on architectures that do not support unaligned long
access
+ (CASSANDRA-6628)
+ * Let scrub optionally skip broken counter partitions (CASSANDRA-5930)
+Merged from 1.2:
* fsync compression metadata (CASSANDRA-6531)
* Validate CF existence on execution for prepared statement (CASSANDRA-6535)
* Add ability to throttle batchlog replay (CASSANDRA-6550)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/58e94818/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
--
diff --cc src/java/org/apache/cassandra/db/AtomicSortedColumns.java
index 1c0bf1b,d6c861b..d3a979c
--- a/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
@@@ -178,19 -194,15 +178,24 @@@ public class AtomicSortedColumns extend
{
sizeDelta = 0;
current = ref.get();
- DeletionInfo newDelInfo =
current.deletionInfo.copy().add(cm.deletionInfo());
+ DeletionInfo newDelInfo =
[1/2] git commit: Fix partition and range deletes not triggering flush
Updated Branches:
refs/heads/cassandra-2.0 49bb972c6 - 58e948185
Fix partition and range deletes not triggering flush
patch by benedict; reviewed by slebresne for CASSANDRA-6655
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/adcb713d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/adcb713d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/adcb713d
Branch: refs/heads/cassandra-2.0
Commit: adcb713d597302a868b6224a87ea6ce38e718e5d
Parents: 16efdf4
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Feb 5 18:34:37 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Feb 5 18:34:37 2014 +0100
--
CHANGES.txt | 3 +++
.../org/apache/cassandra/db/AtomicSortedColumns.java | 7 ++-
src/java/org/apache/cassandra/db/DeletionInfo.java| 14 ++
src/java/org/apache/cassandra/db/Memtable.java| 10 ++
4 files changed, 29 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/adcb713d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0989dc4..cfdd148 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+1.2.16
+ * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
+
1.2.15
* Move handling of migration event source to solve bootstrap race
(CASSANDRA-6648)
* Make sure compaction throughput value doesn't overflow with int math
(CASSANDRA-6647)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/adcb713d/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
b/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
index 9803544..d6c861b 100644
--- a/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
@@ -194,7 +194,12 @@ public class AtomicSortedColumns implements ISortedColumns
{
sizeDelta = 0;
current = ref.get();
-DeletionInfo newDelInfo =
current.deletionInfo.copy().add(cm.getDeletionInfo());
+DeletionInfo newDelInfo = current.deletionInfo;
+if (cm.getDeletionInfo().mayModify(newDelInfo))
+{
+newDelInfo =
current.deletionInfo.copy().add(cm.getDeletionInfo());
+sizeDelta += newDelInfo.dataSize() -
current.deletionInfo.dataSize();
+}
modified = new Holder(current.map.clone(), newDelInfo);
for (IColumn column : cm.getSortedColumns())
http://git-wip-us.apache.org/repos/asf/cassandra/blob/adcb713d/src/java/org/apache/cassandra/db/DeletionInfo.java
--
diff --git a/src/java/org/apache/cassandra/db/DeletionInfo.java
b/src/java/org/apache/cassandra/db/DeletionInfo.java
index e486eeb..91af9fd 100644
--- a/src/java/org/apache/cassandra/db/DeletionInfo.java
+++ b/src/java/org/apache/cassandra/db/DeletionInfo.java
@@ -216,6 +216,20 @@ public class DeletionInfo
return size + (ranges == null ? 0 : ranges.dataSize());
}
+public int rangeCount()
+{
+return ranges == null ? 0 : ranges.size();
+}
+
+/**
+ * Whether this deletion info may modify the provided one if added to it.
+ */
+public boolean mayModify(DeletionInfo delInfo)
+{
+return topLevel.markedForDeleteAt delInfo.topLevel.markedForDeleteAt
+|| ranges == null;
+}
+
@Override
public String toString()
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/adcb713d/src/java/org/apache/cassandra/db/Memtable.java
--
diff --git a/src/java/org/apache/cassandra/db/Memtable.java
b/src/java/org/apache/cassandra/db/Memtable.java
index 817561b..b229060 100644
--- a/src/java/org/apache/cassandra/db/Memtable.java
+++ b/src/java/org/apache/cassandra/db/Memtable.java
@@ -192,6 +192,7 @@ public class Memtable
{
ColumnFamily previous = columnFamilies.get(key);
+long sizeDelta = 0;
if (previous == null)
{
// AtomicSortedColumns doesn't work for super columns (see #3821)
@@ -199,14 +200,15 @@ public class Memtable
// We'll add the columns later. This avoids wasting works if we
get beaten in the putIfAbsent
previous = columnFamilies.putIfAbsent(new DecoratedKey(key.token,
allocator.clone(key.key)), empty);
if (previous == null)
+{
previous = empty;
+sizeDelta +=
[2/3] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0
Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/AtomicSortedColumns.java
src/java/org/apache/cassandra/db/DeletionInfo.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/58e94818
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/58e94818
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/58e94818
Branch: refs/heads/trunk
Commit: 58e948185e214dbdc68e4ce533edb4dfa5430b51
Parents: 49bb972 adcb713
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Feb 5 18:42:00 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Feb 5 18:42:00 2014 +0100
--
CHANGES.txt | 5 +
.../org/apache/cassandra/db/AtomicSortedColumns.java | 7 ++-
src/java/org/apache/cassandra/db/DeletionInfo.java| 14 ++
src/java/org/apache/cassandra/db/Memtable.java| 10 ++
4 files changed, 31 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/58e94818/CHANGES.txt
--
diff --cc CHANGES.txt
index 9599e56,cfdd148..bba5f20
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,29 -1,21 +1,34 @@@
-1.2.16
++2.0.6
++Merged from 1.2:
+ * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
+
-1.2.15
- * Move handling of migration event source to solve bootstrap race
(CASSANDRA-6648)
- * Make sure compaction throughput value doesn't overflow with int math
(CASSANDRA-6647)
-
+
-1.2.14
- * Reverted code to limit CQL prepared statement cache by size
(CASSANDRA-6592)
- * add cassandra.default_messaging_version property to allow easier
- upgrading from 1.1 (CASSANDRA-6619)
- * Allow executing CREATE statements multiple times (CASSANDRA-6471)
- * Don't send confusing info with timeouts (CASSANDRA-6491)
- * Don't resubmit counter mutation runnables internally (CASSANDRA-6427)
- * Don't drop local mutations without a hint (CASSANDRA-6510)
- * Don't allow null max_hint_window_in_ms (CASSANDRA-6419)
- * Validate SliceRange start and finish lengths (CASSANDRA-6521)
+2.0.5
+ * Reduce garbage generated by bloom filter lookups (CASSANDRA-6609)
+ * Add ks.cf names to tombstone logging (CASSANDRA-6597)
+ * Use LOCAL_QUORUM for LWT operations at LOCAL_SERIAL (CASSANDRA-6495)
+ * Wait for gossip to settle before accepting client connections
(CASSANDRA-4288)
+ * Delete unfinished compaction incrementally (CASSANDRA-6086)
+ * Allow specifying custom secondary index options in CQL3 (CASSANDRA-6480)
+ * Improve replica pinning for cache efficiency in DES (CASSANDRA-6485)
+ * Fix LOCAL_SERIAL from thrift (CASSANDRA-6584)
+ * Don't special case received counts in CAS timeout exceptions
(CASSANDRA-6595)
+ * Add support for 2.1 global counter shards (CASSANDRA-6505)
+ * Fix NPE when streaming connection is not yet established (CASSANDRA-6210)
+ * Avoid rare duplicate read repair triggering (CASSANDRA-6606)
+ * Fix paging discardFirst (CASSANDRA-6555)
+ * Fix ArrayIndexOutOfBoundsException in 2ndary index query (CASSANDRA-6470)
+ * Release sstables upon rebuilding 2i (CASSANDRA-6635)
+ * Add AbstractCompactionStrategy.startup() method (CASSANDRA-6637)
+ * SSTableScanner may skip rows during cleanup (CASSANDRA-6638)
+ * sstables from stalled repair sessions can resurrect deleted data
(CASSANDRA-6503)
+ * Switch stress to use ITransportFactory (CASSANDRA-6641)
+ * Fix IllegalArgumentException during prepare (CASSANDRA-6592)
+ * Fix possible loss of 2ndary index entries during compaction
(CASSANDRA-6517)
+ * Fix direct Memory on architectures that do not support unaligned long
access
+ (CASSANDRA-6628)
+ * Let scrub optionally skip broken counter partitions (CASSANDRA-5930)
+Merged from 1.2:
* fsync compression metadata (CASSANDRA-6531)
* Validate CF existence on execution for prepared statement (CASSANDRA-6535)
* Add ability to throttle batchlog replay (CASSANDRA-6550)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/58e94818/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
--
diff --cc src/java/org/apache/cassandra/db/AtomicSortedColumns.java
index 1c0bf1b,d6c861b..d3a979c
--- a/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
@@@ -178,19 -194,15 +178,24 @@@ public class AtomicSortedColumns extend
{
sizeDelta = 0;
current = ref.get();
- DeletionInfo newDelInfo =
current.deletionInfo.copy().add(cm.deletionInfo());
+ DeletionInfo newDelInfo = current.deletionInfo;
-
[jira] [Updated] (CASSANDRA-6658) Nodes flap once at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-6658: Since Version: 2.0.3 (was: 2.0.4) Nodes flap once at startup -- Key: CASSANDRA-6658 URL: https://issues.apache.org/jira/browse/CASSANDRA-6658 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Priority: Minor Upon initially seeing each other, a node will mark another UP, then DOWN, then UP again. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6658) Nodes flap once at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-6658: Attachment: 6658.txt At least one problem is that in CASSANDRA-6385 we never converted the initial value to nanos, so when we divide by the mean when the only sample is the initial value and thus equivalent to it, the resulting phi is inflated. Nodes flap once at startup -- Key: CASSANDRA-6658 URL: https://issues.apache.org/jira/browse/CASSANDRA-6658 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Priority: Minor Attachments: 6658.txt Upon initially seeing each other, a node will mark another UP, then DOWN, then UP again. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6528) TombstoneOverwhelmingException is thrown while populating data in recently truncated CF
[
https://issues.apache.org/jira/browse/CASSANDRA-6528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892376#comment-13892376
]
Machiel Groeneveld commented on CASSANDRA-6528:
---
create table IF NOT EXISTS visits.visits(
id text,
cookie_uuid text, cookie_uuid text, external_click_id text, session_id text,
visitor_ip text, user_agent text, uuid_hash text,
shop_product_id int, channel_id int, shop_id int, shop_category_id int,
type int, medium_id int, campaign_id int, channel_affiliate_id int,
default_cpc float,
created_at timestamp, updated_at timestamp, time_id int,
disabled int, has_referer boolean, known_visitor boolean, marketing boolean,
primary key(time_id, id));
TombstoneOverwhelmingException is thrown while populating data in recently
truncated CF
---
Key: CASSANDRA-6528
URL: https://issues.apache.org/jira/browse/CASSANDRA-6528
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassadra 2.0.3, Linux, 6 nodes
Reporter: Nikolai Grigoriev
Priority: Minor
I am running some performance tests and recently I had to flush the data from
one of the tables and repopulate it. I have about 30M rows with a few columns
in each, about 5kb per row in in total. In order to repopulate the data I do
truncate table from CQLSH and then relaunch the test. The test simply
inserts the data in the table, does not read anything. Shortly after
restarting the data generator I see this on one of the nodes:
{code}
INFO [HintedHandoff:655] 2013-12-26 16:45:42,185 HintedHandOffManager.java
(line 323) Started hinted handoff f
or host: 985c8a08-3d92-4fad-a1d1-7135b2b9774a with IP: /10.5.45.158
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 SliceQueryFilter.java (line
200) Scanned ove
r 10 tombstones; query aborted (see tombstone_fail_threshold)
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 CassandraDaemon.java (line
187) Exception in thread Thread[HintedHandoff:655,1,main]
org.apache.cassandra.db.filter.TombstoneOverwhelmingException
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:201)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:56)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:351)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:309)
at
org.apache.cassandra.db.HintedHandOffManager.access$4(HintedHandOffManager.java:281)
at
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:530)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
INFO [OptionalTasks:1] 2013-12-26 16:45:53,946 MeteredFlusher.java (line 63)
flushing high-traffic column family CFS(Keyspace='test_jmeter',
ColumnFamily='test_profiles') (estimated 192717267 bytes)
{code}
I am inserting the data with CL=1.
It seems to be happening every time I do it. But I do not see any errors on
the client side and the node seems to continue operating, this is why I think
it is not a major issue. Maybe not an issue at all, but the message is logged
as ERROR.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Comment Edited] (CASSANDRA-6528) TombstoneOverwhelmingException is thrown while populating data in recently truncated CF
[
https://issues.apache.org/jira/browse/CASSANDRA-6528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892376#comment-13892376
]
Machiel Groeneveld edited comment on CASSANDRA-6528 at 2/5/14 6:07 PM:
---
create table IF NOT EXISTS visits.visits(
id text,
cookie_uuid text, cookie_uuid text, external_click_id text, session_id text,
visitor_ip text, user_agent text, uuid_hash text,
shop_product_id int, channel_id int, shop_id int, shop_category_id int,
type int, medium_id int, campaign_id int, channel_affiliate_id int,
default_cpc float,
created_at timestamp, updated_at timestamp, time_id int,
disabled int, has_referer boolean, known_visitor boolean, marketing boolean,
primary key(time_id, id));
SELECT * FROM VISITS
was (Author: machielg):
create table IF NOT EXISTS visits.visits(
id text,
cookie_uuid text, cookie_uuid text, external_click_id text, session_id text,
visitor_ip text, user_agent text, uuid_hash text,
shop_product_id int, channel_id int, shop_id int, shop_category_id int,
type int, medium_id int, campaign_id int, channel_affiliate_id int,
default_cpc float,
created_at timestamp, updated_at timestamp, time_id int,
disabled int, has_referer boolean, known_visitor boolean, marketing boolean,
primary key(time_id, id));
TombstoneOverwhelmingException is thrown while populating data in recently
truncated CF
---
Key: CASSANDRA-6528
URL: https://issues.apache.org/jira/browse/CASSANDRA-6528
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassadra 2.0.3, Linux, 6 nodes
Reporter: Nikolai Grigoriev
Priority: Minor
I am running some performance tests and recently I had to flush the data from
one of the tables and repopulate it. I have about 30M rows with a few columns
in each, about 5kb per row in in total. In order to repopulate the data I do
truncate table from CQLSH and then relaunch the test. The test simply
inserts the data in the table, does not read anything. Shortly after
restarting the data generator I see this on one of the nodes:
{code}
INFO [HintedHandoff:655] 2013-12-26 16:45:42,185 HintedHandOffManager.java
(line 323) Started hinted handoff f
or host: 985c8a08-3d92-4fad-a1d1-7135b2b9774a with IP: /10.5.45.158
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 SliceQueryFilter.java (line
200) Scanned ove
r 10 tombstones; query aborted (see tombstone_fail_threshold)
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 CassandraDaemon.java (line
187) Exception in thread Thread[HintedHandoff:655,1,main]
org.apache.cassandra.db.filter.TombstoneOverwhelmingException
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:201)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:56)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:351)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:309)
at
org.apache.cassandra.db.HintedHandOffManager.access$4(HintedHandOffManager.java:281)
at
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:530)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
INFO [OptionalTasks:1] 2013-12-26 16:45:53,946 MeteredFlusher.java (line 63)
flushing high-traffic column family CFS(Keyspace='test_jmeter',
ColumnFamily='test_profiles') (estimated 192717267 bytes)
{code}
I am inserting the data with CL=1.
It seems to be happening every time I do it. But I do not see any errors on
the client side and the node seems to continue operating, this is why I think
it is not a major issue. Maybe not an issue at all, but the message is logged
as ERROR.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6658) Nodes flap once at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-6658: Attachment: (was: 6658.txt) Nodes flap once at startup -- Key: CASSANDRA-6658 URL: https://issues.apache.org/jira/browse/CASSANDRA-6658 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Priority: Minor Attachments: 6658.txt Upon initially seeing each other, a node will mark another UP, then DOWN, then UP again. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6528) TombstoneOverwhelmingException is thrown while populating data in recently truncated CF
[
https://issues.apache.org/jira/browse/CASSANDRA-6528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892380#comment-13892380
]
Aleksey Yeschenko commented on CASSANDRA-6528:
--
Example insert query. Do you ever insert, or update set NULL?
TombstoneOverwhelmingException is thrown while populating data in recently
truncated CF
---
Key: CASSANDRA-6528
URL: https://issues.apache.org/jira/browse/CASSANDRA-6528
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassadra 2.0.3, Linux, 6 nodes
Reporter: Nikolai Grigoriev
Priority: Minor
I am running some performance tests and recently I had to flush the data from
one of the tables and repopulate it. I have about 30M rows with a few columns
in each, about 5kb per row in in total. In order to repopulate the data I do
truncate table from CQLSH and then relaunch the test. The test simply
inserts the data in the table, does not read anything. Shortly after
restarting the data generator I see this on one of the nodes:
{code}
INFO [HintedHandoff:655] 2013-12-26 16:45:42,185 HintedHandOffManager.java
(line 323) Started hinted handoff f
or host: 985c8a08-3d92-4fad-a1d1-7135b2b9774a with IP: /10.5.45.158
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 SliceQueryFilter.java (line
200) Scanned ove
r 10 tombstones; query aborted (see tombstone_fail_threshold)
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 CassandraDaemon.java (line
187) Exception in thread Thread[HintedHandoff:655,1,main]
org.apache.cassandra.db.filter.TombstoneOverwhelmingException
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:201)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:56)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:351)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:309)
at
org.apache.cassandra.db.HintedHandOffManager.access$4(HintedHandOffManager.java:281)
at
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:530)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
INFO [OptionalTasks:1] 2013-12-26 16:45:53,946 MeteredFlusher.java (line 63)
flushing high-traffic column family CFS(Keyspace='test_jmeter',
ColumnFamily='test_profiles') (estimated 192717267 bytes)
{code}
I am inserting the data with CL=1.
It seems to be happening every time I do it. But I do not see any errors on
the client side and the node seems to continue operating, this is why I think
it is not a major issue. Maybe not an issue at all, but the message is logged
as ERROR.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6658) Nodes flap once at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-6658: Attachment: 6658.txt Nodes flap once at startup -- Key: CASSANDRA-6658 URL: https://issues.apache.org/jira/browse/CASSANDRA-6658 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Priority: Minor Attachments: 6658.txt Upon initially seeing each other, a node will mark another UP, then DOWN, then UP again. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Comment Edited] (CASSANDRA-6528) TombstoneOverwhelmingException is thrown while populating data in recently truncated CF
[
https://issues.apache.org/jira/browse/CASSANDRA-6528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892315#comment-13892315
]
Machiel Groeneveld edited comment on CASSANDRA-6528 at 2/5/14 6:10 PM:
---
I have the same issue, after inserting 216258 records (sharing the same
partition key) in a new database (I reinstalled Cassandra) I couldn't run a
select query (something like 'select * from partition_key = x'). Also a
count(*) on the table gives me tombstone warnings. I'm not expecting any
tombstones as they are all inserts (not 100% sure about possible overwriting
though)
In the log I get org.apache.cassandra.db.filter.TombstoneOverwhelmingException
was (Author: machielg):
I have the same issue, after inserting 216258 records (in one row) in a new
database (I removed all files in the data directory files before starting) I
couldn't run a select query (something like 'select * from partition_key = x')
In the log I get org.apache.cassandra.db.filter.TombstoneOverwhelmingException
TombstoneOverwhelmingException is thrown while populating data in recently
truncated CF
---
Key: CASSANDRA-6528
URL: https://issues.apache.org/jira/browse/CASSANDRA-6528
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassadra 2.0.3, Linux, 6 nodes
Reporter: Nikolai Grigoriev
Priority: Minor
I am running some performance tests and recently I had to flush the data from
one of the tables and repopulate it. I have about 30M rows with a few columns
in each, about 5kb per row in in total. In order to repopulate the data I do
truncate table from CQLSH and then relaunch the test. The test simply
inserts the data in the table, does not read anything. Shortly after
restarting the data generator I see this on one of the nodes:
{code}
INFO [HintedHandoff:655] 2013-12-26 16:45:42,185 HintedHandOffManager.java
(line 323) Started hinted handoff f
or host: 985c8a08-3d92-4fad-a1d1-7135b2b9774a with IP: /10.5.45.158
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 SliceQueryFilter.java (line
200) Scanned ove
r 10 tombstones; query aborted (see tombstone_fail_threshold)
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 CassandraDaemon.java (line
187) Exception in thread Thread[HintedHandoff:655,1,main]
org.apache.cassandra.db.filter.TombstoneOverwhelmingException
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:201)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:56)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:351)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:309)
at
org.apache.cassandra.db.HintedHandOffManager.access$4(HintedHandOffManager.java:281)
at
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:530)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
INFO [OptionalTasks:1] 2013-12-26 16:45:53,946 MeteredFlusher.java (line 63)
flushing high-traffic column family CFS(Keyspace='test_jmeter',
ColumnFamily='test_profiles') (estimated 192717267 bytes)
{code}
I am inserting the data with CL=1.
It seems to be happening every time I do it. But I do not see any errors on
the client side and the node seems to continue operating, this is why I think
it is not a major issue. Maybe not an issue at all, but the message is logged
as ERROR.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[
https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-4757:
---
Attachment: 4757-2.0.patch
4757-2.0.patch (and
[branch|https://github.com/thobbs/cassandra/tree/CASSANDRA-4757]) makes the
following changes:
* Add a bulkLoadAsync method to StorageServiceMBean which returns the string
version of the planID
* Print the planID before loading starts in sstableloader
* sstableloader already had a {{--no-progress}} option, but it was being
ignored, so I fixed that as well
Expose bulk loading progress/status over jmx
Key: CASSANDRA-4757
URL: https://issues.apache.org/jira/browse/CASSANDRA-4757
Project: Cassandra
Issue Type: Improvement
Reporter: Nick Bailey
Assignee: Tyler Hobbs
Priority: Minor
Labels: lhf
Fix For: 2.0.6
Attachments: 4757-2.0.patch, CASSANDRA-4757.txt, CASSANDRA-4757v2.txt
The bulk loading interface should be exposing some progress or status
information over jmx. This shouldn't be too difficult and should be exposed
in a way that the information is available whether you are using the separate
sstableloader utility or calling the bulkload jmx call.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-4757: --- Reviewer: Nick Bailey (was: Yuki Morishita) Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Tyler Hobbs Priority: Minor Labels: lhf Fix For: 2.0.6 Attachments: 4757-2.0.patch, CASSANDRA-4757.txt, CASSANDRA-4757v2.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6360) Make nodetool cfhistograms output easily understandable
[ https://issues.apache.org/jira/browse/CASSANDRA-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892394#comment-13892394 ] Tyler Hobbs commented on CASSANDRA-6360: bq. Might be nice to print them side by side? If you don't feel strongly about it, I'd prefer to not do this just to save time, although I agree it might look better. Make nodetool cfhistograms output easily understandable --- Key: CASSANDRA-6360 URL: https://issues.apache.org/jira/browse/CASSANDRA-6360 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Assignee: Tyler Hobbs Priority: Trivial Almost nobody understands the cfhistograms output without googling it. By default, we shouldn't share an axis across all metrics. We can still provide the current format with a --compact option. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6360) Make nodetool cfhistograms output easily understandable
[ https://issues.apache.org/jira/browse/CASSANDRA-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892403#comment-13892403 ] Benedict commented on CASSANDRA-6360: - It involves a lot of scrolling, which isn't ideal for terminal use as things stand. I think it would be nice to either print them side-by-side, or to coalesce adjacent buckets that contain little data. But I don't feel strongly about it, no. Make nodetool cfhistograms output easily understandable --- Key: CASSANDRA-6360 URL: https://issues.apache.org/jira/browse/CASSANDRA-6360 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Assignee: Tyler Hobbs Priority: Trivial Almost nobody understands the cfhistograms output without googling it. By default, we shouldn't share an axis across all metrics. We can still provide the current format with a --compact option. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6360) Make nodetool cfhistograms output easily understandable
[ https://issues.apache.org/jira/browse/CASSANDRA-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892405#comment-13892405 ] Brandon Williams commented on CASSANDRA-6360: - The problem is you'd have to detect how wide the terminal is to know if you've gone too far, otherwise the output is going to be unreadable when it wraps. I'm fine with it the way it is. Make nodetool cfhistograms output easily understandable --- Key: CASSANDRA-6360 URL: https://issues.apache.org/jira/browse/CASSANDRA-6360 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Assignee: Tyler Hobbs Priority: Trivial Almost nobody understands the cfhistograms output without googling it. By default, we shouldn't share an axis across all metrics. We can still provide the current format with a --compact option. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6360) Make nodetool cfhistograms output easily understandable
[ https://issues.apache.org/jira/browse/CASSANDRA-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892408#comment-13892408 ] Benedict commented on CASSANDRA-6360: - bq. The problem is you'd have to detect how wide the terminal is to know if you've gone too far ?? Just printing the two columns we have should be fine for basically any terminal. Anyone with a terminal with ~30 characters can deal with it. Like they would for the compact format (where we assume ~100 chars) Make nodetool cfhistograms output easily understandable --- Key: CASSANDRA-6360 URL: https://issues.apache.org/jira/browse/CASSANDRA-6360 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Assignee: Tyler Hobbs Priority: Trivial Almost nobody understands the cfhistograms output without googling it. By default, we shouldn't share an axis across all metrics. We can still provide the current format with a --compact option. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6360) Make nodetool cfhistograms output easily understandable
[ https://issues.apache.org/jira/browse/CASSANDRA-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892413#comment-13892413 ] Brandon Williams commented on CASSANDRA-6360: - Oh, I thought you meant print as many columns as possible. We have a lot more than two columns, they're just not shown in Tyler's example, and there isn't always a logical pairing between them. Make nodetool cfhistograms output easily understandable --- Key: CASSANDRA-6360 URL: https://issues.apache.org/jira/browse/CASSANDRA-6360 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Assignee: Tyler Hobbs Priority: Trivial Attachments: 6360-2.0.patch Almost nobody understands the cfhistograms output without googling it. By default, we shouldn't share an axis across all metrics. We can still provide the current format with a --compact option. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6360) Make nodetool cfhistograms output easily understandable
[ https://issues.apache.org/jira/browse/CASSANDRA-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892421#comment-13892421 ] Benedict commented on CASSANDRA-6360: - bq. since things are improved in 2.1, it's not terribly important to me. +1 Make nodetool cfhistograms output easily understandable --- Key: CASSANDRA-6360 URL: https://issues.apache.org/jira/browse/CASSANDRA-6360 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Assignee: Tyler Hobbs Priority: Trivial Attachments: 6360-2.0.patch Almost nobody understands the cfhistograms output without googling it. By default, we shouldn't share an axis across all metrics. We can still provide the current format with a --compact option. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6360) Make nodetool cfhistograms output easily understandable
[ https://issues.apache.org/jira/browse/CASSANDRA-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-6360: --- Attachment: 6360-2.0.patch Actually, since the 2.1 output uses percentiles instead of an offsets column, I think it's fine as is. I'm attaching 6360-2.0.patch in case we want to improve this just for the remainder of 2.0, but since things are improved in 2.1, it's not terribly important to me. Make nodetool cfhistograms output easily understandable --- Key: CASSANDRA-6360 URL: https://issues.apache.org/jira/browse/CASSANDRA-6360 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Assignee: Tyler Hobbs Priority: Trivial Attachments: 6360-2.0.patch Almost nobody understands the cfhistograms output without googling it. By default, we shouldn't share an axis across all metrics. We can still provide the current format with a --compact option. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (CASSANDRA-6659) Allow intercepting query by user provided custom classes
Sylvain Lebresne created CASSANDRA-6659:
---
Summary: Allow intercepting query by user provided custom classes
Key: CASSANDRA-6659
URL: https://issues.apache.org/jira/browse/CASSANDRA-6659
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
The idea for this ticket is to abstract the main execution methods of
QueryProcessor into an interface, something like:
{noformat}
public interface QueryHandler
{
public ResultSet process(String query, QueryState state, QueryOptions
options);
public ResultMessage.Prepared prepare(String query, QueryState state);
public ResultSet processPrepared(CQLStatement statement, QueryState state,
QueryOptions options);
public ResultSet processBatch(BatchStatement statement, QueryState state,
BatchQueryOptions options);
}
{noformat}
and to allow users to provide a specific class of their own (implementing said
interface) to which the native protocol would handoff queries to (so by default
queries would go to QueryProcessor, but you would have a way to use a custom
class instead).
A typical use case for that could be to allow some form of custom logging of
incoming queries and/or of their results. But this could probably also have
some application for testing as one could have a handler that completely bypass
QueryProcessor if you want, say, do perf regression tests for a given driver
(and don't want to actually execute the query as you're perf testing the
driver, not C*) without needing to patch the sources. Those being just
examples, the mechanism is generic enough to allow for other ideas.
Most importantly, it requires very little code in C*. As for how users would
register their handler, it can be as simple as a startup flag indicating the
class to use, or a yaml setting, or both.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6659) Allow intercepting query by user provided custom classes
[
https://issues.apache.org/jira/browse/CASSANDRA-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-6659:
Attachment: 6659.txt
Attaching fairly trivial patch to implement this (the patch is against 2.0
because that has virtually no chance to break anything existing so why not).
Note that the patch remove the pre and post execution hooks from QueryProcessor
as those were only here for external tool and, unless I'm missing something
obvious, the mechanism here provides a stricly more general mechanism.
Allow intercepting query by user provided custom classes
--
Key: CASSANDRA-6659
URL: https://issues.apache.org/jira/browse/CASSANDRA-6659
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
Attachments: 6659.txt
The idea for this ticket is to abstract the main execution methods of
QueryProcessor into an interface, something like:
{noformat}
public interface QueryHandler
{
public ResultSet process(String query, QueryState state, QueryOptions
options);
public ResultMessage.Prepared prepare(String query, QueryState state);
public ResultSet processPrepared(CQLStatement statement, QueryState
state, QueryOptions options);
public ResultSet processBatch(BatchStatement statement, QueryState state,
BatchQueryOptions options);
}
{noformat}
and to allow users to provide a specific class of their own (implementing
said interface) to which the native protocol would handoff queries to (so by
default queries would go to QueryProcessor, but you would have a way to use a
custom class instead).
A typical use case for that could be to allow some form of custom logging of
incoming queries and/or of their results. But this could probably also have
some application for testing as one could have a handler that completely
bypass QueryProcessor if you want, say, do perf regression tests for a given
driver (and don't want to actually execute the query as you're perf testing
the driver, not C*) without needing to patch the sources. Those being just
examples, the mechanism is generic enough to allow for other ideas.
Most importantly, it requires very little code in C*. As for how users would
register their handler, it can be as simple as a startup flag indicating
the class to use, or a yaml setting, or both.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6360) Make nodetool cfhistograms output easily understandable
[ https://issues.apache.org/jira/browse/CASSANDRA-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-6360: --- Attachment: 6360-2.0-v2.patch v2 patch (and [branch|https://github.com/thobbs/cassandra/tree/CASSANDRA-6360-rebase]) is rebased for the latest cassandra-2.0. Make nodetool cfhistograms output easily understandable --- Key: CASSANDRA-6360 URL: https://issues.apache.org/jira/browse/CASSANDRA-6360 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Assignee: Tyler Hobbs Priority: Trivial Attachments: 6360-2.0-v2.patch, 6360-2.0.patch Almost nobody understands the cfhistograms output without googling it. By default, we shouldn't share an axis across all metrics. We can still provide the current format with a --compact option. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Assigned] (CASSANDRA-6657) Log the newsize value alongside the heap size at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-6657: - Assignee: Lyuben Todorov Log the newsize value alongside the heap size at startup Key: CASSANDRA-6657 URL: https://issues.apache.org/jira/browse/CASSANDRA-6657 Project: Cassandra Issue Type: Wish Components: Core Reporter: Jeremy Hanna Assignee: Lyuben Todorov Priority: Trivial It would be nice to have the newsize value logged alongside the heap size at startup to more easily track down problems. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6657) Log the newsize value alongside the heap size at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892441#comment-13892441 ] Jonathan Ellis commented on CASSANDRA-6657: --- Suspect you will need to find the MemoryPoolMBean corresponding to young gen. Log the newsize value alongside the heap size at startup Key: CASSANDRA-6657 URL: https://issues.apache.org/jira/browse/CASSANDRA-6657 Project: Cassandra Issue Type: Wish Components: Core Reporter: Jeremy Hanna Assignee: Lyuben Todorov Priority: Trivial It would be nice to have the newsize value logged alongside the heap size at startup to more easily track down problems. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6656) Exception logging
[
https://issues.apache.org/jira/browse/CASSANDRA-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-6656:
--
Reviewer: Mikhail Stepura
Priority: Trivial (was: Major)
Fix Version/s: 2.1
Assignee: Ding Yuan
Can you review, [~mishail]?
Exception logging
-
Key: CASSANDRA-6656
URL: https://issues.apache.org/jira/browse/CASSANDRA-6656
Project: Cassandra
Issue Type: Improvement
Components: Core, Tools
Reporter: Ding Yuan
Assignee: Ding Yuan
Priority: Trivial
Fix For: 2.1
Attachments: trunk-6656.txt
Reporting a few cases where informative exceptions can be silently swallowed.
Attaching a proposed patch.
=
Case 1
Line: 95, File: org/apache/cassandra/utils/Hex.java
An actual failure in the underlying constructor will be lost.
Propose to log it.
{noformat}
try
{
s = stringConstructor.newInstance(0, c.length, c);
+ }
+ catch (InvocationTargetException ite) {
+ // The underlying constructor failed. Unwrapping the
exception.
+ logger.info(Underlying constructor throws exception: ,
ite.getCause());
}
catch (Exception e)
{
// Swallowing as we'll just use a copying constructor
}
return s == null ? new String(c) : s;
{noformat}
==
=
Case 2
Line: 192, File: org/apache/cassandra/db/marshal/DynamicCompositeType.java
The actual cause of comparator error can be lost as it can fail in multiple
locations.
{noformat}
AbstractType? comparator = null;
int header = getShortLength(bb);
if ((header 0x8000) == 0)
{
ByteBuffer value = getBytes(bb, header);
try
{
comparator = TypeParser.parse(ByteBufferUtil.string(value));
}
catch (Exception e)
{
--- can fail here
// we'll deal with this below since comparator == null
}
}
else
{
comparator = aliases.get((byte)(header 0xFF));
--- can fail here
}
if (comparator == null)
throw new MarshalException(Cannot find comparator for component
+ i);
{noformat}
Propose to log the exception.
==
=
Case 3
Line: 239, File: org/apache/cassandra/tools/NodeProbe.java
Exception ignored in finally. Propose log them with debug or trace.
{noformat}
232: finally
233: {
234: try
235: {
236: ssProxy.removeNotificationListener(runner);
236: ssProxy.removeNotificationListener(runner);
237: jmxc.removeConnectionNotificationListener(runner);
238: }
239: catch (Throwable ignored) {}
240: }
{noformat}
Similar case is at line 264 in the same file.
==
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6651) Repair hanging
[ https://issues.apache.org/jira/browse/CASSANDRA-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6651: -- Reproduced In: 2.0.3 Fix Version/s: (was: 2.0.3) Assignee: Yuki Morishita Repair hanging -- Key: CASSANDRA-6651 URL: https://issues.apache.org/jira/browse/CASSANDRA-6651 Project: Cassandra Issue Type: Bug Components: Core Reporter: Eitan Eibschutz Assignee: Yuki Morishita Hi, We have a 12 node cluster in PROD environment and we've noticed that repairs are never finishing. The behavior that we've observed is that a repair process will run until at some point it hangs and no other processing is happening. For example, at the moment, I have a repair process that has been running for two days and not finishing: nodetool tpstats is showing 2 active and 2 pending AntiEntropySessions nodetool compactionstats is showing: pending tasks: 0 Active compaction remaining time :n/a nodetools netstats is showing: Mode: NORMAL Not sending any streams. Read Repair Statistics: Attempted: 0 Mismatch (Blocking): 142110 Mismatch (Background): 0 Pool NameActive Pending Completed Commandsn/a 0 107589657 Responses n/a 0 116430785 The last entry that I see in the log is: INFO [AntiEntropySessions:18] 2014-02-03 04:01:39,145 RepairJob.java (line 116) [repair #ae78c6c0-8c2b-11e3-b950-c3b81a36bc9b] requesting merkle trees for MyCF (to [/x.x.x.x, /y.y.y.y, /z.z.z.z]) The repair started at 4am so it stopped after 1:40 minute. On node y.y.y.y I can see this in the log: INFO [MiscStage:1] 2014-02-03 04:01:38,985 ColumnFamilyStore.java (line 740) Enqueuing flush of Memtable-MyCF@1290890489(2176/5931 serialized/live bytes, 32 ops) INFO [FlushWriter:411] 2014-02-03 04:01:38,986 Memtable.java (line 333) Writing Memtable-MyCF@1290890489(2176/5931 serialized/live bytes, 32 ops) INFO [FlushWriter:411] 2014-02-03 04:01:39,048 Memtable.java (line 373) Completed flushing /var/lib/cassandra/main-db/data/MyKS/MyCF/MyKS-MyCF-jb-518-Data.db (1789 bytes) for commitlog position ReplayPosition(segmentId=1390437013339, position=21868792) INFO [ScheduledTasks:1] 2014-02-03 05:00:04,794 ColumnFamilyStore.java (line 740) Enqueuing flush of Memtable-compaction_history@1649414699(1635/17360 serialized/live bytes, 42 ops) So for some reason the merkle tree for this CF is never sent back to the node being repaired and it's hanging. I've also noticed that sometimes, restarting node y.y.y.y will cause the repair to resume. Another observation is that sometimes when restarting y.y.y.y it will not start with these errors: ERROR 16:34:18,485 Exception encountered during startup java.lang.IllegalStateException: Unfinished compactions reference missing sstables. This should never happen since compactions are marked finished before we start removing the old sstables. at org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:495) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:264) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:461) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:504) java.lang.IllegalStateException: Unfinished compactions reference missing sstables. This should never happen since compactions are marked finished before we start removing the old sstables. at org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:495) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:264) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:461) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:504) Exception encountered during startup: Unfinished compactions reference missing sstables. This should never happen since compactions are marked finished before we start removing the old sstables. And it will only restart after manually cleaning the compactions_in-progress folder. I'm not sure if these two issues are related but we've seen both on all the nodes in our cluster. I'll be happy to provide more info if needed as we are not sure what could cause this behavior. Another thing in our environment is that some of the Cassandra nodes have more than one network interface and RPC is listening on 0.0.0.0, not sure if it has anything to do with this. Thanks, Eitan -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6658) Nodes flap once at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892450#comment-13892450 ] Jonathan Ellis commented on CASSANDRA-6658: --- Nit: shouldn't we use TimeUnit to convert instead of hardcoding extra constants? Nodes flap once at startup -- Key: CASSANDRA-6658 URL: https://issues.apache.org/jira/browse/CASSANDRA-6658 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Priority: Minor Attachments: 6658.txt Upon initially seeing each other, a node will mark another UP, then DOWN, then UP again. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6659) Allow intercepting query by user provided custom classes
[
https://issues.apache.org/jira/browse/CASSANDRA-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-6659:
--
Reviewer: Benjamin Coverston
Tagging [~bcoverston] for review.
Allow intercepting query by user provided custom classes
--
Key: CASSANDRA-6659
URL: https://issues.apache.org/jira/browse/CASSANDRA-6659
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
Attachments: 6659.txt
The idea for this ticket is to abstract the main execution methods of
QueryProcessor into an interface, something like:
{noformat}
public interface QueryHandler
{
public ResultSet process(String query, QueryState state, QueryOptions
options);
public ResultMessage.Prepared prepare(String query, QueryState state);
public ResultSet processPrepared(CQLStatement statement, QueryState
state, QueryOptions options);
public ResultSet processBatch(BatchStatement statement, QueryState state,
BatchQueryOptions options);
}
{noformat}
and to allow users to provide a specific class of their own (implementing
said interface) to which the native protocol would handoff queries to (so by
default queries would go to QueryProcessor, but you would have a way to use a
custom class instead).
A typical use case for that could be to allow some form of custom logging of
incoming queries and/or of their results. But this could probably also have
some application for testing as one could have a handler that completely
bypass QueryProcessor if you want, say, do perf regression tests for a given
driver (and don't want to actually execute the query as you're perf testing
the driver, not C*) without needing to patch the sources. Those being just
examples, the mechanism is generic enough to allow for other ideas.
Most importantly, it requires very little code in C*. As for how users would
register their handler, it can be as simple as a startup flag indicating
the class to use, or a yaml setting, or both.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892474#comment-13892474 ] Nick Bailey commented on CASSANDRA-4757: Patch LGTM. Haven't tested it out though. Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Tyler Hobbs Priority: Minor Labels: lhf Fix For: 2.0.6 Attachments: 4757-2.0.patch, CASSANDRA-4757.txt, CASSANDRA-4757v2.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (CASSANDRA-6660) Make node tool command take a password file
Vishy Kasar created CASSANDRA-6660: -- Summary: Make node tool command take a password file Key: CASSANDRA-6660 URL: https://issues.apache.org/jira/browse/CASSANDRA-6660 Project: Cassandra Issue Type: Improvement Reporter: Vishy Kasar We are sending the jmx password in the clear to the node tool command in production. This is a security risk. Any one doing a 'ps' can see the clear password. Can we change the node tool command to also take a password file argument. This file will list the JMX user and passwords. Example below: cat /cassandra/run/10003004.jmxpasswd monitorRole abc controlRole def Based on the user name provided, node tool can pick up the right password. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6528) TombstoneOverwhelmingException is thrown while populating data in recently truncated CF
[
https://issues.apache.org/jira/browse/CASSANDRA-6528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892475#comment-13892475
]
Machiel Groeneveld commented on CASSANDRA-6528:
---
Only doing inserts (query below), no updates. Not sure about inserting null
values, will get back on that.
BEGIN BATCH
insert into visits (
id, cookie_uuid, uuid_hash,
default_cpc, cookie_uuid,
external_click_id, session_id, visitor_ip, user_agent,
shop_product_id, channel_id, shop_id, shop_category_id,
type, medium_id, campaign_id, channel_affiliate_id,
disabled, has_referer, known_visitor, marketing,
created_at, updated_at, time_id)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?) USING TTL 7776000
insert into visits_by_cookie (visit_id, time_id, cookie_uuid,
shop_id, created_at, enabled_visit)
VALUES(?, ?, ?, ?, ?, ?) USING TTL 7776000
insert into visits_by_hash (visit_id, time_id, uuid_hash, shop_id,
created_at)
VALUES(?, ?, ?, ?, ?) USING TTL 7776000
APPLY BATCH
TombstoneOverwhelmingException is thrown while populating data in recently
truncated CF
---
Key: CASSANDRA-6528
URL: https://issues.apache.org/jira/browse/CASSANDRA-6528
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassadra 2.0.3, Linux, 6 nodes
Reporter: Nikolai Grigoriev
Priority: Minor
I am running some performance tests and recently I had to flush the data from
one of the tables and repopulate it. I have about 30M rows with a few columns
in each, about 5kb per row in in total. In order to repopulate the data I do
truncate table from CQLSH and then relaunch the test. The test simply
inserts the data in the table, does not read anything. Shortly after
restarting the data generator I see this on one of the nodes:
{code}
INFO [HintedHandoff:655] 2013-12-26 16:45:42,185 HintedHandOffManager.java
(line 323) Started hinted handoff f
or host: 985c8a08-3d92-4fad-a1d1-7135b2b9774a with IP: /10.5.45.158
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 SliceQueryFilter.java (line
200) Scanned ove
r 10 tombstones; query aborted (see tombstone_fail_threshold)
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 CassandraDaemon.java (line
187) Exception in thread Thread[HintedHandoff:655,1,main]
org.apache.cassandra.db.filter.TombstoneOverwhelmingException
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:201)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:56)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:351)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:309)
at
org.apache.cassandra.db.HintedHandOffManager.access$4(HintedHandOffManager.java:281)
at
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:530)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
INFO [OptionalTasks:1] 2013-12-26 16:45:53,946 MeteredFlusher.java (line 63)
flushing high-traffic column family CFS(Keyspace='test_jmeter',
ColumnFamily='test_profiles') (estimated 192717267 bytes)
{code}
I am inserting the data with CL=1.
It seems to be happening every time I do it. But I do not see any errors on
the client side and the node seems to continue operating, this is why I think
it is not a major issue. Maybe not an issue at all, but the message is logged
as ERROR.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6528) TombstoneOverwhelmingException is thrown while populating data in recently truncated CF
[
https://issues.apache.org/jira/browse/CASSANDRA-6528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892488#comment-13892488
]
Anne Sullivan commented on CASSANDRA-6528:
--
We can reproduce the tombstone_fail_threshold exception on HintedHandoff replay
consistently. We're using C* 2.0.4. NodeB goes down, and NodeA starts storing
hints for it. Around 400K hints are written, and then NodeB comes back online.
NodeA starts replaying hints for NodeB. It replays 100K hints, and then
NodeB goes down again. When NodeB comes back online, the next HH replay fails
with the tombstone_fail_threshold error.
TombstoneOverwhelmingException is thrown while populating data in recently
truncated CF
---
Key: CASSANDRA-6528
URL: https://issues.apache.org/jira/browse/CASSANDRA-6528
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassadra 2.0.3, Linux, 6 nodes
Reporter: Nikolai Grigoriev
Priority: Minor
I am running some performance tests and recently I had to flush the data from
one of the tables and repopulate it. I have about 30M rows with a few columns
in each, about 5kb per row in in total. In order to repopulate the data I do
truncate table from CQLSH and then relaunch the test. The test simply
inserts the data in the table, does not read anything. Shortly after
restarting the data generator I see this on one of the nodes:
{code}
INFO [HintedHandoff:655] 2013-12-26 16:45:42,185 HintedHandOffManager.java
(line 323) Started hinted handoff f
or host: 985c8a08-3d92-4fad-a1d1-7135b2b9774a with IP: /10.5.45.158
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 SliceQueryFilter.java (line
200) Scanned ove
r 10 tombstones; query aborted (see tombstone_fail_threshold)
ERROR [HintedHandoff:655] 2013-12-26 16:45:42,680 CassandraDaemon.java (line
187) Exception in thread Thread[HintedHandoff:655,1,main]
org.apache.cassandra.db.filter.TombstoneOverwhelmingException
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:201)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:56)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:351)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:309)
at
org.apache.cassandra.db.HintedHandOffManager.access$4(HintedHandOffManager.java:281)
at
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:530)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
INFO [OptionalTasks:1] 2013-12-26 16:45:53,946 MeteredFlusher.java (line 63)
flushing high-traffic column family CFS(Keyspace='test_jmeter',
ColumnFamily='test_profiles') (estimated 192717267 bytes)
{code}
I am inserting the data with CL=1.
It seems to be happening every time I do it. But I do not see any errors on
the client side and the node seems to continue operating, this is why I think
it is not a major issue. Maybe not an issue at all, but the message is logged
as ERROR.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6622) Streaming session failures during node replace using replace_address
[
https://issues.apache.org/jira/browse/CASSANDRA-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892506#comment-13892506
]
Ravi Prasad commented on CASSANDRA-6622:
I'm seeing FailureDetector notifying listeners every second invoked through
GossiperTask's doStatusCheck(). Tested sleeping for RING_DELAY (instead of
BROADCAST_INTERVAL) before bootstrap, works without any stream session closure.
Streaming session failures during node replace using replace_address
Key: CASSANDRA-6622
URL: https://issues.apache.org/jira/browse/CASSANDRA-6622
Project: Cassandra
Issue Type: Bug
Environment: RHEL6, cassandra-2.0.4
Reporter: Ravi Prasad
Assignee: Brandon Williams
Attachments: 0001-don-t-signal-restart-of-dead-states.txt,
6622-2.0.txt, logs.tgz
When using replace_address, Gossiper ApplicationState is set to hibernate,
which is a down state. We are seeing that the peer nodes are seeing streaming
plan request even before the Gossiper on them marks the replacing node as
dead. As a result, streaming on peer nodes convicts the replacing node by
closing the stream handler.
I think, making the StorageService thread on the replacing node, sleep for
BROADCAST_INTERVAL before bootstrapping, would avoid this scenario.
Relevant logs from peer node (see that the Gossiper on peer node mark the
replacing node as down, 2 secs after the streaming init request):
{noformat}
INFO [STREAM-INIT-/x.x.x.x:46436] 2014-01-26 20:42:24,388
StreamResultFuture.java (line 116) [Stream
#5c6cd940-86ca-11e3-90a0-411b913c0e88] Received streaming plan for Bootstrap
INFO [GossipTasks:1] 2014-01-26 20:42:25,240 StreamResultFuture.java (line
181) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Session with /x.x.x.x is
complete
WARN [GossipTasks:1] 2014-01-26 20:42:25,240 StreamResultFuture.java (line
210) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Stream failed
INFO [GossipStage:1] 2014-01-26 20:42:25,242 Gossiper.java (line 850)
InetAddress /x.x.x.x is now DOWN
ERROR [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,766 StreamSession.java (line
410) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Streaming error occurred
java.lang.RuntimeException: Outgoing stream handler has been closed
at
org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:175)
at
org.apache.cassandra.streaming.StreamSession.prepare(StreamSession.java:436)
at
org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:358)
at
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:293)
at java.lang.Thread.run(Thread.java:722)
INFO [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,768 StreamResultFuture.java
(line 181) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Session with
/x.x.x.x is complete
WARN [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,768 StreamResultFuture.java
(line 210) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Stream failed
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6561) Static columns in CQL3
[
https://issues.apache.org/jira/browse/CASSANDRA-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892512#comment-13892512
]
Nicolas Favre-Felix commented on CASSANDRA-6561:
Thanks Sylvain. One more thing: it seems that the static suffix is not
currently added to the column definition printed by DESCRIBE TABLE foo;
Static columns in CQL3
--
Key: CASSANDRA-6561
URL: https://issues.apache.org/jira/browse/CASSANDRA-6561
Project: Cassandra
Issue Type: New Feature
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0.6
I'd like to suggest the following idea for adding static columns to CQL3.
I'll note that the basic idea has been suggested by jhalliday on irc but the
rest of the details are mine and I should be blamed for anything stupid in
what follows.
Let me start with a rational: there is 2 main family of CF that have been
historically used in Thrift: static ones and dynamic ones. CQL3 handles both
family through the presence or not of clustering columns. There is however
some cases where mixing both behavior has its use. I like to think of those
use cases as 3 broad category:
# to denormalize small amounts of not-entirely-static data in otherwise
static entities. It's say tags for a product or custom properties in a
user profile. This is why we've added CQL3 collections. Importantly, this is
the *only* use case for which collections are meant (which doesn't diminishes
their usefulness imo, and I wouldn't disagree that we've maybe not
communicated this too well).
# to optimize fetching both a static entity and related dynamic ones. Say you
have blog posts, and each post has associated comments (chronologically
ordered). *And* say that a very common query is fetch a post and its 50 last
comments. In that case, it *might* be beneficial to store a blog post
(static entity) in the same underlying CF than it's comments for performance
reason. So that fetch a post and it's 50 last comments is just one slice
internally.
# you want to CAS rows of a dynamic partition based on some partition
condition. This is the same use case than why CASSANDRA-5633 exists for.
As said above, 1) is already covered by collections, but 2) and 3) are not
(and
I strongly believe collections are not the right fit, API wise, for those).
Also, note that I don't want to underestimate the usefulness of 2). In most
cases, using a separate table for the blog posts and the comments is The
Right Solution, and trying to do 2) is premature optimisation. Yet, when used
properly, that kind of optimisation can make a difference, so I think having
a relatively native solution for it in CQL3 could make sense.
Regarding 3), though CASSANDRA-5633 would provide one solution for it, I have
the feeling that static columns actually are a more natural approach (in term
of API). That's arguably more of a personal opinion/feeling though.
So long story short, CQL3 lacks a way to mix both some static and dynamic
rows in the same partition of the same CQL3 table, and I think such a tool
could have it's use.
The proposal is thus to allow static columns. Static columns would only
make sense in table with clustering columns (the dynamic ones). A static
column value would be static to the partition (all rows of the partition
would share the value for such column). The syntax would just be:
{noformat}
CREATE TABLE t (
k text,
s text static,
i int,
v text,
PRIMARY KEY (k, i)
)
{noformat}
then you'd get:
{noformat}
INSERT INTO t(k, s, i, v) VALUES (k0, I'm shared, 0, foo);
INSERT INTO t(k, s, i, v) VALUES (k0, I'm still shared, 1, bar);
SELECT * FROM t;
k | s | i |v
k0 | I'm still shared | 0 | bar
k0 | I'm still shared | 1 | foo
{noformat}
There would be a few semantic details to decide on regarding deletions, ttl,
etc. but let's see if we agree it's a good idea first before ironing those
out.
One last point is the implementation. Though I do think this idea has merits,
it's definitively not useful enough to justify rewriting the storage engine
for it. But I think we can support this relatively easily (emphasis on
relatively :)), which is probably the main reason why I like the approach.
Namely, internally, we can store static columns as cells whose clustering
column values are empty. So in terms of cells, the partition of my example
would look like:
{noformat}
k0 : [
(:s - I'm still shared), // the static column
(0: - ) // row marker
(0:v - bar)
(1: - ) // row marker
(1:v - foo)
]
{noformat}
Of course, using empty values for the clustering columns doesn't quite work
because it could conflict with
[jira] [Updated] (CASSANDRA-6658) Nodes flap once at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-6658: Attachment: 6658-v2.txt v2 removes the constant CASSANDRA-4925 added and uses TimeUnit instead. Nodes flap once at startup -- Key: CASSANDRA-6658 URL: https://issues.apache.org/jira/browse/CASSANDRA-6658 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Priority: Minor Attachments: 6658-v2.txt, 6658.txt Upon initially seeing each other, a node will mark another UP, then DOWN, then UP again. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6660) Make node tool command take a password file
[ https://issues.apache.org/jira/browse/CASSANDRA-6660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892570#comment-13892570 ] Brandon Williams commented on CASSANDRA-6660: - Historically (CASSANDRA-5316, CASSANDRA-6279) we've felt that if you need to get this fancy, it's time to be using your own tool, but I wouldn't be against adding it if we had a patch. Probably the best way to do this is with a simple property file that can be specified on the command line. Make node tool command take a password file --- Key: CASSANDRA-6660 URL: https://issues.apache.org/jira/browse/CASSANDRA-6660 Project: Cassandra Issue Type: Improvement Reporter: Vishy Kasar We are sending the jmx password in the clear to the node tool command in production. This is a security risk. Any one doing a 'ps' can see the clear password. Can we change the node tool command to also take a password file argument. This file will list the JMX user and passwords. Example below: cat /cassandra/run/10003004.jmxpasswd monitorRole abc controlRole def Based on the user name provided, node tool can pick up the right password. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6623) Null in a cell caused by expired TTL does not work with IF clause (in CQL3)
[
https://issues.apache.org/jira/browse/CASSANDRA-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892577#comment-13892577
]
Aleksey Yeschenko commented on CASSANDRA-6623:
--
LGTM
Nits:
- I'd rather see {code}if (!(c != null c.isLive(now)
c.value().equals(e.value({code} rewritten as {code}if (c == null ||
c.isMarkedForDelete(now) || !c.value().equals(e.value())){code} in
ThriftCASConditions (as it is more or less in ColumnsConditions, apparently)
- hasLiveColumns() in ColumnsCondition is dead code
- ByteBufferUtil in ModificationStatement; ColumnNameBuilder, NamesQueryFilter,
and SliceQueryFilter in SP are now unused imports
Null in a cell caused by expired TTL does not work with IF clause (in CQL3)
---
Key: CASSANDRA-6623
URL: https://issues.apache.org/jira/browse/CASSANDRA-6623
Project: Cassandra
Issue Type: Bug
Components: Tests
Environment: One cluster with two nodes on a Linux and a Windows
system. cqlsh 4.1.0 | Cassandra 2.0.4 | CQL spec 3.1.1 | Thrift protocol
19.39.0. CQL3 Column Family
Reporter: Csaba Seres
Assignee: Sylvain Lebresne
Priority: Minor
Fix For: 2.0.6
Attachments: 6623.txt
IF onecell=null clause does not work if the onecell has got its null value
from an expired TTL. If onecell is updated with null value (UPDATE) then IF
onecell=null works fine.
This bug is not present when you create a table with COMPACT STORAGE
directive.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
