Re: Calling StorageService.loadNewSSTables function results in deadlock with compaction background task.

issue created https://issues.apache.org/jira/browse/CASSANDRA-14164

Calling StorageService.loadNewSSTables function results in deadlock with compaction background task.

Tested on version 2.2.11 (but seems like trunck 3.x is still the same for the related code path), using nodetool refresh for restoring a snapshot I guess the Cassandra committers can do something with this 😊. Calling StorageService.loadNewSSTables function results in deadlock with compaction b

deadlock cleanup request due to compaction

After adding a 5-th node I started running NodeProbe:: forceKeyspaceCleanup. That function is not returning. Below I added the program stack trace information that shows that in this case and probably others cases it is possible that there is a deadlock In my case, the compactionManager is call

RE: CASSANDRA-7649 : upgrade existing db to 2.0.10

1, 2014 at 8:47 AM, Desimpel, Ignace mailto:ignace.desim...@nuance.com>> wrote: I deploy/distribute the Cassandra database as an embedded service allowing me to create a basic cassandra.yaml file based on the global cluster of machines (seeds, non-seeds, ports, disks, etc…). That allows

CASSANDRA-7649 : upgrade existing db to 2.0.10

I deploy/distribute the Cassandra database as an embedded service allowing me to create a basic cassandra.yaml file based on the global cluster of machines (seeds, non-seeds, ports, disks, etc...). That allows me to configure and upgrade my own software and the cassandra software using the same

FW: Sporadic gossip exception on add node

version for /10.164.8.93 to 7 2014-02-26 10:30:03.131 Shadow request received, adding all states Any more information I can pass? Regards, Ignace From: Desimpel, Ignace [mailto:ignace.desim...@nuance.com] Sent: maandag 24 februari 2014 11:43 To: user@cassandra.apache.org Subject: FW: Sporadic gossip

FW: Sporadic gossip exception on add node

ublic void run() { synchronized( this ) { this.notifyAll(); } That way there is little chance the socket thread is not running yet (should be blocked in the server.accept call() ). Regards, Ignace Desimpel From: Desimpel, Ignace [mailto:ignace.desim...@nuance.com] Sent: d

RE: Thrift execute_cql3_query function seems to require that keyspace name is lowercase and is creating keyspaces with a lower case name

:08 To: user@cassandra.apache.org Subject: Re: Thrift execute_cql3_query function seems to require that keyspace name is lowercase and is creating keyspaces with a lower case name On Fri, Feb 7, 2014 at 2:48 PM, Desimpel, Ignace mailto:ignace.desim...@nuance.com>> wrote: Same for the Column

Thrift execute_cql3_query function seems to require that keyspace name is lowercase and is creating keyspaces with a lower case name

Same for the Column Family name. I thought that CQL names were case sensitive. Also make it impossible to use this interface to query for existing keyspaces that were created with thrift interface and that have capitals in the name. Did I read it wrongly? Simple example : String sCreateKs = "C

RE: exception during add node due to test beforeAppend on SSTableWriter

is issue? On Thursday, February 6, 2014 4:31 AM, "Desimpel, Ignace" mailto:ignace.desim...@nuance.com>> wrote: Also, these nodes and data are entirely created by a 2.0.4 code, so should not really be a 1.1.x related bug. Also, I restarted the whole test, thus completely new datab

FW: exception during add node due to test beforeAppend on SSTableWriter

Also, these nodes and data are entirely created by a 2.0.4 code, so should not really be a 1.1.x related bug. Also, I restarted the whole test, thus completely new database, and I get similar problems. From: Desimpel, Ignace Sent: vrijdag 31 januari 2014 18:02 To: user@cassandra.apache.org

Sporadic gossip exception on add node

Environment : linux, cassandra 2.0.4, 3 node, embedded, byte ordered, LCS When I add a node to the existing 3 node cluster I sometimes get the exception 'Unable to gossip with any seeds ' listed below. If I just restart it without any change then mostly it works. Must be some timing issue. The

Fwd: Unable to gossip with any seeds

Same problem here. But in my case the same configuration does start up when I just retry it a second time. Some more info on my specific configuration : Environment : linux, cassandra 2.0.4, 3 node, embedded, byte ordered, LCS When I add a node to the existing 3 node cluster I sometimes get the

exception during add node due to test beforeAppend on SSTableWriter

't drop in sstables, and have not yet run scrub, you should do so since you may also have rows out-of-order within an sstable From: Desimpel, Ignace Sent: vrijdag 31 januari 2014 17:43 To: user@cassandra.apache.org Subject: exception during add node due to test beforeAppend on SSTableW

exception during add node due to test beforeAppend on SSTableWriter

4 node, byte ordered, LCS, 3 Compaction Executors, replication factor 1 Code is 2.0.4 version but with patch for CASSANDRA-6638 However, no cleanup is run so patch should not play a roll 4 node cluster is started and insert/queries are done u

Nodetool cleanup on vnode cluster removes more data then wanted

Got into a problem when testing a vnode setup. I'm using a byteordered partitioner, linux, code version 2.0.4, replication factor 1, 4 machine All goes ok until I run cleanup, and gets worse when adding / decommissioning nodes. In my opinion the problem can be found in the SSTableScanner:: KeyS

OOM after some days related to RunnableScheduledFuture and meter persistance

Hi, On linux and cassandra version 2.0.2 I had an OOM after a heavy load and then some (15 ) days of idle running (not exactly idle but very very low activity). Two out of a 4 machine cluster had this OOM. I checked the heap dump (9GB) and that tells me : One instance of "java.util.concurrent.S

RE: Endless loop LCS compaction

s, Ignace -Original Message----- From: Desimpel, Ignace Sent: dinsdag 12 november 2013 09:32 To: 'Chris Burroughs' Subject: RE: Endless loop LCS compaction I think that regardless the size, the code should not go into an endless loop. -Original Message- From: Chris Burroughs

FW: Commitlog replay makes dropped and recreated keyspace and column family rows reappear

-- Aaron Morton New Zealand @aaronmorton Co-Founder & Principal Consultant Apache Cassandra Consulting http://www.thelastpickle.com From: Desimpel, Ignace Sent: dinsdag 3 december 2013 14:45 To: user@cassandra.apache.org Subject: Commitlog replay makes dropped and recreated key

Commitlog replay makes dropped and recreated keyspace and column family rows reappear

Hi, I have the impression that there is an issue with dropping a keyspace and then recreating the keyspace (and column families), combined with a restart of the database My test goes as follows: Create keyspace K and column families C. Insert rows X0 column family C0 Query for X0 : found rows

Endless loop LCS compaction

Tested on version 2.0.1 and 2.0.2. At complete idle running (nothing stored nor queried) I see that some random (depending on the tests I do) column family gets compacted over and over again (already 48h) . Total data size is only 3.5GB. Column family was created with SSTableSize : 10 MB Using

FW: Migration LCS from 1.2.X to 2.0.x exception

rename in case of Windows could do the (scary) trick. From: Desimpel, Ignace [mailto:ignace.desim...@nuance.com] Sent: woensdag 25 september 2013 13:13 To: user@cassandra.apache.org Subject: FW: Migration LCS from 1.2.X to 2.0.x exception Same is true if started from 1.2.9 version to 2.0.x Seems

FW: Migration LCS from 1.2.X to 2.0.x exception

Tue, Sep 24, 2013 at 8:57 AM, Desimpel, Ignace mailto:ignace.desim...@nuance.com>> wrote: Tested on WINDOWS : On startup of the 2.0.0 version from 1.2.x files I get an error as listed below. This is due to the code in LeveledManifest:: mutateLevel. The method already has a comment saying

Migration LCS from 1.2.X to 2.0.x exception

Tested on WINDOWS : On startup of the 2.0.0 version from 1.2.x files I get an error as listed below. This is due to the code in LeveledManifest:: mutateLevel. The method already has a comment saying that it is scary ... On windows, one cannot use the File::rename if the target file name already

RE: Too many open files and stopped compaction with many pending compaction tasks

[mailto:jeremy.hanna1...@gmail.com] Sent: donderdag 27 juni 2013 15:36 To: user@cassandra.apache.org Subject: Re: Too many open files and stopped compaction with many pending compaction tasks Are you on SSDs? On 27 Jun 2013, at 14:24, "Desimpel, Ignace" wrote: > On a test with 3 cass

Too many open files and stopped compaction with many pending compaction tasks

On a test with 3 cassandra servers version 1.2.5 with replication factor 1 and leveled compaction, I did a store last night and I did not see any problem with Cassandra. On all 3 machine the compaction is stopped already several hours. However , one machine reports 650 pending compaction tasks (

RE: ordered partitioner

Jonathan Ellis [mailto:jbel...@gmail.com] Sent: maandag 22 april 2013 20:12 To: user Subject: Re: ordered partitioner Not in general, no. There are places, like indexing, that need to use a local partitioner rather than the global one. Which uses of the DK constructor looked erroneous to you?

ordered partitioner

Hi, I was trying to implement my own ordered partitioner and got into problems. The current DecoratedKey is using a ByteBufferUtil.compareUnsigned for comparing the key. I was thinking of having a signed comparison, so I thought of making my own DecoratedKey, Token and Partitioner. That way I wo

RE: Directory structure after upgrading 1.0.8 to 1.2.1

-old CF dir. Does the timestamp indicate it was created the time the server first started as a 1.2 node? Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 7/02/2013, at 10:39 PM, "Desimpel, Ignace" mailto:ign

Directory structure after upgrading 1.0.8 to 1.2.1

After upgrading from 1.0.8 I see that now the directory structure has changed and has a structure like (part of the 1.1.x migration). But I also see that directories appear like , and the content of that 'old' directory is only one file . Questions : Should this xxx-old.json file be in the othe

RE: RE Restore snapshot

the column family MBean, which mean you need to do that per-CF). -- Sylvain On Thu, Aug 2, 2012 at 9:16 AM, Romain HARDOUIN wrote: > > No it's not possible > > "Desimpel, Ignace" a écrit sur 01/08/2012 > 14:58:49 : > >> Hi, >> >> Is it pos

Restore snapshot

Hi, Is it possible to restore a snapshot of a keyspace on a live cassandra cluster (I mean without restarting)?

RE: invalid column name length 0

: invalid column name length 0 Ok. Quick other question then. Did you issue deletion and/or used TTLs for that test ? Also, it's probably worth creating a ticket on https://issues.apache.org/jira/browse/CASSANDRA if you don't mind. -- Sylvain On Wed, Oct 5, 2011 at 2:42 PM, Desimp

RE: invalid column name length 0

d. -Original Message- From: Desimpel, Ignace [mailto:ignace.desim...@nuance.com] Sent: dinsdag 4 oktober 2011 10:48 To: user@cassandra.apache.org Subject: RE: invalid column name length 0 I run the application with the JVM -ea option, so assertions are enabled. I insert records using th

RE: invalid column name length 0

ut such error would typically produce timeout on reads whatever the rpc_timeout value is. -- Sylvain On Mon, Oct 3, 2011 at 4:58 PM, Desimpel, Ignace wrote: > I did an extra test, again starting from scratch but with replication factor > 1. > I still get the dead/up messages and t

RE: invalid column name length 0

mon frames omitted 2011-10-03 14:18:53.436 Saved CodeStructure-Computers-KeyCache (1 items) in 16 ms 2011-10-03 14:19:24.804 Saved KsFullIdx-ReverseLabelValues-KeyCache (123981 items) in 192 ms -Original Message- From: Desimpel, Ignace [mailto:ignace.desim...@nuance.com] Sent: dinsd

RE: invalid column name length 0

art DB (ms) : 2609 2011-09-30 08:24:14.984 Document stored with count records : 53034 Time elapsed (ms) : 3133 2011-09-30 08:24:16.806 Creating new commitlog segment ../../../../data/capd.cassandra.capd/dbcommitlog/CommitLog-1317363856806.log 2011-09-30 08:24:17.153 Received records to store with

RE: invalid column name length 0

al Message- From: Jonathan Ellis [mailto:jbel...@gmail.com] Sent: dinsdag 27 september 2011 17:41 To: user@cassandra.apache.org Subject: Re: invalid column name length 0 Is this upgraded from an earlier version? On Tue, Sep 27, 2011 at 8:51 AM, Desimpel, Ignace wrote: > Version 0.8.6.

invalid column name length 0

Version 0.8.6. After an extreme load to 4 (embedded) cassandra servers with replication factor 3 ( Ubuntu 10.4, dual six core, 64 bit, no swap, 1 15000 rpm commitlog disk, 1 15000 rpm datafile disk, ) I get a fatal exception as listed below. No more messages were found after that. Probably ot

RE: AssertionError

hose function are not called during "normal" query. Sylvain On Wed, May 18, 2011 at 2:40 PM, Desimpel, Ignace wrote: > Hi Sylvain, > > I did the upgrade from 0.7.4 to 0.7.5 and the exception does not occur > anymore (on Windows ...). Thanks for pointing me to the bug fix.

RE: AssertionError

ect if it is not really needed since I know that I will query for a lot of data of that type. Ignace -Original Message----- From: Desimpel, Ignace [mailto:ignace.desim...@nuance.com] Sent: Tuesday, May 17, 2011 3:33 PM To: user@cassandra.apache.org Subject: RE: AssertionError Seems

RE: AssertionError

, 2011 at 1:46 PM, Desimpel, Ignace wrote: > Ok, I will do that (next test will be done on some linux boxes being > installed now, but at this time I need to gone with the current windows > setup). > Question : Can I use the 0.7.4 data files as is? Do I need to backup the > datafiles

RE: AssertionError

e). -- Sylvain On Tue, May 17, 2011 at 12:41 PM, Desimpel, Ignace wrote: > I use a custom comparator class. So I think there is a high chance > that I do something wrong there. I was thinking that the stack trace > could give a clue and help me on the way, maybe because some already go

RE: AssertionError

could happen:) Can you provide some more information about the query and the data, and also the upgrade history for your cluster. Thanks Aaron On 17/05/2011, at 3:07 AM, "Desimpel, Ignace" wrote: Environment : java 64 bit server, java client, thrift get_slice method,

AssertionError

Environment : java 64 bit server, java client, thrift get_slice method, Cassandra 0.7.4, single node Depending on the data I pass for a query on a CF I get the following listed below. Any suggestions what could be wrong based on the stack trace? java.lang.AssertionError at org

NotSerializableException of FutureTask

Using own JMX java code and when using the NodeTool I get the following exception when calling the forceFlush function. But it seems that the flushing itself is started although the exception occurred. Any idea? (running jdk 1.6, 64 bits) Ignace 2011-04-20 16:23:45 INFO ColumnFamilyStor

RE: Question about AbstractType class

s!! Ignace -Original Message- From: Sylvain Lebresne [mailto:sylv...@datastax.com] Sent: Wednesday, April 20, 2011 4:04 PM To: user@cassandra.apache.org Subject: Re: Question about AbstractType class On Wed, Apr 20, 2011 at 3:06 PM, Desimpel, Ignace wrote: > As said above, the remaing bytes won&

RE: Different result after restart

ata was there take a look at bin/sstable2json Hope that helps. Aaron On 20 Apr 2011, at 23:00, Desimpel, Ignace wrote: Cassandra version 0.7.4 Hi, I'm storing (no deletion) in a small test some records to an embedded Cassandra instance. Then I connect using Thri

RE: Question about AbstractType class

-Original Message- From: Sylvain Lebresne [mailto:sylv...@datastax.com] Sent: Wednesday, April 20, 2011 2:07 PM To: user@cassandra.apache.org Subject: Re: Question about AbstractType class On Wed, Apr 20, 2011 at 1:35 PM, Desimpel, Ignace wrote: > Cassandra version 0.7.4 > &g

RE: Different result after restart

and then flush all the changes to SSTables. Which will result in the log files been deleted from disk, and you should see INFO level log messages that say "Discarding obsolete commit log:..." Do you get new SSTables written at start up ? If you wanted to confirm the data wa

Question about AbstractType class

Cassandra version 0.7.4 Hi, I created my own java class as an extension of the AbstractType class. But I'm not sure about the following items related to the compare function : # The remaining bytes of the buffer sometimes is zero during thrift get_slice execution, however I never store any

Different result after restart

Cassandra version 0.7.4 Hi, I'm storing (no deletion) in a small test some records to an embedded Cassandra instance. Then I connect using Thrift and I can retrieve the data as excepted. Then I restart the server with the embedded Cassandra, reconnect using Thrift but now the same quer

RE: FW: Very slow batch insert using version 0.7.2

Thus if needed I can debug the code but I would need some input from your team. Ignace -Original Message- From: Ryan King [mailto:r...@twitter.com] Sent: donderdag 10 maart 2011 18:18 To: user@cassandra.apache.org Cc: Desimpel, Ignace Subject: Re: FW: Very slow batch insert using ve