AW: secondery indexes TTL - strange issues

2012-09-17 Thread Roland Gude
Issue created. Will attach debug logs asap CASSANDRA-4670https://issues.apache.org/jira/browse/CASSANDRA-4670 Von: aaron morton [mailto:aa...@thelastpickle.com] Gesendet: Montag, 17. September 2012 03:46 An: user@cassandra.apache.org Betreff: Re: secondery indexes TTL - strange issues Date

Re: nodetool cfstats and compression

2012-09-17 Thread aaron morton
Yes. It is the space taken up on disk, including compaction. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 15/09/2012, at 6:30 AM, Jim Ancona j...@anconafamily.com wrote: Do the row size stats reported by 'nodetool cfstats' include

Re: minor compaction and delete expired column-tombstones

2012-09-17 Thread aaron morton
Does minor compaction delete expired column-tombstones when the row is also present in another table which is No. Compaction is per Column Family. Tombstones will be expired by Minor Compaction if all fragments of the row are contained in the SSTables being compacted. Cheers

Re: Disk configuration in new cluster node

2012-09-17 Thread aaron morton
4 drives for data and 1 drive for commitlog, How are you configuring the drives ? It's normally best to present one big data volume, e.g. using raid 0, and put the commit log on say the system mirror. will the node balance out the load on the drives, or is it agnostic to usage of drives

Re: Disk configuration in new cluster node

2012-09-17 Thread Robin Verlangen
A word of warning. If you put more than 300GB to 400GB per node you may end experience some issues ... I think this is probably the solution to your multiple disk problem. You could use easily one single disk to store the data on, and one disk for the commitlog. No issues with JBOD, RAID or

Re: minor compaction and delete expired column-tombstones

2012-09-17 Thread Rene Kochen
Oke, thanks! So a column tombstone will only be removed if all row fragments are present in the tables being compacted. I have a row called Index which contains columns like page0, page1, page2, etc. Every several minutes, new columns are created and old ones deleted. The problem is that I now

Re: Query advice to prevent node overload

2012-09-17 Thread André Cruz
On Sep 17, 2012, at 3:04 AM, aaron morton aa...@thelastpickle.com wrote: I have a schema that represents a filesystem and one example of a Super CF is: This may help with some ideas http://www.datastax.com/dev/blog/cassandra-file-system-design In general we advise to avoid Super Columns

Re: Repair: Issue in netstats

2012-09-17 Thread B R
Sorry for the delay; been out of the loop. Could this problem be due to running repair on a node upgraded to 1.0.11 but the other node in the cluster is still at 0.8.x ? On Fri, Sep 7, 2012 at 9:11 PM, Sylvain Lebresne sylv...@datastax.comwrote: That obviously shouldn't happen and I don't

Re: Query advice to prevent node overload

2012-09-17 Thread André Cruz
On Sep 17, 2012, at 3:04 AM, aaron morton aa...@thelastpickle.com wrote: I have a schema that represents a filesystem and one example of a Super CF is: This may help with some ideas http://www.datastax.com/dev/blog/cassandra-file-system-design Could you explain the usage of the sentinel?

Re: Many ParNew collections

2012-09-17 Thread Rene Kochen
Thanks Aaron, I found the problem. It's in this thread: minor compaction and delete expired column-tombstones. The problem was that I have one big row called Index which contains many tombstones. Reading all these tombstones caused the memory issues. I think node 1 and 3 have had enough minor

Re: Repair: Issue in netstats

2012-09-17 Thread Sylvain Lebresne
On Mon, Sep 17, 2012 at 11:06 AM, B R software.research.w...@gmail.com wrote: Could this problem be due to running repair on a node upgraded to 1.0.11 but the other node in the cluster is still at 0.8.x ? Yes, repair (as all operation requiring streaming) doesn't work correctly across major

Re: cassandra/hadoop BulkOutputFormat failures

2012-09-17 Thread Brian Jeltema
As suggested, it was a version-skew problem. Thanks. Brian On Sep 14, 2012, at 11:34 PM, Jeremy Hanna wrote: A couple of guesses: - are you mixing versions of Cassandra? Streaming differences between versions might throw this error. That is, are you bulk loading with one version of

Cassandra Messages Dropped

2012-09-17 Thread Michael Theroux
Hello, While under load, we have occasionally been seeing messages dropped errors in our cassandra log. Doing some research, I understand this is part of Cassandra's design to shed load, and we should look at the tpstats-like output to determine what should be done to resolve the situation.

Astyanax error

2012-09-17 Thread A J
Hello, I am tyring to retrieve a list of Column Names (that are defined as Integer) from a CF with RowKey as Integer as well. (I don't care for the column values that are just nulls) Following is snippet of my Astyanax code. I am getting 0 columns but I know the key that I am querying contains a

Re: Disk configuration in new cluster node

2012-09-17 Thread Casey Deccio
On Mon, Sep 17, 2012 at 1:19 AM, aaron morton aa...@thelastpickle.comwrote: 4 drives for data and 1 drive for commitlog, How are you configuring the drives ? It's normally best to present one big data volume, e.g. using raid 0, and put the commit log on say the system mirror. Given the

Re: minor compaction and delete expired column-tombstones

2012-09-17 Thread Josep Blanquer
We've run exactly into the same problem recently. Some specific keys in a couple CFs accumulate a fair amount of column churn over time. Pre Cassandra 1.x we scheduled full compactions often to purge them. However, when we moved to 1.x but we adopted the recommended practice of avoiding full

Re: minor compaction and delete expired column-tombstones

2012-09-17 Thread Sylvain Lebresne
Is there any JIRA or enhancement to perhaps be able to detect when certain column tombstones can be deleted in minor compactions? The new introduction of SSTable min-max timestamps might help? or perhaps there are new ones coming up that I'm not aware of

persistent compaction issue (1.1.4 and 1.1.5)

2012-09-17 Thread Michael Kjellman
Hi All, I have an issue where each one of my nodes (currently all running at 1.1.5) is reporting around 30,000 pending compactions. I understand that a pending compaction doesn't necessarily mean it is a scheduled task however I'm confused why this behavior is occurring. It is the same on all

Bloom Filters in Cassandra

2012-09-17 Thread Bill Hastings
How are bloom filters used in Cassandra? Is my understanding correct in that there is one per SSTable encapsulating what keys are in the SSTable? Please advise.

Is Cassandra right for me?

2012-09-17 Thread Marcelo Elias Del Valle
Hello, I am new to Cassandra and I am in doubt if Cassandra is the right technology to use in the architecture I am defining. Also, I saw a presentation which said that if I don't have rows with more than a hundred rows in Cassandra, whether I am doing something wrong or I shouldn't be using

are counters stable enough for production?

2012-09-17 Thread Bartłomiej Romański
Hi, Does anyone have any experience with using Cassandra counters in production? We rely heavily on them and recently we've got a few very serious problems. Our counters values suddenly became a few times higher than expected. From the business point of view this is a disaster :/ Also there a

Re: Query advice to prevent node overload

2012-09-17 Thread aaron morton
Could you explain the usage of the sentinel? Queries that use a secondary index must include an equality clause. That's the sentinel is there for… select filename from inode where filename ‘/tmp’ and filename ‘/tmq’ and sentinel = ‘x’; Cheers - Aaron Morton Freelance

Re: Cassandra Messages Dropped

2012-09-17 Thread aaron morton
INFO [ScheduledTasks:1] 2012-09-17 06:28:03,839 StatusLogger.java (line 72) MemtablePostFlusher 1 5 0 INFO [ScheduledTasks:1] 2012-09-17 06:28:03,840 StatusLogger.java (line 72) FlushWriter 1 5 0 Looks suspiciously like

Re: Stream definition is lost after server restart

2012-09-17 Thread Ishan Thilina
Sorry, Forgot to mention that I'm using Cassandra 1.1.3 -- Thank you..! - 071-6372089 Ishan's info: www.ishans.info මගේ සටහන්: www.siblog.ishans.info Ishan's way: www.blog.ishans.info

HTimedOutException and cluster not working

2012-09-17 Thread Jason Wee
Hello, A context to our environment, we have a clusters of 9 nodes with a few keyspaces. The client write to the cluster with consistency level of one to a keyspace in the cluster with a replication factor of 3. The hector client is configured such that all the nodes in cluster is specified and