Re: Concatenating ids with extension to keep multiple rows related to an entity in a single CF

2011-11-04 Thread Aditya Narayan
the data in different rows of an entity is all of similar type but serves different features but still has almost similar storage and retrieval needs thus I wanted to put them in one CF and reduce column families. From my knowledge, I believe compositeType existed for columns as an alternative

Re: Cassandra 1.x and proper JNA setup

2011-11-04 Thread Maciej Miklas
Super - thank you for help :) On Thu, Nov 3, 2011 at 6:55 PM, Jonathan Ellis jbel...@gmail.com wrote: Relying on that was always a terrible idea because you could easily OOM before it could help. There's no substitute for don't make the caches too large in the first place. We're working on

Clear snapshot on windows

2011-11-04 Thread Fredrik L Stigbäck
Clearing snapshots on Windows with nodetool sometimes fails. The reason is because the snapshot files hardlinks into corresponding files in the data directory. When compaction runs, most of the files are merged and the corresponding snapshot files can be deleted but often .e.g. the secondary

Cassandra Counters and TTL

2011-11-04 Thread Vlad Paiu
Hello, I'm a new user of Cassandra and I think it's great. Still, while developing my APP using Cassandra, I got stuck with some things and I'm not really sure that Cassandra can handle them at the moment. So, first of all, does Cassandra allow for Counters and regular Keys to be located in

Re: Cassandra Counters and TTL

2011-11-04 Thread Amit Chavan
Answers inline. On Fri, Nov 4, 2011 at 4:59 PM, Vlad Paiu vladp...@opensips.org wrote: Hello, I'm a new user of Cassandra and I think it's great. Still, while developing my APP using Cassandra, I got stuck with some things and I'm not really sure that Cassandra can handle them at the

Multiget question

2011-11-04 Thread Filipe Gonçalves
Multiget slice queries seem to fetch rows sequentially, at least fromwhat I understood of the sources. This means the node that receives amultiget of N keys does N get operations to the other nodes in thecluster to fetch the remaining keys. Am I right? Is this the way multiget works internally?

Large Increase in SSTable count after Upgrade to 0.6.13

2011-11-04 Thread Jake Maizel
Hello, We moved from 0.6.6 to 0.6.13 recently on an 8 nodes cluster and started to see issues with two nodes where memtables are being flushed at a high rate and compaction seems to have fallen off or behind. A huge number of sstables has accumilated as a result of slowed compaction. We are

Why SSTable is sorted by tokens instead of row keys?

2011-11-04 Thread Gary Shi
I want to save time series event logs into Cassandra, and I need to load them by key range (row key is time-based). But we can't use RandomPartitioner in this way, while OrderPreservingPartitioner leads to hot spot problem. So I wonder why Cassandra save SSTable by sorted row tokens instead of

Re: Why SSTable is sorted by tokens instead of row keys?

2011-11-04 Thread Sylvain Lebresne
On Fri, Nov 4, 2011 at 1:49 PM, Gary Shi gary...@gmail.com wrote: I want to save time series event logs into Cassandra, and I need to load them by key range (row key is time-based). But we can't use RandomPartitioner in this way, while OrderPreservingPartitioner leads to hot spot problem. So

Question about minor compaction

2011-11-04 Thread Rene Kochen
Assume the following default settings: min_compaction_threshold = 4, max_compaction_threshold = 32. When I start a bulk insert in Cassandra, I see minor compactions work: all similar sized files are compacted when there are four of them. However, when files gets larger, Cassandra waits with

Re: Multiget question

2011-11-04 Thread Sylvain Lebresne
2011/11/4 Filipe Gonçalves the.wa.syndr...@gmail.com: Multiget slice queries seem to fetch rows sequentially, at least fromwhat I understood of the sources. This means the node that receives amultiget of N keys does N get operations to the other nodes in thecluster to fetch the remaining keys.

Re: Question about minor compaction

2011-11-04 Thread Radim Kolar
What makes Cassandra decide to wait with this minor compaction? What version do you using? There were some patch for 1.x branch which will do it as you expect. Cassandra 0.8 waited with compactions.

Re: Multiget question

2011-11-04 Thread Filipe Gonçalves
Thanks for the answer. I hadn't realised requests were made in parallel, I first noticed it when multiget's took linear time in machines with high loads. Looking at the code led me to the previous conclusion (N gets for multiget for N keys). I agree it would take a major overhaul of the code to

Re: Using Cli to create a column family with column name metadata question

2011-11-04 Thread Jonathan Ellis
[Moving to user@] Because Cassandra's sparse data model supports using rows as materialized views, having non-UTF8 column names is common and totally valid. On Fri, Nov 4, 2011 at 5:19 AM, Arsene Lee arsene@ruckuswireless.com wrote: Hi, I'm trying to use Column Family's metadata to do

RE: Question about minor compaction

2011-11-04 Thread Rene Kochen
I'm using Cassandra 0.7.9. Ok, so in this version, Cassandra waits with compaction. But when (in my original example) are the four 1GB files compacted? Thanks! -Original Message- From: Radim Kolar [mailto:h...@sendmail.cz] Sent: vrijdag 4 november 2011 15:55 To:

Re: Question about minor compaction

2011-11-04 Thread Radim Kolar
Dne 4.11.2011 16:16, Rene Kochen napsal(a): I'm using Cassandra 0.7.9. Ok, so in this version, Cassandra waits with compaction. But when (in my original example) are the four 1GB files compacted? There are compacted when next file of similar size to 1 GB is created

RE: Question about minor compaction

2011-11-04 Thread Rene Kochen
Thanks for your quick response. I indeed see that similar sized files are compacted. However, for four similar 1GB files, this is not what I see. The documentation states: These parameters set thresholds for the number of similar-sized SSTables that can accumulate before a minor compaction is

Re: Problem after upgrade to 1.0.1

2011-11-04 Thread Jonathan Ellis
One possibility: If you're overloading the cluster, replicas will drop updates to avoid OOMing. (This is logged at WARN level.) Before 1.x Cassandra would just let that slide, but with w/ 1.0 it started recording hints for those. On Thu, Nov 3, 2011 at 7:17 PM, Bryce Godfrey

Re: Question about minor compaction

2011-11-04 Thread Sylvain Lebresne
On Fri, Nov 4, 2011 at 5:22 PM, Rene Kochen rene.koc...@emea.schange.com wrote: Thanks for your quick response. I indeed see that similar sized files are compacted. However, for four similar 1GB files, this is not what I see. The documentation states: These parameters set thresholds for

Re: Question about minor compaction

2011-11-04 Thread Sylvain Lebresne
On Fri, Nov 4, 2011 at 6:01 PM, Rene Kochen rene.koc...@emea.schange.com wrote: Thanks for this very clear explanation. Could it be that Cassandra does not begin a minor compaction if there is memory pressure? No (and in 0.7 Cassandra really does nothing specific under memory pressure except

Re: Debian package jna bug workaroung

2011-11-04 Thread paul cannon
The cassandra-cli tool will show you, if you're using at least cassandra 1.0.1, in a describe command. If not, you can make a thrift describe_keyspace() call some other way, and check the value of the appropriate CfDef's row_cache_provider string. If it's SerializingCacheProvider, it's off-heap.

Re: Why SSTable is sorted by tokens instead of row keys?

2011-11-04 Thread Brandon Williams
On Fri, Nov 4, 2011 at 7:49 AM, Gary Shi gary...@gmail.com wrote: I want to save time series event logs into Cassandra, and I need to load them by key range (row key is time-based). But we can't use RandomPartitioner in this way, while OrderPreservingPartitioner leads to hot spot problem. You

Cassandra Integration w/ SOLR using Virgil

2011-11-04 Thread Brian O'Neill
Up front, I'd like to say this is still pretty raw. We'd love to get feedback (and better yet contributions ;). With that as a disclaimer, I added SOLR integration to Virgil. When you add and delete rows and columns via the REST interface, an index is updated in SOLR. For more information check

The use of SuperColumns

2011-11-04 Thread Dwight Smith
There have been mentions that the use of SuperColumns is not really encouraged, recommendation to use composite columns instead. Will SuperColumns be removed. Any comments greatly appreciated.

Re: Why SSTable is sorted by tokens instead of row keys?

2011-11-04 Thread Gary Shi
Thank you. I agree that request lots of machines process a single query could be slow, if there are hundreds of them instead of dozens. Will a cluster of e.g. 4-20 nodes behave well if we spread the query to all nodes? Many articles suggest model TimeUUID in columns instead of rows, but since

Re: Second Cassandra users survey

2011-11-04 Thread Jim Newsham
- Bulk column deletion by (column name) range. Without this feature, we are forced to perform a range query and iterate over all of the columns, deleting them one by one (we do this in a batch, but it's still a very slow approach). See CASSANDRA-494/3448. If anyone else has a need for

Re: Second Cassandra users survey

2011-11-04 Thread Brandon Williams
On Fri, Nov 4, 2011 at 9:19 PM, Jim Newsham jnews...@referentia.com wrote: - Bulk column deletion by (column name) range.  Without this feature, we are forced to perform a range query and iterate over all of the columns, deleting them one by one (we do this in a batch, but it's still a very

Re: Second Cassandra users survey

2011-11-04 Thread Jim Newsham
On 11/4/2011 4:32 PM, Brandon Williams wrote: On Fri, Nov 4, 2011 at 9:19 PM, Jim Newshamjnews...@referentia.com wrote: - Bulk column deletion by (column name) range. Without this feature, we are forced to perform a range query and iterate over all of the columns, deleting them one by one (we