Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

Thank you DuyHai. I was in two minds about large partitions for my app. I thought upgrading to 3.x would be good and easy option. But now I'm going to work on refactoring my data model :) 2016-10-15 20:38 GMT+09:00 DuyHai Doan : > Yes, more or less. The 100Mb is a rule of

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

Yes, more or less. The 100Mb is a rule of thumb. No one will blame you for storing 200Mb for example. The figure is just given as an example of order of magnitude On Sat, Oct 15, 2016 at 1:37 PM, Kant Kodali wrote: > you mean 100MB (MegaBytes)? Also the data in each of my

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

you mean 100MB (MegaBytes)? Also the data in each of my column is about 1KB so in that case the optimal size 100K columns (since 100K * 1KB = 100MB) right? On Sat, Oct 15, 2016 at 4:26 AM, DuyHai Doan wrote: > "2) so what is optimal limit in terms of data size?" > > -->

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

"2) so what is optimal limit in terms of data size?" --> Usual recommendations for Cassandra 2.1 are: a. max 100Mb per partition size b. or up to 10 000 000 physical columns for a partition (including clustering columns etc ...) Recently, with the work of Robert Stupp (CASSANDRA-11206) and also

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

1) It will be great if someone can confirm that there is no limit 2) so what is optimal limit in terms of data size? Finally, Thanks a lot for pointing out all the operational issues! On Sat, Oct 15, 2016 at 2:39 AM, DuyHai Doan wrote: > "But is there still 2B columns

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

"But is there still 2B columns limit on the Cassandra code?" --> I remember some one the committer saying that this 2B columns limitation comes from the Thrift era where you're limited to max 2B columns to be returned to the client for each request. It also applies to the max size of each "page"

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

compacting 10 sstables each of them have a 15GB partition in what duration? On Fri, Oct 14, 2016 at 11:45 PM, Matope Ono wrote: > Please forget the part in my sentence. > For more correctly, maybe I should have said like "He could compact 10 > sstables each of them have a

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

Please forget the part in my sentence. For more correctly, maybe I should have said like "He could compact 10 sstables each of them have a 15GB partition". What I wanted to say is we can store much more rows(and columns) in a partition than before 3.6. 2016-10-15 15:34 GMT+09:00 Kant Kodali

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

"Robert said he could treat safely 10 15GB partitions at his presentation" This sounds like there is there is a row limit too not only columns?? If I am reading this correctly 10 15GB partitions means 10 partitions (like 10 row keys, thats too small) with each partition of size 15GB. (thats

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

"Robert said he could treat safely 10 15GB partitions at his presentation" This sounds like there is there is a row limit too not only columns?? If I am reading this correctly 10 15GB partitions means 10 partitions (like 10 row keys, thats too small) with each partition of size 15GB. (thats

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

Thanks to CASSANDRA-11206, I think we can have much larger partition than before 3.6. (Robert said he could treat safely 10 15GB partitions at his presentation. https://www.youtube.com/watch?v=N3mGxgnUiRY) But is there still 2B columns limit on the Cassandra code? If so, out of curiosity, I'd

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

The "2 billion column limit" press clipping "puffery". This statement seemingly became popular because highly traffic traffic-ed story, in which a tech reporter embellished on a statement to make a splashy article. The effect is something like this:

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

Well 1) I have not sent it to postgresql mailing lists 2) I thought this is an open ended question as it can involve ideas from everywhere including the Cassandra java driver mailing lists so sorry If that bothered you for some reason. On Wed, Oct 12, 2016 at 1:41 AM, Dorian Hoxha

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

I did mention this in my previous email. This is not time series data. I understand how to structure it if it is a time series data/ What do you mean globally sorted? you mean keeping every partition sorted (since I come from Casandra world)? rowkey 1 -> blob page -> int or long or bigint col1

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

Also, I'm not sure, but I don't think it's "cool" to write to multiple lists in the same message. (based on postgresql mailing lists rules). Example I'm not subscribed to those, and now the messages are separated. On Wed, Oct 12, 2016 at 10:37 AM, Dorian Hoxha wrote: >

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

There are some issues working on larger partitions. Hbase doesn't do what you say! You have also to be carefull on hbase not to create large rows! But since they are globally-sorted, you can easily sort between them and create small rows. In my opinion, cassandra people are wrong, in that they