Re: Inserting list data

2016-10-15 Thread Russell Spitzer
Are you sure you aren't using batches? These will assign the same timestamp to your inserts which can lead to unexpected behaviors. On Fri, Oct 14, 2016 at 9:45 PM Vladimir Yudovin wrote: > Did you try the same quires with Java driver without using prepared > statements? >

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

2016-10-15 Thread Kant Kodali
"Robert said he could treat safely 10 15GB partitions at his presentation" This sounds like there is there is a row limit too not only columns?? If I am reading this correctly 10 15GB partitions means 10 partitions (like 10 row keys, thats too small) with each partition of size 15GB. (thats

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

2016-10-15 Thread Kant Kodali
"Robert said he could treat safely 10 15GB partitions at his presentation" This sounds like there is there is a row limit too not only columns?? If I am reading this correctly 10 15GB partitions means 10 partitions (like 10 row keys, thats too small) with each partition of size 15GB. (thats

Is SASI index in Cassandra efficient for high cardinality columns?

2016-10-15 Thread Kant Kodali
I understand Secondary Indexes in general are inefficient on high cardinality columns but since SASI is built from scratch I wonder if the same argument applies there? If not, Why? Because I believe primary keys in Cassandra are indeed indexed and since Primary key is supposed to be the column

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

2016-10-15 Thread Matope Ono
Please forget the part in my sentence. For more correctly, maybe I should have said like "He could compact 10 sstables each of them have a 15GB partition". What I wanted to say is we can store much more rows(and columns) in a partition than before 3.6. 2016-10-15 15:34 GMT+09:00 Kant Kodali

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

2016-10-15 Thread Kant Kodali
compacting 10 sstables each of them have a 15GB partition in what duration? On Fri, Oct 14, 2016 at 11:45 PM, Matope Ono wrote: > Please forget the part in my sentence. > For more correctly, maybe I should have said like "He could compact 10 > sstables each of them have a

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

2016-10-15 Thread DuyHai Doan
"But is there still 2B columns limit on the Cassandra code?" --> I remember some one the committer saying that this 2B columns limitation comes from the Thrift era where you're limited to max 2B columns to be returned to the client for each request. It also applies to the max size of each "page"

Re: Is SASI index in Cassandra efficient for high cardinality columns?

2016-10-15 Thread DuyHai Doan
Define precisely what you mean by "high cardinality columns". Do you mean: 1) a single indexed value is present in a lot of rows 2) a single indexed value has only a few (if not just one) matching row On Sat, Oct 15, 2016 at 8:37 AM, Kant Kodali wrote: > I understand

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

2016-10-15 Thread Kant Kodali
1) It will be great if someone can confirm that there is no limit 2) so what is optimal limit in terms of data size? Finally, Thanks a lot for pointing out all the operational issues! On Sat, Oct 15, 2016 at 2:39 AM, DuyHai Doan wrote: > "But is there still 2B columns

Re: Is SASI index in Cassandra efficient for high cardinality columns?

2016-10-15 Thread Kant Kodali
Well I went with the definition from wikipedia and that definition rules out #1 so it is #2 and it is just one matching row in my case. On Sat, Oct 15, 2016 at 2:40 AM, DuyHai Doan wrote: > Define precisely what you mean by "high cardinality columns". Do you mean: > > 1)

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

2016-10-15 Thread DuyHai Doan
"2) so what is optimal limit in terms of data size?" --> Usual recommendations for Cassandra 2.1 are: a. max 100Mb per partition size b. or up to 10 000 000 physical columns for a partition (including clustering columns etc ...) Recently, with the work of Robert Stupp (CASSANDRA-11206) and also

Re: Is SASI index in Cassandra efficient for high cardinality columns?

2016-10-15 Thread DuyHai Doan
If each indexed value has very few matching rows, then querying using SASI (or any impl of secondary index) may scan the whole cluster. This is because the index are "distributed" e.g. the indexed values stay on the same nodes as the base data. And even SASI with its own data-structure will not

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

2016-10-15 Thread Kant Kodali
you mean 100MB (MegaBytes)? Also the data in each of my column is about 1KB so in that case the optimal size 100K columns (since 100K * 1KB = 100MB) right? On Sat, Oct 15, 2016 at 4:26 AM, DuyHai Doan wrote: > "2) so what is optimal limit in terms of data size?" > > -->

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

2016-10-15 Thread DuyHai Doan
Yes, more or less. The 100Mb is a rule of thumb. No one will blame you for storing 200Mb for example. The figure is just given as an example of order of magnitude On Sat, Oct 15, 2016 at 1:37 PM, Kant Kodali wrote: > you mean 100MB (MegaBytes)? Also the data in each of my

Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

2016-10-15 Thread Matope Ono
Thank you DuyHai. I was in two minds about large partitions for my app. I thought upgrading to 3.x would be good and easy option. But now I'm going to work on refactoring my data model :) 2016-10-15 20:38 GMT+09:00 DuyHai Doan : > Yes, more or less. The 100Mb is a rule of

Re: Repair in Multi Datacenter - Should you use -dc Datacenter repair or repair with -pr

2016-10-15 Thread Anuj Wadehra
Hi Leena, Do you have a firewall between the two DCs? If yes, connection reset can be caused by Cassandra trying to use a TCP connection which is already closed by the firewall. Please make sure that you set high connection timeout at firewall. Also, make sure your servers are not overloaded.

Re: Scenarios when blocking read repair takes place

2016-10-15 Thread Krishna Chandra Prajapati
Hi which side is this? Mankapur? Krishna On Oct 14, 2016 12:15 PM, "siddharth verma" wrote: > Hi, > Does blocking read repair take place only when we read on the primary key > or > does it take place in the following scenarios as well? > > Consistemcy ALL > 1.