Re: Why does Cassandra need to have 2B column limit? why can't we have unlimited ?

[Matope Ono]

Thanks to CASSANDRA-11206, I think we can have much larger partition than
before 3.6.
(Robert said he could treat safely 10 15GB partitions at his presentation.
https://www.youtube.com/watch?v=N3mGxgnUiRY)

But is there still 2B columns limit on the Cassandra code?
If so, out of curiosity, I'd like to know where the bottleneck is. Could
anyone let me know about it?

Thanks Yasuharu.

2016-10-13 1:11 GMT+09:00 Edward Capriolo :

> The "2 billion column limit" press clipping "puffery". This statement
> seemingly became popular because highly traffic traffic-ed story, in which
> a tech reporter embellished on a statement to make a splashy article.
>
> The effect is something like this:
> http://www.healthnewsreview.org/2012/08/iced-tea-kidney-ston
> es-and-the-study-that-never-existed/
>
> Iced tea does not cause kidney stones! Cassandra does not store rows with
> 2 billion columns! It is just not true.
>
>
>
>
>
>
> On Wed, Oct 12, 2016 at 4:57 AM, Kant Kodali  wrote:
>
>> Well 1) I have not sent it to postgresql mailing lists 2) I thought this
>> is an open ended question as it can involve ideas from everywhere including
>> the Cassandra java driver mailing lists so sorry If that bothered you for
>> some reason.
>>
>> On Wed, Oct 12, 2016 at 1:41 AM, Dorian Hoxha 
>> wrote:
>>
>>> Also, I'm not sure, but I don't think it's "cool" to write to multiple
>>> lists in the same message. (based on postgresql mailing lists rules).
>>> Example I'm not subscribed to those, and now the messages are separated.
>>>
>>> On Wed, Oct 12, 2016 at 10:37 AM, Dorian Hoxha 
>>> wrote:
>>>
 There are some issues working on larger partitions.
 Hbase doesn't do what you say! You have also to be carefull on hbase
 not to create large rows! But since they are globally-sorted, you can
 easily sort between them and create small rows.

 In my opinion, cassandra people are wrong, in that they say "globally
 sorted is the devil!" while all fb/google/etc actually use globally-sorted
 most of the time! You have to be careful though (just like with random
 partition)

 Can you tell what rowkey1, page1, col(x) actually are ? Maybe there is
 a way.
 The most "recent", means there's a timestamp in there ?

 On Wed, Oct 12, 2016 at 9:58 AM, Kant Kodali  wrote:

> Hi All,
>
> I understand Cassandra can have a maximum of 2B rows per partition but
> in practice some people seem to suggest the magic number is 100K. why not
> create another partition/rowkey automatically (whenever we reach a safe
> limit that  we consider would be efficient)  with auto increment bigint  
> as
> a suffix appended to the new rowkey? so that the driver can return the new
> rowkey  indicating that there is a new partition and so on...Now I
> understand this would involve allowing partial row key searches which
> currently Cassandra wouldn't do (but I believe HBASE does) and thinking
> about token ranges and potentially many other things..
>
> My current problem is this
>
> I have a row key followed by bunch of columns (this is not time series
> data)
> and these columns can grow to any number so since I have 100K limit
> (or whatever the number is. say some limit) I want to break the partition
> into level/pages
>
> rowkey1, page1->col1, col2, col3..
> rowkey1, page2->col1, col2, col3..
>
> now say my Cassandra db is populated with data and say my application
> just got booted up and I want to most recent value of a certain partition
> but I don't know which page it belongs to since my application just got
> booted up? how do I solve this in the most efficient that is possible in
> Cassandra today? I understand I can create MV, other tables that can hold
> some auxiliary data such as number of pages per partition and so on..but
> that involves the maintenance cost of that other table which I cannot
> afford really because I have MV's, secondary indexes for other good
> reasons. so it would be great if someone can explain the best way possible
> as of today with Cassandra? By best way I mean is it possible with one
> request? If Yes, then how? If not, then what is the next best way to solve
> this?
>
> Thanks,
> kant
>


>>>
>>
>

Re: Inserting list data

[Vladimir Yudovin]

Did you try the same quires with Java driver without using prepared statements?





Best regards, Vladimir Yudovin, 

Winguzone - Hosted Cloud Cassandra on Azure and SoftLayer.
Launch your cluster in minutes.






 On Fri, 14 Oct 2016 15:13:38 -0400Aoi Kadoya cadyan@gmail.com 
wrote 




Hi Vladimir, 

 

In fact I am having difficulty to reproduce this issue by cqlsh. 

I was reported this issue by one of our developers and he is using his 

client application that uses cassandra java driver 3.0.3. (we're using 

DSE5.0.1) 

 

here is how his application log look like 

 

app A: 

2016-10-11 13:28:23,014 [TRACE] [core.QueryLogger.NORMAL] [cluster1] 

[HOST1/IP1:9042] Query completed normally, took 5 ms: [8 bound values] 

INSERT INTO global.table_name 

("id","alert_to","alert_emails","created_by","created_date","alert_level","updated_by","updated_date")
 

VALUES (?,?,?,?,?,?,?,?); 

[id:25712, alert_to:[2], alert_emails:NULL, 

created_by:'service-worker:ec45afd2-c40a-44d9-a2a1-7416409be6e2', 

created_date:1476160103007, alert_level:2, updated_by:NULL, 

updated_date:NULL] 

 

app B: 

2016-10-11 13:28:23,014 [TRACE] [core.QueryLogger.NORMAL] [cluster1] 

[HOST2/IP2:9042] Query completed normally, took 6 ms: [8 bound values] 

INSERT INTO global.table_name 

("alert_to","alert_emails","created_date","id","created_by","updated_by","updated_date","alert_level")
 

VALUES (?,?,?,?,?,?,?,?); 

[alert_to:[1], alert_emails:NULL, created_date:1476160103007, 

id:25712, created_by:'service-worker:ec45afd2-c40a-44d9-a2a1-7416409be6e2', 

updated_by:NULL, updated_date:NULL, alert_level:1] 

 

and the data on cassandra 

TYPES 

 id bigint, 

 alert_emails list, 

 alert_level int, 

 alert_to list, 

 created_by text, 

 created_date timestamp, 

 updated_by text, 

 updated_date timestamp, 

 PRIMARY KEY (id) 

 

 

SELECT id, alert_level, alert_to FROM global.table_name WHERE id=25712; 

| id | alert_level | alert_to | 

| 25712 | 2 | [2, 1] | 

 

but when I threw the queries like below from cqlsh from different 

nodes at the same time in my testing environment, the data(alert_to) 

was just [1], which is expected behavior. 

 

on host 1 

cqlsh INSERT INTO global.table_name 

("id","alert_to","alert_emails","created_by","created_date","alert_level","updated_by","updated_date")
 

VALUES 
(25712,[2],NULL,'service-worker:ec45afd2-c40a-44d9-a2a1-7416409be6e2',1476160103007,2,NULL,NULL);
 

on host2 

cqlsh INSERT INTO global.table_name 

("alert_to","alert_emails","created_date","id","created_by","updated_by","updated_date","alert_level")
 

VALUES 
([1],NULL,1476160103007,25712,'service-worker:ec45afd2-c40a-44d9-a2a1-7416409be6e2',NULL,NULL,1);
 

 

 

so I wonder if this is something wrong with java driver but I cannot 

figure out the way to break this down further. 

 

 

@Andrew 

we're not using UDT..but appreciate if you could share your case, too. 

 

Thanks, 

Aoi 

 

2016-10-13 11:26 GMT-07:00 Andrew Baker baker...@gmail.com: 

 I saw evidence of this behavior, but when we created a test to try to make 

 it happen it never did, we assumed it was UDT related and lost interest, 

 since it didn't have a big impact. I will try to carve some time to look 

 into this some more and let you know if I find anything. 

 

 On Wed, Oct 12, 2016 at 9:24 PM Vladimir Yudovin 
vla...@winguzone.com 

 wrote: 

 

 The data is actually appended. not overwritten. 

 Strange, can you send exactly operators? 

 

 Here is example I do: 

 CREATE KEYSPACE events WITH replication = {'class': 'SimpleStrategy', 

 'replication_factor': 1}; 

 CREATE TABLE events.data (id int primary key, events 
listtext); 

 INSERT INTO events.data (id, events) VALUES ( 0, ['a']); 

 SELECT * FROM events.data ; 

 id | events 

 + 

 0 | ['a'] 

 

 (1 rows) 

 

 INSERT INTO events.data (id, events) VALUES ( 0, ['b']); 

 SELECT * FROM events.data ; 

 id | events 

 + 

 0 | ['b'] 

 

 (1 rows) 

 

 As you see, 'a' was overwritten by 'b' 

 

 

 Best regards, Vladimir Yudovin, 

 Winguzone - Hosted Cloud Cassandra on Azure and SoftLayer. 

 Launch your cluster in minutes. 

 

 

  On Wed, 12 Oct 2016 23:58:23 -0400Aoi Kadoya 
cadyan@gmail.com 

 wrote  

 

 yes, that's what I thought. but, when I use these forms, 

 INSERT ... ['A'] 

 INSERT ... ['B'] 

 

 The data is actually appended. not overwritten. 

 so I guess this is something unexpected? 

 

 Thanks, 

 Aoi 

 

 2016-10-12 20:55 GMT-07:00 Vladimir Yudovin 
vla...@winguzone.com: 

  If you use form 

  INSERT ... ['A'] 

  INSERT ... ['B'] 

  

  latest INSERT will overwrite first, because this insert the whole 
list. 

  It's 

  better to use UPDATE like: 

  UPDATE ... SET events = events + ['A'] 

  UPDATE ... SET events = events + ['B'] 

  These operations add new elements to the end of existing list. 

  

  

  From here 

  
https://docs.datastax.com/en/cql/3.0/cql/cql_using/use_list_t.html 

  : 

Repair: huge boost on C* 2.1 with CASSANDRA-12580

[Romain Hardouin]

Hi all,

Many people here have troubles with repair so I would like to share my 
experience regarding the backport of CASSANDRA-12580 "Fix merkle tree size 
calculation" (thanks Paulo!) in our C* 2.1.16. I was expecting some minor 
improvements but the results are impressive on some tables.

Because of a slow VPN between our EU and US AWS DCs, the massive drop of 
overstreaming is a big win for us. On top of that, before the backport I used 
to see many RepairException that increased during each repair. With this fix 
the graph shows only one exception on one node, so we can say it's negligible. 
Such exceptions are not critical because Cassandra-reaper makes a retry but 
it's a waste of time.


I run a repair on tables set by set (some sets of tables being more critical, 
etc.).
The most impressive result so far for a set is:
* Before: 23 days (days, not hours)
* With CASSANDRA-12580: 16 hours (yes, hours!)

The improvement is not always dramatic (e.g. 8 hours instead of 39 hours on 
another set) but still significant and valuable.

Moreover, considering that:
* repair is a mandatory operation in many use cases
* Paulo already made the patch for 2.1
* C* 2.1 is widely used (the most used?)
I think this bugfix is critical - from an Ops point of view - and should land 
in 2.1.17 to be available to people that don't deploy from sources.

Best,

Romain

Re: Inserting list data

[Aoi Kadoya]

Hi Vladimir,

In fact I am having difficulty to reproduce this issue by cqlsh.
I was reported this issue by one of our developers and he is using his
client application that uses cassandra java driver 3.0.3. (we're using
DSE5.0.1)



app A:
2016-10-11 13:28:23,014 [TRACE] [core.QueryLogger.NORMAL] [cluster1]
[HOST1/IP1:9042] Query completed normally, took 5 ms: [8 bound values]
INSERT INTO global.table_name
("id","alert_to","alert_emails","created_by","created_date","alert_level","updated_by","updated_date")
VALUES (?,?,?,?,?,?,?,?);
[id:25712, alert_to:[2], alert_emails:NULL,
created_by:'service-worker:ec45afd2-c40a-44d9-a2a1-7416409be6e2',
created_date:1476160103007, alert_level:2, updated_by:NULL,
updated_date:NULL]

app B:
2016-10-11 13:28:23,014 [TRACE] [core.QueryLogger.NORMAL] [cluster1]
[HOST2/IP2:9042] Query completed normally, took 6 ms: [8 bound values]
INSERT INTO global.table_name
("alert_to","alert_emails","created_date","id","created_by","updated_by","updated_date","alert_level")
VALUES (?,?,?,?,?,?,?,?);
[alert_to:[1], alert_emails:NULL, created_date:1476160103007,
id:25712, created_by:'service-worker:ec45afd2-c40a-44d9-a2a1-7416409be6e2',
updated_by:NULL, updated_date:NULL, alert_level:1]



id bigint,
alert_emails list,
alert_level int,
alert_to list,
created_by text,
created_date timestamp,
updated_by text,
updated_date timestamp,
PRIMARY KEY (id)


SELECT id, alert_level, alert_to FROM global.table_name WHERE id=25712;
| id | alert_level | alert_to |
| 25712   | 2   | [2, 1]   |

but when I threw the queries like below from cqlsh from different
nodes at the same time in my testing environment, the data(alert_to)
was just [1], which is expected behavior.

on host 1
cqlsh> INSERT INTO global.table_name
("id","alert_to","alert_emails","created_by","created_date","alert_level","updated_by","updated_date")
VALUES 
(25712,[2],NULL,'service-worker:ec45afd2-c40a-44d9-a2a1-7416409be6e2',1476160103007,2,NULL,NULL);
on host2
cqlsh> INSERT INTO global.table_name
("alert_to","alert_emails","created_date","id","created_by","updated_by","updated_date","alert_level")
VALUES 
([1],NULL,1476160103007,25712,'service-worker:ec45afd2-c40a-44d9-a2a1-7416409be6e2',NULL,NULL,1);


so I wonder if this is something wrong with java driver but I cannot
figure out the way to break this down further.


@Andrew
we're not using UDT..but appreciate if you could share your case, too.

Thanks,
Aoi

2016-10-13 11:26 GMT-07:00 Andrew Baker :
> I saw evidence of this behavior, but when we created a test to try to make
> it happen it never did, we assumed it was UDT related and lost interest,
> since it didn't have a big impact. I will try to carve some time to look
> into this some more and let you know if I find anything.
>
> On Wed, Oct 12, 2016 at 9:24 PM Vladimir Yudovin 
> wrote:
>>
>> The data is actually appended. not overwritten.
>> Strange, can you send exactly operators?
>>
>> Here is example I do:
>> CREATE KEYSPACE events WITH replication = {'class': 'SimpleStrategy',
>> 'replication_factor': 1};
>> CREATE TABLE events.data (id int primary key, events list);
>> INSERT INTO events.data (id, events) VALUES ( 0, ['a']);
>> SELECT * FROM events.data ;
>>  id | events
>> +
>>   0 |  ['a']
>>
>> (1 rows)
>>
>> INSERT INTO events.data (id, events) VALUES ( 0, ['b']);
>> SELECT * FROM events.data ;
>>  id | events
>> +
>>   0 |  ['b']
>>
>> (1 rows)
>>
>> As you see, 'a' was overwritten by 'b'
>>
>>
>> Best regards, Vladimir Yudovin,
>> Winguzone - Hosted Cloud Cassandra on Azure and SoftLayer.
>> Launch your cluster in minutes.
>>
>>
>>  On Wed, 12 Oct 2016 23:58:23 -0400Aoi Kadoya 
>> wrote 
>>
>> yes, that's what I thought. but, when I use these forms,
>> INSERT ... ['A']
>> INSERT ... ['B']
>>
>> The data is actually appended. not overwritten.
>> so I guess this is something unexpected?
>>
>> Thanks,
>> Aoi
>>
>> 2016-10-12 20:55 GMT-07:00 Vladimir Yudovin :
>> > If you use form
>> > INSERT ... ['A']
>> > INSERT ... ['B']
>> >
>> > latest INSERT will overwrite first, because this insert the whole list.
>> > It's
>> > better to use UPDATE like:
>> > UPDATE ... SET events = events + ['A']
>> > UPDATE ... SET events = events + ['B']
>> > These operations add new elements to the end of existing list.
>> >
>> >
>> > From here
>> > https://docs.datastax.com/en/cql/3.0/cql/cql_using/use_list_t.html
>> > :
>> >
>> > These update operations are implemented internally without any
>> > read-before-write. Appending and prepending a new element to the list
>> > writes
>> > only the new element.
>> >
>> >
>> > Best regards, Vladimir Yudovin,
>> > Winguzone - Hosted Cloud Cassandra on Azure and SoftLayer.
>> > Launch your cluster in minutes.
>> >
>> >
>> >  On Wed, 12 Oct 2016 17:39:46 -0400Aoi Kadoya 
>> > wrote 
>> >
>> > Hi,
>> >
>> > 

Re: corrupted gossip generation

[Yucheng Liu]

I resolved this by doing more rolling restarts on the nodes that had this
WARN it's just more restarts than I thought I would have to do.
 annoying!

On Wed, Oct 12, 2016 at 1:08 PM, Yucheng Liu  wrote:

> *Env: * apache cassandra 2.1.8, 6-nodes
>
> *Problem: *one node had kernel panic and crashed twice this morning.
>  seems gossip generation was messed up.  all nodes are flooded with
> "received an invalid gossip generation for peer" warning messages.
>  multiple rolling restarts only fixed "nodetool status".  the warning
> messages are still happening. and gossipinfo is showing "shutdown" for some
> nodes.
>
> *Question: *Does anyone know how to get rid of this warning messages?
> take the whole cluster down is not desired as this is production.
>
> *$ nodetool gossipinfo|grep STATUS*
>   STATUS:NORMAL,3074457345618258600
>   STATUS:NORMAL,-3074457345618258604
>   STATUS:NORMAL,6148914691236517202
>   STATUS:shutdown,true
>   STATUS:shutdown,true
>   STATUS:NORMAL,-9223372036854775808
>
> *system.log:*
>
> WARN  [GossipStage:1] 2016-10-12 09:58:02,913 Gossiper.java:1078 -
> received an invalid gossip generation for peer /10.150.12.118; local
> generation = 144263, received generation = 1476286723
>
> cqlsh> select gossip_generation from system.local;
>
> *1476286662   (I don't see where 144263 is from...)*
>
>
>

Re: cassandra dump file path

[Romain Hardouin]

Hi Jean,
I had the same problem, I removed the lines in /etc/init.d/cassandra template 
(we use Chef to deploy) and now the HeapDumpPath is not overridden anymore.The 
same goes for -XX:ErrorFile. 
Best,
Romain

Le Mardi 4 octobre 2016 9h25, Jean Carlo  a 
écrit :
 

 Yes, we did it. 

So if the parameter in cassandra-env.sh is used only if we have a OOM, what is 
for the definition of -XX:HeapDumpPath=/var/lib/ cassandra/java_1475461286. 
hprof in /etc/init.d/cassandra for?


Saludos

Jean Carlo
"The best way to predict the future is to invent it" Alan Kay

On Tue, Oct 4, 2016 at 2:58 AM, Yabin Meng  wrote:

Have you restarted Cassandra after making changes in cassandra-env.sh?
Yabin
On Mon, Oct 3, 2016 at 7:44 AM, Jean Carlo  wrote:

OK I got the response to one of my questions. In the script 
/etc/init.d/cassandra we set the path for the heap dump by default in the 
cassandra_home.

Now the thing I don't understand is, why do the dumps are located by the file 
set by /etc/init.d/cassandra and not by the  conf file cassandra-env.sh?

Anyone any idea?


Saludos

Jean Carlo
"The best way to predict the future is to invent it" Alan Kay

On Mon, Oct 3, 2016 at 12:00 PM, Jean Carlo  wrote:


Hi

I see in the log of my node cassandra that the parameter -XX:HeapDumpPath is 
charged two times.

INFO  [main] 2016-10-03 04:21:29,941 CassandraDaemon.java:205 - JVM Arguments: 
[-ea, -javaagent:/usr/share/cassandr a/lib/jamm-0.3.0.jar, 
-XX:+CMSClassUnloadingEnabled, -XX:+UseThreadPriorities, 
-XX:ThreadPriorityPolicy=42, -Xms6G, -Xmx6G, -Xmn600M, 
-XX:+HeapDumpOnOutOfMemoryErro r, -XX:HeapDumpPath=/cassandra/du 
mps/cassandra-1475461287-pid34 435.hprof, -Xss256k, 
-XX:StringTableSize=103, -XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, 
-XX:+CMSParallelRemarkEnabled, -XX:SurvivorRatio=8, -XX:MaxTenuringThreshold=1, 
-XX:CMSInitiatingOccupancyFrac tion=30, -XX:+UseCMSInitiatingOccupancy Only, 
-XX:+UseTLAB, -XX:CompileCommandFile=/etc/ca ssandra/hotspot_compiler, 
-XX:CMSWaitDuration=1, -XX:+CMSParallelInitialMarkEna bled, 
-XX:+CMSEdenChunksRecordAlways , -XX:CMSWaitDuration=1, 
-XX:+UseCondCardMark, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, 
-XX:+PrintGCApplicationStopped Time, -Xloggc:/var/opt/hosting/log/c 
assandra/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=20, 
-XX:GCLogFileSize=20M, -Djava.net.preferIPv4Stack=tru e, 
-Dcom.sun.management.jmxremote .port=7199, -Dcom.sun.management.jmxremote 
.rmi.port=7199, -Dcom.sun.management.jmxremote .ssl=false, 
-Dcom.sun.management.jmxremote .authenticate=false, 
-Dcom.sun.management.jmxremote .password.file=/etc/cassandra/ 
jmxremote.password, -Djava.io.tmpdir=/var/opt/host ing/db/cassandra/tmp, 
-javaagent:/usr/share/cassandr a/lib/jolokia-jvm-1.0.6-agent. 
jar=port=8778,host=0.0.0.0, -Dcassandra.auth_bcrypt_gensal t_log2_rounds=4, 
-Dlogback.configurationFile=lo gback.xml, -Dcassandra.logdir=/var/log/ca 
ssandra, -Dcassandra.storagedir=, -Dcassandra-pidfile=/var/run/c 
assandra/cassandra.pid, -XX:HeapDumpPath=/var/lib/cass 
andra/java_1475461286.hprof, -XX:ErrorFile=/var/lib/cassand 
ra/hs_err_1475461286.log]

This option is defined in cassandra-env.sh

if [ "x$CASSANDRA_HEAPDUMP_DIR" != "x" ]; then
    JVM_OPTS="$JVM_OPTS -XX:HeapDumpPath=$CASSANDRA_HE 
APDUMP_DIR/cassandra-`date +%s`-pid$$.hprof"
fi
 and we defined before the value of CASSANDRA_HEAPDUMP_DIR before to 
/cassandra/dumps/

It is seems that cassandra does not care about the conf in cassandra-env.sh and 
he only takes in account the last set for HeapDumpPath 
/var/lib/cassandra/java_147546 1286.hprof

This causes problems when we have to dump the heap because cassandra uses the 
disk not suitable to do it.

Is  XX:HeapDumpPath set in another place/file that I dont know?

Thxs

Jean Carlo
"The best way to predict the future is to invent it" Alan Kay








   

Re: Repair in Multi Datacenter - Should you use -dc Datacenter repair or repair with -pr

[Leena Ghatpande]

Thank you for the update.


The repair fails with the Error 'Failed Creating merkle tree' but does not give 
any additional details.


With -pr running on all DC nodes, we see a peer connection reset error, which 
then results in hanged repair process even though the TCP connection settings 
looks good on all nodes.



From: Anuj Wadehra 
Sent: Wednesday, October 12, 2016 2:41 PM
To: user
Subject: Re: Repair in Multi Datacenter - Should you use -dc Datacenter repair 
or repair with -pr

Hi Leena,

First thing you should be concerned about is : Why the repair -pr operation 
doesnt complete ?
Second comes the question : Which repair option is best?


One probable cause of stuck repairs is : if the firewall between DCs is closing 
TCP connections and Cassandra is trying to use such connections, repairs will 
hang. Please refer 
https://docs.datastax.com/en/cassandra/2.0/cassandra/troubleshooting/trblshootIdleFirewall.html
 . We faced that.

Also make sure you comply with basic bandwidth requirement between DCs. 
Recommended is 1000 Mb/s (1 gigabit) or greater.

Answers for specific questions:
1.As per my understanding, all replicas will not participate in dc local 
repairs and thus repair would be ineffective. You need to make sure that all 
replicas of a data in all dcs are in sync.

2. Every DC is not a ring. All DCs together form a token ring. So, I think yes 
you should run repair -pr on all nodes.

3. Yes. I dont have experience with incremental repairs. But you can run repair 
-pr on all nodes of all DCs.

Regarding Best approach of repair, you should see some repair presentations of 
Cassandra Summit 2016. All are online now.

I attended the summit and people using large clusters generally use sub range 
repairs to repair their clusters. But such large deployments are on older 
Cassandra versions and these deployments generally dont use vnodes. So people 
know easily which nodes hold which token range.



Thanks
Anuj



From: Leena Ghatpande ;
To: user@cassandra.apache.org ;
Subject: Repair in Multi Datacenter - Should you use -dc Datacenter repair or 
repair with -pr
Sent: Wed, Oct 12, 2016 2:15:51 PM


Please advice. Cannot find any clear documentation on what is the best strategy 
for repairing nodes on a regular basis with multiple datacenters involved.


We are running cassandra 3.7 in multi datacenter with 4 nodes in each data 
center. We are trying to run repairs every other night to keep the nodes in 
good state.We currently run repair with -pr option , but the repair process 
gets hung and does not complete gracefully. Dont see any errors in the logs 
either.


What is the best way to perform repairs on multiple data centers on large 
tables.

1. Can we run Datacenter repair using -dc option for each data center? Do we 
need to run repair on each node in that case or will it repair all nodes within 
the datacenter?

2. Is running repair with -pr across all nodes required , if we perform the 
step 1 every night?

3. Is cross data center repair required and if so whats the best option?


Thanks


Leena

Re: Cassandra data model right definition

[selcuk mart]

unsubscribe


3.10.2016 16:25 tarihinde Edward Capriolo yazdı:
The phrase is defensible, but that is the root of the problem. Take 
for example a skateboard.


"A skateboard is like a bike because it has wheels and you ride on it."

That is true and defensively true. :) However with not much more text 
you can accurately describe what it is, as opposed to something it is 
almost like.


"A skateboard is a thin piece of wood on top of four small wheels that 
you stand on and ride"


The old sentence Cassandra statement was something to the effect of 
"with the storage model of big table and the consistency model of 
dynamo". This accurately described the system and gave reference to 
specific known quantities (bigtable/dynamo) in which white papers 
existed for further reading.


On Mon, Oct 3, 2016 at 6:24 AM, Benedict Elliott Smith 
> wrote:


While that sentence leaves a lot to be desired (for me because it
confers a different meaning on row store), it doesn't say
"Cassandra is like a RDBMS" - it says "like an RDBMS, it organises
data by rows and columns" - i.e., in this regard only it is like
an RDBMS, not more generally.

I believe it was meant to help people, especially those afraid of
the NoSQL thrift world, understand that it still uses the basic
concept of a rows and columns they are used to.  I agree it could
be improved to minimise the chance of misreading it, and I'm
certain contributions would be welcome here.

I don't personally want to get bogged down in analysing every
piece of text anyone has ever written, so I'll bow out of further
discussion on this.  These phrases may all be suboptimal, but they
are certainly defensible.  Column store is not, that's all I
wanted to contribute here.





On 1 October 2016 at 19:35, Peter Lin > wrote:

I'll second Ed's comment.

The documentation should be more careful when using phrases
"like relational databases". When we look at the history of
relational databases, people expect certain things like ACID
transactions, primary/foriegn key constraints, query planners,
joins and relational algebra. Clearly Cassandra's storage
engine does not follow most of those principals for a good reason.

The term row oriented storage would be more descriptive and
appropriate. It avoids conflating Cassandra storage engine
with "traditional" relational storage engines. Those of us
that have spent over a decade using IBM DB2, Oracle, Sql
Server and Sybase tend to think of relational databases in a
certain way. If we go back to 1998, most RDBMS storage engine
had a max row size limit. Databases like Sybase before version
9 preferred RAW disk for optimal performance. I can go on and
on, but there's no point really.

Cassandra's storage engine is "row oriented", but it's not
relational in RDBMS sense. We do everyone a huge disservice by
using confusing terminology and then making fun of those who
get confused. No one wins when that happens. At the end of the
day, what differentiates cassandra's storage engine is it
support static and dynamic columns, which traditional RDBMS
don't support today. Calling Cassandra storage "distributed
tables" doesn't really help in my bias opinion.

For example, if you tell a SqlServer or Oracle RAC admin
"cassandra uses distributed tables" they might answer "so
what, sql server and oracle can do that too." The difference
is with RDBMS the partitioning is optional and requires more
work to configure. Whereas with Cassandra you can have
everything in 1 node, which means there is only 1 partition
and no different to 1 instance of sql server. Where you win is
when you need to add 2 more nodes, Cassandra makes this easier
whereas with SqlServer and Oracle you have to do a little bit
more work. I've lost count of how many times I've to explained
noSql databases to RDBMS admins and had to explain the
official docs are stupid.



On Sat, Oct 1, 2016 at 11:31 AM, Edward Capriolo
> wrote:

https://github.com/apache/cassandra


Row store
 means that
like relational databases, Cassandra organizes data by
rows and columns. The Cassandra Query Language (CQL) is a
close relative of SQL.

I generally do not know what to say about these high level
"oversimplifications" like "firewalls block hackers". Are
there "firewalls" or do they mean IP routers 

Scenarios when blocking read repair takes place

[siddharth verma]

Hi,
Does blocking read repair take place only when we read on the primary key or
does it take place in the following scenarios as well?

Consistemcy ALL
1. select * from ks.table_name
2. select * from ks.table_name where token(pk) >= ? and token(pk) <= ?

While using manual paging or automatic paging in either of the scenarios.

Thanks
Siddharth Verma
(Visit https://github.com/siddv29/cfs for a high speed cassandra full table
scan)