Size of a single Data Row?

2018-06-10 Thread Ralph Soika
Hi, I have a general question concerning the Cassandra technology. I already read 2 books but after all I am more and more confused about the question if Cassandra is the right technology. My goal is to store Business Data form a workflow engine into Cassandra. I want to use Cassandra as a

Re: data consistency without using nodetool repair

2018-06-10 Thread Jeff Jirsa
If you have two replicas A and B, and you write at ONE A acknowledged the write, B is running but drops write, the write succeeds A fails 30 seconds later. The data is lost because it had no chance to be repaired to the other host via hints / repair / readrepair 10 days is a big window -

Re: Size of a single Data Row?

2018-06-10 Thread Ralph Soika
Hi Eevee, thanks for your response. Low latency is not an issue because I do read only in rarely cases and also I write rarely cases. But for me it is important to have a high data consistency over a decentralized cluster. And Cassandra fills that perfectly. Hadoop is much more complex in

Re: Size of a single Data Row?

2018-06-10 Thread daemeon reiydelle
I'd like to split your question into two parts. Part one is around recovery. If you lose a copy of the underlying data because a note fails and let's assume you have three copies, how long can you tolerate the time to restore the third copy? The second question is about the absolute length of a

Re: Size of a single Data Row?

2018-06-10 Thread Evelyn Smith
Hi Ralph, Yes, having partitions of 100mb will seriously hit your performance. But usually the issue here is for people handling large numbers of transactions and aiming for low latency. My understanding is the column value up to 2GB is it’s max. Like after that the system would start to fail,

Re: Size of a single Data Row?

2018-06-10 Thread Jeff Jirsa
Let's talk about what the real limitations are. There are two here that you should care about: 1) Cassandra runs in the JVM. When you read and write to Cassandra, those objects end up in the heap as byte arrays. If you're regularly reading and writing 100MB byte arrays, it's easy to see

Re: Size of a single Data Row?

2018-06-10 Thread Ralph Soika
Thanks for your answer. Ok - I think I understand your points and the worries you have about my architecture. To give more inside information: We are working on the Open Source Project Imixs-Workflow . This is a human-centric workflow engine based on Java EE. The engine