Re: Size of a single Data Row?

2018-06-11 Thread Ralph Soika
Hi Jeff, thanks for that answer. I understand the problem now much better. As you explain the problem also exists in the VM and so also in the 'other' part of my application which is running on JavaEE/JPA. At the end the 100MB byte arrays also cause a HeapSpace problem there. So Cassandra is

Re: Size of a single Data Row?

2018-06-10 Thread Jeff Jirsa
Let's talk about what the real limitations are. There are two here that you should care about: 1) Cassandra runs in the JVM. When you read and write to Cassandra, those objects end up in the heap as byte arrays. If you're regularly reading and writing 100MB byte arrays, it's easy to see

Re: Size of a single Data Row?

2018-06-10 Thread Ralph Soika
Thanks for your answer. Ok - I think I understand your points and the worries you have about my architecture. To give more inside information: We are working on the Open Source Project Imixs-Workflow . This is a human-centric workflow engine based on Java EE. The engine

Re: Size of a single Data Row?

2018-06-10 Thread daemeon reiydelle
I'd like to split your question into two parts. Part one is around recovery. If you lose a copy of the underlying data because a note fails and let's assume you have three copies, how long can you tolerate the time to restore the third copy? The second question is about the absolute length of a

Re: Size of a single Data Row?

2018-06-10 Thread Ralph Soika
Hi Eevee, thanks for your response. Low latency is not an issue because I do read only in rarely cases and also I write rarely cases. But for me it is important to have a high data consistency over a decentralized cluster. And Cassandra fills that perfectly. Hadoop is much more complex in

Re: Size of a single Data Row?

2018-06-10 Thread Evelyn Smith
Hi Ralph, Yes, having partitions of 100mb will seriously hit your performance. But usually the issue here is for people handling large numbers of transactions and aiming for low latency. My understanding is the column value up to 2GB is it’s max. Like after that the system would start to fail,

Size of a single Data Row?

2018-06-10 Thread Ralph Soika
Hi, I have a general question concerning the Cassandra technology. I already read 2 books but after all I am more and more confused about the question if Cassandra is the right technology. My goal is to store Business Data form a workflow engine into Cassandra. I want to use Cassandra as a