Self-healing data integrity?

2017-09-09 Thread Ralph Soika
Hi, I am searching for a big data storage solution for the Imixs-Workflow project. I started with Hadoop until I became aware of the 'small-file-problem'. So I am considering using Cassandra now. But Hadoop has one important feature for me. The replicator continuously examines whether data

Re: Self-healing data integrity?

2017-09-09 Thread Jeff Jirsa
Cassandra doesn't do that automatically - it can guarantee consistency on read or write via ConsistencyLevel on each query, and it can run active (AntiEntropy) repairs. But active repairs must be scheduled (by human or cron or by third party script like http://cassandra-reaper.io/), and to be

Re: New Column addition and Locking

2017-09-09 Thread Myrle Krantz
There's a tool in the early stages of development here: https://github.com/mifosio/cassandra/tree/develop/src/main/java/io/mifos/core/cassandra/core That is part of a mass of code which is currently under (slow) discussion to be added to the Apache Fineract project. ALV2 license. I wouldn't be

Re: Self-healing data integrity?

2017-09-09 Thread Jeff Jirsa
There is, but they aren't consulted on the streaming paths (only on normal reads) -- Jeff Jirsa > On Sep 9, 2017, at 12:02 PM, DuyHai Doan wrote: > > Jeff, > > With default compression enabled on each table, isn't there CRC files > created along side with SSTables

Re: Lightweight transaction in Multi DC

2017-09-09 Thread kurt greaves
Yes it will "slow down" as more nodes need to be involved. Yes you will need to use SERIAL for both reads and writes. On 9 Sep. 2017 08:49, "Charulata Sharma (charshar)" wrote: > Thanks for your reply. I understand that LOCAL_SERIAL is for within a DC , > will setting up

Re: Self-healing data integrity?

2017-09-09 Thread DuyHai Doan
Jeff, With default compression enabled on each table, isn't there CRC files created along side with SSTables that can help detecting bit-rot ? On Sat, Sep 9, 2017 at 7:50 PM, Jeff Jirsa wrote: > Cassandra doesn't do that automatically - it can guarantee consistency on >

Re: Self-healing data integrity?

2017-09-09 Thread Jeff Jirsa
(Which isn't to say that someone shouldn't implement this; they should, and there's probably a JIRA to do so already written, but it's a project of volunteers, and nobody has volunteered to do the work yet) -- Jeff Jirsa > On Sep 9, 2017, at 12:59 PM, Jeff Jirsa wrote: >

Re: Lightweight transaction in Multi DC

2017-09-09 Thread DuyHai Doan
Using SERIAL is of course much more expensive, but then the trade-off is that you are guaranteed to have linearizability cross data-centers. Please note that when using Lightweight Transactions, there 2 two distinct consistency levels to be set: 1) The Paxos phase consistency level: SERIAL or