unit test failing when pull is taken

2015-06-04 Thread Amit Singh F
Hi All, I have taken pull from Cassandra branch and when I try to run test cases , they start failing(around 13 test suites are failing). Below are some traces of some test suite which are failing : * [junit] Testsuite: org.apache.cassandra.db.compaction.BlacklistingCompactionsTest

Re: Throttle Heavy Read / Write Loads

2015-06-04 Thread Anishek Agarwal
may be just increase the read and write timeouts at cassandra currently at 5 sec i think. i think the datastax java client driver provides ability to say how many max requests per connection are to be sent, you can try and lower that to limit excessive requests along with limiting the number of con

Re: Reading too many tombstones

2015-06-04 Thread Alain RODRIGUEZ
Actually what happen is that STC as well as LCS mix old and fresh data during the compaction process. So all the fragments of the same row that you deleted (or reached the TTL of), are spread among multiple sstables. The point is that they need to be gathered all in the same compaction to be reall

Re: Reading too many tombstones

2015-06-04 Thread Sebastian Estevez
Check out the compaction subproperties for tombstones. http://docs.datastax.com/en/cql/3.1/cql/cql_reference/compactSubprop.html?scroll=compactSubprop__compactionSubpropertiesDTCS On Jun 4, 2015 1:29 PM, "Aiman Parvaiz" wrote: > Thanks Carlos for pointing me in that direction, I have some intere

Re: Reading too many tombstones

2015-06-04 Thread Aiman Parvaiz
Thanks Carlos for pointing me in that direction, I have some interesting findings to share. So in December last year there was a redesign of home_feed and it was migrated to a new CF. Initially all the data in home_feed had a TTL of 1 year but migrated data was inserted with TTL of 30days. Now on d

Re: Reading too many tombstones

2015-06-04 Thread Carlos Rolo
The TTL data will only be removed after the gc_grace_seconds. So your data with 30 days TTL will be still in Cassandra for 10 days more (40 in total). Is your data being there for more than that? Otherwise it is expected behaviour and probably you should do something on your data model to avoid sca

Throttle Heavy Read / Write Loads

2015-06-04 Thread Anuj Wadehra
We are using Cassandra 2.0.14 with Hector as client ( will be gradually moving to CQL Driver ).  Often we see that heavy read and write loads lead to Cassandra timeouts and unpredictable results due to gc pauses and request timeouts. We need to know the best way to throttle read and write load

Re: Reading too many tombstones

2015-06-04 Thread Aiman Parvaiz
yeah we don't update old data. One thing I am curious about is why are we running in to so many tombstones with compaction happening normally. Is compaction not removing tombstomes? On Thu, Jun 4, 2015 at 11:25 AM, Jonathan Haddad wrote: > DateTiered is fantastic if you've got time series, TTLed

Re: Reading too many tombstones

2015-06-04 Thread Jonathan Haddad
DateTiered is fantastic if you've got time series, TTLed data. That means no updates to old data. On Thu, Jun 4, 2015 at 10:58 AM Aiman Parvaiz wrote: > Hi everyone, > We are running a 10 node Cassandra 2.0.9 without vnode cluster. We are > running in to a issue where we are reading too many to

Reading too many tombstones

2015-06-04 Thread Aiman Parvaiz
Hi everyone, We are running a 10 node Cassandra 2.0.9 without vnode cluster. We are running in to a issue where we are reading too many tombstones and hence getting tons of WARN messages and some ERROR query aborted. cass-prod4 2015-06-04 14:38:34,307 WARN ReadStage:

Re: sstableloader usage doubts

2015-06-04 Thread Robert Coli
On Thu, Jun 4, 2015 at 5:39 AM, ZeroUno wrote: > while defining backup and restore procedures for a Cassandra cluster I'm > trying to use sstableloader for restoring a snapshot from a backup, but I'm > not sure I fully understand the documentation on how it should be used. > http://www.pythian.c

Re: sstableloader usage doubts

2015-06-04 Thread Sebastian Estevez
You don't need sstable loader if your topology hasn't changed and you have all your sstables backed up for each node. SStableloader actually streams data to all the nodes in a ring (this is what OpsCenter backup restore does). So you can actually restore to a larger or smaller cluster or a cluster

sstableloader usage doubts

2015-06-04 Thread ZeroUno
Hi, while defining backup and restore procedures for a Cassandra cluster I'm trying to use sstableloader for restoring a snapshot from a backup, but I'm not sure I fully understand the documentation on how it should be used. Looking at the examples in the doc at http://docs.datastax.com/en/ca

com/datastax/driver/core/policies/LoadBalancingPolicy

2015-06-04 Thread Marko Dinic
Hello everyone, I'm new to Cassandra and I'm trying to use it as input for Hadoop. For some reason I'm getting the following exception while trying to use Cassandra as input to Hadoop Exception in thread "main" java.lang.NoClassDefFoundError: com/datastax/driver/core/policies/LoadBalancingPo

RE: Different number of records from COPY command

2015-06-04 Thread Vanlerberghe, Luc
You’re probably hitting https://issues.apache.org/jira/browse/CASSANDRA-8940: Inconsistent select count and select distinct It’s resolved (as I understand, a non-thread-safe object was shared between threads) and the patch will be included in 2.1.6 and 2.0.16 It’s a showstopper for me too: while