Re: Controlling the MAX SIZE of sstables after compaction

2015-01-28 Thread Daniel Chia
gt; wrote: > >> I believe Aegisthus is open sourced. >> >> >> >> Mohammed >> >> >> >> *From:* Jan [mailto:cne...@yahoo.com] >> *Sent:* Monday, January 26, 2015 11:20 AM >> *To:* user@cassandra.apache.org >> *Subject:* Re: Con

Re: Controlling the MAX SIZE of sstables after compaction

2015-01-27 Thread Mikhail Strebkov
ry 26, 2015 11:20 AM > *To:* user@cassandra.apache.org > > *Subject:* Re: Controlling the MAX SIZE of sstables after compaction > > > > Parth et al; > > > > the folks at Netflix seem to have built a solution for your problem. > > The Netflix Tech Blog: Aegisthus - A

RE: Controlling the MAX SIZE of sstables after compaction

2015-01-27 Thread Mohammed Guller
I believe Aegisthus is open sourced. Mohammed From: Jan [mailto:cne...@yahoo.com] Sent: Monday, January 26, 2015 11:20 AM To: user@cassandra.apache.org Subject: Re: Controlling the MAX SIZE of sstables after compaction Parth et al; the folks at Netflix seem to have built a solution for your

Re: Controlling the MAX SIZE of sstables after compaction

2015-01-26 Thread Jan
Parth  et al;  the folks at Netflix seem to have built a solution for your problem. The Netflix Tech Blog: Aegisthus - A Bulk Data Pipeline out of Cassandra |   | |   | |   |   |   |   |   | | The Netflix Tech Blog: Aegisthus - A Bulk Data Pipeline ...By Charles Smith and Jeff Magnusson | | |

Re: Controlling the MAX SIZE of sstables after compaction

2015-01-26 Thread Robert Coli
On Sun, Jan 25, 2015 at 10:40 PM, Parth Setya wrote: > 1. Is there a way to configure the size of sstables created after > compaction? > No, won'tfix : https://issues.apache.org/jira/browse/CASSANDRA-4897. You could use the "sstablesplit" utility on your One Big SSTable to split it into files o

Re: Controlling the MAX SIZE of sstables after compaction

2015-01-26 Thread Eric Stevens
If you're concerned about impacting production performance, the steps of compacting and sstable2json will almost certainly also cause performance problems if performed on the same hardware. You won't get away from a production performance impact as long as you're using production hardware. If you

Re: Controlling the MAX SIZE of sstables after compaction

2015-01-26 Thread Jens Rantil
Parth, > So are you saying that I should query cassandra right away? Well, don’t take my word for it, but it definitely sounds like a more simple approach. > If yes, like I mentioned, I have to run this during traffic hours. Isnt there > a possibility then that my traffic to the db m

Re: Controlling the MAX SIZE of sstables after compaction

2015-01-26 Thread Parth Setya
hey Jens Thank you so much for the advise and reading through. So are you saying that I should query cassandra right away? If yes, like I mentioned, I have to run this during traffic hours. Isnt there a possibility then that my traffic to the db may get impacted? also is it okay to use hector to t

Re: Controlling the MAX SIZE of sstables after compaction

2015-01-26 Thread Jens Rantil
Hi Parth, I’ll take your questions in order: 1. Have a look at the compaction subproperties for STCS: http://datastax.com/documentation/cql/3.1/cql/cql_reference/compactSubprop.html 2. Why not talk to Cassandra when generating the report? It will be waaay faster (and easier!); Cassandra

Controlling the MAX SIZE of sstables after compaction

2015-01-25 Thread Parth Setya
Hi *Setup* *3 Node Cluster* Api- * Hector*CL- * QUORUM* RF- *3* Compaction Strategy- *Size Tiered Compaction* *Use Case* I have about *320 million rows*(~12 to 15 columns each) worth of data stored in Cassandra. In order to generate a report containing ALL that data, I do the following: 1. Run