Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Ben Coverston
Every time a repair creates a new SSTable segment you'll get another index build compaction for the keys in that segment. On Tue, Apr 17, 2012 at 8:32 AM, Maxim Potekhin wrote: > The "offending" CF only has one. The other one, that seems to behave well, > has nine. > > Maxim > > > > On 4/17/201

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Maxim Potekhin
The "offending" CF only has one. The other one, that seems to behave well, has nine. Maxim On 4/17/2012 10:20 AM, Jake Luciani wrote: How many indexes are there? On Tue, Apr 17, 2012 at 10:16 AM, Maxim Potekhin > wrote: Yes. Sorry I didn't mention this, but of c

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Jake Luciani
How many indexes are there? On Tue, Apr 17, 2012 at 10:16 AM, Maxim Potekhin wrote: > Yes. Sorry I didn't mention this, but of course I'm checking on indexes > once in a while. > So yes, they are marked as built. > > All of this started happening after a few days of continuous loading > process

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Maxim Potekhin
Yes. Sorry I didn't mention this, but of course I'm checking on indexes once in a while. So yes, they are marked as built. All of this started happening after a few days of continuous loading process. Since the nodes have good hardware (24 cores + SSD), the apparent load on each node was nothi

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Jake Luciani
Hmm that does sound fishy. When you run show keyspaces from cassandra-cli it shows which indexes are built. Are they marked built in your column family? -Jake On Tue, Apr 17, 2012 at 10:09 AM, Maxim Potekhin wrote: > I understand that indexes are CFs. But the compaction stats says it's > bui

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Maxim Potekhin
I understand that indexes are CFs. But the compaction stats says it's building the index, not compacting the corresponding CF. Either that's an ambiguous diagnostic, or indeed something is not right with my rig as of late. Maxim On 4/17/2012 10:05 AM, Jake Luciani wrote: Well, the since the

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Jake Luciani
Well, the since the secondary indexes are themselves column families they too are compacted along with everything else. On Tue, Apr 17, 2012 at 10:02 AM, Maxim Potekhin wrote: > Thanks Jake. Then I am definitely seeing weirdness, as there are tons of > "pending tasks" in compaction stats, and t

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Maxim Potekhin
Thanks Jake. Then I am definitely seeing weirdness, as there are tons of "pending tasks" in compaction stats, and tons of index files created in the data directory. Plus it does tell me that it is building the secondary index, and that seems to be happening at an amazingly glacial pace. I have

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Jake Luciani
No, the indexes are not rebuilt every compaction. Only if you manually rebuild or bootstrap a new node does it use compaction manager to rebuild. On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin wrote: > Thanks Aaaron. Just to be clear, every time I do a compaction, > I rebuild all indexes from

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Maxim Potekhin
Thanks Aaaron. Just to be clear, every time I do a compaction, I rebuild all indexes from scratch. Right? Maxim On 4/17/2012 6:16 AM, aaron morton wrote: Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton htt

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread aaron morton
Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote: > I noticed that "nodetool compactionstats" shows the building of the secondary

Is the secondary index re-built under compaction?

2012-04-16 Thread Maxim Potekhin
I noticed that "nodetool compactionstats" shows the building of the secondary index while I initiate compaction. Is this to be expected? Cassandra version 0.8.8. Thank you Maxim