Re: Is the secondary index re-built under compaction?

2012-04-17 Thread aaron morton
Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote: I noticed that nodetool compactionstats shows the building of the secondary

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Maxim Potekhin
Thanks Aaaron. Just to be clear, every time I do a compaction, I rebuild all indexes from scratch. Right? Maxim On 4/17/2012 6:16 AM, aaron morton wrote: Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Jake Luciani
No, the indexes are not rebuilt every compaction. Only if you manually rebuild or bootstrap a new node does it use compaction manager to rebuild. On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin potek...@bnl.gov wrote: Thanks Aaaron. Just to be clear, every time I do a compaction, I rebuild

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Maxim Potekhin
Thanks Jake. Then I am definitely seeing weirdness, as there are tons of pending tasks in compaction stats, and tons of index files created in the data directory. Plus it does tell me that it is building the secondary index, and that seems to be happening at an amazingly glacial pace. I have 2

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Jake Luciani
Well, the since the secondary indexes are themselves column families they too are compacted along with everything else. On Tue, Apr 17, 2012 at 10:02 AM, Maxim Potekhin potek...@bnl.gov wrote: Thanks Jake. Then I am definitely seeing weirdness, as there are tons of pending tasks in compaction

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Maxim Potekhin
I understand that indexes are CFs. But the compaction stats says it's building the index, not compacting the corresponding CF. Either that's an ambiguous diagnostic, or indeed something is not right with my rig as of late. Maxim On 4/17/2012 10:05 AM, Jake Luciani wrote: Well, the since the

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Jake Luciani
Hmm that does sound fishy. When you run show keyspaces from cassandra-cli it shows which indexes are built. Are they marked built in your column family? -Jake On Tue, Apr 17, 2012 at 10:09 AM, Maxim Potekhin potek...@bnl.gov wrote: I understand that indexes are CFs. But the compaction stats

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Maxim Potekhin
Yes. Sorry I didn't mention this, but of course I'm checking on indexes once in a while. So yes, they are marked as built. All of this started happening after a few days of continuous loading process. Since the nodes have good hardware (24 cores + SSD), the apparent load on each node was

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Jake Luciani
How many indexes are there? On Tue, Apr 17, 2012 at 10:16 AM, Maxim Potekhin potek...@bnl.gov wrote: Yes. Sorry I didn't mention this, but of course I'm checking on indexes once in a while. So yes, they are marked as built. All of this started happening after a few days of continuous

Re: Is the secondary index re-built under compaction?

2012-04-17 Thread Maxim Potekhin
The offending CF only has one. The other one, that seems to behave well, has nine. Maxim On 4/17/2012 10:20 AM, Jake Luciani wrote: How many indexes are there? On Tue, Apr 17, 2012 at 10:16 AM, Maxim Potekhin potek...@bnl.gov mailto:potek...@bnl.gov wrote: Yes. Sorry I didn't mention

Is the secondary index re-built under compaction?

2012-04-16 Thread Maxim Potekhin
I noticed that nodetool compactionstats shows the building of the secondary index while I initiate compaction. Is this to be expected? Cassandra version 0.8.8. Thank you Maxim