Every time a repair creates a new SSTable segment you'll get another index build compaction for the keys in that segment.
On Tue, Apr 17, 2012 at 8:32 AM, Maxim Potekhin <potek...@bnl.gov> wrote: > The "offending" CF only has one. The other one, that seems to behave well, > has nine. > > Maxim > > > > On 4/17/2012 10:20 AM, Jake Luciani wrote: > > How many indexes are there? > > On Tue, Apr 17, 2012 at 10:16 AM, Maxim Potekhin <potek...@bnl.gov> wrote: > >> Yes. Sorry I didn't mention this, but of course I'm checking on indexes >> once in a while. >> So yes, they are marked as built. >> >> All of this started happening after a few days of continuous loading >> process. Since >> the nodes have good hardware (24 cores + SSD), the apparent load on each >> node >> was nothing remarkable, even at 20kHz insertion rate. But maybe I'm being >> overoptimistic. >> >> Maxim >> >> >> >> On 4/17/2012 10:12 AM, Jake Luciani wrote: >> >> Hmm that does sound fishy. >> >> When you run show keyspaces from cassandra-cli it shows which indexes >> are built. Are they marked built in your column family? >> >> -Jake >> >> On Tue, Apr 17, 2012 at 10:09 AM, Maxim Potekhin <potek...@bnl.gov>wrote: >> >>> I understand that indexes are CFs. But the compaction stats says it's >>> building the >>> index, not compacting the corresponding CF. Either that's an ambiguous >>> diagnostic, >>> or indeed something is not right with my rig as of late. >>> >>> Maxim >>> >>> >>> >>> >>> On 4/17/2012 10:05 AM, Jake Luciani wrote: >>> >>> Well, the since the secondary indexes are themselves >>> column families they too are compacted along with everything else. >>> >>> On Tue, Apr 17, 2012 at 10:02 AM, Maxim Potekhin <potek...@bnl.gov>wrote: >>> >>>> Thanks Jake. Then I am definitely seeing weirdness, as there are tons >>>> of >>>> "pending tasks" in compaction stats, and tons of index files created in >>>> the >>>> data directory. Plus it does tell me that it is building the secondary >>>> index, >>>> and that seems to be happening at an amazingly glacial pace. >>>> >>>> I have 2 CFs there, with multiple secondary indexes. I'll try >>>> to compact the CF one by one, reboot and see if that helps. >>>> >>>> Maxim >>>> >>>> >>>> >>>> On 4/17/2012 9:53 AM, Jake Luciani wrote: >>>> >>>> No, the indexes are not rebuilt every compaction. Only if you manually >>>> rebuild or bootstrap a new node does it use compaction manager to rebuild. >>>> >>>> On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin <potek...@bnl.gov>wrote: >>>> >>>>> Thanks Aaaron. Just to be clear, every time I do a compaction, >>>>> I rebuild all indexes from scratch. Right? >>>>> >>>>> Maxim >>>>> >>>>> >>>>> >>>>> On 4/17/2012 6:16 AM, aaron morton wrote: >>>>> >>>>> Yes secondary index builds are done via the compaction manager. >>>>> >>>>> Cheers >>>>> >>>>> ----------------- >>>>> Aaron Morton >>>>> Freelance Developer >>>>> @aaronmorton >>>>> http://www.thelastpickle.com >>>>> >>>>> On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote: >>>>> >>>>> I noticed that "nodetool compactionstats" shows the building of the >>>>> secondary index while >>>>> I initiate compaction. Is this to be expected? Cassandra version 0.8.8. >>>>> >>>>> Thank you >>>>> >>>>> Maxim >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> http://twitter.com/tjake >>>> >>>> >>>> >>> >>> >>> -- >>> http://twitter.com/tjake >>> >>> >>> >> >> >> -- >> http://twitter.com/tjake >> >> >> > > > -- > http://twitter.com/tjake > > > -- Ben Coverston DataStax -- The Apache Cassandra Company
