?????? secondary index creation causes C* oom

[Peng Xiao]

Thanks Kurt.




--  --
??: "kurt";;
: 2018??1??11??(??) 11:46
??: "User";

: Re: secondary index creation causes C* oom




1.not sure if secondary index creation is the same as index rebuild

Fairly sure they are the same. 
2.we noticed that the memory table flush looks still working,not the same as 
CASSANDRA-12796 mentioned,but the compactionExecutor pending is increasing.

Do you by chance have concurrent_compactors only set to 2? Seems like your 
index builds are blocking other compactions from taking place.
Seems that maybe postflush is backed up because it's blocked on writes 
generated from the rebuild? Maybe, anyway.
3.I'm wondering if the block only blocks the specified table which is creating 
secondary index?

If all your flush writers/post flushers are blocked I assume no other flushes 
will be able to take place, regardless of table. 


Seems like CASSANDRA-12796 is related but not sure why it didn't get fixed in 
2.1.

Re: secondary index creation causes C* oom

[kurt greaves]

> 1.not sure if secondary index creation is the same as index rebuild
>
Fairly sure they are the same.

> 2.we noticed that the memory table flush looks still working,not the same
> as CASSANDRA-12796 mentioned,but the compactionExecutor pending is
> increasing.
>
Do you by chance have concurrent_compactors only set to 2? Seems like your
index builds are blocking other compactions from taking place.
Seems that maybe postflush is backed up because it's blocked on writes
generated from the rebuild? Maybe, anyway.

> 3.I'm wondering if the block only blocks the specified table which is
> creating secondary index?
>
If all your flush writers/post flushers are blocked I assume no other
flushes will be able to take place, regardless of table.

Seems like CASSANDRA-12796
 is related but not
sure why it didn't get fixed in 2.1.

secondary index creation causes C* oom

[Peng Xiao]

Dear All,


We met some C* nodes oom during secondary index creation with C* 2.1.18.
As per https://issues.apache.org/jira/browse/CASSANDRA-12796,the flush writer 
will be blocked by index rebuild.but we still have some confusions:
1.not sure if secondary index creation is the same as index rebuild
2.we noticed that the memory table flush looks still working,not the same as 
CASSANDRA-12796 mentioned,but the compactionExecutor pending is increasing.
3.I'm wondering if the block only blocks the specified table which is creating 
secondary index?


Could anyone please explain?





Thanks