[
https://issues.apache.org/jira/browse/CASSANDRA-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202070#comment-15202070
]
DOAN DuyHai commented on CASSANDRA-11383:
-----------------------------------------
I try to create indices ONE by ONE, but same result.
new JVM settings:
- CMS
- Xms8G, Xmx8G
after some time, the node goes long Old GC and then crashes:
{noformat}
632936; Par Survivor Space: 125829120 -> 120349752
WARN [Service Thread] 2016-03-18 21:02:45,237 GCInspector.java:282 -
ConcurrentMarkSweep GC in 10417ms. Par Eden Space: 1006632952 -> 1006632928;
Par Survivor Space: 125829120 -> 120921320
WARN [Service Thread] 2016-03-18 21:02:56,969 GCInspector.java:282 -
ConcurrentMarkSweep GC in 11675ms. Par Survivor Space: 125829112 -> 121748224
WARN [Service Thread] 2016-03-18 21:03:07,359 GCInspector.java:282 -
ConcurrentMarkSweep GC in 10327ms. CMS Old Gen: 7331643344 -> 7331643392; Par
Eden Space: 1006632960 -> 1006632864; Par Survivor Space: 125828928 -> 122147720
WARN [Service Thread] 2016-03-18 21:03:50,019 GCInspector.java:282 -
ConcurrentMarkSweep GC in 42574ms. CMS Old Gen: 7331643392 -> 7331643368; Par
Eden Space: 1006632960 -> 1006632824; Par Survivor Space: 125829120 -> 122651640
WARN [Service Thread] 2016-03-18 21:05:01,704 GCInspector.java:282 -
ConcurrentMarkSweep GC in 71592ms. Par Eden Space: 1006632960 -> 1006632928;
Par Survivor Space: 125829112 -> 123069400
{noformat}
> SASI index build leads to massive OOM
> -------------------------------------
>
> Key: CASSANDRA-11383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11383
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
> Environment: C* 3.4
> Reporter: DOAN DuyHai
> Attachments: system.log_sasi_build_oom
>
>
> 13 bare metal machines
> - 6 cores CPU (12 HT)
> - 64Gb RAM
> - 4 SSD in RAID0
> JVM settings:
> - G1 GC
> - Xms32G, Xmx32G
> Data set:
> - ≈ 100Gb/per node
> - 1.3 Tb cluster-wide
> - ≈ 20Gb for all SASI indices
> C* settings:
> - concurrent_compactors: 1
> - compaction_throughput_mb_per_sec: 256
> - memtable_heap_space_in_mb: 2048
> - memtable_offheap_space_in_mb: 2048
> I created 9 SASI indices
> - 8 indices with text field, NonTokenizingAnalyser, PREFIX mode,
> case-insensitive
> - 1 index with numeric field, SPARSE mode
> After a while, the nodes just gone OOM.
> I attach log files. You can see a lot of GC happening while index segments
> are flush to disk. At some point the node OOM ...
> /cc [~xedin]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
