[
https://issues.apache.org/jira/browse/CASSANDRA-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17183177#comment-17183177
]
ZhaoYang edited comment on CASSANDRA-16071 at 8/24/20, 11:57 AM:
-----------------------------------------------------------------
{code:java}
long maxMemMb = indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION) == null
? (long) (1048576 * INDEX_MAX_FLUSH_DEFAULT_MULTIPLIER) // 1G default for
memtable
: Long.parseLong(indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION));
{code}
[~mck] I think the default {{"1048576 * 0.15 mb" (153GB)}} may still cause OOM.
How about we rename {{"maxMemMb"}} to {{"maxMemBytes"}} and rename
{{"IndexMode#maxCompactionFlushMemoryInMb"}} to
{{"maxCompactionFlushMemoryInBytes"}}. So it should be :
{code:java}
long maxMemBytes = indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION) == null
? (long) (1073741824 * INDEX_MAX_FLUSH_DEFAULT_MULTIPLIER) // 1G default for
memtable
: 1048576L * Long.parseLong(indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION));
{code}
was (Author: jasonstack):
{code:java}
long maxMemMb = indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION) == null
? (long) (1048576 * INDEX_MAX_FLUSH_DEFAULT_MULTIPLIER) // 1G default for
memtable
: Long.parseLong(indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION));
{code}
[~mck] I think the default {{"1048576 * 0.15 mb" (153GB)}} may still cause OOM.
How about we rename {{"maxMemMb"}} to {{"maxMemBytes" and rename
{{"IndexMode#maxCompactionFlushMemoryInMb"}} to
{{"maxCompactionFlushMemoryInBytes"}}. So it should be :
{code:java}
long maxMemBytes = indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION) == null
? (long) (1073741824 * INDEX_MAX_FLUSH_DEFAULT_MULTIPLIER) // 1G default for
memtable
: 1048576L * Long.parseLong(indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION));
{code}
> max_compaction_flush_memory_in_mb is interpreted as bytes
> ---------------------------------------------------------
>
> Key: CASSANDRA-16071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16071
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SASI
> Reporter: Michael Semb Wever
> Assignee: Michael Semb Wever
> Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> In CASSANDRA-12662, [~scottcarey]
> [reported|https://issues.apache.org/jira/browse/CASSANDRA-12662?focusedCommentId=17070055&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17070055]
> that the {{max_compaction_flush_memory_in_mb}} setting gets incorrectly
> interpreted in bytes rather than megabytes as its name implies.
> {quote}
> 1. the setting 'max_compaction_flush_memory_in_mb' is a misnomer, it is
> actually memory in BYTES. If you take it at face value, and set it to say,
> '512' thinking that means 512MB, you will produce a million temp files
> rather quickly in a large compaction, which will exhaust even large values of
> max_map_count rapidly, and get the OOM: Map Error issue above and possibly
> have a very difficult situation to get a cluster back into a place where
> nodes aren't crashing while initilaizing or soon after. This issue is minor
> if you know about it in advance and set the value IN BYTES.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
