[
https://issues.apache.org/jira/browse/ACCUMULO-4083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated ACCUMULO-4083:
---------------------------------
Description:
Noticed that the TServer tries to fail fast if caches and memory maps are
configured to use more memory than the JVM has given to it via -Xmx. However,
if native maps are being used, this check fails. I recently had OOMEs kill
TServers because their data block cache was 16gb and the max heap was 16gb.
Here's the check:
{code:title=TabletServerResourceManager.java}
Runtime runtime = Runtime.getRuntime();
if (!usingNativeMap && maxMemory + dCacheSize + iCacheSize >
runtime.maxMemory()) {
throw new IllegalArgumentException(String.format(
"Maximum tablet server map memory %,d and block cache sizes %,d is
too large for this JVM configuration %,d", maxMemory, dCacheSize + iCacheSize,
runtime.maxMemory()));
}
{code}
I think the check should still happen for index and data caches if native maps
are being used.
was:
Noticed that the TServer tries to fail fast if caches and memory maps are
configured to use more memory than the JVM has given to it via -Xmx. However,
if native maps are being used, this check fails. I recently had OOMEs kill
TServers because their data block cache was 16gb and the max heap was 16gb.
Here's the check:
{code:title=TabletServer.java}
Runtime runtime = Runtime.getRuntime();
if (!usingNativeMap && maxMemory + dCacheSize + iCacheSize >
runtime.maxMemory()) {
throw new IllegalArgumentException(String.format(
"Maximum tablet server map memory %,d and block cache sizes %,d is
too large for this JVM configuration %,d", maxMemory, dCacheSize + iCacheSize,
runtime.maxMemory()));
}
{code}
I think the check should still happen for index and data caches if native maps
are being used.
> TabletServerResourceManager doesn't sanity check memory settings if native
> maps are being used
> ----------------------------------------------------------------------------------------------
>
> Key: ACCUMULO-4083
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4083
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
> Affects Versions: 1.6.4
> Reporter: William Slacum
> Assignee: Josh Elser
> Priority: Minor
> Fix For: 1.6.5, 1.7.1, 1.8.0
>
>
> Noticed that the TServer tries to fail fast if caches and memory maps are
> configured to use more memory than the JVM has given to it via -Xmx. However,
> if native maps are being used, this check fails. I recently had OOMEs kill
> TServers because their data block cache was 16gb and the max heap was 16gb.
> Here's the check:
> {code:title=TabletServerResourceManager.java}
> Runtime runtime = Runtime.getRuntime();
> if (!usingNativeMap && maxMemory + dCacheSize + iCacheSize >
> runtime.maxMemory()) {
> throw new IllegalArgumentException(String.format(
> "Maximum tablet server map memory %,d and block cache sizes %,d is
> too large for this JVM configuration %,d", maxMemory, dCacheSize + iCacheSize,
> runtime.maxMemory()));
> }
> {code}
> I think the check should still happen for index and data caches if native
> maps are being used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
