[
https://issues.apache.org/jira/browse/FLINK-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nico Kruber updated FLINK-5973:
-------------------------------
Description:
A user on the mailing list ran into the problem that the {{directMemorySize}}
was incorrectly set too high which may happen if the following code path gets
{{maxMemory}} from 1/4*<physical mem size>> instead of the calculation,
{{taskmanager.sh}} is doing (in his case via the discouraged {{start-local.sh}}
script).
Other code paths may also exhibit this issue, which should be checked.
{code:title=TaskManagerServices#createMemoryManager()}
} else if (memType == MemoryType.OFF_HEAP) {
// The maximum heap memory has been adjusted according to the fraction
long maxMemory = EnvironmentInformation.getMaxJvmHeapMemory();
long directMemorySize = (long) (maxMemory / (1.0 - memoryFraction) *
memoryFraction);
if (preAllocateMemory) {
LOG.info("Using {} of the maximum memory size for managed
off-heap memory ({} MB)." ,
memoryFraction, directMemorySize >> 20);
} else {
LOG.info("Limiting managed memory to {} of the maximum memory
size ({} MB)," +
" memory will be allocated lazily.", memoryFraction,
directMemorySize >> 20);
}
memorySize = directMemorySize;
} else {
{code}
was:
A user on the mailing list ran into the problem that the {{directMemorySize}}
was incorrectly set too high which may happen if the following code path gets
{{maxMemory}} from 1/4*<physical mem size>> instead of the calculation,
{{taskmanager.sh}} is doing (in his case via the discouraged {{start-local.sh}}
script).
It be the case that other code paths also exhibit this issue, which should be
checked.
{code:title=TaskManagerServices#createMemoryManager()}
} else if (memType == MemoryType.OFF_HEAP) {
// The maximum heap memory has been adjusted according to the fraction
long maxMemory = EnvironmentInformation.getMaxJvmHeapMemory();
long directMemorySize = (long) (maxMemory / (1.0 - memoryFraction) *
memoryFraction);
if (preAllocateMemory) {
LOG.info("Using {} of the maximum memory size for managed
off-heap memory ({} MB)." ,
memoryFraction, directMemorySize >> 20);
} else {
LOG.info("Limiting managed memory to {} of the maximum memory
size ({} MB)," +
" memory will be allocated lazily.", memoryFraction,
directMemorySize >> 20);
}
memorySize = directMemorySize;
} else {
{code}
> check whether the direct memory size is always correctly calculated
> -------------------------------------------------------------------
>
> Key: FLINK-5973
> URL: https://issues.apache.org/jira/browse/FLINK-5973
> Project: Flink
> Issue Type: Bug
> Components: Local Runtime
> Reporter: Nico Kruber
> Priority: Minor
>
> A user on the mailing list ran into the problem that the {{directMemorySize}}
> was incorrectly set too high which may happen if the following code path gets
> {{maxMemory}} from 1/4*<physical mem size>> instead of the calculation,
> {{taskmanager.sh}} is doing (in his case via the discouraged
> {{start-local.sh}} script).
> Other code paths may also exhibit this issue, which should be checked.
> {code:title=TaskManagerServices#createMemoryManager()}
> } else if (memType == MemoryType.OFF_HEAP) {
> // The maximum heap memory has been adjusted according to the fraction
> long maxMemory = EnvironmentInformation.getMaxJvmHeapMemory();
> long directMemorySize = (long) (maxMemory / (1.0 - memoryFraction) *
> memoryFraction);
> if (preAllocateMemory) {
> LOG.info("Using {} of the maximum memory size for managed
> off-heap memory ({} MB)." ,
> memoryFraction, directMemorySize >> 20);
> } else {
> LOG.info("Limiting managed memory to {} of the maximum memory
> size ({} MB)," +
> " memory will be allocated lazily.", memoryFraction,
> directMemorySize >> 20);
> }
> memorySize = directMemorySize;
> } else {
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
