[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated YARN-10863:
--
Labels: pull-request-available (was: )
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
> Labels: pull-request-available
> Attachments: YARN-10863.001-1.patch, YARN-10863.002.patch,
> YARN-10863.004.patch, YARN-10863.005.patch, YARN-10863.006.patch,
> YARN-10863.007.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
> logic have some problem. The return condition is strictMemoryEnforcement is
> true and elasticMemoryEnforcement is false. So, following the document set
> use elastic memory control, the check logic will continue, when container
> memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuoGe updated YARN-10863:
-
Attachment: YARN-10863.007.patch
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
> Attachments: YARN-10863.001-1.patch, YARN-10863.002.patch,
> YARN-10863.004.patch, YARN-10863.005.patch, YARN-10863.006.patch,
> YARN-10863.007.patch
>
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
> logic have some problem. The return condition is strictMemoryEnforcement is
> true and elasticMemoryEnforcement is false. So, following the document set
> use elastic memory control, the check logic will continue, when container
> memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuoGe updated YARN-10863:
-
Attachment: YARN-10863.006.patch
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
> Attachments: YARN-10863.001-1.patch, YARN-10863.002.patch,
> YARN-10863.004.patch, YARN-10863.005.patch, YARN-10863.006.patch
>
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
> logic have some problem. The return condition is strictMemoryEnforcement is
> true and elasticMemoryEnforcement is false. So, following the document set
> use elastic memory control, the check logic will continue, when container
> memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuoGe updated YARN-10863:
-
Attachment: YARN-10863.005.patch
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
> Attachments: YARN-10863.001-1.patch, YARN-10863.002.patch,
> YARN-10863.004.patch, YARN-10863.005.patch
>
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
> logic have some problem. The return condition is strictMemoryEnforcement is
> true and elasticMemoryEnforcement is false. So, following the document set
> use elastic memory control, the check logic will continue, when container
> memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuoGe updated YARN-10863:
-
Attachment: YARN-10863.004.patch
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
> Attachments: YARN-10863.001-1.patch, YARN-10863.002.patch,
> YARN-10863.004.patch
>
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
> logic have some problem. The return condition is strictMemoryEnforcement is
> true and elasticMemoryEnforcement is false. So, following the document set
> use elastic memory control, the check logic will continue, when container
> memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuoGe updated YARN-10863:
-
Attachment: (was: YARN-10863.003.patch)
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
> Attachments: YARN-10863.001-1.patch, YARN-10863.002.patch
>
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
> logic have some problem. The return condition is strictMemoryEnforcement is
> true and elasticMemoryEnforcement is false. So, following the document set
> use elastic memory control, the check logic will continue, when container
> memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuoGe updated YARN-10863:
-
Attachment: YARN-10863.003.patch
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
> Attachments: YARN-10863.001-1.patch, YARN-10863.002.patch,
> YARN-10863.003.patch
>
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
> logic have some problem. The return condition is strictMemoryEnforcement is
> true and elasticMemoryEnforcement is false. So, following the document set
> use elastic memory control, the check logic will continue, when container
> memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuoGe updated YARN-10863:
-
Attachment: YARN-10863.002.patch
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
> Attachments: YARN-10863.001-1.patch, YARN-10863.002.patch
>
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
> logic have some problem. The return condition is strictMemoryEnforcement is
> true and elasticMemoryEnforcement is false. So, following the document set
> use elastic memory control, the check logic will continue, when container
> memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuoGe updated YARN-10863:
-
Attachment: (was: YARN-10863.001.patch)
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
> Attachments: YARN-10863.001-1.patch
>
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
> logic have some problem. The return condition is strictMemoryEnforcement is
> true and elasticMemoryEnforcement is false. So, following the document set
> use elastic memory control, the check logic will continue, when container
> memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuoGe updated YARN-10863:
-
Attachment: YARN-10863.001.patch
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
> Attachments: YARN-10863.001.patch
>
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
> logic have some problem. The return condition is strictMemoryEnforcement is
> true and elasticMemoryEnforcement is false. So, following the document set
> use elastic memory control, the check logic will continue, when container
> memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuoGe updated YARN-10863:
-
Attachment: (was: YARN-10863.000.patch)
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
> logic have some problem. The return condition is strictMemoryEnforcement is
> true and elasticMemoryEnforcement is false. So, following the document set
> use elastic memory control, the check logic will continue, when container
> memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuoGe updated YARN-10863:
-
Attachment: YARN-10863.000.patch
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
> Attachments: YARN-10863.000.patch
>
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
> logic have some problem. The return condition is strictMemoryEnforcement is
> true and elasticMemoryEnforcement is false. So, following the document set
> use elastic memory control, the check logic will continue, when container
> memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuoGe updated YARN-10863:
-
Description:
When following the
[documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
configuring elastic memory resource control,
yarn.nodemanager.elastic-memory-control.enabled set true,
yarn.nodemanager.resource.memory.enforced set to false,
yarn.nodemanager.pmem-check-enabled set true, and
yarn.nodemanager.resource.memory.enabled set true to use cgroup control memory,
but elastic memory control is not work.
I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
logic have some problem. The return condition is strictMemoryEnforcement is
true and elasticMemoryEnforcement is false. So, following the document set use
elastic memory control, the check logic will continue, when container memory
used over limit will killed by checkLimit.
{code:java}
if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
// When cgroup-based strict memory enforcement is used alone without
// elastic memory control, the oom-kill would take care of it.
// However, when elastic memory control is also enabled, the oom killer
// would be disabled at the root yarn container cgroup level (all child
// cgroups would inherit that setting). Hence, we fall back to the
// polling-based mechanism.
return;
}
{code}
was:
When following the
[documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
configuring elastic memory resource control,
yarn.nodemanager.elastic-memory-control.enabled set true,
yarn.nodemanager.resource.memory.enforced set to false,
yarn.nodemanager.pmem-check-enabled set true, and
yarn.nodemanager.resource.memory.enabled set true to use cgroup control memory,
but elastic memory control is not work.
so, I see the code ContainersMonitorImpl.java, in checkLimit function, the
skip logic have some problem. The return condition is strictMemoryEnforcement
is true and elasticMemoryEnforcement is false. So, following the document set
use elastic memory control, the check logic will continue, when container
memory used over limit will killed by checkLimit.
{code:java}
if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
// When cgroup-based strict memory enforcement is used alone without
// elastic memory control, the oom-kill would take care of it.
// However, when elastic memory control is also enabled, the oom killer
// would be disabled at the root yarn container cgroup level (all child
// cgroups would inherit that setting). Hence, we fall back to the
// polling-based mechanism.
return;
}
{code}
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
> logic have some problem. The return condition is strictMemoryEnforcement is
> true and elasticMemoryEnforcement is false. So, following the document set
> use elastic memory control, the check logic will continue, when container
> memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10863) CGroupElasticMemoryController is not work
[
https://issues.apache.org/jira/browse/YARN-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuoGe updated YARN-10863:
-
Description:
When following the
[documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
configuring elastic memory resource control,
yarn.nodemanager.elastic-memory-control.enabled set true,
yarn.nodemanager.resource.memory.enforced set to false,
yarn.nodemanager.pmem-check-enabled set true, and
yarn.nodemanager.resource.memory.enabled set true to use cgroup control memory,
but elastic memory control is not work.
so, I see the code ContainersMonitorImpl.java, in checkLimit function, the
skip logic have some problem. The return condition is strictMemoryEnforcement
is true and elasticMemoryEnforcement is false. So, following the document set
use elastic memory control, the check logic will continue, when container
memory used over limit will killed by checkLimit.
{code:java}
if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
// When cgroup-based strict memory enforcement is used alone without
// elastic memory control, the oom-kill would take care of it.
// However, when elastic memory control is also enabled, the oom killer
// would be disabled at the root yarn container cgroup level (all child
// cgroups would inherit that setting). Hence, we fall back to the
// polling-based mechanism.
return;
}
{code}
was:
When following the
[documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
configuring elastic memory resource control,
yarn.nodemanager.elastic-memory-control.enabled set true,
yarn.nodemanager.resource.memory.enforced set to false,
yarn.nodemanager.pmem-check-enabled set true, and
yarn.nodemanager.resource.memory.enabled set true to use cgroup control memory,
but elastic memory control is not work.
so, I see the code ContainersMonitorImpl.java, in checkLimit function, the skip
logic have some problem. The return condition is strictMemoryEnforcement is
true and elasticMemoryEnforcement is false. So, following the document set use
elastic memory control, the check logic will continue, when container memory
used over limit will killed by checkLimit.
{code:java}
if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
// When cgroup-based strict memory enforcement is used alone without
// elastic memory control, the oom-kill would take care of it.
// However, when elastic memory control is also enabled, the oom killer
// would be disabled at the root yarn container cgroup level (all child
// cgroups would inherit that setting). Hence, we fall back to the
// polling-based mechanism.
return;
}
{code}
> CGroupElasticMemoryController is not work
> -
>
> Key: YARN-10863
> URL: https://issues.apache.org/jira/browse/YARN-10863
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.3.1
>Reporter: LuoGe
>Priority: Major
>
> When following the
> [documentation|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html]
> configuring elastic memory resource control,
> yarn.nodemanager.elastic-memory-control.enabled set true,
> yarn.nodemanager.resource.memory.enforced set to false,
> yarn.nodemanager.pmem-check-enabled set true, and
> yarn.nodemanager.resource.memory.enabled set true to use cgroup control
> memory, but elastic memory control is not work.
> so, I see the code ContainersMonitorImpl.java, in checkLimit function, the
> skip logic have some problem. The return condition is
> strictMemoryEnforcement is true and elasticMemoryEnforcement is false. So,
> following the document set use elastic memory control, the check logic will
> continue, when container memory used over limit will killed by checkLimit.
> {code:java}
> if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
> // When cgroup-based strict memory enforcement is used alone without
> // elastic memory control, the oom-kill would take care of it.
> // However, when elastic memory control is also enabled, the oom killer
> // would be disabled at the root yarn container cgroup level (all child
> // cgroups would inherit that setting). Hence, we fall back to the
> // polling-based mechanism.
> return;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
