singer-bin opened a new pull request, #8573:
URL: https://github.com/apache/hadoop/pull/8573
### Description of PR
`ContainersMonitorImpl#checkLimit()` skips the polling-based pmem/vmem memory
check whenever `strictMemoryEnforcement && !elasticMemoryEnforcement`,
relying
on the kernel OOM killer (introduced in YARN-8930).
`strictMemoryEnforcement` was derived only from
`yarn.nodemanager.resource.memory.enforced` (default **true**), but the
CGroups
memory hard limit is only written when
`yarn.nodemanager.resource.memory.enabled` (default **false**) is set. With
the
default configuration and polling enabled (`pmem-check-enabled=true`),
neither
the OOM killer nor the polling check enforces the limit, so containers can
exceed their requested memory unbounded (e.g. a 4 GB Spark executor growing
to
20 GB+).
This PR gates `strictMemoryEnforcement` on **both**
`yarn.nodemanager.resource.memory.enabled` and
`yarn.nodemanager.resource.memory.enforced`. The polling check is now skipped
only when CGroups memory enforcement is actually in effect. The
`yarn.nodemanager.resource.memory.enforced` description in yarn-default.xml
is
updated accordingly.
### How was this patch tested?
- Added
`TestContainersMonitor#testStrictMemoryEnforcementRequiresMemoryEnabled`.
- `./mvnw -pl
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
test
-Dtest=TestContainersMonitor#testStrictMemoryEnforcementRequiresMemoryEnabled`
→ Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
### For code changes:
- [x] Does the title of this PR start with the corresponding JIRA issue id?
- [x] If applicable, have you updated the LICENSE/NOTICE files? N/A
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
