singer-bin opened a new pull request, #8575:
URL: https://github.com/apache/hadoop/pull/8575
### Description of PR
Backport of #8573 to `branch-3.2`.
`ContainersMonitorImpl#checkLimit()` skips the polling-based pmem/vmem check
whenever `strictMemoryEnforcement` is on. However `strictMemoryEnforcement` was
derived only from `yarn.nodemanager.resource.memory.enforced` (default `true`),
while the CGroups memory hard limit is only written when
`yarn.nodemanager.resource.memory.enabled` (default `false`) is set.
So with the default config and polling enabled, neither the CGroups memory
controller nor the polling check enforces limits, leaving container memory
unbounded (e.g. a 4 GB Spark executor growing past 20 GB).
The fix gates `strictMemoryEnforcement` in `serviceInit` on both
`memory.enabled && memory.enforced`, so the polling check is skipped only when
CGroups memory enforcement is actually in effect.
### Note for branch-3.2
`branch-3.2` predates YARN-8930, so there is no `if (strictMemoryEnforcement
&& !elasticMemoryEnforcement) { return; }` early-return. Instead the skip
happens through the older YARN-8461 path in `checkLimit`: the `else if
(strictMemoryEnforcement || elasticMemoryEnforcement)` branch sets
`isMemoryOverLimit = Optional.of(false)`, which bypasses the polling check. The
root cause and fix are identical.
### How was this patch tested?
Added
`TestContainersMonitor#testStrictMemoryEnforcementRequiresMemoryEnabled`
(JUnit4). Built the full dependency chain and ran the test locally: `Tests run:
1, Failures: 0, Errors: 0`.
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
