Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/1906#issuecomment-54311053
@copester Any chance you can upmerge this and update based on the jira
conversation. I think the issue is actually with yarn alpha version now. The
yarn alpha version of this shouldn't subtract out the memoryOverhead since its
using that directly as the heap size and adding in the memory overhead when it
requests the AM container ( memoryResource.setMemory(args.amMemory +
memoryOverhead)).
We also don't really need to do the logic there of rounding based on the
getMinimumResourceCapability because Yarn will handle that for us. So I think
we can actually remove the calculateAMMemory routines and just use
args.amMemory.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
