[
https://issues.apache.org/jira/browse/SPARK-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves resolved SPARK-1287.
----------------------------------
Resolution: Duplicate
duplicate of SPARK-2140
> yarn alpha and stable Client calculateAMMemory routines are different
> ---------------------------------------------------------------------
>
> Key: SPARK-1287
> URL: https://issues.apache.org/jira/browse/SPARK-1287
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 0.9.0, 1.0.0
> Reporter: Thomas Graves
>
> The yarn alpha version of calculateAMMemory takes into account the minimum
> resource capability and also subtracts out
> YarnAllocationHandler.MEMORY_OVERHEAD.
> The yarn stable version just sets the -Xmx to whatever is passed in by the
> user.
> The 2 of these should be the same.
>
> Personally I also think its weird how spark currently takes whatever user
> passes in for memory and adds YarnAllocationHandler.MEMORY_OVERHEAD to
> request to RM. This can be confusing to users. We should revisit all of this
> and commonize the stable/alpha code where possible.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
