[
https://issues.apache.org/jira/browse/SPARK-11154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14961195#comment-14961195
]
Dustin Cote commented on SPARK-11154:
-------------------------------------
That's absolutely a good point. It might be more confusing to have a default
'untagged' number be specified in megabytes and allow option appended 'g' for
gigabytes. I think it would be safe to allow people to specify the 'm' at the
end (this currently doesn't work), but I agree it would be a bad thing to move
from a default of megabytes to bytes as it wouldn't be backward compatible.
This might just be a usability quirk, but I figured I'd call it out since I see
customers trying to use this like Xmx all the time.
> make specificaition spark.yarn.executor.memoryOverhead consistent with
> typical JVM options
> ------------------------------------------------------------------------------------------
>
> Key: SPARK-11154
> URL: https://issues.apache.org/jira/browse/SPARK-11154
> Project: Spark
> Issue Type: Improvement
> Components: Documentation, Spark Submit
> Reporter: Dustin Cote
> Priority: Minor
>
> spark.yarn.executor.memoryOverhead is currently specified in megabytes by
> default, but it would be nice to allow users to specify the size as though it
> were a typical -Xmx option to a JVM where you can have 'm' and 'g' appended
> to the end to explicitly specify megabytes or gigabytes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
