GitHub user djalova opened a pull request:
https://github.com/apache/spark/pull/11255
[SPARK-12759][Core][Spark should fail fast if --executor-memory is too
small for spark to start]
Added an exception to be thrown in UnifiedMemoryManager.scala if the
configuration given for executor memory is too low. Also modified the exception
message thrown when driver memory is too low.
This patch was tested manually by passing in config options to Spark shell.
I also added a test in UnifiedMemoryManagerSuite.scala
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/djalova/spark SPARK-12759
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11255.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11255
----
commit 39c058c58772d8124daaa2b09c1a6a3c866d0f19
Author: Daniel Jalova <[email protected]>
Date: 2016-02-15T22:46:26Z
Added exception to be thrown if executor memory is too small
commit 154b57bbe316cb12301bbbc0c6e042e41cbb7837
Author: Daniel Jalova <[email protected]>
Date: 2016-02-16T19:08:35Z
Added test for executor memory.
commit 7f6dc296368bee1c54f269f7a0d49d19f4c74024
Author: Daniel Jalova <[email protected]>
Date: 2016-02-18T19:12:08Z
Refactoring and added to exception message
commit 134d5304b396b25431c3f473ad77bd7f642778c0
Author: Daniel Jalova <[email protected]>
Date: 2016-02-18T19:56:45Z
Minor fix and refactoring
commit f4587130be54f657ce0259f95b013f773c885d32
Author: Daniel Jalova <[email protected]>
Date: 2016-02-18T20:17:21Z
Fixed test
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
