GitHub user Pashugan opened a pull request:
https://github.com/apache/spark/pull/16913
[SPARK-15531] [DEPLOY] Complement launcher JVM memory settings to avoid the
conflict with system-wide settings
## What changes were proposed in this pull request?
It is well-known that having a lot of memory on a server JVM tries to
allocate a huge chunk of it for the heap by default when it starts.
I'm using the JAVA_TOOL_OPTIONS environment variable to override default
JVM settings and set an adequate system-wide heap size. When command line
options merge with JAVA_TOOL_OPTIONS I'm getting the following error because my
initial heap size (1g for instance) exceeds the maximum heap size (128m)
provided in the code:
```
Error occurred during initialization of VM
Incompatible minimum and maximum heap sizes specified
```
I debugged the launcher scripts and, since there is no visible way to fix
this outside the code, my proposal is to provide an initial heap size in the
code too.
## How was this patch tested?
Jenkins tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Pashugan/spark fix_java_options_conflict
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16913.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16913
----
commit be6deabcf30b2ceda475e31d45626e211af7d195
Author: Pavel Knoblokh <[email protected]>
Date: 2017-02-13T08:13:04Z
Complement launcher JVM memory settings to avoid the conflict with
system-wide settings
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
