mridulm commented on code in PR #41806:
URL: https://github.com/apache/spark/pull/41806#discussion_r1247544031
##########
launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java:
##########
@@ -311,11 +311,17 @@ private List<String> buildSparkSubmitCommand(Map<String,
String> env)
}
private void checkJavaOptions(String javaOptions) {
- if (!isEmpty(javaOptions) && javaOptions.contains("Xmx")) {
- String msg = String.format("Not allowed to specify max heap(Xmx) memory
settings through " +
- "java options (was %s). Use the corresponding --driver-memory or " +
- "spark.driver.memory configuration instead.", javaOptions);
- throw new IllegalArgumentException(msg);
+ System.out.println(javaOptions);
Review Comment:
Please remove this
##########
launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java:
##########
@@ -311,11 +311,17 @@ private List<String> buildSparkSubmitCommand(Map<String,
String> env)
}
private void checkJavaOptions(String javaOptions) {
- if (!isEmpty(javaOptions) && javaOptions.contains("Xmx")) {
- String msg = String.format("Not allowed to specify max heap(Xmx) memory
settings through " +
- "java options (was %s). Use the corresponding --driver-memory or " +
- "spark.driver.memory configuration instead.", javaOptions);
- throw new IllegalArgumentException(msg);
+ System.out.println(javaOptions);
+
+ if (!isEmpty(javaOptions)) {
+ for (String javaOption: javaOptions.split(" ")) {
Review Comment:
This is not robust - you can have space within the value of a property by
quoting it.
```
$ scala -Dprop="abc 123"
Welcome to Scala 2.12.14 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_212).
Type in expressions for evaluation. Or try :help.
scala> System.getProperty("prop")
res0: String = abc 123
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
