Repository: zeppelin Updated Branches: refs/heads/master 4ef14956a -> d9a201388
ZEPPELIN-2045. Pass interpreter properties with "spark." as prefix to SparkConf ### What is this PR for? Minor change to only pass interpreter properties with "spark." as prefix to SparkConf. Other properties is used by zeppelin interpreter process, so don't need to be passed to SparkConf. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2045 ### How should this be tested? Tested manually, this is the log after this PR ``` INFO [2017-02-03 09:05:33,664] ({pool-2-thread-2} SparkInterpreter.java[createSparkContext_1]:384) - ------ Create new SparkContext yarn-client ------- DEBUG [2017-02-03 09:05:33,668] ({pool-2-thread-2} SparkInterpreter.java[createSparkContext_1]:467) - SparkConf: key = [spark.cores.max], value = [2] DEBUG [2017-02-03 09:05:33,668] ({pool-2-thread-2} SparkInterpreter.java[createSparkContext_1]:467) - SparkConf: key = [spark.app.name], value = [Zeppelin] ``` ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <[email protected]> Closes #1970 from zjffdu/ZEPPELIN-2045 and squashes the following commits: 3a146d3 [Jeff Zhang] ZEPPELIN-2045. Pass interpreter properties with "spark." as prefix to SparkConf Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/d9a20138 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/d9a20138 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/d9a20138 Branch: refs/heads/master Commit: d9a20138855f8b3b58f5dbe90bb49427a10157ae Parents: 4ef1495 Author: Jeff Zhang <[email protected]> Authored: Fri Feb 3 09:01:09 2017 +0800 Committer: Lee moon soo <[email protected]> Committed: Wed Feb 8 14:41:48 2017 +0900 ---------------------------------------------------------------------- .../main/java/org/apache/zeppelin/spark/SparkInterpreter.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/d9a20138/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java ---------------------------------------------------------------------- diff --git a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java index 3c1288e..8ce2ba3 100644 --- a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java +++ b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java @@ -364,7 +364,7 @@ public class SparkInterpreter extends Interpreter { for (Object k : intpProperty.keySet()) { String key = (String) k; String val = toString(intpProperty.get(key)); - if (!key.startsWith("spark.") || !val.trim().isEmpty()) { + if (key.startsWith("spark.") && !val.trim().isEmpty()) { logger.debug(String.format("SparkConf: key = [%s], value = [%s]", key, val)); conf.set(key, val); } @@ -495,7 +495,7 @@ public class SparkInterpreter extends Interpreter { for (Object k : intpProperty.keySet()) { String key = (String) k; String val = toString(intpProperty.get(key)); - if (!key.startsWith("spark.") || !val.trim().isEmpty()) { + if (key.startsWith("spark.") && !val.trim().isEmpty()) { logger.debug(String.format("SparkConf: key = [%s], value = [%s]", key, val)); conf.set(key, val); }
