Repository: zeppelin Updated Branches: refs/heads/branch-0.7 dc14599c9 -> b5ac7b885
ZEPPELIN-2045. Pass interpreter properties with "spark." as prefix to SparkConf ### What is this PR for? Minor change to only pass interpreter properties with "spark." as prefix to SparkConf. Other properties is used by zeppelin interpreter process, so don't need to be passed to SparkConf. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2045 ### How should this be tested? Tested manually, this is the log after this PR ``` INFO [2017-02-03 09:05:33,664] ({pool-2-thread-2} SparkInterpreter.java[createSparkContext_1]:384) - ------ Create new SparkContext yarn-client ------- DEBUG [2017-02-03 09:05:33,668] ({pool-2-thread-2} SparkInterpreter.java[createSparkContext_1]:467) - SparkConf: key = [spark.cores.max], value = [2] DEBUG [2017-02-03 09:05:33,668] ({pool-2-thread-2} SparkInterpreter.java[createSparkContext_1]:467) - SparkConf: key = [spark.app.name], value = [Zeppelin] ``` ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <[email protected]> Closes #1970 from zjffdu/ZEPPELIN-2045 and squashes the following commits: 3a146d3 [Jeff Zhang] ZEPPELIN-2045. Pass interpreter properties with "spark." as prefix to SparkConf (cherry picked from commit d9a20138855f8b3b58f5dbe90bb49427a10157ae) Signed-off-by: Lee moon soo <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/b5ac7b88 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/b5ac7b88 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/b5ac7b88 Branch: refs/heads/branch-0.7 Commit: b5ac7b885f8b284824baa50116e4328cbbae684b Parents: dc14599 Author: Jeff Zhang <[email protected]> Authored: Fri Feb 3 09:01:09 2017 +0800 Committer: Lee moon soo <[email protected]> Committed: Wed Feb 8 14:45:02 2017 +0900 ---------------------------------------------------------------------- .../main/java/org/apache/zeppelin/spark/SparkInterpreter.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/b5ac7b88/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java ---------------------------------------------------------------------- diff --git a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java index 0584a30..1aecec4 100644 --- a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java +++ b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java @@ -332,7 +332,7 @@ public class SparkInterpreter extends Interpreter { for (Object k : intpProperty.keySet()) { String key = (String) k; String val = toString(intpProperty.get(key)); - if (!key.startsWith("spark.") || !val.trim().isEmpty()) { + if (key.startsWith("spark.") && !val.trim().isEmpty()) { logger.debug(String.format("SparkConf: key = [%s], value = [%s]", key, val)); conf.set(key, val); } @@ -463,7 +463,7 @@ public class SparkInterpreter extends Interpreter { for (Object k : intpProperty.keySet()) { String key = (String) k; String val = toString(intpProperty.get(key)); - if (!key.startsWith("spark.") || !val.trim().isEmpty()) { + if (key.startsWith("spark.") && !val.trim().isEmpty()) { logger.debug(String.format("SparkConf: key = [%s], value = [%s]", key, val)); conf.set(key, val); }
