Repository: zeppelin Updated Branches: refs/heads/master d2f476d20 -> abd95fa5e
[HOTFIX] Set default ZEPPELIN_INTP_MEM ### What is this PR for? This PR sets default value for ZEPPELIN_INTP_MEM to avoid OOM Exception in SparkInterpreter when Zeppelin has zero configuration. This PR should be merged to both branch-0.6 and master. ### What type of PR is it? Bug Fix ### How should this be tested? 1. Build with: ``` mvn clean package -DskipTests -pl '!zeppelin-distribution,!file,!alluxio,!livy,!hbase,!bigquery,!python,!jdbc,!ignite,!lens,!postgresql,!cassandra,!kylin,!elasticsearch,!flink,!markdown,!shell,!angular' ``` 2. Unset SPARK_HOME in conf/zeppelin-env.sh if you have. 3. Run Zeppelin with java 1.7. 4. Run tutorial and see if it doesn't hang. ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Mina Lee <mina...@apache.org> Closes #1505 from minahlee/hotfix/default_intp_jvm and squashes the following commits: 0dfda4f [Mina Lee] Set default ZEPPELIN_INTP_MEM Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/abd95fa5 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/abd95fa5 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/abd95fa5 Branch: refs/heads/master Commit: abd95fa5e4a760ee19eb5597bf1bcaabdcd7b0ea Parents: d2f476d Author: Mina Lee <mina...@apache.org> Authored: Tue Oct 11 19:13:52 2016 +0900 Committer: Mina Lee <mina...@apache.org> Committed: Wed Oct 12 16:44:53 2016 +0900 ---------------------------------------------------------------------- bin/common.cmd | 4 ++++ bin/common.sh | 6 +++++- conf/zeppelin-env.cmd.template | 4 ++-- conf/zeppelin-env.sh.template | 4 ++-- docs/install/upgrade.md | 4 ++-- 5 files changed, 15 insertions(+), 7 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/abd95fa5/bin/common.cmd ---------------------------------------------------------------------- diff --git a/bin/common.cmd b/bin/common.cmd index b4fb6bf..745750f 100644 --- a/bin/common.cmd +++ b/bin/common.cmd @@ -69,6 +69,10 @@ if not defined ZEPPELIN_MEM ( set ZEPPELIN_MEM=-Xms1024m -Xmx1024m -XX:MaxPermSize=512m ) +if not defined ZEPPELIN_INTP_MEM ( + set ZEPPELIN_INTP_MEM=-Xms1024m -Xmx1024m -XX:MaxPermSize=512m +) + if not defined ZEPPELIN_JAVA_OPTS ( set ZEPPELIN_JAVA_OPTS=-Dfile.encoding=%ZEPPELIN_ENCODING% %ZEPPELIN_MEM% ) else ( http://git-wip-us.apache.org/repos/asf/zeppelin/blob/abd95fa5/bin/common.sh ---------------------------------------------------------------------- diff --git a/bin/common.sh b/bin/common.sh index b69f28c..486d2b1 100644 --- a/bin/common.sh +++ b/bin/common.sh @@ -113,10 +113,14 @@ if [[ -z "${ZEPPELIN_ENCODING}" ]]; then export ZEPPELIN_ENCODING="UTF-8" fi -if [[ -z "$ZEPPELIN_MEM" ]]; then +if [[ -z "${ZEPPELIN_MEM}" ]]; then export ZEPPELIN_MEM="-Xms1024m -Xmx1024m -XX:MaxPermSize=512m" fi +if [[ -z "${ZEPPELIN_INTP_MEM}" ]]; then + export ZEPPELIN_INTP_MEM="-Xms1024m -Xmx1024m -XX:MaxPermSize=512m" +fi + JAVA_OPTS+=" ${ZEPPELIN_JAVA_OPTS} -Dfile.encoding=${ZEPPELIN_ENCODING} ${ZEPPELIN_MEM}" JAVA_OPTS+=" -Dlog4j.configuration=file://${ZEPPELIN_CONF_DIR}/log4j.properties" export JAVA_OPTS http://git-wip-us.apache.org/repos/asf/zeppelin/blob/abd95fa5/conf/zeppelin-env.cmd.template ---------------------------------------------------------------------- diff --git a/conf/zeppelin-env.cmd.template b/conf/zeppelin-env.cmd.template index 4d697fc..cb7f87c 100644 --- a/conf/zeppelin-env.cmd.template +++ b/conf/zeppelin-env.cmd.template @@ -19,8 +19,8 @@ REM REM set JAVA_HOME= REM set MASTER= REM Spark master url. eg. spark://master_addr:7077. Leave empty if you want to use local mode. REM set ZEPPELIN_JAVA_OPTS REM Additional jvm options. for example, set ZEPPELIN_JAVA_OPTS="-Dspark.executor.memory=8g -Dspark.cores.max=16" -REM set ZEPPELIN_MEM REM Zeppelin jvm mem options Default -Xmx1024m -XX:MaxPermSize=512m -REM set ZEPPELIN_INTP_MEM REM zeppelin interpreter process jvm mem options. +REM set ZEPPELIN_MEM REM Zeppelin jvm mem options Default -Xms1024m -Xmx1024m -XX:MaxPermSize=512m +REM set ZEPPELIN_INTP_MEM REM zeppelin interpreter process jvm mem options. Default -Xmx1024m -Xms1024m -XX:MaxPermSize=512m REM set ZEPPELIN_INTP_JAVA_OPTS REM zeppelin interpreter process jvm options. REM set ZEPPELIN_LOG_DIR REM Where log files are stored. PWD by default. http://git-wip-us.apache.org/repos/asf/zeppelin/blob/abd95fa5/conf/zeppelin-env.sh.template ---------------------------------------------------------------------- diff --git a/conf/zeppelin-env.sh.template b/conf/zeppelin-env.sh.template index e814115..5263cad 100644 --- a/conf/zeppelin-env.sh.template +++ b/conf/zeppelin-env.sh.template @@ -19,8 +19,8 @@ # export JAVA_HOME= # export MASTER= # Spark master url. eg. spark://master_addr:7077. Leave empty if you want to use local mode. # export ZEPPELIN_JAVA_OPTS # Additional jvm options. for example, export ZEPPELIN_JAVA_OPTS="-Dspark.executor.memory=8g -Dspark.cores.max=16" -# export ZEPPELIN_MEM # Zeppelin jvm mem options Default -Xmx1024m -XX:MaxPermSize=512m -# export ZEPPELIN_INTP_MEM # zeppelin interpreter process jvm mem options. +# export ZEPPELIN_MEM # Zeppelin jvm mem options Default -Xms1024m -Xmx1024m -XX:MaxPermSize=512m +# export ZEPPELIN_INTP_MEM # zeppelin interpreter process jvm mem options. Default -Xms1024m -Xmx1024m -XX:MaxPermSize=512m # export ZEPPELIN_INTP_JAVA_OPTS # zeppelin interpreter process jvm options. # export ZEPPELIN_SSL_PORT # ssl port (used when ssl environment variable is set to true) http://git-wip-us.apache.org/repos/asf/zeppelin/blob/abd95fa5/docs/install/upgrade.md ---------------------------------------------------------------------- diff --git a/docs/install/upgrade.md b/docs/install/upgrade.md index 6e23694..c218b4c 100644 --- a/docs/install/upgrade.md +++ b/docs/install/upgrade.md @@ -49,6 +49,6 @@ So, copying `notebook` and `conf` directory should be enough. ### Upgrading from Zeppelin 0.6 to 0.7 - - From 0.7, we don't use `ZEPPELIN_JAVA_OPTS` as default value of `ZEPPELIN_INTP_JAVA_OPTS` and also the same for `ZEPPELIN_MEM`/`ZEPPELIN_INTP_MEM`. If user want to configure the jvm opts of interpreter process, please set `ZEPPELIN_INTP_JAVA_OPTS` and `ZEPPELIN_INTP_MEM` explicitly. + - From 0.7, we don't use `ZEPPELIN_JAVA_OPTS` as default value of `ZEPPELIN_INTP_JAVA_OPTS` and also the same for `ZEPPELIN_MEM`/`ZEPPELIN_INTP_MEM`. If user want to configure the jvm opts of interpreter process, please set `ZEPPELIN_INTP_JAVA_OPTS` and `ZEPPELIN_INTP_MEM` explicitly. If you don't set `ZEPPELIN_INTP_MEM`, Zeppelin will set it to `-Xms1024m -Xmx1024m -XX:MaxPermSize=512m` by default. - Mapping from `%jdbc(prefix)` to `%prefix` is no longer available. Instead, you can use %[interpreter alias] with multiple interpreter setttings on GUI. - - Usage of `ZEPPELIN_PORT` is not supported in ssl mode. Instead use `ZEPPELIN_SSL_PORT` to configure the ssl port. Value from `ZEPPELIN_PORT` is used only when `ZEPPELIN_SSL` is set to `false`. \ No newline at end of file + - Usage of `ZEPPELIN_PORT` is not supported in ssl mode. Instead use `ZEPPELIN_SSL_PORT` to configure the ssl port. Value from `ZEPPELIN_PORT` is used only when `ZEPPELIN_SSL` is set to `false`.
