About setting memory, This thread may help. http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Can-not-configure-driver-memory-size-td1513.html
Thanks, moon On Wed, Nov 25, 2015 at 2:49 AM Jakub Liska <liska.ja...@gmail.com> wrote: > Btw one more similar thing is that env var ZEPPELIN_MEM should be > reflected in JAVA_OPTS here : > > https://github.com/apache/incubator-zeppelin/blob/master/bin/common.sh#L92 > > but if I declare it in the docker container and run zeppelin.sh, it always > leads to default -Xmx1024m : > > /usr/jdk1.8.0_31/bin/java -Dspark.executor.memory=4g > -Dspark.serializer=org.apache.spark.serializer.KryoSerializer > -Dfile.encoding=UTF-8 -Xmx1024m > -Dzeppelin.log.file=/opt/zeppelin/logs/zeppelin-root-e352da1028c0.log -cp > ::/opt/zeppelin/zeppelin-server/target/lib/*:/opt/zeppelin/zeppelin-zengine/target/lib/*:/opt/zeppelin/zeppelin-interpreter/target/lib/*:/opt/zeppelin/*::/opt/zeppelin/conf:/opt/zeppelin/zeppelin-interpreter/target/classes:/opt/zeppelin/zeppelin-zengine/target/classes:/opt/zeppelin/zeppelin-server/target/classes > org.apache.zeppelin.server.ZeppelinServer start > > > Even though for instance ZEPPELIN_JAVA_OPTS are reflected : -e > ZEPPELIN_JAVA_OPTS="-Dspark.executor.memory=5g" > > I cannot figure that out, it's like if this condition succeeded and the > default value was set : > > https://github.com/apache/incubator-zeppelin/blob/master/bin/common.sh#L88 > > On Mon, Nov 23, 2015 at 11:58 PM, Jakub Liska <liska.ja...@gmail.com> > wrote: > >> You saved my day, >> >> this was the culprit >> https://github.com/datalayer/zeppelin-docker/issues/1#issuecomment-159094193 >> >> The image contains hard-coded settings in interpreter.json. >> >> Thank you, Jakub >> >> On Mon, Nov 23, 2015 at 11:24 PM, moon soo Lee <m...@apache.org> wrote: >> >>> Hi, >>> >>> You'll need to set 'master' property in 'Interpreter' menu, in your GUI. >>> >>> MASTER env variable set default value for this 'master' property. So >>> once 'master' is decided and saved in conf/interpreter.json, it'll not be >>> changed by MASTER env variable unless you remove conf/interpreter.json. >>> >>> Hope this helps >>> >>> Best, >>> moon >>> >>> On Tue, Nov 24, 2015 at 4:29 AM Jakub Liska <liska.ja...@gmail.com> >>> wrote: >>> >>>> Hey, >>>> >>>> I'm starting zeppelin.sh from within docker container: >>>> >>>> /usr/bin/docker run \ >>>> -p 4040:4040 \ >>>> -p 8080:8080 \ >>>> -e ZEPPELIN_JAVA_OPTS="-Dspark.executor.memory=5g" \ >>>> -e ZEPPELIN_PORT=8080 \ >>>> -e ZEPPELIN_NOTEBOOK_DIR=foo/bar \ >>>> -e MASTER=spark://${SPARK_MASTER_IP}:7077 \ >>>> foo/zeppelin >>>> >>>> And since my Master IP is dynamic, I need to set it via Environment >>>> variable as you can see above... >>>> >>>> root@3d1c2c3bc389:/opt/zeppelin# printenv | grep MASTER >>>> MASTER=spark://10.1.4.4:7077 >>>> >>>> But Zeppelin doesn't use it, I see this in it's logs : >>>> >>>> ------ Create new SparkContext local[*] ------- >>>> >>>> Any idea what might be wrong? I cannot use zeppelin-env.sh because the >>>> IP address is dynamic. >>>> >>> >> >
