Hello,
on the presentation on Spark Summit Amsterdam Moon showcased that it is
possible to change a location of the storage of the notebooks to S3 (so
they are not stored locally on the zeppelin machine).
I am running a standard Amazon EMR cluster with Zeppelin 0.6 SNAPSHOT
(default Amazon installation).
I tried to modify the /etc/zeppelin/conf.dist/zeppelin-env.sh file
changing
export ZEPPELIN_NOTEBOOK_DIR=/var/lib/zeppelin/notebook
to the following:
export ZEPPELIN_NOTEBOOK_DIR=s3:/bucket/folder
export ZEPPELIN_NOTEBOOK_DIR=s3://bucket/folder
export ZEPPELIN_NOTEBOOK_DIR=s3n://bucket/folder
and each time when I restart Zeppelin using
sudo /usr/lib/zeppelin/bin/zeppelin-daemon.sh stop
sudo /usr/lib/zeppelin/bin/zeppelin-daemon.sh start
I get error:
"Notebook dir doesn't exist, create"
when looking in to the zeppelin-env.sh I noticed it only checks for the
local folder.
if [[ ! -d "${ZEPPELIN_NOTEBOOK_DIR}" ]]; then
echo "Notebook dir doesn't exist, create ${ZEPPELIN_NOTEBOOK_DIR}"
$(mkdir -p "${ZEPPELIN_NOTEBOOK_DIR}")
fi
So I assume it is not the way :)
Can you please tell me how can I do it? I want to store all my notebooks on
s3 folder, rather than locally.
Thanks a lot!
Josef
