zjffdu commented on pull request #4022: URL: https://github.com/apache/zeppelin/pull/4022#issuecomment-764741093
> > > I also set the following environment variable in my Dockerfile, but I am using Spark from a different location. > > > Are the Python libraries that Zeppelin provides loaded? > > > ``` > > > PYTHONPATH=${SPARK_HOME}/python/lib/pyspark.zip:${SPARK_HOME}/python/lib/py4j-0.10.8.1-src.zip:/opt/conda/lib/python3.7/site-packages/ \ > > > ``` > > > > > > I didn't find this in Dockerfile, do you mean you pass this env when starting docker container ? > > No, > I added this line to my private Dockerfile because I use a different Apache Spark version, which is located at `${SPARK_HOME}`. > With this extra line, pyspark is able to use the correct Apache Spark version. > If we run pyspark with the Apache Spark version that is included in Apache Zeppelin, does Zeppelin add the pyspark libs? Yes, zeppelin would load pyspark libs in all cases. BTW, what is your command to launch zeppelin docker container ? here's what I use: ``` docker run -p 8080:8080 --rm -e ZEPPELIN_IN_DOCKER=true --name zeppelin -v $PWD/flink-1.11.2:/flink -v $PWD/spark-2.4.6-bin-hadoop2.7:/spark 67ed2d8ec2e0 ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org