Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/19840
I didn't read the entire thread here but what you want is this:
--archives hdfs:///python36/python36.tgz#python36 --conf
spark.pyspark.python=./python36/bin/python3.6 --conf
spark.executorEnv.LD_LIBRARY_PATH=./python36/lib --driver-library-path
/opt/python36/lib --conf spark.pyspark.driver.python=/opt/python36/bin/python3.6
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
