Hello, Can anyone confirm here please?
Regards Rajat On Sat, Jan 16, 2021 at 11:46 PM rajat kumar <kumar.rajat20...@gmail.com> wrote: > Hey Users, > > I want to run spark job from virtual environment using Python. > > Please note I am creating virtual env (using python3 -m venv env) > > I see that there are 3 variables for PYTHON which we have to set: > PYTHONPATH > PYSPARK_DRIVER_PYTHON > PYSPARK_PYTHON > > I have 2 doubts: > 1. If i want to use Virtual env, do I need to point python path of virtual > environment to all these variables? > 2. Should I set these variables in spark-env.sh or should I set them using > export statements. > > Regards > Rajat > > >
