I don't think it's propagated automatically. Try this: spark-submit --conf "spark.executorEnv.PYTHONPATH=..." ...
On Wed, Jun 10, 2015 at 8:15 AM, Bob Corsaro <rcors...@gmail.com> wrote: > I'm setting PYTHONPATH before calling pyspark, but the worker nodes aren't > inheriting it. I've tried looking through the code and it appears that it > should, I can't find the bug. Here's an example, what am I doing wrong? > > https://gist.github.com/dokipen/84c4e4a89fddf702fdf1 > -- Marcelo
