nanav yorbiz created SPARK-28652:
------------------------------------
Summary: spark.kubernetes.pyspark.pythonVersion is never passed to
executors
Key: SPARK-28652
URL: https://issues.apache.org/jira/browse/SPARK-28652
Project: Spark
Issue Type: Bug
Components: Kubernetes
Affects Versions: 2.4.3
Reporter: nanav yorbiz
I suppose this may not be a priority with Python2 on its way out, but given
that this setting is only ever sent to the driver and not the executors, no
actual work can be performed when the versions don't match, which will tend to
be *always* with the default setting for the driver being changed from 2 to 3,
and the executors using `python`, which defaults to v2, by default.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
