Hyukjin Kwon created SPARK-35946:
------------------------------------
Summary: Respect Py4J server if InheritableThread API
Key: SPARK-35946
URL: https://issues.apache.org/jira/browse/SPARK-35946
Project: Spark
Issue Type: Improvement
Components: PySpark
Affects Versions: 3.2.0
Reporter: Hyukjin Kwon
Currently we sets the enviornment variables at the client side of Py4J
(python/pyspark/util.py ). If the Py4J gateway is created somewhere else (e.g.,
Zeppelin, etc), it could introduce a breakage at:
{code}
from pyspark import SparkContext
jvm = SparkContext._jvm
thread_connection = jvm._gateway_client.get_thread_connection()
# ^ the MLlibMLflowIntegrationSuite test suite failed at this line
# `AttributeError: 'GatewayClient' object has no attribute
'get_thread_connection'`
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
