grundprinzip commented on code in PR #41013:
URL: https://github.com/apache/spark/pull/41013#discussion_r1194574096
##########
python/pyspark/context.py:
##########
@@ -180,7 +180,7 @@ def __init__(
udf_profiler_cls: Type[UDFBasicProfiler] = UDFBasicProfiler,
memory_profiler_cls: Type[MemoryProfiler] = MemoryProfiler,
):
- if "SPARK_REMOTE" in os.environ and "SPARK_LOCAL_REMOTE" not in
os.environ:
Review Comment:
The difference is that spark remote indicates one particular endpoint and at
the same time makes PySpark use spark connect.
With this change spark remote enables the spark connect mode but does not
modify the environment variable when a new session is created. I will clarify
this more in the commit message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
