ericm-db commented on code in PR #56907:
URL: https://github.com/apache/spark/pull/56907#discussion_r3547843457
##########
python/pyspark/sql/connect/session.py:
##########
@@ -1247,6 +1249,372 @@ def _start_connect_server(master: str, opts: Dict[str,
Any]) -> None:
messageParameters={},
)
+ # Opt-in reuse of a persistent local Spark Connect server. By default
``.remote("local[*]")``
+ # boots a fresh in-process server every process (see
``_start_connect_server`` above); when
+ # reuse is enabled, the first run starts a detached server
(``connect/local_server.py``) and
+ # records it in a discovery file, and later runs reconnect to it instead
of re-paying the cold
+ # start.
+
+ @staticmethod
+ def _local_connect_discovery_path() -> str:
+ """Location of the discovery file describing the running persistent
local server."""
+ override = os.environ.get("SPARK_LOCAL_CONNECT_DISCOVERY")
Review Comment:
Yup done, exactly that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
