Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r212168597
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +99,143 @@ def getLocalProperty(self, key):
Get a local property set upstream in the driver, or None if it is
missing.
"""
return self._localProperties.get(key, None)
+
+
+BARRIER_FUNCTION = 1
+
+
+def _load_from_socket(port, auth_secret):
+ """
+ Load data from a given socket, this is a blocking method thus only
return when the socket
+ connection has been closed.
+
+ This is copied from context.py, while modified the message protocol.
--- End diff --
It would be nicer if we can deduplciate it later.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
