Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/21437#discussion_r191548734
--- Diff: python/pyspark/tests.py ---
@@ -543,6 +543,15 @@ def test_tc_on_driver(self):
tc = TaskContext.get()
self.assertTrue(tc is None)
+ def test_get_local_property(self):
+ """Verify that local properties set on the driver are available in
TaskContext."""
+ self.sc.setLocalProperty("testkey", "testvalue")
--- End diff --
there isnt a clear way to clear local property in the spark context? what
do you propose is the right approach here? set the local property as "None"?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
