Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22962#discussion_r232528655
--- Diff: python/pyspark/tests.py ---
@@ -618,10 +618,13 @@ def test_barrier_with_python_worker_reuse(self):
"""
Verify that BarrierTaskContext.barrier() with reused python worker.
"""
+ self.sc._conf.set("spark.python.work.reuse", "true")
--- End diff --
@xuanyuanking, this will probably need a separate suite case since it's
also related with how we start the worker or not. You can make a new class, run
a simple job to make sure workers are created and being resued, test it and
stop.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
