viirya commented on a change in pull request #31723:
URL: https://github.com/apache/spark/pull/31723#discussion_r586178392
##########
File path: python/pyspark/tests/test_taskcontext.py
##########
@@ -282,6 +281,11 @@ def context(iterator):
self.assertTrue(bps == (-1, -1))
for pid in pids:
self.assertTrue(pid in worker_pids)
+ return True
+
+ def test_task_context_correct_with_python_worker_reuse(self):
+ eventually(
+ self.check_task_context_correct_with_python_worker_reuse,
catch_assertions=True)
Review comment:
nit: we may add few comment like the one in the description, e.g.
"Retrying the check as reusing Python workers actually sort of makes the best
effort reuse the Python workers."
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
