This is an automated email from the ASF dual-hosted git repository.
ruifengz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new f1bab5a9e14c [SPARK-54989][PYTHON][TEST] Improve eventually util
f1bab5a9e14c is described below
commit f1bab5a9e14c5d1d0e660c90d9cdda184e36a22f
Author: Tian Gao <[email protected]>
AuthorDate: Mon Jan 12 16:28:32 2026 +0800
[SPARK-54989][PYTHON][TEST] Improve eventually util
### What changes were proposed in this pull request?
* Add a `quiet` arg to `eventually` and default to `True` - it should not
print attempts be default
* Make `interval` controllable and set default to `0.1` instead of `0.01`.
`0.1` is pretty fast in our tests. `0.01` is a bit too much.
### Why are the changes needed?
When I worked with `eventually` in my previous experience, it filled my
screen and I can't check other information. Most of the time we don't care how
many attempts there are.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Locally it does not print stuff anymore by default.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #53755 from gaogaotiantian/improve-eventually.
Authored-by: Tian Gao <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
---
python/pyspark/testing/utils.py | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/python/pyspark/testing/utils.py b/python/pyspark/testing/utils.py
index 4286a55bb699..438143d66c66 100644
--- a/python/pyspark/testing/utils.py
+++ b/python/pyspark/testing/utils.py
@@ -184,6 +184,8 @@ def eventually(
timeout=30.0,
catch_assertions=False,
catch_timeout=False,
+ quiet=True,
+ interval=0.1,
):
"""
Wait a given amount of time for a condition to pass, else fail with an
error.
@@ -209,10 +211,17 @@ def eventually(
If False (default), do not catch TimeoutError.
If True, catch TimeoutError; continue, but save
error to throw upon timeout.
+ quiet : bool
+ If True (default), do not print any output.
+ If False, print output.
+ interval : float
+ Number of seconds to wait between attempts. Default 0.1 seconds.
"""
assert timeout > 0
assert isinstance(catch_assertions, bool)
assert isinstance(catch_timeout, bool)
+ assert isinstance(quiet, bool)
+ assert isinstance(interval, float)
def decorator(condition: Callable) -> Callable:
assert isinstance(condition, Callable)
@@ -241,8 +250,9 @@ def eventually(
if lastValue is True or lastValue is None:
return
- print(f"\nAttempt #{numTries} failed!\n{lastValue}")
- sleep(0.01)
+ if not quiet:
+ print(f"\nAttempt #{numTries} failed!\n{lastValue}")
+ sleep(interval)
if isinstance(lastValue, (AssertionError, TimeoutError)):
raise lastValue
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
