gemini-code-assist[bot] commented on code in PR #612:
URL: https://github.com/apache/tvm-ffi/pull/612#discussion_r3367193192
##########
tests/python/test_dataclass_hash.py:
##########
@@ -862,8 +862,12 @@ def test_shared_dag_hash_scaling_not_exponential() -> None:
RecursiveHash(d19)
t19 = (time.perf_counter() - t0) / repeats
- # With memoization this ratio should stay close to 1x; 1.6x leaves buffer
for noise.
- assert t19 <= t18 * 2.0, f"Unexpected super-linear scaling: d18={t18:.6f}s
d19={t19:.6f}s"
+ # With memoization this ratio should stay close to 1x. Threshold 4x leaves
+ # buffer for microsecond-scale measurement noise (CI runs frequently see
+ # t18~4us, t19~8us where small jitter blows past a 2x ceiling) while
+ # still catching true exponential blowup, which compounds far past 4x
+ # within a few depth steps.
+ assert t19 <= t18 * 4.0, f"Unexpected super-linear scaling: d18={t18:.6f}s
d19={t19:.6f}s"
Review Comment:
While relaxing the threshold to `4.0` helps mitigate flakiness, the root
cause of the microsecond-scale measurement noise is the very low number of
iterations (`repeats = 10` on line 854). At 10 repeats, the total execution
time is only around 40 microseconds, making it highly susceptible to CPU
scheduling and OS jitter.
To make this test truly robust, consider:
1. Increasing `repeats` to `1000` or `5000` (defined on line 854) to average
out the noise. This will still run in a few milliseconds.
2. Adding a small absolute tolerance (e.g., `1e-5` seconds) to the assertion
to handle cases where both times are extremely close to zero.
```suggestion
assert t19 <= t18 * 4.0 + 1e-5, f"Unexpected super-linear scaling:
d18={t18:.6f}s d19={t19:.6f}s"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
