gaogaotiantian commented on code in PR #58990:
URL: https://github.com/apache/spark/pull/58990#discussion_r4137148622
##########
python/pyspark/eval_handlers/tests/test_arrow_eval_type_handlers.py:
##########
@@ -57,6 +61,42 @@ class _RunnerConf:
map_in_batch_legacy_accept_any_iterable = False
# One bound type per window UDF; the bounded test overrides this per
instance.
window_bound_types = ["unbounded"]
+ # Map-side PARTIAL buffer cap; the flush tests override this per instance.
+ arrow_max_records_per_batch = 10000
+
+
+class _EvalConf:
Review Comment:
Let's not hack this - it's not really necessary. We are shadowing the real
`EvalConf` and `RunnerConf` - if these actual conf changed in the future, we
could miss the change here because they are not even used.
If you really need a quick reference to a conf with the confs you need, just
create an object and use that. If you need it to be configurable, at least
inherit the real `RunnerConf` and `EvalConf` so the code changes there will be
reflected in this test. Use their `__init__` mechanism to initialize the conf
and read with their approach as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
