Akash3121 commented on code in PR #11033:
URL: https://github.com/apache/paimon/pull/11033#discussion_r4507385634
##########
paimon-python/pypaimon/execution/python/read/reader/aggregate/aggregators.py:
##########
@@ -801,6 +816,11 @@ def __init__(
)
self.distinct = options.field_collect_distinct(field_name)
+ self.count_limit = options.field_collect_agg_count_limit(field_name)
+ element_base = _atomic_base_name(field_type.element)
+ self._set_dedupe = (
+ self.distinct and element_base in _COLLECT_SET_DEDUPE_TYPES
+ )
def agg_reversed(self, accumulator: Any, input_field: Any) -> Any:
Review Comment:
`FieldCollectAgg.agg_reversed()` delegates to `self.agg(accumulator,
input_field)`, but the Java implementation swaps the operands because reverse
traversal supplies the newer partial result as the accumulator and the older
value as the input. With `older=[1,2]`, `newer=[3,4]`, and limit `2`, forward
aggregation returns `[1,2]`, while the equivalent reverse aggregation
incorrectly returns `[3,4]`; with limit `-2`, it incorrectly returns `[1,2]`
instead of `[3,4]`. This affects distinct and non-distinct modes. Change it to
aggregate `input_field` before `accumulator` and add Java-equivalent
forward-versus-reversed count-limit tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]