uros-db commented on PR #46670: URL: https://github.com/apache/spark/pull/46670#issuecomment-2123327859
Yes, aggregation buffers are InternalRows in Spark, however - I would not suggest extending stuff as you described in order to get this working. This approach turned out to not be as straight-forward as we'd have hoped, and after all - this is yet another approach with an aux structure for proper collation-aware aggregation in Mode, so since we don't really have guarantees of better performance, I'd suggest sticking with GroupMapReduce for now (https://github.com/apache/spark/pull/46597). Sorry for leading you the wrong way with this one, I hoped this one would turn out better / easier to implement! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
