NEUpanning commented on issue #8227: URL: https://github.com/apache/incubator-gluten/issues/8227#issuecomment-2559640412
@zhztheplayer Thanks for your thoughts. > I think a complete solution should involve changes to Velox code to make sure Velox's collect functions use binary intermediate buffer This solution sounds good to me. I think what we need to do is (correct me if I am wrong) (a) change the `IntermediateType` and `OutputType` of collect_list/collect_set to `VARBINARY` (b) utilize `UnsafeRowDeserializer` and `UnsafeRowFast` to do the SerDe (c) remove the rewrite rules for collect_list/collect_set in gluten Should we need to open an issue in Velox? > velox_collect_set is now slow when the resident aggregate operator fell back. And could you provide more details about this? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
