mzabaluev commented on issue #22682: URL: https://github.com/apache/datafusion/issues/22682#issuecomment-4995768374
@Rich-T-kid #23128 resolves the all-or-nothing problem, but as far as I understand, the arrow-row based implementation always allocates and converts to an intermediate row buffer, so it can be less optimal and somewhat more memory-hungry than a purely inspective approach. I have benchmarked a similar change and found that it does not bring much improvement at least on some workloads. The code bloat argument against `FixedSizeList` does not apply to `List` / `LargeList`: the `GroupColumn` implementation is able to dispatch dynamically on the child array, the only monomorphizations are for the two offset types. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
