tustvold commented on issue #6692: URL: https://github.com/apache/arrow-rs/issues/6692#issuecomment-2462625408
> This is reasonable -- though I could imagine adding type erased builders like DynListBuilder for this usecase This sort of partially type-erased API seems like the worst of both worlds, you either want something that is completely type-erased (e.g. MutableArrayData), or fully typed (e.g. ListBuilder). > Is there some fundamental reason the builders can't made faster Not without changing their APIs :sweat_smile:. For the primitive builders one _could_ simply move the current kernel implementations into the builders, but this doesn't really achieve much IMO. > A builder based approach could help (e.g. optimize for the case where the input batches had the same dictionary and handle the case where they didn't -- either via deferred computation or on the fly or something else) Yeah, it gets very complicated and fiddly. A similar challenge likely exists for StringView, although I'm not sure what level of sophistication we've reached w.r.t automatic GC. > Does approach seems like that for filter? That would be a very naive way to implement the filter kernel, I would encourage looking at what the selection kernels actually do. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
