alamb commented on issue #24111: URL: https://github.com/apache/datafusion/issues/24111#issuecomment-5416573631
> > I do think Dictionary support is important > > Thank you [@alamb](https://github.com/alamb) and since you already like the general direction. I will work with [@Rich-T-kid](https://github.com/Rich-T-kid) to move this forward. Thank yoU @kumarUjjawal and @Rich-T-kid One concern I have is that we don't explode the code size too much -- for example if every kernel now gets dictionary handling that is likely to generate many many copies of the code For example, something like this is happening in IN LISTs where each specialization also has a dictioanry dispatch - https://github.com/apache/datafusion/issues/24658 In general I think the pattern pattern for Dictionary types should be to apply the operation to the values array, and then apply via `take` and the indices. For example to support hashing on a dictionary array, the values should be hashed and then when a key hash is needed, look up in the corresponding values hashs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
