zhuqi-lucas commented on PR #23128: URL: https://github.com/apache/datafusion/pull/23128#issuecomment-4903901308
Thanks @alamb. Context: this is PR 2 of #22715, split out from the original mega-PR #22706 at @adriangb's request (that PR added FSL / List / LargeList / Struct all in one, 4 files / +2468). #22751 landed as PR 1 (dispatcher + factory framework, primitives only). #23128 is the smallest additive next step — one new builder + two dispatcher entries. Motivation: production workloads that GROUP BY a mix of primitives plus a nested column (Struct<List<Struct>> in the case I benchmarked) fall entirely into GroupValuesRows because of the one nested column. In my measurements the fallback ran at ~25 GiB peak vs ~15 GiB with the nested column removed — about 40% of peak was fallback tax, not the nested data itself. The EPIC brings nested types onto the column-wise path so mixed shapes stop falling back. On binary size: the 12 arms mirror the existing `instantiate_primitive!` dispatch already in GroupValuesColumn (int8..64, uint8..64, float32/64, date32/64). Same type set, one wrap per primitive — not broadening the monomorphization surface. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
