wgtmac commented on PR #45085: URL: https://github.com/apache/arrow/pull/45085#issuecomment-2556588198
> The slowdown on lists is a bit surprising, is it because of levels histograms? Did you mean the slowdown from which one? 1. T -> List[T] for same level. 2. Level::None -> Level::ColumnChunk for List[T]. For 1, I think it is due to explosion of element sizes. For 2, I think it is due to levels histograms because the string and integer types seem to have similar regression. The data size is large enough that most iteration numbers are 1, which may affect judgement. > Does the non-list case have nulls? Yes, the null probability is hard-coded to 50%. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
