adriangb commented on PR #24526: URL: https://github.com/apache/datafusion/pull/24526#issuecomment-5433437007
Filed the follow-ups from review that I (somewhat arbitrarily) felt are worth tracking: #24707 — extend the pruning benchmark: small N, container count, value length and shared-prefix length, input type, duplicate ratio. Blocks the next two, since neither can really be decided without numbers. #24709 — extend the compact path beyond strings. Tracking issue: binary looks nearly free, the integer-likes bring dates/times/timestamps/durations along with them, and decimals and floats each have their own wrinkle. It also holds the representation question (Arc<[String]> vs an Arrow array vs an order-preserving byte encoding such as arrow::row), since that seems to decide whether this is one change or several. #24710 — revisit the 20-value lower bound. #24708 — enclosing-range fallback above the cap, where there's currently no container pruning at all. #24711 — NOT IN and NULL-containing lists, kept as two separate problems per your explanation. Thanks again for the thorough responses on all of this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
