zhuqi-lucas commented on PR #24426:
URL: https://github.com/apache/datafusion/pull/24426#issuecomment-5370788440
Thanks @alamb, great work — following the Q25 dig here, a precise
DataFusion-side gate that stacks with the arrow-rs work, in case it's useful:
The row-level RowFilter is pure overhead in exactly one shape: when the
(dynamic) filter column **is the sole projected column**. Then there's no
"other column" for the selection to save a decode on; the filter column must be
fully decoded to evaluate the threshold regardless; and for a high-cardinality
key like `SearchPhrase`, RG pruning drops nothing — so the selection/mask
construction is paid for zero payback.
That's a sharper predicate than "< 3 non-filter columns": decline only the
*row-level* RowFilter (keep RG-level dynamic pruning) when `dynamic filter
column == the sole projected column`. It hits Q25 (`ORDER BY SearchPhrase`,
projection `{SearchPhrase}`) and leaves Q24 (`ORDER BY EventTime`, `EventTime`
∉ projection → `SearchPhrase` is a savable "other column") and any multi-column
projection untouched — those always have a ≠-filter column for the selection to
earn its keep.
Orthogonal to the selection-construction speedups in #10776 — if those land
this becomes unnecessary; until then it's a zero-risk way to drop the Q25
regression specifically.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
