viirya commented on code in PR #25693:
URL: https://github.com/apache/datafusion/pull/25693#discussion_r4176308635
##########
datafusion/physical-plan/src/joins/utils.rs:
##########
@@ -1391,12 +1391,19 @@ pub(crate) fn build_null_aware_left_mark_column(
probe_side_has_null: bool,
probe_side_non_empty: bool,
) -> ArrayRef {
+ let build_key_nulls = build_key_column.logical_nulls();
Review Comment:
Thanks, this is valid. I reproduced the repeated full-dictionary scan with
524,288 rows and batch size 8192. The diagnostic runtime dropped from about
693ms to 141ms after limiting logical_nulls() to the current contiguous output
chunk. Correlated joins now skip this computation and continue using their
precomputed bitmap. All 104 null-aware tests and the complete
datafusion-physical-plan all-features suite pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
