SteNicholas commented on code in PR #1487:
URL: https://github.com/apache/paimon-rust/pull/1487#discussion_r4657246331


##########
crates/paimon/src/table/de_vector_read.rs:
##########
@@ -2540,6 +2540,18 @@ fn collect_raw_batch_vector_batch(
             });
         }
         let row_id = row_id_to_u64(row_ids.value(row))?;
+        let candidate_queries = 
scoring_plan.candidate_query_indices.get(&row_id);

Review Comment:
   **A NULL vector now incurs a candidate-map lookup before the collector skips 
it.** I measured the collector on 262,144 rows in 1,024-row Arrow batches, with 
one non-NULL vector in 31 and two queries selecting even and odd row IDs. The 
optimized arm64 scan took 7.5 ms on this commit and 1.8 ms on the base. Moving 
the outer list's null check above this lookup brought it to 1.9 ms with 
identical results.
   
   Please check the outer null bit immediately after validating `_ROW_ID`. The 
candidate filter can still run before reading offsets or validating child 
coordinates, preserving the fix for excluded rows.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to