jaingaurav-dev opened a new pull request, #27585:
URL: https://github.com/apache/datafusion/pull/27585

   ## Which issue does this PR close?
   
   - None. There is no existing issue for this change.
   
   ## Rationale for this change
   
   The TopK operator keeps the current top `k` rows in a `BinaryHeap<TopKRow>`. 
`TopKRow` holds `row: Vec<u8>`, `batch_id: u32`, and `index: usize`. On 64-bit 
targets, the `usize` index plus alignment padding makes the struct 40 bytes. 
`index` names a row inside one Arrow batch, so `u32` is enough. With a `u32` 
index it packs next to `batch_id`, and `TopKRow` shrinks from 40 to 32 bytes. 
Heap entries get smaller, and each sift-down moves less memory.
   
   This PR does not claim a measured speedup. No benchmark results are included.
   
   Assumption to call out: the two narrowing conversions are plain `as u32` 
casts, one for a row position within an input batch and one for a position 
within the compacted heap batch. They assume a single batch has fewer than 
`u32::MAX` rows. If reviewers prefer, a checked conversion such as 
`u32::try_from` is an alternative.
   
   ## What changes are included in this PR?
   
   - `TopKRow.index` changes from `usize` to `u32`. `TopKRow::new`, 
`TopKRow::replace_with`, and `TopKHeap::add` now take a `u32` index.
   - `find_new_topk_items` and heap compaction cast the row position to `u32` 
when they store it.
   - The sites that pass the index to Arrow cast it back to `usize`. These are 
`RecordBatch::slice` for the heap boundary, `interleave_record_batch` in 
compaction, and the common-prefix row conversion.
   - The `heap_snapshot` test helper returns a `u32` index.
   
   `TopKRow` and `TopKHeap` are private to the `topk` module, so no public API 
changes.
   
   ## What is the testing strategy for this PR?
   
   This changes a type without changing behavior. The existing TopK unit tests 
in `datafusion/physical-plan/src/topk/mod.rs` and the TopK sqllogictests (for 
example `topk.slt`) cover it. No new tests are added. Locally, `cargo check -p 
datafusion-physical-plan --tests` and the TopK unit tests (`cargo test -p 
datafusion-physical-plan --lib topk`) pass.
   
   ## Are there any user-facing changes?
   
   No.
   
   ---
   
   AI disclosure: This change and description were prepared with an AI coding 
assistant (Cursor) at the author's direction, following the [DataFusion AI 
policy](https://datafusion.apache.org/contributor-guide/ai-policy.html). It 
contains no third-party code.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to