0lai0 opened a new pull request, #1462: URL: https://github.com/apache/mahout/pull/1462
### Related Issues <!-- Closes #123 --> Closes #1436 ### Changes - [ ] Bug fix - [ ] New feature - [x] Refactoring - [ ] Documentation - [x] Test - [ ] CI/CD pipeline - [ ] Other ### Why <!-- Why is this change needed? --> `StreamingProducer` tracked consumed elements with a `buffer_cursor` and, once the cursor passed the halfway mark (`BUFFER_COMPACT_DENOM`), reclaimed the prefix with `Vec::drain(..cursor)` — an O(n) memmove of the retained tail, on the streaming hot path. A `VecDeque` advances its head instead, so discarding a consumed prefix never shifts the data that is still live: O(1) amortized buffer advance, no periodic compaction pass, and no change in output. ### How - `buffer: Vec<T>` + `buffer_cursor: usize` → `buffer: VecDeque<T>`; removed the now-unused `BUFFER_COMPACT_DENOM` compaction heuristic - `produce()` copies the batch out of `as_slices()` (stitching both halves when a batch straddles the ring's wrap boundary), then `drain(..take)`s the consumed prefix. Copying via the slices keeps the batch copy on `extend_from_slice`'s bulk path — `Drain` is not `TrustedLen`, so `extend(drain)` would copy element by element. Recycled batch buffers are still reused, so the batch copy itself stays allocation-free - Refills use `extend(&scratch[..written])` rather than `.iter().copied()`: `VecDeque` specializes `Extend<&T> for T: Copy` into a bulk `copy_slice` - The build path pre-reserves the steady state (`batch_size * sample_size + initial_cap`, the peak live length) so the ring never reallocates mid-run. `VecDeque::from(Vec)` reuses the existing allocation, so this only tops the capacity up. ## Checklist - [x] Added or updated unit tests for all changes - [x] Added or updated documentation for all changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
