leaves12138 commented on code in PR #128:
URL:
https://github.com/apache/paimon-vector-index/pull/128#discussion_r4584017754
##########
core/src/diskann_io.rs:
##########
@@ -1693,6 +1766,58 @@ impl<R: SeekRead> DiskAnnIndexReader<R> {
self.last_search_stats
}
+ pub(crate) fn with_graph_cache_budget<T>(
+ &mut self,
+ borrow_raw: bool,
+ operation: impl FnOnce(Option<Arc<Vec<u8>>>) -> io::Result<T>,
+ ) -> io::Result<T> {
+ if !borrow_raw {
+ return operation(None);
+ }
+ let resident = self
+ .resident
+ .as_ref()
+ .ok_or_else(|| invalid_data("DiskANN resident state is missing"))?
+ .clone();
+ let cache_bytes = resident
+ .raw_vector_cache
+ .total_capacity()
+ .saturating_add(resident.adjacency_cache.total_capacity());
+ let length = cache_bytes.saturating_sub(64).min(16 * 1024 * 1024).min(
+ (self.header.sections.adjacency.length as usize)
+ .saturating_sub(self.hot_adjacency.len()),
+ ) / DISKANN_PAGE_SIZE as usize
+ * DISKANN_PAGE_SIZE as usize;
+ let mut loan = AdjacencyCacheLoan::new(
+ &resident.adjacency_cache,
+ &resident.raw_vector_cache,
+ if length > 0 { cache_bytes } else { 0 },
+ )?;
+ let extension = if length > 0 {
+ let mut payload = Vec::new();
+ payload
+ .try_reserve_exact(length)
Review Comment:
Verified fixed in abdc8ac. I reran the original independent reproduction
against the unchanged new production modules: with a 65 KiB combined cache
allowance and a 16 KiB prefix, the previously retained raw-vector allocations
are now all released (0 live bytes). The new regression also covers parent,
retained and active worker references, failure/unwind restoration, and the
no-prefix path. The full release workspace suite passes. Thanks for addressing
this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]