etseidl commented on code in PR #8671:
URL: https://github.com/apache/arrow-rs/pull/8671#discussion_r2448744488
##########
parquet/src/file/metadata/memory.rs:
##########
@@ -50,6 +51,16 @@ impl<T: HeapSize> HeapSize for Vec<T> {
}
}
+impl<K: HeapSize, V: HeapSize> HeapSize for HashMap<K, V> {
Review Comment:
Yeah, if the point is to not overrun available memory, it's probably safer
to overestimate.
##########
parquet/src/encryption/decrypt.rs:
##########
@@ -547,6 +572,21 @@ impl PartialEq for FileDecryptor {
}
}
+/// Estimate the size in bytes required for the file decryptor.
+/// This is important to track the memory usage of cached Parquet meta data,
+/// and is used via [`crate::file::metadata::ParquetMetaData::memory_size`].
+/// Note that when a [`KeyRetriever`] is used, its heap size won't be included
+/// and the result will be an underestimate.
+/// If the [`FileDecryptionProperties`] are shared between multiple files then
the
+/// heap size may also be an overestimate.
Review Comment:
❤️
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
