c-thiel opened a new issue, #1720: URL: https://github.com/apache/iceberg-rust/issues/1720
### Apache Iceberg Rust version None ### Describe the bug Currently we use a weighed moka Cache to store read objects: https://github.com/apache/iceberg-rust/blob/dc349284a4204c1a56af47fb3177ace6f9e899a0/crates/iceberg/src/io/object_cache.rs#L63-L69 However, the current implementation only takes the stack size of the struct itself into account. Thus, a cached `ManifestList` would return a low number, while its many `entries` could allocate a significant number of storage on the heap. I am unsure if exact calculation is feasable. I would opt for a rough estimate, such as 2KB for a list and 8KB for a Manfest. Let me know what you think! ### To Reproduce Create a ManifestList with many entries. Read it cached, watch the memory usage significantly overshoot the set max_capacity. ### Expected behavior _No response_ ### Willingness to contribute None -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
