adamreeve commented on issue #8472: URL: https://github.com/apache/arrow-rs/issues/8472#issuecomment-3387858071
> I have no idea how to handle the LessSafeKey. I got a bit lost trying to understand what's actually stored in there too. If it's important to have the memory usage be accurate, we could change `FileDecryptor` to only store the key bytes and create the `RingGcmBlockDecryptor` on demand? We might just want to check the performance overhead on a file that uses uniform encryption, but I think it should be negligible. Handling the `KeyRetriever` seems tricky. It's possible that the same retriever could be shared across multiple files, so even if we knew the size of the `KeyRetriever` it might not be correct to add it to the memory used by each file. Although at least in our `parquet-key-management` crate we do currently [create a new KeyRetriever instance per file](https://github.com/G-Research/parquet-key-management-rs/blob/165b7e193f992586be31887b3286e70ce7aa3567/parquet-key-management/src/crypto_factory.rs#L306), and this holds an Arc to some state that's shared between retriever instances. Maybe it's fine to add a new `heap_size` method to the trait and have a default implementation that returns 0, and document that if a user creates a retriever per file and needs accurate memory accounting, they should implement this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
