tustvold commented on PR #7387: URL: https://github.com/apache/arrow-rs/pull/7387#issuecomment-2789020286
I'm afraid I've not really been following this effort closely, and so I may be missing something but I would have thought this would need to be async to accommodate external stores. Taking a step back though I wonder if this makes sense to include in the parquet crate proper, or if it could be some third-party crate. Is there some way we could add the necessary hooks to parquet-rs, if they don't already exist, and have this be an external project. I suspect arrow-cpp bundles all this largely for packaging reasons that don't apply to arrow-rs. I say this for a few reasons * Supporting external KMS providers is a monumental undertaking * There is very limited arrow-rs review budget * Key management is hard and needs to be done and reviewed with care Basically I'm a little concerned that the complexity and risks involved in this outstrip the arrow-rs project's ability to effectively review it... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
