adamreeve commented on PR #7387: URL: https://github.com/apache/arrow-rs/pull/7387#issuecomment-2789417669
> I'm afraid I've not really been following this effort closely, and so I may be missing something but I would have thought this would need to be async to accommodate external stores. This is a good point but I don't think that's really feasible, it would require changing all code paths where encryption is used to be async which I expect would be a very large and breaking change. Providing a non-async API doesn't prevent you from communicating with external stores, you can always use `block_on`, but I can see how some users might prefer to avoid that. > I wonder if this makes sense to include in the parquet crate proper, or if it could be some third-party crate. Yes this could easily be a third party crate. My concern with that is that it would be less discoverable, and users would assume that if they want Parquet encryption they should use the existing lower level API where encryption keys are specified directly. But I believe users should be directed to using this KMS based API if possible to push them towards better security practices. > Supporting external KMS providers is a monumental undertaking I don't think any KMS specific clients should be included in the Parquet crate, those should definitely be third party crates. Arrow C++ and PyArrow don't include any KMS client implementations either. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
