alamb commented on issue #15216: URL: https://github.com/apache/datafusion/issues/15216#issuecomment-2836077703
> With the KMS API not being included in arrow-rs but being built as a third-party crate (https://github.com/apache/arrow-rs/pull/7387#issuecomment-2819908130), I would assume we probably don't want to depend on that in Datafusion, but keep the encryption integration more flexible to allow other approaches? I think that is correct. I agree the key question here will be "how to integrate / use a KMS system without making DataFusion dependent on such a system" As I think you have been discussing the major challenge will be configuration. I personally suggest using the `Arc<dyn Any>` approach, but I don't really understand the requirements enough to know What i suggest as a next step is to make a public example of reading encrypted parquet files files, and then try to integrate that example with the KMS -- to get the example working you'll very likely have to add some new APIs to DataFusion, but the example will drive that implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
