adamreeve commented on issue #47435: URL: https://github.com/apache/arrow/issues/47435#issuecomment-3234977585
For a bit more context, here's the old [mailing list discussion](https://lists.apache.org/thread/l7tltjf9w3lcjvngq35fzqsddmylkt3k) and [issue](https://github.com/apache/arrow/issues/24254) where it was decided to only expose the higher level in Python. The main arguments against exposing the low level in Python (from @ggershinsky ) were: > security: low-level encryption API is easy to misuse (eg giving the same keys for a number of different files; this'd break the AES GCM cipher). The high-level encryption layer handles that by applying envelope encryption and other best practices in data security. Also, this layer is maintained by the community, meaning that future improvements and security fixes can be upstreamed by anyone, and available to all. > > compatibility: parquet-mr implements the high-level encryption layer. If we want the files produced by Spark/Presto/etc to be readable by pandas/PyArrow (and vice versa), we need to provide the Arrow users with the high-level API. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
