scoder commented on PR #34616: URL: https://github.com/apache/arrow/pull/34616#issuecomment-1753236007
> It will not be a `def` function but a `cdef` function (actually, several of them) since it has to handle C++ types. Given the length of the discussion so far, I'm sure I'm missing something, but aren't we talking about these bits? https://github.com/apache/arrow/blob/d2f15841b32571b5ead6ec50c638b6e6a291856c/python/pyarrow/_dataset_parquet.pyx#L847-L849 https://github.com/apache/arrow/blob/d2f15841b32571b5ead6ec50c638b6e6a291856c/python/pyarrow/_dataset_parquet.pyx#L876-L877 I would write this as a (Cython compiled) Python function as follows (using badly chosen names): ```cython def set_encryption_config(pfso: ParquetFragmentScanOptions, pdc: ParquetDecryptionConfig): cdef shared_ptr[CParquetDecryptionConfig] c_config = pdc.unwrap() pfso.parquet_options.parquet_decryption_config = c_config ``` Then you can declare `ParquetFragmentScanOptions._parquet_decryption_config` as a plain Python object attribute, just to keep the reference. From what I read in this PR, `ParquetFragmentScanOptions` does not need to know how Parquet encryption works, as long as it knows how to call a Python function and use Python objects that handle it for it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
