jorisvandenbossche commented on PR #34616:
URL: https://github.com/apache/arrow/pull/34616#issuecomment-1753226304
So I have a somewhat working implementation where I moved the conditional
parts to a `_dataset_parquet_encryption.pyx` file, with a shim
`_dataset_parquet_no_encryption.pyx` that contains just dummies with a runtime
error message in case encryption is not available. During compilation, we
switch the source files for the cythonize step of a
`_dataset_parquet_encryption` module between those two.
The `pxd` file for this essentially looks like:
```
from pyarrow.includes.libarrow_dataset_parquet cimport
CParquetFragmentScanOptions
cdef bint is_encryption_enabled()
cdef set_decryption_config(CParquetFragmentScanOptions * parquet_options,
config)
```
However, what I cannot yet figure out is how to deal with this pxd file.
Should I have two versions of that `_dataset_parquet_encryption.pxd` and
`_dataset_parquet_no_encryption.pxd`? (although they are exactly the same)
But how does the cythonize step deal with that?
If I only have a single `_dataset_parquet_encryption.pxd`, then compiling
`_dataset_parquet_no_encryption.pxd` to a `_dataset_parquet_encryption` module,
I get errors like "pyarrow/_dataset_parquet_encryption.pxd' not found".
However, when I add a `_dataset_parquet_no_encryption.pxd` with the same
content, I get a huge amount of seemingly unrelated compiler errors like:
```
Error compiling Cython file:
------------------------------------------------------------
...
bint operator>= (const string&)
bint operator>= (const char*)
string to_string(int val) except +
string to_string(long val) except +
^
------------------------------------------------------------
/home/joris/miniconda3/envs/arrow-dev/lib/python3.10/site-packages/Cython/Includes/libcpp/string.pxd:290:20:
Function signature does not match previous declaration
....
Error compiling Cython file:
------------------------------------------------------------
...
cdef extern from "arrow/util/iterator.h" namespace "arrow" nogil:
cdef cppclass CIterator" arrow::Iterator"[T]:
CResult[T] Next()
CStatus Visit[Visitor](Visitor&& visitor)
^
------------------------------------------------------------
pyarrow/includes/libarrow.pxd:2721:38: Expected ')', found '&&'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
