danepitkin commented on code in PR #37097:
URL: https://github.com/apache/arrow/pull/37097#discussion_r1298761676
##########
python/pyarrow/_dataset_parquet.pyx:
##########
@@ -741,7 +737,15 @@ cdef class ParquetFragmentScanOptions(FragmentScanOptions):
thrift_string_size_limit=self.thrift_string_size_limit,
thrift_container_size_limit=self.thrift_container_size_limit,
)
- return type(self)._reconstruct, (kwargs,)
+ return _reconstruct_parquet_fragment_scan_options, (kwargs,)
+
+
+def _reconstruct_parquet_fragment_scan_options(kwargs):
+ # __reduce__ doesn't allow passing named arguments directly to the
+ # reconstructor, hence this wrapper.
+ # In Cython >= 3.0.0, function binding is turned on by default, so
+ # a global static method is used (instead of a class method) for pickling.
Review Comment:
Ahh I remember what happened. This works for Cython 3, but not Cython
0.29.36.
With Cython 0.29.36, we just need to add the `binding=True` compiler option.
my_test.pyx
```
# cython : language_level = 3
import cython
cdef class Foo():
@staticmethod
@cython.binding(True) # This works
def _reconstruct():
return Foo()
def __reduce__(self):
return Foo._reconstruct, tuple()
@staticmethod
def _reconstruct(x):
return Bar(x)
cdef class Bar():
@staticmethod # This does not work
def _reconstruct():
return Bar()
def __reduce__(self):
return Bar._reconstruct, tuple()
```
Usage here:
```
In [1]: from my_test import Foo, Bar
In [2]: f = Foo()
In [3]: b = Bar()
In [4]: import pickle
In [5]: pickle.dumps(f)
Out[5]:
b'\x80\x04\x95#\x00\x00\x00\x00\x00\x00\x00\x8c\x07my_test\x94\x8c\x10Foo._reconstruct\x94\x93\x94)R\x94.'
In [6]: pickle.dumps(b)
---------------------------------------------------------------------------
PicklingError Traceback (most recent call last)
Cell In[6], line 1
----> 1 pickle.dumps(b)
PicklingError: Can't pickle <built-in function _reconstruct>: it's not the
same object as my_test._reconstruct
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
