lidavidm commented on code in PR #13454:
URL: https://github.com/apache/arrow/pull/13454#discussion_r913924665
##########
python/pyarrow/types.pxi:
##########
@@ -902,6 +902,18 @@ cdef class ExtensionType(BaseExtensionType):
"""
return ExtensionArray
+ def scalar_as_py(self, scalar):
+ """Convert scalar to a Python type.
+
+ This method can be overridden in subclasses to customize what type
+ scalars are converted to.
+
+ Parameters
+ ----------
+ scalar : pyarrow.ExtensionScalar
+ The scalar to be converted to a Python object.
+ """
+ return scalar.as_py() if scalar is not None else None
Review Comment:
This should no longer be able to be None right?
Also is the type ExtensionScalar? It seems we pass the underlying value
right?
##########
python/pyarrow/tests/test_extension_type.py:
##########
@@ -261,16 +297,19 @@ def test_ext_scalar_from_array():
assert s.type == ty1
if val is not None:
assert s.value == pa.scalar(val, storage.type)
+ assert s.as_py() == UUID(bytes=val)
else:
assert s.value is None
- assert s.as_py() == val
scalars_b = list(b)
assert len(scalars_b) == 4
for sa, sb in zip(scalars_a, scalars_b):
assert sa.is_valid == sb.is_valid
- assert sa.as_py() == sb.as_py()
+ if sa.as_py() is None:
+ assert sa.as_py() == sb.as_py()
+ else:
+ assert sa.as_py().bytes == sb.as_py()
assert sa != sb
Review Comment:
Is there still a test of converting a type that uses the default
`scalar_as_py`?
##########
docs/source/python/extending_types.rst:
##########
@@ -286,6 +286,33 @@ are available).
The same ``__arrow_ext_class__`` specialization can be used with custom types
defined
by subclassing :class:`ExtensionType`.
+Custom scalar conversion
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you want scalars of your custom extension type to convert to a custom type
when
+:meth:`ExtensionScalar.as_py()` is called, you can override the
:meth:`ExtensionType.scalar_as_py()`
+method. For example, if we wanted the above example 3D point type to return a
custom
+3D point class instead of a list, we would implement::
+
+ Point3D = namedtuple("Point3D", ["x", "y", "z"])
+
+ class Point3DType(pa.PyExtensionType):
+ def __init__(self):
+ pa.PyExtensionType.__init__(self, pa.list_(pa.float32(), 3))
+
+ def __reduce__(self):
+ return Point3DType, ()
+
+ def scalar_as_py(self, scalar):
Review Comment:
```suggestion
def scalar_as_py(self, scalar: ListScalar) -> Point3D:
```
maybe for clarity about types?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
