[
https://issues.apache.org/jira/browse/ARROW-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16881581#comment-16881581
]
Joris Van den Bossche commented on ARROW-5610:
----------------------------------------------
I am trying to wrap my head around what is needed here. For the "send" part, we
want to create a RecordBatch with an array that has an ExtensionType with a
custom name. Currently, in C++ to have an ExtensionType with a custom name, you
need to create and register a subclass (eg the {{UUIDType}} in the tests).
So if we want to create and register such a subclass from python, does that
mean we need to make a new C++ subclass from cython (not sure if that is
actually possible) ? Or, are there ways to circumvent the need of a new class
in C++ per extension type? Eg by having a single C++ class that takes the name
as constructor argument (so where the {{extension_name}} attribute is not a
constant), and then register an instance of that?
Currently, the Python types can be constructed from the C++ type because the
pickled type is stored as the serialized metadata. For such a generic extension
type that can be send to or come from other languages, we can't use pickle.
Would we then need some *python-specific* registry (different from the C++
registry of extension types) for mapping extension names to a python
constructor?
> [Python] Define extension type API in Python to "receive" or "send" a foreign
> extension type
> --------------------------------------------------------------------------------------------
>
> Key: ARROW-5610
> URL: https://issues.apache.org/jira/browse/ARROW-5610
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Reporter: Wes McKinney
> Priority: Major
> Fix For: 1.0.0
>
>
> In work in ARROW-840, a static {{arrow.py_extension_type}} name is used.
> There will be cases where an extension type is coming from another
> programming language (e.g. Java), so it would be useful to be able to "plug
> in" a Python extension type subclass that will be used to deserialize the
> extension type coming over the wire. This has some different API requirements
> since the serialized representation of the type will not have knowledge of
> Python pickling, etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
