[
https://issues.apache.org/jira/browse/ARROW-16688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17544033#comment-17544033
]
Dewey Dunnington commented on ARROW-16688:
------------------------------------------
cc [~jorisvandenbossche]
> [R][Python] Extension types cannot be registered in both R and Python
> ---------------------------------------------------------------------
>
> Key: ARROW-16688
> URL: https://issues.apache.org/jira/browse/ARROW-16688
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python, R
> Reporter: Dewey Dunnington
> Priority: Major
>
> When registering extension types as is now possible in the R bindings, it
> looks as though we cannot register an extension type in R and Python at the
> same time:
> {code:R}
> # apache/arrow@master
> library(arrow, warn.conflicts = FALSE)
> library(reticulate)
> # this is a virtualenv with pyarrow installed against the same commit
> use_virtualenv(
> "/Users/deweydunnington/Desktop/rscratch/pyarrow-dev",
> required = TRUE
> )
> pa <- import("pyarrow")
> pa[["__version__"]]
> #> [1] "9.0.0.dev131+g8a36f0f6c"
> py_run_string("
> import pyarrow as pa
> class TestExtensionType(pa.ExtensionType):
>
> def __init__(self):
> super().__init__(pa.int32(), 'arrow.test_type')
>
> def __arrow_ext_serialize__(self):
> return b''
> @classmethod
> def __arrow_ext_deserialize__(cls, storage_type, serialized):
> return cls()
> pa.register_extension_type(TestExtensionType())
> ")
> arrow::register_extension_type(
> arrow::new_extension_type(int32(), "arrow.test_type")
> )
> #> Error: Key error: A type extension with name arrow.test_type already
> defined
> {code}
> I also get a segfault if I try to surface a Python type into R (probably
> because the R bindings mistakenly assume that if {{type.id() ==
> Type::EXTENSION}} then it is safe to cast to our own {{ExtensionType}} C++
> subclass that implements R-specific things.
> This came about because the 'geoarrow' Python and 'geoarrow' R packages both
> register a number of extension type definitions.
> - geoarrow's Python registration:
> https://github.com/jorisvandenbossche/python-geoarrow/blob/main/src/geoarrow/extension_types.py#L108-L117
> - geoarrow's R registration:
> https://github.com/paleolimbot/geoarrow/blob/master/R/pkg-arrow.R#L208-L223
> I can also force an interaction if I build GDAL against the same Arrow that
> the arrow R package is linked against and attempt to load a Feather file
> saved with an extension type using the sf package. I will attempt to recreate
> that interaction as well in both R and Python.
> I don't know enough about linking to know to what extent this is linked to my
> own development setup/build of the R package, although I think there are at
> least some environments where a shared library is picked up first by the R
> config script (fedora36, for example). It does look like my own R package
> build is dynamically linking to libarrow.dylib.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
