[
https://issues.apache.org/jira/browse/ARROW-12609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17337452#comment-17337452
]
Sergey Mozharov commented on ARROW-12609:
-----------------------------------------
Iteration of invalid list scalars is also broken. In the first example from the
description:
{code:java}
assert hasattr(missing_scalar, '__iter__')
assert len(list(missing_scalar)) == 0 # -> TypeError: 'NoneType' object is not
iterable
{code}
I assume that an invalid ListScalar should be iterable and should produce an
empty iterable object. But I am not sure if this is the right way to think
about it and if it is specified anywhere.
Another example: should __getitem__ with an invalid ListScalar raise an
IndexError instead of TypeError?
{code:java}
missing_scalar[0] # -> TypeError: 'NoneType' object is not subscriptable{code}
> TypeError when accessing length of an invalid ListScalar
> --------------------------------------------------------
>
> Key: ARROW-12609
> URL: https://issues.apache.org/jira/browse/ARROW-12609
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 3.0.0, 4.0.0
> Environment: Windows 10
> python=3.9.2
> pyarrow=4.0.0 (3.0.0 has the same behavior)
> Reporter: Sergey Mozharov
> Priority: Major
>
> For List-like data types, the scalar corresponding to a missing value has
> '___len___' attribute, but TypeError is raised when it is accessed
> {code:java}
> import pyarrow as pa
> data_type = pa.list_(pa.struct([
> ('a', pa.int64()),
> ('b', pa.bool_())
> ]))
> data = [[{'a': 1, 'b': False}, {'a': 2, 'b': True}], None]
> arr = pa.array(data, type=data_type)
> missing_scalar = arr[1] # <pyarrow.ListScalar: None>
> assert hasattr(missing_scalar, '__len__')
> assert len(missing_scalar) == 0 # --> TypeError: object of type 'NoneType'
> has no len()
> {code}
> Expected behavior: length is expected to be 0.
> This issue causes several pandas unit tests to fail when an ExtensionArray
> backed by arrow array with this data type is built.
> This behavior is also inconsistent with a similar example where the data type
> is a struct:
> {code:java}
> import pyarrow as pa
> data_type = pa.struct([
> ('a', pa.int64()),
> ('b', pa.bool_())
> ])
> data = [{'a': 1, 'b': False}, None]
> arr = pa.array(data, type=data_type)
> missing_scalar = arr[1] # <pyarrow.StructScalar: None>
> assert hasattr(missing_scalar, '__len__')
> assert len(missing_scalar) == 0 # Ok
> {code}
> In this second example the TypeError is not raised.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
