[
https://issues.apache.org/jira/browse/ARROW-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114580#comment-16114580
]
Wes McKinney commented on ARROW-1309:
-------------------------------------
Here's the repro; one of the columns is an array type and has all nulls
{code}
In [8]: pa.Array.from_pandas(np.array([[None, None], None], dtype=object))
---------------------------------------------------------------------------
ArrowNotImplementedError Traceback (most recent call last)
<ipython-input-8-c3bdca54ea4c> in <module>()
----> 1 pa.Array.from_pandas(np.array([[None, None], None], dtype=object))
~/code/arrow/python/pyarrow/array.pxi in pyarrow.lib.Array.from_pandas
(/home/wesm/code/arrow/python/build/temp.linux-x86_64-3.5/lib.cxx:22323)()
174 c_type = type.sp_type
175 with nogil:
--> 176 check_status(PandasObjectsToArrow(
177 pool, values, mask, c_type, &chunked_out))
178
~/code/arrow/python/pyarrow/error.pxi in pyarrow.lib.check_status
(/home/wesm/code/arrow/python/build/temp.linux-x86_64-3.5/lib.cxx:7711)()
76 raise ArrowKeyError(message)
77 elif status.IsNotImplemented():
---> 78 raise ArrowNotImplementedError(message)
79 elif status.IsTypeError():
80 raise ArrowTypeError(message)
ArrowNotImplementedError: NotImplemented: null
{code}
> pyarrow.lib.ArrowNotImplementedError: NotImplemented: null
> ----------------------------------------------------------
>
> Key: ARROW-1309
> URL: https://issues.apache.org/jira/browse/ARROW-1309
> Project: Apache Arrow
> Issue Type: Bug
> Environment: centos 7.3
> Reporter: Luke Higgins
> Priority: Minor
> Fix For: 0.6.0
>
>
> I have an avro file in hdfs that I am reading in using fastavro, converting
> to a pandas dataframe and then trying to create an arrow table and get as
> error:
> >>> table=pyarrow.Table.from_pandas(my_dataframe)
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "pyarrow/table.pxi", line 746, in pyarrow.lib.Table.from_pandas
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:34089)
> File "pyarrow/table.pxi", line 346, in pyarrow.lib._dataframe_to_arrays
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:30476)
> File "pyarrow/array.pxi", line 182, in pyarrow.lib.Array.from_pandas
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:22110)
> File "pyarrow/error.pxi", line 66, in pyarrow.lib.check_status
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:7702)
> pyarrow.lib.ArrowNotImplementedError: NotImplemented: null
> The avro schema indeed has null fields possible. Is this not implemented? I
> am using pyarrow 0.5.0. Also, for what I am doing I am not using pandas at
> all, I just read in the avro and I have a list of dicts and really want to
> write them to disk in parquet format and am utilizing these steps (which
> isn't optimal but may be necessary without writing more code of my own).
> thanks,
> Luke
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
