shner-elmo opened a new issue, #36865:
URL: https://github.com/apache/arrow/issues/36865
### Describe the enhancement requested
First of all thanks for the great project which Pyarrow is.
When I attempt to create a table from a list it works fine, but when I
create it from a generator it complains:
```py
>>> data: list[dict]
>>> table = pa.Table.from_pylist(data, schema=SCHEMA)
>>> table = pa.Table.from_pylist((dct for dct in data), schema=SCHEMA)
Traceback (most recent call last):
File
"/home/shner/.cache/pypoetry/virtualenvs/ame-wikipedia-pipeline-FN8t627X-py3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py",
line 3508, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-15-87d9c5bc5106>", line 1, in <module>
table = pa.Table.from_pylist((dct for dct in data), schema=SCHEMA)
File "pyarrow/table.pxi", line 3906, in pyarrow.lib.Table.from_pylist
File "pyarrow/table.pxi", line 5460, in pyarrow.lib._from_pylist
File "pyarrow/table.pxi", line 3798, in pyarrow.lib.Table.from_arrays
File "pyarrow/table.pxi", line 2962, in pyarrow.lib.Table.validate
File "pyarrow/error.pxi", line 100, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Column 1 named year expected length 389612 but got
length 0
```
I guess that kind of is expected since the method is called `from_pylist()`,
but usually most functions that take in some data will accept more general
types, it would be great if you guys could extend this to accept:
`Iterable[dict[str, ...]] | Iterator[dict[str, ...]]`,
or at least to specify which methods and properties the data structure
should have, for example: `SupportslenAndGetitem`, `typing.Collection` or
`typing.Sequence` and so on.
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
