Marc Garcia created ARROW-10419:
-----------------------------------
Summary: Add max_rows parameter to pyarrow.csv.ReadOptions
Key: ARROW-10419
URL: https://issues.apache.org/jira/browse/ARROW-10419
Project: Apache Arrow
Issue Type: New Feature
Components: Python
Reporter: Marc Garcia
I'm trying to read only the first 1,000 rows of a huge CSV with PyArrow.
I don't see a way to do this with Arrow. I guess it should be easy to implement
by adding a `max_rows` parameter to pyarrow.csv.ReadOptions.
After reading the first 1,000, it should be possible to load the next 1,000 (or
any other chunk) by using both the new `max_rows` together with `skip_rows`
(e.g. `pyarrow.csv.read_csv(path, pyarrow.csv.ReadOption(skip_rows=1_000,
max_rows=1_000)` would read from 1,000 to 2,000).
Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
