jorisvandenbossche commented on issue #49232: URL: https://github.com/apache/arrow/issues/49232#issuecomment-5129130975
> Would it still make sense to do what [@tadeja](https://github.com/tadeja) is proposing [@jorisvandenbossche](https://github.com/jorisvandenbossche) ? That would be _my_ preference, but I think that requires some other maintainers with an opinion to chime in as well To clarify a bit more the differences between `pyarrow.feather` and `pyarrow.ipc` from a user point of view: - The Feather API provides "one-liner" options (i.e. just doing a `read_table` / `write_feather` single function call), while `pyarrow.ipc` uses a file-like interface where you first have to open a file to then read from it / write to it (https://arrow.apache.org/docs/python/ipc.html#writing-and-reading-random-access-files) - The Feather API allows to read a subset of the columns by providing a `columns=[..]` argument. The IPC interface does allow you to select fields as well (through `IPCReadOptions::included_fields`, but you have to first manually translate the column names to the field indices --- I think the questions for us to discuss / answer are: - Do we want such a more "direct" (single read/write call) API for Arrow IPC files as well? (we do have that for other IO formats like Parquet or csv as well) - If we want the above, do we keep that under the "Feather" name, or do we expand the `pyarrow.ipc` interface with such functions / options? For the "Feather" term, there was some discussion about this in 2022 (https://lists.apache.org/thread/d2l7179yd41gfdhktrl1h06gh31l8l4l), and I think the general agreement is that this naming is deprecated and should be discouraged. But the question about what term to use instead was not really finalized IMO (there was the idea to use "Arrow File" without the "IPC" in it) So if the outcome would be to add something like `pyarrow.read_arrow_file` or `pyarrow.ipc.read_file`, that ideally happens before deprecating the Feather V2 reader/writer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
