Just started with arrow...
I wrote a record batch to a file using ipc::MakeFileWriter to create a writer
and writer->WriteRecordBatch in a C++ program and tried to read it in python
with:
[] import pyarrow as pa
[] reader = pa.ipc.open_file("myfile")
It raises the ArrowInvalid with the message "not an arrow file".
If I write it out as a Table in feather format, I can read it in python. But I
want to write large files on the order of 100GB or more and then read them back
into python as pandas dataframes or something similar.
So, I switched to using an ipc writer.
Can something point me in the right direction? Thanks.
Ken
