Thanks for the pointer, Aldrin! Works like a charm. On Fri, Jul 5, 2024 at 8:05 PM Aldrin <octalene....@pm.me> wrote:
> Instead of using RecordBatchFileReader, maybe you can try ipc::ReadTensor > [1]. > > [1]: https://github.com/apache/arrow/blob/main/cpp/src/arrow/ipc/reader.h > #L583-L595 > > Sent from Proton Mail <https://proton.me/mail/home> for iOS > > > On Fri, Jul 5, 2024 at 02:53, Benjamin Tan <benjaminhm...@gmail.com > <On+Fri,+Jul+5,+2024+at+02:53,+Benjamin+Tan+%3C%3Ca+href=>> wrote: > > Hi > > I am trying to use Apache Arrow as a way to pass tensors between Python > and C++. > Both are Version 16.1.0 though installed separately (apt-get for C++ and > pip for Python). > Currently, I am getting the error "Invalid: Not an Arrow file" from using > the code below. > > Thanks in advance for your help! > Benjamin > > Tried to use the following functions in pyarrow.ipc to write pa.Tensors to > disk: > with pa.OSFile(test_prefix + weight_filename, "wb") as sink: > pa.ipc.write_tensor(arrow_weight, sink) > with pa.OSFile(test_prefix + bias_filename, "wb") as sink: > pa.ipc.write_tensor(arrow_bias, sink) > > And load from C++ with: > std::shared_ptr<arrow::io::ReadableFile> infile; > ARROW_ASSIGN_OR_RAISE( > infile, arrow::io::ReadableFile::Open(test_prefix + data_path, > arrow::default_memory_pool())); > ARROW_ASSIGN_OR_RAISE(auto ipc_reader, > arrow::ipc::RecordBatchFileReader::Open(infile)); > > > > >
