mohabouje commented on issue #37144:
URL: https://github.com/apache/arrow/issues/37144#issuecomment-1677158079
The code does not compile because the
`::arrow::Table::FromRecordBatcheReader` is expecting a `RecordBatchReader`,
while we are building a `RecordBatchFileReader`.
Using the buffer `::arrow::io::BufferReader` to avoid dumping the data into
a file definitely helped. The missing part is the need to create a vector
holding all the `::arrow::RecordBatch` to create an `::arrow::Table` after. Any
idea of how to avoid this intermediate step?
```cpp
auto read(std::span<const char> buffer) -> std::shared_ptr<arrow::Table> {
::arrow::Buffer arrow_buffer(buffer.data(), buffer.size());
::arrow::io::BufferReader buffer_reader(arrow_buffer);
auto const ipc_reader =
::arrow::ipc::RecordBatchFileReader::Open(&buffer_reader);
if (!ipc_reader.ok()) {
return nullptr;
}
auto const reader = ipc_reader.ValueOrDie();
auto const num_record_batches = reader->num_record_batches();
auto batches =
std::vector<std::shared_ptr<::arrow::RecordBatch>>(num_record_batches);
for (auto i = 0; i < num_record_batches; ++i) {
auto const batch = reader->ReadRecordBatch(i);
if (!batch.ok()) {
return nullptr;
}
batches[i] = batch.ValueOrDie();
}
auto const creation = ::arrow::Table::FromRecordBatches(batches);
if (!creation.ok()) {
return nullptr;
}
return creation.ValueOrDie();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
