westonpace commented on issue #15151: URL: https://github.com/apache/arrow/issues/15151#issuecomment-1369035221
Yes, I think it's entirely appropriate to put RecordBatchReader->source node in the C++ code. On the output side we have collector variants for tables (DeclarationToTable), vector of record batches (DeclarationToBatches) and record batch reader (DeclarationToReader). We already have source node variants for accepting data from a table (`table_source`) and a vector of record batches (`record_batch_source`). So I think it would be a good addition to add `record_batch_reader_source`. However, I'm also not sure why we wouldn't expose the default I/O pool (`arrow::io::internal::GetIOThreadPool`) in the public headers given that we have public methods for getting and setting the size. As a short term hack you can do: ``` #include "arrow/io/type_fwd.h" arrow::io::IOContext io_context = arrow::io::default_io_context(); arrow::internal::Executor* io_executor = io_context.executor(); ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
