mapleFU commented on PR #37514:
URL: https://github.com/apache/arrow/pull/37514#issuecomment-1724829474
Hi @bkietz , I've write a test using a extended file-reader, it can
reproduce the case.
```
class AsyncBufferReader : public ::arrow::io::BufferReader {
public:
explicit AsyncBufferReader(std::shared_ptr<Buffer> buffer, const
::arrow::io::IOContext& ctx): ::arrow::io::BufferReader(std::move(buffer)),
ctx_(ctx) {}
/// EXPERIMENTAL: Read data asynchronously.
Future<std::shared_ptr<Buffer>> ReadAsync(const ::arrow::io::IOContext&
ctx, int64_t position,
int64_t nbytes) override {
auto self = checked_pointer_cast<AsyncBufferReader>(shared_from_this());
return DeferNotOk(ctx.executor()->Submit([self, position, nbytes]() {
return self->ReadAt(position, nbytes);
}));
}
const ::arrow::io::IOContext& io_context() const override {
return ctx_;
}
private:
::arrow::io::IOContext ctx_;
};
```
However, default io-pool is widely used in scanner, it's a bit hard to hook
all io_context( Might need to change `EnsureCompleteMetadata` to below) :
```c++
Status EnsureCompleteMetadata(parquet::arrow::FileReader* reader = NULLPTR,
const std::shared_ptr<ScanOptions>& scan_options=NULLPTR);
```
So I change the global thread pool during testing, would you mind take a
look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
