zeroshade opened a new pull request, #12876: URL: https://github.com/apache/arrow/pull/12876
Currently the BufferSize in the `ReaderProperties` isn't utilized properly when enabling BufferedStreams. This fixes that issue so that enabling `BufferedStream` reading via the properties will correctly utilize the given buffer size when reading. The default buffersize is currently 16K, so reads that are larger than that will ignore the buffering and just pull directly from the underlying reader when BufferedStream is enabled, pulling the entire page or otherwise from the reader if BufferedStream is not enabled. The buffer size can be set larger so that controlled reads can improve performance on high-latency readers without having to use the memory to read the entire column/page/row group into memory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
