westonpace commented on PR #13442: URL: https://github.com/apache/arrow/pull/13442#issuecomment-1170226163
> Would the "better" approach be something like determine optimal block_size based on blob size unless manually specified on ReadOptions? @raulcd I think so. Either grow the block size when a large blob is encountered or gather the blocks in a queue until read to process. It should be fairly straightforward for a single-threaded reader but might be more complicated if we have a multi-threaded JSONL reader (I don't recall which the current reader is). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
