jorisvandenbossche commented on code in PR #37822:
URL: https://github.com/apache/arrow/pull/37822#discussion_r1651076210
##########
python/pyarrow/_dataset_parquet.pyx:
##########
@@ -737,6 +737,8 @@ cdef class ParquetFragmentScanOptions(FragmentScanOptions):
new CParquetFragmentScanOptions()))
self.use_buffered_stream = use_buffered_stream
self.buffer_size = buffer_size
+ if pre_buffer and not is_threading_enabled():
+ pre_buffer = False
Review Comment:
One problem is that `pre_buffer` defaults to True in some cases, and we
don't want to raise an error by default but we also want to keep that default,
I think.
Adding a documentation note that this keyword only has effect when running
with threading enabled sounds sufficient to me.
(you can argue the same for the `use_threads` keyword)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
