slfan1989 opened a new issue, #3591:
URL: https://github.com/apache/iceberg-rust/issues/3591
### Apache Iceberg Rust version
V0.10.1
### Describe the bug
Setting `ArrowReaderBuilder::with_data_file_concurrency_limit(0)` causes the
resulting record batch stream to remain pending indefinitely, even when the
input task stream is empty.
The builder accepts zero unchanged and passes it to
`try_buffer_unordered(0)`. This prevents the input stream from being polled, so
the reader cannot observe that there are no tasks and finish.
This reproduces without data files, delete files, or network requests.
Setting the concurrency limit to 1 allows the same empty stream to complete.
### To Reproduce
Add the following test as `crates/iceberg/tests/zero_concurrency_probe.rs`:
```
use futures::{FutureExt, TryStreamExt};
use iceberg::Runtime;
use iceberg::arrow::ArrowReaderBuilder;
use iceberg::io::FileIO;
use iceberg::scan::FileScanTaskStream;
#[tokio::test]
async fn zero_concurrency_empty_stream_completes() {
let reader =
ArrowReaderBuilder::new(FileIO::new_with_fs(), Runtime::current())
.with_data_file_concurrency_limit(0)
.build();
let tasks =
Box::pin(futures::stream::empty()) as FileScanTaskStream;
let batches = reader
.read(tasks)
.unwrap()
.stream()
.try_collect::<Vec<_>>()
.now_or_never()
.expect("empty task stream should complete")
.unwrap();
assert!(batches.is_empty());
}
```
The test fails at `expect` because the stream returns `Pending`. Replacing
`0` with `1` makes it pass. Removing `now_or_never()` and awaiting collection
instead leaves the test waiting indefinitely.
Both cases were verified against the commit above.
### Expected behavior
A zero concurrency limit should be handled explicitly, either by treating it
as a positive minimum such as `1`, or by returning a validation error. It
should not cause an indefinite wait.
### Willingness to contribute
I can contribute a fix for this bug independently
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
