sandy-sachin7 opened a new pull request, #10168:
URL: https://github.com/apache/arrow-rs/pull/10168
## Which issue does this PR close?
Closes #8231.
## Rationale for this change
When `ParquetObjectReader` is constructed without an explicit `runtime`
(i.e., via `ParquetRecordBatchStreamBuilder::new(reader)`), the `spawn` method
falls through to the `None` branch and runs async closures inline. For object
store implementations (S3, HTTP, GCS via `object_store` crate), this breaks
connection pooling and DNS resolution because the underlying `reqwest` client
relies on `tokio::spawn` to propagate the runtime context.
The result is a panic: *"there is no reactor running, must be called from
the context of a Tokio 1.x runtime"*
## What changes are included in this PR?
Changed `ParquetObjectReader::spawn` to first check `Handle::try_current()`
when `self.runtime` is `None`, discovering an ambient tokio runtime if one
exists. Only falls back to inline execution when no tokio runtime context is
available at all.
Before:
```rust
match &self.runtime {
Some(handle) => { handle.spawn(...) }
None => { /* inline — breaks S3/HTTP */ }
}
```
After:
```rust
let handle = self.runtime.clone().or_else(|| Handle::try_current().ok());
match handle {
Some(handle) => { handle.spawn(...) }
None => { /* inline — only when no runtime at all */ }
}
```
## Are these changes tested?
1166 parquet lib tests pass. No new tests added as the existing test
infrastructure doesn't exercise the `spawn()` method with async object store
implementations (those are integration-level tests).
## Are there any user-facing changes?
`ParquetObjectReader` will now correctly discover a tokio runtime when used
inside `#[tokio::main]` or any other tokio runtime context, even without an
explicit runtime being configured.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
