Kontinuation commented on code in PR #362:
URL: https://github.com/apache/sedona-db/pull/362#discussion_r2579539114
##########
rust/sedona-spatial-join/src/build_index.rs:
##########
@@ -33,7 +33,72 @@ use crate::{
spatial_predicate::SpatialPredicate,
};
-pub(crate) async fn build_index(
+/// Synchronous version of build_index that doesn't spawn tasks
+/// Used in execution contexts without async runtime support (e.g.,
Spark/Comet JNI)
+pub async fn build_index_sync(
Review Comment:
I tried to reproduce this by spawning some async tasks in the native comet
scan operator, but Comet tests involving native scan operator finished
successfully and did not hang. I have also tried setting env
`COMET_WORKER_THREADS=1` and still could not observe hanging. The problem could
be in somewhere else and not related to spawning async tasks.
```rust
impl Stream for ScanStream<'_> {
type Item = DataFusionResult<RecordBatch>;
fn poll_next(self: Pin<&mut Self>, ctx: &mut Context<'_>) ->
Poll<Option<Self::Item>> {
// ...
println!("polling batches from scan: spawn an async task");
let mut join_set = JoinSet::new();
for k in 0..10 {
let res = join_set.spawn(async move {
println!("hello {}", k);
k
});
println!("spawned async task: {:?}", res);
}
let fut = join_set.join_all();
let pin_fut = pin!(fut);
let res = ready!(pin_fut.poll(ctx));
println!("async task result: {:?}", res);
// ...
}
// ...
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
