djanderson commented on PR #14286: URL: https://github.com/apache/datafusion/pull/14286#issuecomment-2655024463
> What is the panic error message? Panic error message is just > error=status: DataLoss, message: "data write failed: IO error: Error joining spawned task: task 93 panicked with message \"A Tokio 1.x context was found, but timers are disabled. Call `enable_time` on the runtime builder to enable timers.\" > Maybe you need to run data_sink.write_all(stream, &datafusion.task_ctx()).await }) somehow on the iO thread? I am nto sure what it is doing Yeah, without dedicated executor, this is where I hit executor starvation, so I can't wrap the whole call in spawn_io, I don't think... otherwise it would be the same as no having the dedicated executor at all. I thought this would work first review because ParquetSink.write_all [retrieves the object_store from the runtime_env](https://github.com/apache/datafusion/blob/branch-44/datafusion/core/src/datasource/file_format/parquet.rs#L793) and and uses it to initialize the async arrow writer. Maybe there's some interaction with the `file_write_tasks` JoinSet that I don't understand? https://github.com/apache/datafusion/blob/branch-44/datafusion/core/src/datasource/file_format/parquet.rs#L865 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
