djanderson commented on PR #14286:
URL: https://github.com/apache/datafusion/pull/14286#issuecomment-2654795222
Is there extra work I'd need to do to get this to work with the
`ParquetSink`? This isn't a full reproducer but just a quick copy of the
relevant section of the `do_put_statement_ingest()` method in my flightsql
server.
I get
> flight_sql: writing data to disk
thread 'tokio-runtime-worker' panicked at
/Users/dja/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5:
A Tokio 1.x context was found, but timers are disabled. Call `enable_time`
on the runtime builder to enable timers.
```rust
let data_sink = ParquetSink::new(file_sink_config, table_options);
let record_batch_stream =
FlightRecordBatchStream::new_from_flight_data(flight_data_stream.map_err(|e|
e.into()));
let stream = Box::pin(RecordBatchStreamAdapter::new(
schema,
record_batch_stream.map_err(|e|
DataFusionError::External(Box::new(e))),
));
// This displays "IoObjectStore"
info!(
"{}",
self.datafusion_state
.runtime_env()
.object_store(object_store_url.as_ref())
.unwrap()
);
info!("writing data to disk");
let rows_written = self
.dedicated_rt
.spawn(async move { data_sink.write_all(stream,
&datafusion.task_ctx()).await })
.await
.map_err(|e| Status::data_loss(format!("error processing stream:
{e}")))?
.map_err(|e| Status::data_loss(format!("data write failed:
{e}")))?;
info!("wrote {rows_written} rows");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
