alamb commented on PR #27110:
URL: https://github.com/apache/datafusion/pull/27110#issuecomment-6661954541
As an alternative, we could maybe even add a more general API for tracing to
cover other important events (like the sql planning, analyzer, optimizer
passes, etc)
Something like
```rust
#[non_exhaustive]
pub enum TraceEvent<'a> {
// ---- planning ----
/// Span: parsing SQL text into a statement
ParseSql { sql: &'a str },
/// Event: one analyzer rule ran
AnalyzerRule { rule: &'a str, plan: &'a dyn Display },
...
// ---- parquet scan ----
/// Event: the scan started opening a file
ParquetOpenFile { path: &'a str, size: u64 },
/// Span: footer metadata load; `cached` says whether the metadata cache
hit
ParquetLoadMetadata { path: &'a str, cached: bool },
...
}
/// Registered once on the session; every subsystem reports to it
pub trait TraceObserver: Debug + Send + Sync {
...
}
// ---- registration: one place ----
let ctx = SessionContext::new_with_state(
SessionStateBuilder::new()
.with_trace_observer(Arc::new(MyObserver)) // stored on
SessionState
.build(),
);
// SessionState::task_ctx() copies it onto TaskContext, like the function
registry.
```
Then we could instrument call sites like
Planner
```rust
// EXPLAIN VERBOSE and tracing consume the same points.
self.optimizer().optimize(plan, self, |plan, rule| {
observer.event(&TraceEvent::OptimizerRule { rule: rule.name(), pass,
transformed, plan });
})?;
```
or in the Parquet opener
```rust
observer.event(&TraceEvent::ParquetOpenFile { path: &file_name, size });
...
let _guard = observer.span(&TraceEvent::ParquetRowGroupsPruned { .. }); //
sync regio
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
