ernestprovo23 commented on PR #20767: URL: https://github.com/apache/datafusion/pull/20767#issuecomment-4090132443
rebased on latest main — the stream construction was refactored to use `PushDecoderStreamState` with `unfold`, so I adapted the fix to wrap the compute path (`project_batch` + `copy_arrow_reader_metrics`) inside `transition()` with the `elapsed_compute` timer. the I/O path (`NeedsData` / `get_byte_ranges`) is intentionally excluded. also updated the `.slt` test in `dynamic_filter_pushdown_config.slt` to use `<slt:ignore>` for the now-populated `elapsed_compute` value. @mbutrovich — good point on timer overhead. the timer here fires once per decoded batch (same granularity as every other operator using `BaselineMetrics`). the `Instant::now()` cost (~25ns on most platforms) is negligible relative to the per-batch projection work. DataFusion already uses this pattern across `FilterExec`, `ProjectionExec`, `SortExec`, etc., so this keeps ParquetExec consistent with the rest of the execution engine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
