alamb commented on PR #5790: URL: https://github.com/apache/arrow-datafusion/pull/5790#issuecomment-1499464262
> Thanks @alamb. My previous comment may be misleading. The main controversial is the metrics of time_elapsed_scanning_total contains the times cost by its parent operators, which may be misleading. By the above metrics, developers may think the bottleneck is the parquet scanning. However, it should be the aggregation whose time is wrongly counted in the metrics time_elapsed_scanning_total I think the parent/child relationships of where time is being spent will be hard to capture in scalar metrics (aka any sort of "elapsed time" metrics). I think to get that level of detail, DataFusion would likely need some sort of tracing / span abstraction that can show the heirarchy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
