andygrove opened a new issue, #1110:
URL: https://github.com/apache/datafusion-comet/issues/1110
### Describe the bug
Here is the native plan for a join. The join metrics of `build_time` and
`join_time` get reported in the Spark UI but we do not report the metrics for
fetching the input batches from the JVM or for unpacking dictionaries and
performing deep copies where needed.
For this example it means we are reporting a time of ~410ms when the actual
time is closer to ~600ms, and this is just for one partition.
```
HashJoinExec: metrics=[build_time=400.827077ms, join_time=8.557039ms]
CopyExec [UnpackOrDeepCopy], metrics=[elapsed_compute=18.643737ms]
ScanExec: source=[ShuffleQueryStage],
metrics=[elapsed_compute=186.719525ms]
CopyExec [UnpackOrDeepCopy], metrics=[..., elapsed_compute=293.113µs]
ScanExec: source=[ShuffleQueryStage ...],
metrics=[elapsed_compute=5.906924ms]
```
### Steps to reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
