n0r0shi opened a new pull request, #11724:
URL: https://github.com/apache/incubator-gluten/pull/11724
## Summary
Surfaces Velox's `cpuWallTiming.cpuNanos` as a separate Spark SQL metric
(`cpu time`) on `InputIteratorTransformer`. Currently only `wallNanos` and
`cpuCount` are exposed — `cpuNanos` is available from Velox but not propagated
to Gluten. This change threads it through: C++ → JNI → Java → Scala SQLMetric.
The metric automatically appears in Spark UI and event logs.
## Motivation
Related to #10618. `InputIteratorTransformer` sits at the boundary between
Spark (shuffle/broadcast) and Velox. Its wall time includes I/O waits, not just
CPU work. Exposing `cpuNanos` separately lets users distinguish compute-bound
vs I/O-bound bottlenecks. For most other operators, `cpuNanos ≈ wallNanos` —
this boundary operator is where they diverge the most.
Velox already tracks `cpuNanos` on every operator — exposing it as-is is the
simplest approach and better than not surfacing CPU time at all.
## Open questions
- Any concerns with the metric name / label (`cpu time`)?
- Should any other operators also expose `cpuNanos`? (e.g., operators that
spill to disk)
Tests will be added once the approach is confirmed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
