Github user arunmahadevan commented on the issue:
https://github.com/apache/spark/pull/21919
`numOutputRows` makes sense for all sinks, but I agree the counting should
be done at the framework and not by individual sinks. For metrics that does not
apply to all sinks, they could report it as some custom metrics if they want
to. Heres a proposal to add collect and report custom metrics for sources and
sinks - https://github.com/apache/spark/pull/21721
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
