Github user jose-torres commented on the issue:
https://github.com/apache/spark/pull/21919
I like the idea of doing this, but I don't think it really belongs as part
of the WriterCommitMessage interface. Every connector shouldn't have to
independently count its rows; the execution framework should do the counting
automatically, and send an independent StreamWriterCommitProgress to the driver
along with each WriterCommitMessage.
Note that we'll probablywant to extend StreamWriterCommitProgress soon to
carry metrics for continuous processing.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
