Cq-study commented on code in PR #681:
URL:
https://github.com/apache/doris-flink-connector/pull/681#discussion_r3683304007
##########
flink-doris-connector/flink-doris-connector-base/src/main/java/org/apache/doris/flink/sink/writer/DorisWriteMetrics.java:
##########
@@ -187,8 +192,10 @@ public void register(SinkWriterMetricGroup
sinkMetricGroup) {
private void flushSuccessLoad(RespContent responseContent) {
Optional.ofNullable(responseContent.getLoadBytes()).ifPresent(totalFlushLoadBytes::inc);
+
Optional.ofNullable(responseContent.getLoadBytes()).ifPresent(numBytesSend::inc);
Optional.ofNullable(responseContent.getNumberLoadedRows())
.ifPresent(totalFlushLoadedRows::inc);
+
Optional.ofNullable(responseContent.getNumberLoadedRows()).ifPresent(numRecordsSend::inc);
Review Comment:
Updated — the counting is moved to the send path in
`DorisWriter#writeOneDorisRecord`: `numRecordsSend` is incremented per record
and `numBytesSend` by the payload size when the record is written to the stream
load, so records of failed loads are also counted and the metrics update
continuously instead of only at checkpoint flush. The connector-specific
counters (`totalFlushLoadedRows`, etc.) keep reporting what Doris actually
loaded. PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
