lindong28 commented on code in PR #21579:
URL: https://github.com/apache/flink/pull/21579#discussion_r1062396813
##########
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/OperatorChain.java:
##########
@@ -622,6 +627,43 @@ private Map<StreamConfig.SourceInputConfig, ChainedSource>
createChainedSources(
return chainedSourceInputs;
}
+ // Get the numRecordsOut counter for the operator represented by the given
config. And re-use
+ // this operator-level counter for the task-level numRecordsOut counter if
this operator
+ // is at the end of the operator chain.
+ //
+ // Return null if we should not use the numRecordsOut counter to track the
records emitted
+ // by this operator.
+ private Counter getOperatorRecordsOutCounter(
+ StreamTask<?, ?> containingTask, StreamConfig operatorConfig) {
+ InternalOperatorMetricGroup operatorMetricGroup =
+ containingTask
+ .getEnvironment()
+ .getMetricGroup()
+ .getOrAddOperator(
+ operatorConfig.getOperatorID(),
operatorConfig.getOperatorName());
+ if (operatorConfig.isChainEnd()) {
+ operatorMetricGroup.getIOMetricGroup().reuseOutputMetricsForTask();
+ }
+ ClassLoader userCodeClassloader =
containingTask.getUserCodeClassLoader();
+
+ /**
+ * Do not use the numRecordsOut counter on output if this operator is
SinkWriterOperator.
+ *
+ * <p>Metric "numRecordsOut" is defined as the total number of records
written to the
+ * external system in FLIP-33, but this metric is occupied in
AbstractStreamOperator as the
+ * number of records sent to downstream operators, which is number of
Committable batches
+ * sent to SinkCommitter. So we skip registering this metric on output
and leave this metric
+ * to sink writer implementations to report.
+ */
Review Comment:
Thanks. It is fixed now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
