junaiddshaukat commented on code in PR #39341:
URL: https://github.com/apache/beam/pull/39341#discussion_r3597242397
##########
runners/kafka-streams/src/main/java/org/apache/beam/runners/kafka/streams/translation/ExecutableStageProcessor.java:
##########
@@ -179,11 +190,25 @@ public <OutputT> FnDataReceiver<OutputT> create(String
pCollectionId) {
};
}
};
+ // Fold the harness's reported metrics into this stage's container when
each bundle completes.
+ // Only the completion response is applied: it carries the bundle's final
cumulative values, and
+ // the container's update() adds counter values, so also applying
mid-bundle progress snapshots
+ // would double-count them. Live mid-bundle metrics can come later if a
use appears.
+ BundleProgressHandler progressHandler =
+ new BundleProgressHandler() {
+ @Override
+ public void onProgress(ProcessBundleProgressResponse progress) {
+ // Deliberately not folded into the container; see comment above.
Review Comment:
Done, logs the progress response at DEBUG with the level check and
TextFormat.
##########
runners/kafka-streams/src/main/java/org/apache/beam/runners/kafka/streams/translation/ExecutableStageProcessor.java:
##########
@@ -179,11 +190,25 @@ public <OutputT> FnDataReceiver<OutputT> create(String
pCollectionId) {
};
}
};
+ // Fold the harness's reported metrics into this stage's container when
each bundle completes.
+ // Only the completion response is applied: it carries the bundle's final
cumulative values, and
+ // the container's update() adds counter values, so also applying
mid-bundle progress snapshots
+ // would double-count them. Live mid-bundle metrics can come later if a
use appears.
+ BundleProgressHandler progressHandler =
+ new BundleProgressHandler() {
+ @Override
+ public void onProgress(ProcessBundleProgressResponse progress) {
+ // Deliberately not folded into the container; see comment above.
+ }
+
+ @Override
+ public void onCompleted(ProcessBundleResponse response) {
Review Comment:
Done here too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
