vvcephei commented on a change in pull request #8697:
URL: https://github.com/apache/kafka/pull/8697#discussion_r430846939
##########
File path:
streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java
##########
@@ -586,6 +606,7 @@ public boolean process(final long wallClockTime) {
log.trace("Start processing one record [{}]", record);
updateProcessorContext(record, currNode, wallClockTime);
+ maybeRecordE2ELatency(record.timestamp, wallClockTime,
currNode.name());
Review comment:
Sorry for my ambiguity. Please let me clarify my terms. Currently if you
wait until the end of the "process" method, you wind up including the call to
forward, which recursively calls process on all descendents of the source node.
This is _not_ what I was talking about. I meant only the time spent _just_ in
processing the SourceNode, excluding the time in "forward". What shall we call
this? Maybe "actual", or "proper", or "internal" processing time?
So, my comment was that, given that we know the implementation of
SourceNode, we know that it's "actual", "proper", "internal" processing time is
going to be very small, probably far less than a single millisecond. So it
doesn't make any practical difference whether we measure before the call for
just the special case of source nodes, or magically solve the problem of
measuring the e2e latency after internal processing, but not including the
calls to "forward".
This is why I think it's fine to measure SourceNodes _before_ the call to
process, even though the KIP technically specifies that processors' end-to-end
latencies should include processing latency. We're making a simplifying
assumption that for source nodes specifically, the processing latency would be
`<< 1`, so we can ignore it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
