Lemarais commented on pull request #317: URL: https://github.com/apache/incubator-nemo/pull/317#issuecomment-899417187
[flink style latency](https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/metrics/) is >Note that the latency markers are not accounting for the time user records spend in operators as they are bypassing them. In particular the markers are not accounting for the time records spend for example in window buffers. Only if operators are not able to accept new records, thus they are queuing up, the latency measured using the markers will reflect that. Can you explain what event means? First I tried to use watermark to calculate latency, But timestamp didn't always mean the time it was created. When using bounded sources, sometimes different values are intentionally assigned. when watermarks are reached at aggregation vertex, watermarks with long max values or long min values were created, and this watermark was transferred to the next task alone. So there is a limit to calculate latency using only watermarks. So I added new marker to calculate latency. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
