StephanEwen commented on pull request #13920: URL: https://github.com/apache/flink/pull/13920#issuecomment-723222632
A thought on performance: There are some metrics that use `volatile` variables. So far we always avoided that. Even "infrequently accessed" fields, like watermarks, are quite frequent in some use cases. And a single volatile write has a very significant overhead in a high-performance pipeline. It is the number one thing we try to avoid, because it breaks the processor pipeline and puts pressure on the memory bus and processor's cache coherency mechanism. For the metrics, performance (least impact on main data paths) should have clear precedence. For metrics, we don't need a hard "happens before" relationship or any "immediate visibility". If the visibility is some 100s of milliseconds later (which would be a lot), it is no issue for the metrics at all, which are periodically reported every few seconds. The case where the JIT inlines the variable and eliminates cross-thread visibility entirely is AFAIK not realistic here. And if we ever find that the metric gets not reported, we can still use an occasional volatile write (like we do occasional volatile exception checks, every 1000 records, in the network stack). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
