GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/3106
[FLINK-????] FIx inconsistent numBytesIn/Out metrics This PR fixes _some_ jira issue (i can't find the correct issue ID right now since JIRA is being stupid) regarding inconsistent numBytesIn/Out metrics. Given 2 subsequent tasks A->B the numBytesOut count of A was lower than the numBytesIn(Local/Remote) count of B by a huge margin, although they should be (nearly) identical. The problem is that A was counting how much bytes the serialized records were using, whereas B was counting how large the ```Buffer```s were that it received. A ```Buffer``` contains the following data: ``` |size_of_R1|R1|size_of_R2|R2|... ``` where as R1/R2 are serialized records and the sizes denote the serialized length of their respective record. So, while A was adding ```sizeOf(R1)```, B was adding sizeOf(size_of_R1) + sizeOf(R1). A was simply not accounting for the added bytes that size_of_RX were using, which is what this PR is fixing by adding ```4``` after the serialization of each record. cc @uce You can merge this pull request into a Git repository by running: $ git pull https://github.com/zentol/flink _metrics_incon Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3106.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3106 ---- commit 5300a35ac6ee8815d07a739c9b370ae475f0d9a0 Author: zentol <ches...@apache.org> Date: 2017-01-12T13:59:22Z [FLINK-????] FIx inconsistent numBytesIn/Out metrics ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---