zentol commented on a change in pull request #6850: [FLINK-10252] Handle
oversized metric messges
URL: https://github.com/apache/flink/pull/6850#discussion_r225473371
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/metrics/dump/MetricQueryService.java
##########
@@ -109,7 +122,17 @@ public void onReceive(Object message) {
}
} else if (message instanceof CreateDump) {
MetricDumpSerialization.MetricSerializationResult dump =
serializer.serialize(counters, gauges, histograms, meters);
- getSender().tell(dump, getSelf());
+
+ int realMsgSize = dump.serializedMetrics.length;
+
+ if (realMsgSize > maximumFramesize) {
+ String overSizeErrorMsg = "The metric
dump message size : " + realMsgSize
+ + " exceeds the maximum akka
framesize : " + maximumFramesize + ".";
+ LOG.error(overSizeErrorMsg);
+ getSender().tell(new Status.Failure(new
IOException(overSizeErrorMsg)), getSelf());
Review comment:
well then don't pass incomplete data. metrics are serialized one at a time,
so you can just remember during serialization the last metrics that fit into
the buffer, and once you go over the limit go back to that.
You can also use other interesting heuristics like dropping histograms first
(since they are significantly larger).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
