Hello everyone, I'd like to start a discussion on KIP-1398, which proposes changes to the KIP-714 client telemetry protocol so that clients can send more data, in a more deterministic way.
Today a client sends all its metrics for an interval in a single request, capped at telemetry.max.bytes. A client whose subscription selects more series than fit, which is easy to reach with per-partition or per-connection metrics, has its push rejected every interval and gets none of its metrics through. The KIP lets a client split one push across several requests, up to a limit the broker sets, and writes the payload more compactly so that each request holds more metrics. If a client still has more than fits, operators can prioritize subscriptions, and the client always drops the lowest-priority metrics first. KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1398%3A+Extending+the+client+telemetry+protocol Looking forward to your feedback. Thanks, Nikita
