divijvaidya commented on code in PR #14322:
URL: https://github.com/apache/kafka/pull/14322#discussion_r1313393480
##########
docs/design.html:
##########
@@ -136,8 +136,9 @@ <h4 class="anchor-heading"><a id="design_compression"
class="anchor-link"></a><a
the user can always compress its messages one at a time without any
support needed from Kafka, but this can lead to very poor compression ratios as
much of the redundancy is due to repetition between messages of
the same type (e.g. field names in JSON or user agents in web logs or
common string values). Efficient compression requires compressing multiple
messages together rather than compressing each message individually.
<p>
- Kafka supports this with an efficient batching format. A batch of messages
can be clumped together compressed and sent to the server in this form. This
batch of messages will be written in compressed form and will
- remain compressed in the log and will only be decompressed by the consumer.
+ Kafka supports this with an efficient batching format. A batch of messages
can be grouped together, compressed, and sent to the server in this form. The
broker decompresses the batch in order to validate it and
Review Comment:
maybe worth adding an example of validation here since people might confuse
it with schema validation. For example you can add that it validates that the
number of records in batch is same as what batch header states or maybe a
better one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
