Lorenzo Alberton created KAFKA-406: -------------------------------------- Summary: Gzipped payload is a full wrapped Message (with headers), not just payload Key: KAFKA-406 URL: https://issues.apache.org/jira/browse/KAFKA-406 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.7.1 Environment: N/A Reporter: Lorenzo Alberton
When creating a gzipped MessageSet, the collection of Messages is passed to CompressionUtils.compress(), where each message is serialised [1] into a buffer (not just the payload, the full Message with headers, uncompressed), then gripped, and finally wrapped into another Message [2]. In other words, the consumer has to unwrap the Message flagged as gzipped, unzip the payload, and unwrap the unzipped payload again as a non-compressed Message. Is this double-wrapping the intended behaviour? [1] messages.foreach(m => m.serializeTo(messageByteBuffer)) [2] new Message(outputStream.toByteArray, compressionCodec) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira