ssbytedev opened a new pull request, #23260: URL: https://github.com/apache/kafka/pull/23260
Implements [KIP-1337](https://cwiki.apache.org/confluence/spaces/KAFKA/pages/421958161/KIP-1337+Add+Protobuf+MessageFormatter+for+Kafka+Console+Consumer) (currently Under Discussion) as a reference implementation to make the proposal concrete for discussion. ## What Adds ProtobufMessageFormatter for kafka-console-consumer, implementing the existing org.apache.kafka.common.MessageFormatter extension point. ## Why Allows inspecting Protobuf payloads from the console consumer using local .proto schema files, without requiring a schema registry. ## How --formatter org.apache.kafka.tools.consumer.ProtobufMessageFormatter --formatter-property proto.dir=/path/to/protos --formatter-property message.type=com.example.MyEvent At configure() time, compiles every *.proto file under proto.dir via protoc (--descriptor_set_out, --include_imports), resolves the configured message.type against the resulting descriptors, and caches both. At writeTo() time, decodes each record with DynamicMessage.parseFrom and prints canonical ProtoJSON via protobuf-java-util JsonFormat. Requires protoc on PATH. ## Testing - ProtobufMessageFormatterTest: happy path, missing/invalid config, no .proto files found, invalid .proto syntax, message.type not found, malformed record bytes, null (tombstone) and empty-value records - compiling real .proto sources via protoc, not synthetic descriptors - ConsoleConsumerOptionsTest: --formatter/--formatter-property CLI wiring without --skip-message-on-error, tombstone) - checkstyleMain/checkstyleTest, spotlessCheck, spotbugsMain/spotbugsTest all pass ## Open questions from the KIP not yet resolved here - Import resolution across multiple proto.dir files/subdirectories is supported via protoc --proto_path but not exercised by a dedicated test yet - google.protobuf.Any scope is not addressed - Tombstone (null value) behavior (currently prints the literal null) is a placeholder pending KIP agreement -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
