[
https://issues.apache.org/jira/browse/NIFI-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18014967#comment-18014967
]
Lukas Kucharski commented on NIFI-14424:
----------------------------------------
Updated ticket description. Resolving.
> Support for Confluent schema registry Protobuf wire format in
> StandardProtobufReader
> ------------------------------------------------------------------------------------
>
> Key: NIFI-14424
> URL: https://issues.apache.org/jira/browse/NIFI-14424
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Alaksiej Ščarbaty
> Assignee: Lukas Kucharski
> Priority: Major
> Attachments: image-2025-07-16-12-52-20-296.png,
> image-2025-07-17-10-32-37-291.png, image-2025-07-17-14-40-58-260.png,
> image-2025-07-17-14-42-31-326.png, image-2025-07-17-14-43-58-023.png,
> image-2025-07-17-14-45-47-577.png, image-2025-07-17-14-51-24-943.png,
> image-2025-07-17-14-52-18-792.png
>
> Time Spent: 13h 20m
> Remaining Estimate: 0h
>
> h2. Summary
> Add Confluent Protobuf Wire Format Support - Foundation Services and APIs
> h2. Description
> This enhancement introduces foundational support for deserializing Protobuf
> messages encoded using the Confluent protobuf wire format in NiFi. The
> existing Confluent Schema Registry integration only supported Avro data
> decoding. Confluent's Protobuf serialization adds a variable-length byte
> section to the message header containing message indexes that must be parsed
> and interpreted.
> This ticket represents Phase 1 of the implementation, providing the core
> services and API extensions. Phase 2 will introduce the new
> StandardProtobufReader component.
> h2. Key Components Implemented
> h3. API Extensions (nifi-schema-registry-service-api)
> - {*}SchemaDefinition{*}: New abstraction representing raw schema forms
> with schema text and referenced schema collections
> - {*}MessageNameResolver{*}: Controller service interface for translating
> encoded Protobuf message indexes into message names
> - {*}MessageName{*}: Abstraction for values returned by MessageNameResolver
> h3. Confluent Platform Bundle Enhancements
> - {*}ConfluentProtobufMessageNameResolver{*}: Service that decodes
> Confluent-standardized message name indexes from wire format using zigzag
> encoded varints
> - Enhanced {*}RestSchemaRegistryClient{*}: Extended to support recursive
> fetching of referenced/imported Protobuf schemas
> - {*}ProtobufMessageSchema{*}: Schema representation supporting nested
> message parsing
> - ANTLR Parser Integration: Added Proto3 grammar parser for schema
> structure analysis without requiring full compilation
> h3. Wire Format Processing
> - Message Index Decoding: Parses variable-length message indexes (e.g.,
> [0][2][0] or [1]) from Confluent wire format headers
> - Nested Message Support: Resolves complex message paths like
> MessageA.MessageE.MessageG
> - Schema Structure Analysis: Parses Protobuf schemas to locate correct
> message definitions for deserialization
> h2. Next Phase
> Phase 2 (linked issue) will implement *StandardProtobufReader* component
> that utilizes these foundational services to provide full Protobuf record
> processing capabilities with Schema Registry integration.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
