[
https://issues.apache.org/jira/browse/NIFI-16349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18118442#comment-18118442
]
ASF subversion and git services commented on NIFI-16349:
--------------------------------------------------------
Commit 4abef54d295f2c3f55c8d09fb1e61f78625259eb in nifi's branch
refs/heads/main from Alaksiej Ščarbaty
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4abef54d295 ]
NIFI-16349 Strip custom Protobuf options before Wire schema compilation (#11710)
Signed-off-by: David Handermann <[email protected]>
> StandardProtobufReader fails to compile Confluent Protobuf schemas with
> implicit field_meta options
> ---------------------------------------------------------------------------------------------------
>
> Key: NIFI-16349
> URL: https://issues.apache.org/jira/browse/NIFI-16349
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Wojciech Trefon
> Assignee: Alaksiej Ščarbaty
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> h2. Summary
> {{StandardProtobufReader}} fails to compile Protobuf schemas retrieved from
> Confluent Schema Registry when a schema contains the
> {{(confluent.field_meta)}} custom option without an explicit
> {{confluent/meta.proto}} import.
> The failure occurs during schema compilation, before the message payload is
> decoded.
> h2. Background
> Confluent Schema Registry treats {{confluent/meta.proto}} as a built-in known
> dependency. Schemas generated from Kafka Connect logical types can therefore
> contain {{confluent.field_meta}} without listing {{confluent/meta.proto}} as
> a normal Schema Registry reference or explicit import.
> h2. Example schema
> {code:protobuf}
> syntax = "proto3";
> package example;
> message Value {
> optional int32 enabled = 31 [(confluent.field_meta) = {
> params: [
> {
> key: "connect.type",
> value: "int16"
> }
> ]
> }];
> }
> {code}
> h2. Steps to reproduce
> # Store or generate a Protobuf schema in Confluent Schema Registry that
> contains {{confluent.field_meta}} without an explicit
> {{confluent/meta.proto}} import or schema reference.
> # Serialize a message using the Confluent Protobuf wire format and the
> affected schema.
> # Configure {{StandardProtobufReader}} to use
> {{ConfluentEncodedSchemaReferenceReader}}, {{ConfluentSchemaRegistry}}, and
> {{ConfluentProtobufMessageNameResolver}}.
> # Read the serialized message.
> h2. Actual behavior
> Schema compilation fails with an error similar to:
> {code}
> com.squareup.wire.schema.SchemaException: unable to resolve option
> confluent.field_meta
> for field enabled
> in message example.Value
> {code}
> The message payload is not decoded.
> h2. Expected behavior
> {{StandardProtobufReader}} should be able to compile and read a schema
> accepted by Confluent Schema Registry when the schema uses Confluent's
> built-in metadata options.
> h2. Impact
> Messages using affected schemas cannot be consumed as records. Existing
> messages remain affected because their Confluent wire headers continue to
> reference the original schema identifiers.
> h2. Technical context
> {{ProtobufSchemaCompiler}} provides selected Wire runtime definitions for
> standard Protobuf types. It does not currently make {{confluent/meta.proto}}
> or its {{google/protobuf/descriptor.proto}} dependency available during
> compilation. Wire also requires the source schema to import the file that
> defines a custom option.
> Observed with the {{StandardProtobufReader}} implementation using Square Wire
> 6.4.6.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
