DanielLeens opened a new pull request, #11706:
URL: https://github.com/apache/seatunnel/pull/11706
### Purpose of this pull request
A user asked (via a support-bot Q&A export I was reviewing for documentation
gaps) whether
SeaTunnel's Kafka source `avro` format supports Confluent
Schema-Registry-encoded Avro messages
("confluent-avro"). The existing FAQ entry lists `avro` as a supported
format but doesn't clarify
this, so the question keeps coming up.
Verified against current source: `MessageFormat.AVRO` deserializes via
`AvroDeserializationSchema`
(`connector-kafka/.../serialize` and `.../source/KafkaSourceConfig.java`,
`case AVRO:`), which has no
schema-registry-header-stripping logic. This is different from `PROTOBUF`,
which explicitly supports
it via `strip_schema_registry_header` /
`SchemaRegistryAwareProtobufDeserializationSchema` (see the
existing "Protobuf with Schema Registry wire format" section in the same
doc). Concretely: if a topic
was produced with Confluent's `KafkaAvroSerializer`, each message carries a
5-byte wire-format prefix
(magic byte + schema id) before the Avro payload. `format = avro` does not
strip that prefix, so
deserialization will fail or produce corrupted data.
Extended the existing "What message formats does Kafka Source support?" FAQ
entry (`docs/en` and
`docs/zh`) to state this limitation explicitly, at the same level of detail
as the existing
Protobuf-with-Schema-Registry section, and cross-linked to it.
### Does this PR introduce _any_ user-facing change?
No functional change. Documentation-only addition.
### How was this patch tested?
Documentation-only change, so `./mvnw spotless:apply` is a no-op (spotless
in this repo only formats
`<java>` and `<pom>`, not markdown). The technical claim was verified
directly against current `dev`
HEAD source:
-
`seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/config/MessageFormat.java`
-- confirms the enum values, including the absence of a distinct
Confluent-Avro variant.
-
`seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/source/KafkaSourceConfig.java`
-- confirms the `AVRO` case only consults `avro_schema` / catalog schema,
with no schema-registry
header handling, versus the `PROTOBUF` case which explicitly branches on
`STRIP_SCHEMA_REGISTRY_HEADER`.
- Confirmed the cross-reference anchor
`#protobuf-with-schema-registry-wire-format` matches the
existing `### Protobuf with Schema Registry wire format` heading in both
`docs/en` and `docs/zh`.
### Check list
* [x] No new JAR binary package is added.
* [x] English and Chinese documentation both updated.
* [x] No incompatible configuration or API change is introduced (docs-only).
* [x] N/A -- documentation-only change, no test coverage applicable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
