[
https://issues.apache.org/jira/browse/CAMEL-27130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-27130:
--------------------------------
Description:
Using a schema registry with camel-kafka today means setting the serde classes
of the registry vendor (valueSerializer / valueDeserializer) and the registry
properties under their vendor-specific names (additionalProperties...), and
knowing which jars to add. camel-kafka has schemaRegistryURL and
specificAvroReader, but schemaRegistryURL only sets the Confluent property
schema.registry.url.
*Proposal:* two options that let camel-kafka configure the registry itself:
* schemaRegistryVendor: confluent, apicurio, aws-glue, azure or custom.
* schemaFormat: avro, json or protobuf. Default avro; not every vendor supports
every format.
>From these, camel-kafka sets the serializer and deserializer classes and the
>vendor's registry properties, such as the registry URL and the auto-register
>and use-latest flags, using the existing schemaRegistryURL as the URL. The
>classes are only names (strings), so camel-kafka needs no compile dependency
>on any vendor. Options the user sets explicitly (valueSerializer,
>valueDeserializer, additionalProperties) still win.
The vendors, with what was checked on 2026-11-17:
|| Vendor || Serde artifacts || On Maven Central || Licence ||
| confluent | io.confluent:kafka-avro-serializer and others | No, only
https://packages.confluent.io/maven/ (public, no login) | Avro and client:
Apache 2.0. JSON Schema and Protobuf: Confluent Community License and Apache
2.0 |
| apicurio | io.apicurio:apicurio-registry-avro-serde-kafka 3.0.8 and others
(Apicurio 3 renamed the 2.x artifacts) | Yes | Apache 2.0 |
| aws-glue | software.amazon.glue:schema-registry-serde 1.1.25 | Yes | Apache
2.0 |
| azure | com.microsoft.azure:azure-schemaregistry-kafka-avro 1.1.2 | Yes | MIT
|
Several registries speak the Confluent registry API, so confluent would also
cover, for example, Karapace, the Redpanda schema registry and the
Confluent-compatible API of Apicurio.
*Dependencies.* camel-kafka does not depend on the vendor jars. The user adds
them:
* For confluent, the user is responsible for adding the Confluent Maven
repository, or for having the jars in their own pom.xml, as they are not on
Maven Central.
* Camel JBang could add the jars of the chosen vendor through its dependency
detection, from the option value.
* Camel JBang gets a --repos=confluent shorthand for
https://packages.confluent.io/maven/, like the existing --repos=redhat, so that
a user can write camel run --repos=confluent ... instead of the full URL.
*Kamelets.* With this, the Kafka Kamelets of camel-kamelets can offer a schema
registry with a few optional options (registry, URL, format) instead of one
Kamelet per vendor and format. A Kamelet template cannot choose classes or
dependencies from an option value by itself: nested placeholders and
placeholders in Kamelet dependencies do not resolve. See
apache/camel-kamelets#3183 (design and spike result), #3182 and #3144.
Related: CAMEL-24172 (proposal for a camel-apicurio-registry component).
_Claude Code on behalf of Claus Ibsen_
was:
Using a schema registry with camel-kafka today means setting the serde classes
of the registry vendor (valueSerializer / valueDeserializer) and the registry
properties under their vendor-specific names (additionalProperties...), and
knowing which jars to add. camel-kafka has schemaRegistryURL and
specificAvroReader, but schemaRegistryURL only sets the Confluent property
schema.registry.url.
*Proposal:* two options that let camel-kafka configure the registry itself:
* schemaRegistryVendor: confluent, apicurio, aws-glue, azure or custom.
* schemaFormat: avro, json or protobuf. Default avro; not every vendor supports
every format.
>From these, camel-kafka sets the serializer and deserializer classes and the
>vendor's registry properties, such as the registry URL and the auto-register
>and use-latest flags, using the existing schemaRegistryURL as the URL. The
>classes are only names (strings), so camel-kafka needs no compile dependency
>on any vendor. Options the user sets explicitly (valueSerializer,
>valueDeserializer, additionalProperties) still win.
The vendors, with what was checked on 2026-11-17:
|| Vendor || Serde artifacts || On Maven Central || Licence ||
| confluent | io.confluent:kafka-avro-serializer and others | No, only
https://packages.confluent.io/maven/ (public, no login) | Avro and client:
Apache 2.0. JSON Schema and Protobuf: Confluent Community License and Apache
2.0 |
| apicurio | io.apicurio:apicurio-registry-avro-serde-kafka 3.0.8 and others
(Apicurio 3 renamed the 2.x artifacts) | Yes | Apache 2.0 |
| aws-glue | software.amazon.glue:schema-registry-serde 1.1.25 | Yes | Apache
2.0 |
| azure | com.microsoft.azure:azure-schemaregistry-kafka-avro 1.1.2 | Yes | MIT
|
Several registries speak the Confluent registry API, so confluent would also
cover, for example, Karapace, the Redpanda schema registry and the
Confluent-compatible API of Apicurio.
*Dependencies.* camel-kafka does not depend on the vendor jars. The user adds
them:
* For confluent, the user is responsible for adding the Confluent Maven
repository, or for having the jars in their own pom.xml, as they are not on
Maven Central.
* Camel JBang could add the jars of the chosen vendor through its dependency
detection, from the option value. For confluent it would also need the
Confluent repository, given with --repos.
*Kamelets.* With this, the Kafka Kamelets of camel-kamelets can offer a schema
registry with a few optional options (registry, URL, format) instead of one
Kamelet per vendor and format. A Kamelet template cannot choose classes or
dependencies from an option value by itself: nested placeholders and
placeholders in Kamelet dependencies do not resolve. See
apache/camel-kamelets#3183 (design and spike result), #3182 and #3144.
Related: CAMEL-24172 (proposal for a camel-apicurio-registry component).
_Claude Code on behalf of Claus Ibsen_
> camel-kafka - a schemaRegistryVendor option that sets up the serde classes
> and registry properties of a schema registry
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-27130
> URL: https://issues.apache.org/jira/browse/CAMEL-27130
> Project: Camel
> Issue Type: New Feature
> Components: camel-kafka
> Reporter: Claus Ibsen
> Priority: Major
>
> Using a schema registry with camel-kafka today means setting the serde
> classes of the registry vendor (valueSerializer / valueDeserializer) and the
> registry properties under their vendor-specific names
> (additionalProperties...), and knowing which jars to add. camel-kafka has
> schemaRegistryURL and specificAvroReader, but schemaRegistryURL only sets the
> Confluent property schema.registry.url.
> *Proposal:* two options that let camel-kafka configure the registry itself:
> * schemaRegistryVendor: confluent, apicurio, aws-glue, azure or custom.
> * schemaFormat: avro, json or protobuf. Default avro; not every vendor
> supports every format.
> From these, camel-kafka sets the serializer and deserializer classes and the
> vendor's registry properties, such as the registry URL and the auto-register
> and use-latest flags, using the existing schemaRegistryURL as the URL. The
> classes are only names (strings), so camel-kafka needs no compile dependency
> on any vendor. Options the user sets explicitly (valueSerializer,
> valueDeserializer, additionalProperties) still win.
> The vendors, with what was checked on 2026-11-17:
> || Vendor || Serde artifacts || On Maven Central || Licence ||
> | confluent | io.confluent:kafka-avro-serializer and others | No, only
> https://packages.confluent.io/maven/ (public, no login) | Avro and client:
> Apache 2.0. JSON Schema and Protobuf: Confluent Community License and Apache
> 2.0 |
> | apicurio | io.apicurio:apicurio-registry-avro-serde-kafka 3.0.8 and others
> (Apicurio 3 renamed the 2.x artifacts) | Yes | Apache 2.0 |
> | aws-glue | software.amazon.glue:schema-registry-serde 1.1.25 | Yes | Apache
> 2.0 |
> | azure | com.microsoft.azure:azure-schemaregistry-kafka-avro 1.1.2 | Yes |
> MIT |
> Several registries speak the Confluent registry API, so confluent would also
> cover, for example, Karapace, the Redpanda schema registry and the
> Confluent-compatible API of Apicurio.
> *Dependencies.* camel-kafka does not depend on the vendor jars. The user adds
> them:
> * For confluent, the user is responsible for adding the Confluent Maven
> repository, or for having the jars in their own pom.xml, as they are not on
> Maven Central.
> * Camel JBang could add the jars of the chosen vendor through its dependency
> detection, from the option value.
> * Camel JBang gets a --repos=confluent shorthand for
> https://packages.confluent.io/maven/, like the existing --repos=redhat, so
> that a user can write camel run --repos=confluent ... instead of the full URL.
> *Kamelets.* With this, the Kafka Kamelets of camel-kamelets can offer a
> schema registry with a few optional options (registry, URL, format) instead
> of one Kamelet per vendor and format. A Kamelet template cannot choose
> classes or dependencies from an option value by itself: nested placeholders
> and placeholders in Kamelet dependencies do not resolve. See
> apache/camel-kamelets#3183 (design and spike result), #3182 and #3144.
> Related: CAMEL-24172 (proposal for a camel-apicurio-registry component).
> _Claude Code on behalf of Claus Ibsen_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)