[
https://issues.apache.org/jira/browse/CASSSIDECAR-450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSSIDECAR-450:
------------------------------------------
Description:
I had to add extra libs to Sidecar's CP
{code}
avro-1.12.0.jar
common-utils-8.1.2.jar
httpcore5-5.3.4.jar
jackson-datatype-jdk8-2.16.1.jar
kafka-schema-registry-client-8.1.2.jar
common-config-8.1.2.jar
httpclient5-5.5.jar
httpcore5-h2-5.3.4.jar
kafka-avro-serializer-8.1.2.jar
kafka-schema-serializer-8.1.2.jar
{code}
This failed later on this:
{code}
❯ [root@spark-master-1 /]#
CP="/opt/cassandra-sidecar/latest/lib/*:/opt/cassandra-sidecar/latest/extra-libs/*"
[root@spark-master-1 /]# echo
"Class.forName(\"io.confluent.kafka.schemaregistry.avro.AvroSchemaUtils\");" |
jshell --class-path "$CP" -s 2>&1
May 05, 2026 6:06:53 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
-> | Exception java.lang.NoSuchMethodError: 'void
org.apache.avro.Schema$Parser.<init>(org.apache.avro.NameValidator)'
| at AvroSchema.getParser (AvroSchema.java:232)
| at AvroSchema.<init> (AvroSchema.java:119)
| at AvroSchema.<init> (AvroSchema.java:108)
| at AvroSchema.<init> (AvroSchema.java:100)
| at AvroSchema.<init> (AvroSchema.java:93)
| at AvroSchemaUtils.createPrimitiveSchema (AvroSchemaUtils.java:234)
| at AvroSchemaUtils.<clinit> (AvroSchemaUtils.java:222)
| at Class.forName0 (Native Method)
| at Class.forName (Class.java:423)
| at Class.forName (Class.java:414)
| at (#1:1)
{code}
So I looked who is messing with this:
{code}
❯
CP="/opt/cassandra-sidecar/latest/lib/*:/opt/cassandra-sidecar/latest/extra-libs/*"
echo
"System.out.println(org.apache.avro.Schema.Parser.class.getProtectionDomain().getCodeSource().getLocation());"
| jshell --class-path "$CP" -s 2>&1
-> file:/opt/cassandra-sidecar/latest/lib/datahub-client-0.15.0-3.jar
{code}
Right ... datahub. I attempted to just remove whole jar, that did not work
because we have {{SchemaReportingModule}}.
This did the trick
{code}
# shadowing avro-1.12 in extra-libs. Strip those bundled Avro classes so the
JAR's own classes
# (datahub.client.*, referenced by SidecarModules) stay reachable while the
real Avro wins.
for j in /opt/cassandra-sidecar/latest/lib/datahub-client-*.jar; do
[ -f "$j" ] && zip -d "$j" 'org/apache/avro/*' >/dev/null 2>&1 || true
done
{code}
Basically, datahub uses some version of avro which is incompatible with avro
confluent uses and if i put libs from confluent on cp to use that serializer
then it will use avro from datahub instead of what it needs. So I just forcibly
removed classes from datahub jar. Not sure what is the proper solution to this.
Maybe some shadowing? Update of datahub to something newer? Confluent uses 1.12
Avro.
was:
I had to add extra libs to Sidecar's CP
{code}
avro-1.12.0.jar
common-utils-8.1.2.jar
httpcore5-5.3.4.jar
jackson-datatype-jdk8-2.16.1.jar
kafka-schema-registry-client-8.1.2.jar
common-config-8.1.2.jar
httpclient5-5.5.jar
httpcore5-h2-5.3.4.jar
kafka-avro-serializer-8.1.2.jar
kafka-schema-serializer-8.1.2.jar
{code}
This failed later on this:
{code}
❯ [root@spark-master-1 /]#
CP="/opt/cassandra-sidecar/latest/lib/*:/opt/cassandra-sidecar/latest/extra-libs/*"
[root@spark-master-1 /]# echo
"Class.forName(\"io.confluent.kafka.schemaregistry.avro.AvroSchemaUtils\");" |
jshell --class-path "$CP" -s 2>&1
May 05, 2026 6:06:53 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
-> | Exception java.lang.NoSuchMethodError: 'void
org.apache.avro.Schema$Parser.<init>(org.apache.avro.NameValidator)'
| at AvroSchema.getParser (AvroSchema.java:232)
| at AvroSchema.<init> (AvroSchema.java:119)
| at AvroSchema.<init> (AvroSchema.java:108)
| at AvroSchema.<init> (AvroSchema.java:100)
| at AvroSchema.<init> (AvroSchema.java:93)
| at AvroSchemaUtils.createPrimitiveSchema (AvroSchemaUtils.java:234)
| at AvroSchemaUtils.<clinit> (AvroSchemaUtils.java:222)
| at Class.forName0 (Native Method)
| at Class.forName (Class.java:423)
| at Class.forName (Class.java:414)
| at (#1:1)
{code}
So I looked who is messing with this:
{code}
❯
CP="/opt/cassandra-sidecar/latest/lib/*:/opt/cassandra-sidecar/latest/extra-libs/*"
echo
"System.out.println(org.apache.avro.Schema.Parser.class.getProtectionDomain().getCodeSource().getLocation());"
| jshell --class-path "$CP" -s 2>&1
-> file:/opt/cassandra-sidecar/latest/lib/datahub-client-0.15.0-3.jar
{code}
Right ... datahub. I attempted to just remove whole jar, that did not work
because we have {{SchemaReportingModule}}.
This did the trick
{code}
# shadowing avro-1.12 in extra-libs. Strip those bundled Avro classes so the
JAR's own classes
# (datahub.client.*, referenced by SidecarModules) stay reachable while the
real Avro wins.
for j in /opt/cassandra-sidecar/latest/lib/datahub-client-*.jar; do
[ -f "$j" ] && zip -d "$j" 'org/apache/avro/*' >/dev/null 2>&1 || true
done
{code}
Dasically, datahub uses some version of avro which is incompatible with avro
confluent uses and if i put libs from confluent on cp to use that serializer
then it will use avro from datahub instead of what it needs. So I just forcibly
removed classes from datahub jar. Not sure what is the proper solution to this.
Maybe some shadowing? Update of datahub to something newer? Confluent uses 1.12
Avro.
> Unable to use io.confluent.kafka.serializers.KafkaAvroSerializer
> -----------------------------------------------------------------
>
> Key: CASSSIDECAR-450
> URL: https://issues.apache.org/jira/browse/CASSSIDECAR-450
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: CDC
> Reporter: Stefan Miklosovic
> Assignee: Bernardo Botella Corbi
> Priority: Major
>
> I had to add extra libs to Sidecar's CP
> {code}
> avro-1.12.0.jar
> common-utils-8.1.2.jar
> httpcore5-5.3.4.jar
> jackson-datatype-jdk8-2.16.1.jar
> kafka-schema-registry-client-8.1.2.jar
> common-config-8.1.2.jar
> httpclient5-5.5.jar
> httpcore5-h2-5.3.4.jar
> kafka-avro-serializer-8.1.2.jar
> kafka-schema-serializer-8.1.2.jar
> {code}
> This failed later on this:
> {code}
> ❯ [root@spark-master-1 /]#
> CP="/opt/cassandra-sidecar/latest/lib/*:/opt/cassandra-sidecar/latest/extra-libs/*"
>
> [root@spark-master-1 /]# echo
> "Class.forName(\"io.confluent.kafka.schemaregistry.avro.AvroSchemaUtils\");"
> | jshell --class-path "$CP" -s 2>&1
> May 05, 2026 6:06:53 PM java.util.prefs.FileSystemPreferences$1 run
>
>
> INFO: Created user preferences directory.
>
>
> -> | Exception java.lang.NoSuchMethodError: 'void
> org.apache.avro.Schema$Parser.<init>(org.apache.avro.NameValidator)'
>
> | at AvroSchema.getParser (AvroSchema.java:232)
>
>
> | at AvroSchema.<init> (AvroSchema.java:119)
>
>
> | at AvroSchema.<init> (AvroSchema.java:108)
>
>
> | at AvroSchema.<init> (AvroSchema.java:100)
>
>
> | at AvroSchema.<init> (AvroSchema.java:93)
>
>
> | at AvroSchemaUtils.createPrimitiveSchema
> (AvroSchemaUtils.java:234)
>
> | at AvroSchemaUtils.<clinit> (AvroSchemaUtils.java:222)
>
>
> | at Class.forName0 (Native Method)
>
>
> | at Class.forName (Class.java:423)
>
>
> | at Class.forName (Class.java:414)
>
>
> | at (#1:1)
> {code}
> So I looked who is messing with this:
> {code}
> ❯
> CP="/opt/cassandra-sidecar/latest/lib/*:/opt/cassandra-sidecar/latest/extra-libs/*"
> echo
>
>
> "System.out.println(org.apache.avro.Schema.Parser.class.getProtectionDomain().getCodeSource().getLocation());"
> | jshell --class-path "$CP" -s 2>&1
> -> file:/opt/cassandra-sidecar/latest/lib/datahub-client-0.15.0-3.jar
> {code}
> Right ... datahub. I attempted to just remove whole jar, that did not work
> because we have {{SchemaReportingModule}}.
> This did the trick
> {code}
> # shadowing avro-1.12 in extra-libs. Strip those bundled Avro classes so the
> JAR's own classes
> # (datahub.client.*, referenced by SidecarModules) stay reachable while the
> real Avro wins.
> for j in /opt/cassandra-sidecar/latest/lib/datahub-client-*.jar; do
> [ -f "$j" ] && zip -d "$j" 'org/apache/avro/*' >/dev/null 2>&1 || true
> done
> {code}
> Basically, datahub uses some version of avro which is incompatible with avro
> confluent uses and if i put libs from confluent on cp to use that serializer
> then it will use avro from datahub instead of what it needs. So I just
> forcibly removed classes from datahub jar. Not sure what is the proper
> solution to this. Maybe some shadowing? Update of datahub to something newer?
> Confluent uses 1.12 Avro.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
