This is an automated email from the ASF dual-hosted git repository.
pjfanning pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pekko-connectors.git
The following commit(s) were added to refs/heads/main by this push:
new 40741ebc3 Kinesis: resolve mbknor-jackson-jsonschema for the build's
Scala version (#1925)
40741ebc3 is described below
commit 40741ebc393752a7dc57419db3e92d18a6bb4846
Author: PJ Fanning <[email protected]>
AuthorDate: Tue Sep 8 08:51:04 2026 +0100
Kinesis: resolve mbknor-jackson-jsonschema for the build's Scala version
(#1925)
Motivation:
The kinesis connector depends on
software.amazon.kinesis:amazon-kinesis-client,
which pulls in software.amazon.glue:schema-registry-serde. That POM
hardcodes
com.kjetland:mbknor-jackson-jsonschema_2.12 - the artifactId carries the
2.12
suffix literally, so it does not follow the Scala version the connector is
built for. Every pekko-connectors-kinesis user, on any Scala version, gets a
Scala 2.12 jar and its org.scala-lang:scala-library:2.12.10 on the compile
classpath.
This was reported in #479 in Feb 2024 against KCL 2.5.4 / GSR 1.1.17. It is
unchanged at KCL 3.5.1 / GSR 1.1.27. Upstream is not moving:
awslabs/aws-glue-schema-registry#315, the PR that would fix the suffix, is
still an open draft last touched in Feb 2025.
Modification:
Exclude the transitive mbknor-jackson-jsonschema_2.12 from
amazon-kinesis-client and declare the dependency ourselves with %%, so it
resolves to _2.13 on Scala 2.13. CrossVersion.for3Use2_13 covers Scala 3,
which is what raboof suggested on the issue - there is no _3 build, but the
_2.13 build exists at the same 1.0.39 version, published the same day as the
_2.12 one.
The substitution is safe for the way Glue uses the library. Exactly one
class
in schema-registry-serde-1.1.27.jar references it -
com.amazonaws.services.schemaregistry.serializers.json.JsonSerializer - and
it
touches only two members:
new JsonSchemaGenerator(ObjectMapper)
JsonSchemaGenerator.generateJsonSchema(Class): JsonNode
Both descriptors are pure Java/Jackson types with no Scala collections, and
both are present unchanged in the _2.13 jar. (JsonSchemaGenerator also
exposes
a scala.Option overload of generateJsonSchema, but Glue does not call it.)
Nothing in this connector's own sources references Glue or the schema
registry at all.
This fixes the Scala version mismatch only. mbknor-jackson-jsonschema is
still
unmaintained - last released in 2020 - so its stale jackson-databind 2.10.1
and kotlin-scripting-compiler-embeddable 1.3.50 remain in the closure,
evicted
upward by our own Jackson. Dropping the library entirely is up to AWS.
Result:
pekko-connectors-kinesis no longer puts a Scala 2.12 artifact and
scala-library 2.12.10 on the classpath of Scala 2.13 and Scala 3 users.
Tests:
- sbt "print kinesis/Compile/externalDependencyClasspath" "++3.3.8" "print
kinesis/Compile/externalDependencyClasspath" "kinesis/Test/compile" -
both
classpaths resolve mbknor-jackson-jsonschema_2.13 and carry only
scala-library-2.13.18; scala-library-2.12.10 is gone.
kinesis/Test/compile
is clean on Scala 3.3.8.
- scalafmt --test project/Dependencies.scala - clean.
- Dependency-resolution change only, so no new tests. The kinesis
integration
tests exercise KCL but not the Glue serde, which is the only consumer of
this jar.
References:
Fixes #479
Refs https://github.com/awslabs/aws-glue-schema-registry/pull/315
---
project/Dependencies.scala | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/project/Dependencies.scala b/project/Dependencies.scala
index 66e2957f8..70dea7b54 100644
--- a/project/Dependencies.scala
+++ b/project/Dependencies.scala
@@ -402,7 +402,11 @@ object Dependencies {
"software.amazon.kinesis" % "amazon-kinesis-client" % "3.5.2").map(
_.excludeAll(
ExclusionRule("software.amazon.awssdk", "apache-client"),
- ExclusionRule("software.amazon.awssdk", "netty-nio-client"))) ++ Seq(
+ ExclusionRule("software.amazon.awssdk", "netty-nio-client"),
+ // software.amazon.glue:schema-registry-serde hardcodes the Scala 2.12
build of this
+ // jar, so it does not follow the Scala version the connector is built
for - see #479
+ ExclusionRule("com.kjetland", "mbknor-jackson-jsonschema_2.12"))) ++
Seq(
+ ("com.kjetland" %% "mbknor-jackson-jsonschema" %
"1.0.39").cross(CrossVersion.for3Use2_13),
"org.slf4j" % "slf4j-api" % Slf4jVersion % Test,
"ch.qos.logback" % "logback-classic" % LogbackVersion % Test) ++ Mockito)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
