pjfanning opened a new pull request, #1925:
URL: https://github.com/apache/pekko-connectors/pull/1925
Fixes #479.
## Problem
`pekko-connectors-kinesis` depends on
`software.amazon.kinesis:amazon-kinesis-client`, which pulls in
`software.amazon.glue:schema-registry-serde`. That POM hardcodes the Scala 2.12
build:
```
pekko-connectors-kinesis
└── software.amazon.kinesis:amazon-kinesis-client:3.5.1
└── software.amazon.glue:schema-registry-serde:1.1.27 (compile scope)
└── com.kjetland:mbknor-jackson-jsonschema_2.12:1.0.39
└── org.scala-lang:scala-library:2.12.10
```
The `_2.12` suffix is literal in the artifactId, so it does not follow the
Scala version we build for — every user gets a Scala 2.12 jar and
`scala-library:2.12.10` on the compile classpath regardless.
#479 reported this in Feb 2024 against KCL 2.5.4 / GSR 1.1.17; it is
unchanged at KCL 3.5.1 / GSR 1.1.27. Upstream is not moving —
[awslabs/aws-glue-schema-registry#315](https://github.com/awslabs/aws-glue-schema-registry/pull/315),
the PR that would add proper suffixes, is still an open draft last touched Feb
2025.
## Fix
Exclude the transitive `_2.12` artifact and declare the dependency
ourselves, so `%%` resolves `_2.13` on Scala 2.13 and
`CrossVersion.for3Use2_13` covers Scala 3 — [as @raboof suggested on the
issue](https://github.com/apache/pekko-connectors/issues/479#issuecomment-1938262211).
There is no `_3` build, but the `_2.13` build exists at the same 1.0.39
version, published the same day as `_2.12`.
## Why the substitution is safe
Exactly one class in `schema-registry-serde-1.1.27.jar` references the
library —
`com.amazonaws.services.schemaregistry.serializers.json.JsonSerializer` — and
it touches only two members:
```
new JsonSchemaGenerator(ObjectMapper)
JsonSchemaGenerator.generateJsonSchema(Class): JsonNode
```
Both descriptors are pure Java/Jackson types with no Scala collections, and
both are present unchanged in the `_2.13` jar. (`JsonSchemaGenerator` also
exposes a `scala.Option` overload of `generateJsonSchema`, but Glue does not
call it.) Nothing in this connector's own sources references Glue or the schema
registry at all.
## Scope
This fixes the Scala version mismatch only. `mbknor-jackson-jsonschema` is
still unmaintained — last released in 2020 — so its stale
`jackson-databind:2.10.1` and `kotlin-scripting-compiler-embeddable:1.3.50`
remain in the closure, evicted upward by our own Jackson. Dropping the library
entirely is AWS's call.
## Tests
- `sbt "print kinesis/Compile/externalDependencyClasspath" "++3.3.8" "print
kinesis/Compile/externalDependencyClasspath" "kinesis/Test/compile"` — both
classpaths resolve `mbknor-jackson-jsonschema_2.13` and carry only
`scala-library-2.13.18`; `scala-library-2.12.10` is gone.
`kinesis/Test/compile` clean on Scala 3.3.8.
- `scalafmt --test project/Dependencies.scala` — clean.
- Dependency-resolution change only, so no new tests. The kinesis
integration tests exercise KCL but not the Glue serde, which is the only
consumer of this jar.
## References
Fixes #479
Refs https://github.com/awslabs/aws-glue-schema-registry/pull/315
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
