piotr-szuberski commented on a change in pull request #13026:
URL: https://github.com/apache/beam/pull/13026#discussion_r524248460
##########
File path: sdks/java/io/kafka/build.gradle
##########
@@ -65,26 +76,68 @@ dependencies {
testCompile library.java.junit
testCompile library.java.powermock
testCompile library.java.powermock_mockito
+ testCompile library.java.testcontainers_kafka
testRuntimeOnly library.java.slf4j_jdk14
testRuntimeOnly project(path: ":runners:direct-java", configuration:
"shadow")
- kafkaVersion210 "org.apache.kafka:kafka-clients:2.1.0"
+ kafkaVersions.each {"kafkaVersion$it.key"
"org.apache.kafka:kafka-clients:$it.value"}
Review comment:
That's super useful! Thank you!
I can't see any Kafka clients in the created jar. Here is the output of
:sdks:java:io:kafka:dependencies
```
compileClasspath - Compile classpath for source set 'main'.
\--- org.apache.beam:beam-sdks-java-io-kafka:2.27.0-SNAPSHOT
+--- org.checkerframework:checker-qual:3.7.0 FAILED
+--- org.apache.beam:beam-vendor-guava-26_0-jre:0.1
+--- org.apache.beam:beam-sdks-java-core:2.27.0-SNAPSHOT FAILED
+--- org.apache.beam:beam-sdks-java-expansion-service:2.27.0-SNAPSHOT
FAILED
+--- org.slf4j:slf4j-api:1.7.30
+--- joda-time:joda-time:2.10.5
+--- com.fasterxml.jackson.core:jackson-annotations:2.10.2
+--- com.fasterxml.jackson.core:jackson-databind:2.10.2
| +--- com.fasterxml.jackson.core:jackson-annotations:2.10.2
| \--- com.fasterxml.jackson.core:jackson-core:2.10.2
+--- org.springframework:spring-expression:4.3.18.RELEASE FAILED
\--- io.confluent:kafka-avro-serializer:5.3.2
+--- org.apache.avro:avro:1.8.1
| +--- org.codehaus.jackson:jackson-core-asl:1.9.13
| +--- org.codehaus.jackson:jackson-mapper-asl:1.9.13
| | \--- org.codehaus.jackson:jackson-core-asl:1.9.13
| +--- com.thoughtworks.paranamer:paranamer:2.7
| +--- org.xerial.snappy:snappy-java:1.1.1.3
| +--- org.apache.commons:commons-compress:1.8.1
| +--- org.tukaani:xz:1.5
| \--- org.slf4j:slf4j-api:1.7.7 -> 1.7.30
+--- io.confluent:kafka-schema-registry-client:5.3.2
| +--- io.confluent:common-config:5.3.2
| | +--- io.confluent:common-utils:5.3.2
| | | +--- org.slf4j:slf4j-api:1.7.26 -> 1.7.30
| | | \--- com.101tec:zkclient:0.10
| | | \--- org.slf4j:slf4j-api:1.6.1 -> 1.7.30
| | \--- org.slf4j:slf4j-api:1.7.26 -> 1.7.30
| +--- org.apache.avro:avro:1.8.1 (*)
| +--- com.fasterxml.jackson.core:jackson-databind:2.9.10.1 ->
2.10.2 (*)
| \--- io.confluent:common-utils:5.3.2 (*)
+--- io.confluent:common-config:5.3.2 (*)
\--- io.confluent:common-utils:5.3.2 (*)
compileOnly - Compile only dependencies for source set 'main'. (n)
No dependencies
default - Configuration for default artifacts. (n)
No dependencies
implementation - Implementation only dependencies for source set 'main'. (n)
No dependencies
runtimeClasspath - Runtime classpath of source set 'main'.
\--- org.apache.beam:beam-sdks-java-io-kafka:2.27.0-SNAPSHOT
+--- org.checkerframework:checker-qual:3.7.0 FAILED
+--- org.apache.beam:beam-vendor-guava-26_0-jre:0.1
+--- org.apache.beam:beam-sdks-java-core:2.27.0-SNAPSHOT FAILED
+--- org.apache.beam:beam-sdks-java-expansion-service:2.27.0-SNAPSHOT
FAILED
+--- org.slf4j:slf4j-api:1.7.30
+--- joda-time:joda-time:2.10.5
+--- com.fasterxml.jackson.core:jackson-annotations:2.10.2
+--- com.fasterxml.jackson.core:jackson-databind:2.10.2
| +--- com.fasterxml.jackson.core:jackson-annotations:2.10.2
| \--- com.fasterxml.jackson.core:jackson-core:2.10.2
+--- org.springframework:spring-expression:4.3.18.RELEASE FAILED
\--- io.confluent:kafka-avro-serializer:5.3.2
+--- org.apache.avro:avro:1.8.1
| +--- org.codehaus.jackson:jackson-core-asl:1.9.13
| +--- org.codehaus.jackson:jackson-mapper-asl:1.9.13
| | \--- org.codehaus.jackson:jackson-core-asl:1.9.13
| +--- com.thoughtworks.paranamer:paranamer:2.7
| +--- org.xerial.snappy:snappy-java:1.1.1.3
| +--- org.apache.commons:commons-compress:1.8.1
| +--- org.tukaani:xz:1.5
| \--- org.slf4j:slf4j-api:1.7.7 -> 1.7.30
+--- io.confluent:kafka-schema-registry-client:5.3.2
| +--- io.confluent:common-config:5.3.2
| | +--- io.confluent:common-utils:5.3.2
| | | +--- org.slf4j:slf4j-api:1.7.26 -> 1.7.30
| | | \--- com.101tec:zkclient:0.10
| | | \--- org.slf4j:slf4j-api:1.6.1 -> 1.7.30
| | \--- org.slf4j:slf4j-api:1.7.26 -> 1.7.30
| +--- org.apache.avro:avro:1.8.1 (*)
| +--- com.fasterxml.jackson.core:jackson-databind:2.9.10.1 ->
2.10.2 (*)
| \--- io.confluent:common-utils:5.3.2 (*)
+--- io.confluent:common-config:5.3.2 (*)
\--- io.confluent:common-utils:5.3.2 (*)
runtimeElements - Elements of runtime for main. (n)
No dependencies
runtimeOnly - Runtime only dependencies for source set 'main'. (n)
No dependencies
testAnnotationProcessor - Annotation processors and their dependencies for
source set 'test'.
No dependencies
testCompileClasspath - Compile classpath for source set 'test'.
\--- org.apache.beam:beam-sdks-java-io-kafka:2.27.0-SNAPSHOT
+--- org.checkerframework:checker-qual:3.7.0 FAILED
+--- org.apache.beam:beam-vendor-guava-26_0-jre:0.1
+--- org.apache.beam:beam-sdks-java-core:2.27.0-SNAPSHOT FAILED
+--- org.apache.beam:beam-sdks-java-expansion-service:2.27.0-SNAPSHOT
FAILED
+--- org.slf4j:slf4j-api:1.7.30
+--- joda-time:joda-time:2.10.5
+--- com.fasterxml.jackson.core:jackson-annotations:2.10.2
+--- com.fasterxml.jackson.core:jackson-databind:2.10.2
| +--- com.fasterxml.jackson.core:jackson-annotations:2.10.2
| \--- com.fasterxml.jackson.core:jackson-core:2.10.2
+--- org.springframework:spring-expression:4.3.18.RELEASE FAILED
\--- io.confluent:kafka-avro-serializer:5.3.2
+--- org.apache.avro:avro:1.8.1
| +--- org.codehaus.jackson:jackson-core-asl:1.9.13
| +--- org.codehaus.jackson:jackson-mapper-asl:1.9.13
| | \--- org.codehaus.jackson:jackson-core-asl:1.9.13
| +--- com.thoughtworks.paranamer:paranamer:2.7
| +--- org.xerial.snappy:snappy-java:1.1.1.3
| +--- org.apache.commons:commons-compress:1.8.1
| +--- org.tukaani:xz:1.5
| \--- org.slf4j:slf4j-api:1.7.7 -> 1.7.30
+--- io.confluent:kafka-schema-registry-client:5.3.2
| +--- io.confluent:common-config:5.3.2
| | +--- io.confluent:common-utils:5.3.2
| | | +--- org.slf4j:slf4j-api:1.7.26 -> 1.7.30
| | | \--- com.101tec:zkclient:0.10
| | | \--- org.slf4j:slf4j-api:1.6.1 -> 1.7.30
| | \--- org.slf4j:slf4j-api:1.7.26 -> 1.7.30
| +--- org.apache.avro:avro:1.8.1 (*)
| +--- com.fasterxml.jackson.core:jackson-databind:2.9.10.1 ->
2.10.2 (*)
| \--- io.confluent:common-utils:5.3.2 (*)
+--- io.confluent:common-config:5.3.2 (*)
\--- io.confluent:common-utils:5.3.2 (*)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
