Hi Matthias, Sorry for my miss leading. I mean kafka-schema-serializer rather than kafka-avro-serializer.
io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe is in kafka-schema-serializer and kafka-schema-serializer should be a dependency of kafka-avro-serializer according to their pom.xml files(see [1], [2]). I couldn't resolve a valid kafka-avro-serializer.jar in my mirror so I downloaded it manually from [3] and installed it using: mvn install:install-file -DgroupId=io.confluent -DartifactId=kafka-avro-serializer -Dversion=5.5.2 -Dpackaging=jar -Dfile=kafka-avro-serializer-5.5.2.jar After that, I tried to build Flink and got the above exceptions. Then I tried to add the dependency of kafka-schema-serializer to flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/pom.xml (also manually installed it to my local maven repo) and everything went well. I also tried to remove it from the pom.xml after installing, and the exception came back. Maybe there was something wrong with the manually-installed kafka-avro-serializer? [1]. https://mvnrepository.com/artifact/io.confluent/kafka-schema-serializer/usages [2]. https://packages.confluent.io/maven/io/confluent/kafka-avro-serializer/5.5.2/kafka-avro-serializer-5.5.2.pom [3]. https://mvnrepository.com/artifact/io.confluent/kafka-avro-serializer At 2021-01-22 21:22:51, "Matthias Pohl" <matth...@ververica.com> wrote: Hi Smile, Have you used a clean checkout? I second Robert's statement considering that the dependency you're talking about is already part of flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/pom.xml. It also has the correct scope set both in master and release-1.12. Best, Matthias On Fri, Jan 22, 2021 at 10:04 AM Smile@LETTers <letters_sm...@163.com> wrote: Yes, I've tried from both the root directory and the sub module. Neither or them works. And the error messages are the same. At 2021-01-21 23:22:12, "Robert Metzger" <rmetz...@apache.org> wrote: Since our CI system is able to build Flink, I believe it's a local issue. Are you sure that the build is failing when you build Flink from the root directory (not calling maven from within a maven module?) On Tue, Jan 19, 2021 at 11:19 AM Smile@LETTers <letters_sm...@163.com> wrote: Hi, I got an error when tried to compile & package Flink (version 1.12 & current master). It can be reproduced by run 'mvn clean test' under flink-end-to-end-tests/flink-end-to-end-tests-common-kafka. It seems that a necessary dependency for test scope was missing and some classes can not be found. After adding the dependency kafka-avro-serializer to the pom of flink-end-to-end-tests/flink-end-to-end-tests-common-kafka everything goes well. And I just wonder that is this a bug or I missed some local setting? Best regards. Smile Error logs attached: [INFO] --------< org.apache.flink:flink-end-to-end-tests-common-kafka >-------- [INFO] Building Flink : E2E Tests : Common Kafka 1.13-SNAPSHOT [INFO] --------------------------------[ jar ]--------------------------------- Downloading ... [INFO] [INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ flink-end-to-end-tests-common-kafka --- [INFO] Deleting /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target [INFO] [INFO] --- maven-checkstyle-plugin:2.17:check (validate) @ flink-end-to-end-tests-common-kafka --- [INFO] [INFO] --- spotless-maven-plugin:2.4.2:check (spotless-check) @ flink-end-to-end-tests-common-kafka --- [INFO] [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven-version) @ flink-end-to-end-tests-common-kafka --- [INFO] [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @ flink-end-to-end-tests-common-kafka --- [INFO] [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-snakeyaml) @ flink-end-to-end-tests-common-kafka --- [INFO] [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-jackson) @ flink-end-to-end-tests-common-kafka --- [INFO] [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (forbid-log4j-1) @ flink-end-to-end-tests-common-kafka --- [INFO] [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ flink-end-to-end-tests-common-kafka --- [INFO] [INFO] --- gmavenplus-plugin:1.8.1:execute (merge-categories) @ flink-end-to-end-tests-common-kafka --- [INFO] Using plugin classloader, includes GMavenPlus classpath. [INFO] Using Groovy 2.5.12 to perform execute. includes: org.apache.flink.tests.util.categories.Dummy excludes: [INFO] [INFO] --- directory-maven-plugin:0.1:highest-basedir (directories) @ flink-end-to-end-tests-common-kafka --- [INFO] Highest basedir set to: /Users/smile/Downloads/W/code/flink/apache/master/flink [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ flink-end-to-end-tests-common-kafka --- [INFO] [INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ flink-end-to-end-tests-common-kafka --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ flink-end-to-end-tests-common-kafka --- [INFO] Compiling 5 source files to /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/classes [INFO] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java: /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java uses unchecked or unsafe operations. [INFO] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java: Recompile with -Xlint:unchecked for details. [INFO] [INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ flink-end-to-end-tests-common-kafka --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 2 resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ flink-end-to-end-tests-common-kafka --- [INFO] Compiling 4 source files to /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/test-classes [INFO] ------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20] cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found [INFO] 1 error [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 29.842 s [INFO] Finished at: 2021-01-19T18:04:39+08:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project flink-end-to-end-tests-common-kafka: Compilation failure [ERROR] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20] cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe [ERROR] class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found [ERROR] [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
