[
https://issues.apache.org/jira/browse/FLINK-17417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093475#comment-17093475
]
Wei Zhong commented on FLINK-17417:
-----------------------------------
Hi [~twalthr] [~pnowojski] [~chesnay], it seems the code of packaging sql-jar
of flink-avro is involved in this PR:
[https://github.com/apache/flink/pull/6218]
Is this a expected behavior to avoid any bugs? If not, I would like to fix this
problem as it causes the test failure of PyFlink. Currently we need to manually
download the Apache Avro jar file and add it to the classpath of the java
process to avoid the failure.
> The flink-avro-sql-jar is not really sql-jar
> --------------------------------------------
>
> Key: FLINK-17417
> URL: https://issues.apache.org/jira/browse/FLINK-17417
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Affects Versions: 1.9.3, 1.10.0, 1.11.0
> Reporter: Wei Zhong
> Priority: Major
>
> The flink-avro-\{flink.version}-sql-jar.jar is not really sql-jar. It does
> not contains the transitive dependence of flink-avro, i.e. Apache Avro. It
> can be reproduced by the following code:
> {code:java}
> URL flinkAvroSqlJar = new
> URL("file:///Users/zhongwei/flink/flink-formats/flink-avro/target/flink-avro-1.11-SNAPSHOT-sql-jar.jar");
> URLClassLoader classLoader = new URLClassLoader(new URL[]{flinkAvroSqlJar},
> Thread.currentThread().getContextClassLoader());
> Thread.currentThread().setContextClassLoader(classLoader);
> StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> List<Tuple2> list = new ArrayList<>();
> list.add(new Tuple2(1, 2));
> list.add(new Tuple2(3, 4));
> DataStreamUtils.collect(env.fromCollection(list,
> Types.GENERIC(Tuple2.class)));
> {code}
> The exception is:
> {code:java}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/avro/generic/GenericData$Array
> at
> org.apache.flink.formats.avro.utils.AvroKryoSerializerUtils.addAvroGenericDataArrayRegistration(AvroKryoSerializerUtils.java:69)
> at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.buildKryoRegistrations(KryoSerializer.java:565)
> at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.<init>(KryoSerializer.java:132)
> at
> org.apache.flink.api.java.typeutils.GenericTypeInfo.createSerializer(GenericTypeInfo.java:90)
> at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.fromCollection(StreamExecutionEnvironment.java:814)
> at com.example.TestArgs.main(TestArgs.java:24)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.avro.generic.GenericData$Array
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 6 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
