[
https://issues.apache.org/jira/browse/FLINK-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17164263#comment-17164263
]
Dawid Wysakowicz edited comment on FLINK-18478 at 7/24/20, 8:55 AM:
--------------------------------------------------------------------
Yes I also got to the same conclusions yesterday. Generally speaking
{{avrohugger}} (or rather scala) does not work with avro native libraries. You
never know what will blow up as some of the methods in avro simply do not work
with {{avrohugger}}.
We can workaround this particular problem in {{AvroDeserializationSchema}} by
passing the schema in the ctor of {{SpecificDatumReader}}. Nevertheless we can
not make it work with schema registry as we do not the method {{setSchema}} to
work...
I'd suggest to implement this fix (passing schema in a ctor of
{{SpecificDatumReader}}) for {{AvroDeserializationSchema}}, but I'd add a note
that we discourage usage of avrohugger as it is incompatible with avro native
libraries and we do not guarantee it works properly in all cases.
was (Author: dawidwys):
Yes I also got to the same conclusions yesterday. Generally speaking
{{avrohugger}} (or rather scala) does not work with avro native libraries. You
never know what will blow up as some of the methods in avro simply do not work
with {{avrohugger}}.
We can workaround this particular problem in {{AvroDeserializationSchema}} by
passing the schema in ctor of {{SpecificDatumReader}}. Nevertheless we can not
make it work with schema registry as we do not the method {{setSchema}} to
work...
I'd suggest to implement this fix (passing schema in a ctor of
{{SpecificDatumReader}}) for {{AvroDeserializationSchema}}, but I'd add a note
that we discourage usage of avrohugger as it is incompatible with avro native
libraries and we do not guarantee it works properly in all cases.
> AvroDeserializationSchema does not work with types generated by avrohugger
> --------------------------------------------------------------------------
>
> Key: FLINK-18478
> URL: https://issues.apache.org/jira/browse/FLINK-18478
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Reporter: Aljoscha Krettek
> Priority: Major
> Labels: pull-request-available
>
> The main problem is that the code in {{SpecificData.createSchema()}} tries to
> reflectively read the {{SCHEMA$}} field, that is normally there in Avro
> generated classes. However, avrohugger generates this field in a companion
> object, which the reflective Java code will therefore not find.
> This is also described in these ML threads:
> *
> [https://lists.apache.org/thread.html/5db58c7d15e4e9aaa515f935be3b342fe036e97d32e1fb0f0d1797ee@%3Cuser.flink.apache.org%3E]
> *
> [https://lists.apache.org/thread.html/cf1c5b8fa7f095739438807de9f2497e04ffe55237c5dea83355112d@%3Cuser.flink.apache.org%3E]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
