[
https://issues.apache.org/jira/browse/FLINK-17091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105210#comment-17105210
]
Paul Lin commented on FLINK-17091:
----------------------------------
Maybe I could add more background information. Avro requires long-based
timestamps for serialization, but the query result would be Timestamp backed by
LocalDateTime or java.sql.Timestamp, so we need to convert them to long in
AvroRowSerializationSchema. That's the way it always has been doing.
The problem is that previous it only needs to deal with java.sql.Timestamp (see
[code|[https://github.com/apache/flink/blob/master/flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/AvroRowSerializationSchema.java#L251]]),
but after FLINK-14645 it also needs to deal with LocalDateTime and.
The conversion I proposed is not for the generation use case of SQL data types,
but just for the serialization of Avro, to bridge between the physical classes
of the SQL data types and the classes that a specific serializer requires.
That's why the conversion happens in AvroRow(De)SerializationSchema.
> AvroRow(De)SerializationSchema doesn't support new Timestamp conversion
> classes
> -------------------------------------------------------------------------------
>
> Key: FLINK-17091
> URL: https://issues.apache.org/jira/browse/FLINK-17091
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Affects Versions: 1.10.0
> Reporter: Paul Lin
> Priority: Major
>
> AvroRow(De)SerializationSchema doesn't know how to convert the new physical
> classes of Timestamp (eg. java.time.Date) to/from Avro's int/long based
> timestamp. Currently, when encountering objects of the new physical classes,
> AvroRow(De)SerializationSchema just ignores them and passes them to Avro's
> GenericDatumWriter/Reader, which leads to ClassCastException thrown by
> GenericDatumWriter/Reader. See
> [AvroRowSerializationSchema|https://github.com/apache/flink/blob/master/flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/AvroRowSerializationSchema.java#L251].
> To fix this problem, we should support LocalTime/LocalDate/LocalDateTime
> conversion to int/long in AvroRowSerializationSchema, and support int/long
> conversion to LocalTime/LocalDate/LocalDateTime based on logical
> types(Types.LOCAL_TIME/Types.LOCAL_DATE/Types.LOCAL_DATE_TIME) in
> AvroRowDeserializationSchema.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
