[
https://issues.apache.org/jira/browse/FLINK-40264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40264:
-----------------------------------
Labels: pull-request-available (was: )
> nested legacyTimestampMapping
> -----------------------------
>
> Key: FLINK-40264
> URL: https://issues.apache.org/jira/browse/FLINK-40264
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Affects Versions: 1.20.0, 2.2.0, 2.3.0
> Reporter: Pritam Kumar
> Assignee: Pritam Kumar
> Priority: Major
> Labels: pull-request-available
>
> h2. Problem
> {{AvroToRowDataConverters#createConverter}} dispatches a nested {{ROW}} to
> the single argument overload:
> {code:java}
> case ROW:
> return createRowConverter((RowType) type);
> {code}
> {{createRowConverter(RowType)}} defaults {{legacyTimestampMapping}} to
> {{true}}, so the flag is dropped for every nesting level below the top one.
> Since {{TIMESTAMP_LTZ}} is only supported by the non legacy mapping, a nested
> {{TIMESTAMP_LTZ}}
> column fails even when the non legacy mapping was explicitly requested:
> {noformat}
> java.lang.UnsupportedOperationException: Unsupported type: TIMESTAMP_LTZ(3)
> NOT NULL
> {noformat}
> The failure happens while the deserialization schema is being constructed,
> so the job never starts.
> h2. Reproduction
> {code:java}
> DataType dataType =
> ROW(
> FIELD("id", INT().notNull()),
> FIELD("nested", ROW(FIELD("ltz3",
> TIMESTAMP_LTZ(3).notNull())).notNull()))
> .notNull();
> RowType rowType = (RowType) dataType.getLogicalType();
> new AvroRowDataDeserializationSchema(
> rowType,
> InternalTypeInfo.of(rowType),
> AvroEncoding.BINARY,
> false /* legacyTimestampMapping */); // throws
> {code}
> {{AvroSchemaConverter#convertToSchema}} handles the same type correctly
> with {{legacyTimestampMapping = false}}, and {{RowDataToAvroConverters}}
> propagates the flag properly, so only the deserialization side is affected.
> h2. Fix
> Propagate the flag:
> {code:java}
> case ROW:
> return createRowConverter((RowType) type, legacyTimestampMapping);
> {code}
> Strictly a widening: the only code paths whose behaviour changes are ones
> that throw today.
> h2. Introduced by
> FLINK-33198, first released in 1.19.0.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
