swapna267 commented on code in PR #18525:
URL: https://github.com/apache/iceberg/pull/18525#discussion_r4300780310
##########
flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordInternalSerializer.java:
##########
@@ -94,21 +94,9 @@ public void serialize(DynamicRecordInternal toSerialize,
DataOutputView dataOutp
}
dataOutputView.writeInt(toSerialize.writerKey());
- final RowDataSerializer rowDataSerializer;
- if (writeSchemaAndSpec) {
- rowDataSerializer =
- serializerCache.serializer(
- toSerialize.tableName(), toSerialize.schema(),
toSerialize.spec());
- } else {
- // Check that the schema id can be resolved. Not strictly necessary for
serialization.
- Tuple3<RowDataSerializer, Schema, PartitionSpec> serializer =
- serializerCache.serializerWithSchemaAndSpec(
- toSerialize.tableName(),
- toSerialize.schema().schemaId(),
- toSerialize.spec().specId());
- rowDataSerializer = serializer.f0;
- }
-
+ RowDataSerializer rowDataSerializer =
+ serializerCache.serializer(
+ toSerialize.tableName(), toSerialize.schema(), toSerialize.spec());
Review Comment:
In worst case, when user sets a new Schema object reference for every record,
Case1: If Record schema matches the existing table schema - Cached [Schema
reference](https://github.com/apache/iceberg/blob/main/flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/TableMetadataCache.java#L260)
is passed.
Case2: Schema evolved and ImmediateTableUpdate is true - Table is evolved
locally and cached schema reference is used.
Case3: Schema evolved and ImmediateTableUpdate is false - This path hasn't
changed in this PR. With writeSchemaAndSpec set to true for side output, we
used the schema from incoming record , as schema id is not valid.
So the scenario where Schema ID was being passed before, it was always the
Schema loaded from Table, not the schema from incoming record.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
