yihua opened a new issue, #19563:
URL: https://github.com/apache/hudi/issues/19563
### Describe the problem you faced
Merging a Debezium Postgres change event that carries a non-null `BYTES`
column (or a union containing `BYTES`) fails with a `ClassCastException`:
```
java.lang.ClassCastException:
org.apache.hudi.common.model.SerializableIndexedRecord
cannot be cast to org.apache.avro.generic.GenericData$Record
at
org.apache.hudi.common.model.debezium.PostgresDebeziumAvroPayload.containsBytesToastedValues(PostgresDebeziumAvroPayload.java:143)
```
`PostgresDebeziumAvroPayload` casts records to the concrete class
`GenericData.Record` in `containsBytesToastedValues` and in
`mergeToastedValuesIfPresent`. Since #13987 ("perf: Lazy deserialization of
Avro indexed record"), `BaseAvroPayload.getRecord` returns a
`SerializableIndexedRecord` whenever the payload's record does not already
carry the exact `Schema` instance being merged with, which is the ordinary case
in production and after any Kryo round trip. `SerializableIndexedRecord`
implements `GenericRecord` but does not extend `GenericData.Record`, so the
cast fails.
That PR converted the casts in `containsStringToastedValues` to the
`GenericRecord` interface but left the two in `containsBytesToastedValues` and
one in `mergeToastedValuesIfPresent`, which is why only tables with a `BYTES`
column are affected.
The existing `TestPostgresDebeziumAvroPayload#testMergeWithToastedValues`
misses this because it constructs the record and merges with the very same
`Schema` instance, so the payload hands back the original `GenericData.Record`
and never takes the lazy path.
### To Reproduce
1. Ingest a Postgres table through the Debezium source with
`hoodie.datasource.write.payload.class=org.apache.hudi.common.model.debezium.PostgresDebeziumAvroPayload`.
2. Give the table a `BYTES` (or nullable `BYTES`) column with a non-null
value.
3. Apply an update whose LSN is ahead of the record already in storage, so
the incoming event wins the LSN comparison and the TOAST check runs.
### Expected behavior
The TOAST check works against any `GenericRecord` implementation, and the
merge preserves the previous value for toasted columns.
### Environment Description
* Hudi version: master (`c63c9bfa79f` and later)
### Additional context
Affects any `IndexedRecord` implementation that is not a
`GenericData.Record`, so the same latent problem exists on `branch-0.x` even
though the lazy-deserialization path that triggers it is only on master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
