hutiefang76 opened a new issue, #12879: URL: https://github.com/apache/seatunnel/issues/12879
### Search before asking - [x] I searched the issues and open/closed PRs for Avro nullable rows. #11660 concerns Parquet model allocation and #11493 adds a JSON logical type; neither addresses this generated-schema case. ### What happened The generated Avro schema wraps primitive, array, and map types in a nullable union, but emits a nested SeaTunnel `ROW` as a bare Avro record. Consequently, a null ROW field fails before binary serialization. The same issue appears for a null element of `ARRAY<ROW<...>>` and a null value of `MAP<STRING, ROW<...>>`. I added three round-trip cases to the existing `AvroSerializationSchemaTest`. All three fail on current dev, while the seven existing cases pass: ```text 10 tests, 0 assertion failures, 3 errors, 0 skipped AvroRuntimeException: Field payload type:RECORD pos:0 does not accept null values NullPointerException: null value for (non-nullable) items at SeaTunnelRecord.items[array][1] NullPointerException: null value for (non-nullable) payload_map at SeaTunnelRecord.payload_map[map]["absent"] ``` Each case builds a SeaTunnel row type, sets the nested ROW value to null, and runs the existing Avro serializer/deserializer. The ARRAY and MAP cases include a non-null sibling as a control. ### SeaTunnel Version Current dev e2957d27afa29d4de4324cf642450757d865b902. ### SeaTunnel Config Format-level reproduction; no engine job, external database, credentials, or container is needed. ```text ROW<payload ROW<value INT>> ROW<items ARRAY<ROW<value INT>>> ROW<payload_map MAP<STRING, ROW<value INT>>> ``` ### Running Command ```shell ./mvnw -B -pl seatunnel-formats/seatunnel-format-avro -am -Dtest=AvroSerializationSchemaTest -Dsurefire.failIfNoSpecifiedTests=false package ``` ### Error Exception See the three actual errors above. The failure is reproducible with the project Avro dependency and the existing format test class. ### Compatibility question Making the ROW branch nullable would fix these cases, but a record-to-union change adds a union branch index to the binary encoding, including for non-null records. Replacing the generated schema directly could therefore affect existing unframed Avro payloads when the old writer schema is unavailable. I have not changed production code for that reason. Would an opt-in nullable-ROW schema mode be preferable, or should this wait for an explicitly documented format migration? I can implement the agreed direction and include old-writer/new-reader compatibility cases. For now, this report records the reproducible failure and the wire-format constraint rather than proposing a default schema change. ### Java or Scala Version Java 17. ### Are you willing to submit PR? - [x] Yes, after the schema compatibility direction is agreed. ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
