DanielLeens commented on issue #12940: URL: https://github.com/apache/seatunnel/issues/12940#issuecomment-6340369899
Thanks for keeping this performance investigation separate from the nullable-ROW correctness work. Classification: E / Format Avro nested-ROW serialization allocation. Current `RowToAvroConverter.resolveObject` does build a nested Avro schema for each non-null ROW value, including rows reached through ARRAY and MAP recursion, so the allocation concern is real. The source currently has no such cache. Please keep this issue investigation-first for now. #12900 is still open and changes the same Avro conversion/schema paths for nullable ROW layout, so do not begin an overlapping optimization PR until that correctness work is merged and the current behavior is rechecked. Then capture an allocation/CPU baseline for realistic nested data, compare a structural schema cache with reuse of enclosing-schema branches, and only propose an approach that preserves default bytes, nullable layout, legacy record names, and distinct same-named rows under different parents. Existing byte-compatibility and nested Kafka/Pulsar round trips are the minimum regression boundary; no field-name-only cache is acceptable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
