voonhous commented on issue #20630:
URL: https://github.com/apache/hudi/issues/20630#issuecomment-7031125835

   ### Avro conversions per row, before and after #20758
   
   Applies only to MOR file slices with log files. COW, `_ro` and slices 
without log files take the base-file-only page source and convert nothing in 
either case.
   
   | Row | Before (`IndexedRecord` context) | After (`TrinoRow` context) | 
Change |
   |---|---|---|---|
   | Base row, no log update | page to Avro (`serialize()`), then Avro to page 
(`buildRecordInPage()`) | none: copied by position | optimized: both 
conversions removed |
   | Base and log row, commit-time or event-time merge | base row page to Avro; 
ordering values compared on Avro; winner Avro to page | none: the ordering 
field is read from the block, the winner is copied by position | optimized: 
conversions removed |
   | Log row from a native (Parquet) log block | page to Avro; Avro to page if 
emitted | none: read into pages, copied into single-row blocks while buffered | 
optimized: conversions replaced by a block copy |
   | Log row from a classic Avro log block | decoded to Avro by hudi-common; 
Avro to page if emitted | decoded to Avro by hudi-common; Avro to `TrinoRow` 
once on read | same: the conversion moves from output to read, so a log row 
that loses its merge is now converted too |
   | Base and log row, payload merge | base row page to Avro; payload runs; 
result Avro to page | both rows to Avro; payload runs; result back to a 
`TrinoRow` | same: one extra conversion when the log row came from a classic 
Avro log block |
   | Custom `HoodieRecordMerger` table | as above | unchanged: stays on the 
`IndexedRecord` context | same |
   | Row spilled to disk | Avro record written with hudi-common's default 
serializer (#20648 replaces it) | row written as Avro binary and converted back 
on read | changed, not measured; to be aligned with #20648 |
   
   Commit-time and event-time ordering are the defaults, and the built-in 
payloads (`OverwriteWithLatestAvroPayload`, `DefaultHoodieRecordPayload`) 
resolve to them, so the payload row applies only to other payload classes. Avro 
remains where hudi-common makes it the interface: classic Avro log blocks, 
payload classes, custom mergers and spill.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to