yihua opened a new issue, #20175: URL: https://github.com/apache/hudi/issues/20175
A merge-on-read update appends the new version of a record to the log of its current file group. Hudi's own readers merge that correctly, but storage layouts and engines that model a table as immutable data files plus positional delete files cannot represent such a log until the next compaction. The pluggable table format (RFC-93) therefore publishes merge-on-read deltacommits as empty changes, and readers through it see stale data between compactions. Compaction and merge-on-read rollback also complete without notifying the table format. Proposal: a new operation mode in which an update becomes a positional delete against the record's current file group plus an insert into a new file slice, so log files carry positional deletes only and every live record stays in a base file; the table format completion hook also fires on compaction, log compaction and merge-on-read rollback. Design in RFC-114 (number claimed in #20174); writer changes in #20167. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
