minihippo commented on PR #5629:
URL: https://github.com/apache/hudi/pull/5629#issuecomment-1191692939
> How much gains do we get after the patch, feel nervous about this change,
let's pay more attention about stability first.
@danny0405 The current benchmark result of RFC-46:
- no overhead introduced of avro log read/write with new merger
- Compare with avro record in reading/writing parquet log
- for read, spark record improves ~33% (after e2e duration-before e2e
duration)*before
- for write, spark record improves 7%~9%. In the stage that transforms
input to hoodie record, it improves ~25%. But the shuffle data becomes bigger
that before. It costs more in the following shuffle read stage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
