kazdy commented on issue #8274: URL: https://github.com/apache/hudi/issues/8274#issuecomment-1480907317
Hudi does a lot of additional stuff when compared to Iceberg. Eg metadata table maintenance itself in my case takes the same time as read transform and write part (around 3m in CoW for inline metadata table maintenence) have you tried to disable metadata table or use async metadata table service? In my case microbatch processing time dropped from 7m to 3m once I enabled it. If you want higher throughput you can also disable cleaning, compaction and clustering and run it in a separate job. Have you tried these things? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
