nsivabalan commented on issue #7246: URL: https://github.com/apache/hudi/issues/7246#issuecomment-1352178930
do you have a rough estimate of the size of commit meta files that are being archived. the first 10 commits in your active timeline will be the ones likely going to be archived. I can't think of any reason other than those 10 commit files put together is causing the OOM. may be you can try reducing the value for `hoodie.commits.archival.batch` to 5 may be and see what happens. In general, how many file groups you have in ur table and on an avg how many files are touched for every commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
