kasured commented on issue #7246: URL: https://github.com/apache/hudi/issues/7246#issuecomment-1352420251
Sorry for the late update. I should have followed up with that before. Right after the issue was created we decided to go with the presumably least risky option of decreasing the batch size 'hoodie.commits.archival.batch'. It helped to eliminate the issue with that particular table. However, the remaining concern for us at the moment is that regardless the options I have listed (unless there are some other) either the number of archive files will keep increasing (if the archive merge is disabled) or the overall archival size will be accumulating (if archival is enabled). Therefore we can close the issue as there is no OOM anymore, on the other hand there seems to be no way to control the growth of the archival files/overall size. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
