Hugo-WB commented on PR #13647: URL: https://github.com/apache/iceberg/pull/13647#issuecomment-4432217771
Looking forward to re-merging this on 1.11! When expiring snapshots on large tables (100k+) snapshots to remove. Our REST Catalog struggles a lot due to how many times it hits `rewriteSnapshotsInternal`. Very slow + very large memory usage. Temporary fix for us has been was coalescing the RemoveSnapshots into a single large RemoveSnapshot which fixed the slowness. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
