Cqz666 commented on issue #4875: URL: https://github.com/apache/iceberg/issues/4875#issuecomment-1139372412
@kbendick My main problem was solved via `partial-progress.enabled `and `max-concurrent-file-group-rewrites`. :+1: But I still have some confusion, when does it make sense to do the 'removeOrphanFilesAction' job, because you know that rewriting a file leaves an old, useless small file that I don't want to keep forever, but don't know when to do the 'removeOrphanFilesAction' job. I have a simple idea, after the rewrite job is done,then trigger the 'removeOrphanFilesAction' task, or is there a better suggestion? And whether the metadata catalog will continue to swell to a large order of magnitude as streaming tasks continue to run, assuming they continue to run for a long period of time, creating performance issues and becoming difficult to maintain. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
