kbendick commented on PR #4736: URL: https://github.com/apache/iceberg/pull/4736#issuecomment-1131020429
Thanks for tagging me Anton. I think this is a good idea as well. I won’t repeat others but I have some questions and possibly a few additional ideas but I need to go through some theoretical cases on paper first. > One potential problem with that is that we will load the manifest list for every expired snapshot on the driver, which can become a bottleneck if we expire a lot of snapshots. I've seen such cases. Two thoughts: 1) We should add an event / metric describing this replanning work. Could be used as a signal to perform table maintenance. 2) We might be able to track a metric to determine if we should do this initial replanning work in a distributed manner. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
