RussellSpitzer commented on issue #3409: URL: https://github.com/apache/iceberg/issues/3409#issuecomment-953958541
The data rewrite does not remove files, it is a purely additive operation like almost all Iceberg operations. The new snapshot created will refer only to the new files which should be compacted versions of the old small files. To remove the old small files you must erase the history of the table with `expire snapshots`, only when the table no longer has a history which references the pre-compacted files will it physically remove them. Be sure to check the parameters of `expire snapshots` when using it as the defaults are conservative to prevent erasing history which may be in use. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
