yguy-ryft commented on code in PR #13432: URL: https://github.com/apache/iceberg/pull/13432#discussion_r2229069060
########## docs/docs/maintenance.md: ########## @@ -63,20 +63,16 @@ Expiring old snapshots removes them from metadata, so they are no longer availab ### Remove old metadata files -Iceberg keeps track of table metadata using JSON files. Each change to a table produces a new metadata file to provide atomicity. +Iceberg keeps track of table metadata using JSON files. Each change to a table produces a new metadata file to provide atomicity. Old metadata files are kept for history by default, and are tracked in the `metadata-log` field of the metadata file. Tables with frequent commits, like those written by streaming jobs, may need to regularly clean metadata files to reduce metadata overhead. -Old metadata files are kept for history by default. Tables with frequent commits, like those written by streaming jobs, may need to regularly clean metadata files. +The number of metadata files being **tracked** is defined by `write.metadata.previous-versions-max` (default is 100). -To automatically clean metadata files, set `write.metadata.delete-after-commit.enabled=true` in table properties. This will keep some metadata files (up to `write.metadata.previous-versions-max`) and will delete the oldest metadata file after each new one is created. +To automatically delete older metadata files when they become untracked, set `write.metadata.delete-after-commit.enabled=true` in table properties. This will keep some metadata files as tracked (up to `write.metadata.previous-versions-max`), and will delete the oldest metadata file every time a new one is created. +Alternatively, untracked metadata files can be deleted as part of [orphan file deletion](#delete-orphan-files). Review Comment: fixed 👍 ########## docs/docs/maintenance.md: ########## @@ -63,20 +63,16 @@ Expiring old snapshots removes them from metadata, so they are no longer availab ### Remove old metadata files -Iceberg keeps track of table metadata using JSON files. Each change to a table produces a new metadata file to provide atomicity. +Iceberg keeps track of table metadata using JSON files. Each change to a table produces a new metadata file to provide atomicity. Old metadata files are kept for history by default, and are tracked in the `metadata-log` field of the metadata file. Tables with frequent commits, like those written by streaming jobs, may need to regularly clean metadata files to reduce metadata overhead. -Old metadata files are kept for history by default. Tables with frequent commits, like those written by streaming jobs, may need to regularly clean metadata files. +The number of metadata files being **tracked** is defined by `write.metadata.previous-versions-max` (default is 100). -To automatically clean metadata files, set `write.metadata.delete-after-commit.enabled=true` in table properties. This will keep some metadata files (up to `write.metadata.previous-versions-max`) and will delete the oldest metadata file after each new one is created. +To automatically delete older metadata files when they become untracked, set `write.metadata.delete-after-commit.enabled=true` in table properties. This will keep some metadata files as tracked (up to `write.metadata.previous-versions-max`), and will delete the oldest metadata file every time a new one is created. Review Comment: fixed 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
