xtern commented on code in PR #4325: URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753182568
########## modules/catalog-compaction/README.md: ########## @@ -1 +1,101 @@ # Catalog compaction module + +> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies, +as it requires some components that may themselves depend on the catalog module. Please +refer to the catalog's module [readme](../catalog/README.md) to see more details about +catalog service and update log. + +## Overview + +During schema changes, the catalog update log stores incremental updates. Each update +increases the catalog version. Over time, the log may grow to a humongous size. To +address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing +incremental updates with a snapshot. + +But different components can refer to a specific version of the catalog. Until they +finish their work with this version, it cannot be truncated. + +This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) +component. This component handles periodical catalog compaction, ensuring that dropped versions +of the catalog are no longer needed by any component in the cluster. + +## Compaction restrictions + +1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less + or equal to the earliest active transaction begin timestamp. +2. Catalog must not be compacted by version which can be required to replay the raft log during recovery. Review Comment: Thanks, fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
