korlov42 commented on code in PR #3585:
URL: https://github.com/apache/ignite-3/pull/3585#discussion_r1567377675
##########
modules/catalog/README.md:
##########
@@ -1,7 +1,54 @@
# Catalog module
-This module provides database catalog service implementation and descriptor
for catalog objects.
+Catalog is a component that responsible for managing descriptors of objects
available in cluster,
+as well as serving as single source of truth for other components to acquire
an actual state of
+the schema.
-See
[CatalogService](src/main/java/org/apache/ignite/internal/catalog/CatalogServiceImpl.java)
+This module provides implementation of catalog service as well as internal API
for accessing the
+objects descriptors.
-TODO: IGNITE-19082 Add comprehensive description.
\ No newline at end of file
+## Base components description
+
+*
[CatalogObjectDescriptor](src/main/java/org/apache/ignite/internal/catalog/descriptors/CatalogObjectDescriptor.java)
--
+ base class for objects managed by catalog, like tables, indexes, zones, etc.
+*
[CatalogService](src/main/java/org/apache/ignite/internal/catalog/CatalogService.java)
-- provides
+ methods to access catalog's objects' descriptors of exact version and/or
last actual version at
+ given timestamp, which is logical point-in-time.
+*
[CatalogCommand](src/main/java/org/apache/ignite/internal/catalog/CatalogCommand.java)
-- denotes
+ particular modification of the catalog, like creation of particular table,
for example.
+*
[CatalogManager](src/main/java/org/apache/ignite/internal/catalog/CatalogManager.java)
-- provides
+ methods for object manipulation (like creation of new object and/or
modification of existing ones),
+ also takes care of component lifecycle.
+*
[UpdateEntry](src/main/java/org/apache/ignite/internal/catalog/storage/UpdateEntry.java)
--
+ result of applying
[CatalogCommand](src/main/java/org/apache/ignite/internal/catalog/CatalogCommand.java),
+ represents delta required to move current version of catalog to state
defined in the given command.
+*
[UpdateLog](src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLog.java)
-- distributed
+ log of incremental updates.
+
+## How it works
+
+### CatalogManager
+
+Every modification of catalog is split on two phases: accept and apply. During
accept phase, given command
+is validated, then used to generate a list of update entries to save, and,
finally, list is saved to
+distributed update log. Below is a sequence diagram describing accept phase:
+
+
+After update entries are saved to the log, it is the job of the update log to
propagate updates across the
+cluster. On every node, update log notifies catalog manager about new update
entries, and latter applies
+them and stores new version of a catalog in a local cache. Below is a sequence
diagram describing apply phase:
+
+
+### UpdateLog
+
+Current implementation of update log based on a metastorage. Update entries of
version N are stored by
+`catalog.update.{N}` key. Also, the latest known version is stored by
`catalog.version` key. Updates
+are saved on CAS manner with condition `newVersion == value(catalog.version)`.
Review Comment:
> I expect to see some guatantees here
good point! added `Guarantee` section
> mentioning particular metastorage keys name is too detailed
perhaps, you are right. But since you don't mind to leave it as is, I would
prefer to go this way. We always can get rid of this section later if it
appears too expensive to keep this up to date
##########
modules/catalog/README.md:
##########
@@ -1,7 +1,54 @@
# Catalog module
-This module provides database catalog service implementation and descriptor
for catalog objects.
+Catalog is a component that responsible for managing descriptors of objects
available in cluster,
+as well as serving as single source of truth for other components to acquire
an actual state of
+the schema.
-See
[CatalogService](src/main/java/org/apache/ignite/internal/catalog/CatalogServiceImpl.java)
+This module provides implementation of catalog service as well as internal API
for accessing the
+objects descriptors.
-TODO: IGNITE-19082 Add comprehensive description.
\ No newline at end of file
+## Base components description
+
+*
[CatalogObjectDescriptor](src/main/java/org/apache/ignite/internal/catalog/descriptors/CatalogObjectDescriptor.java)
--
+ base class for objects managed by catalog, like tables, indexes, zones, etc.
+*
[CatalogService](src/main/java/org/apache/ignite/internal/catalog/CatalogService.java)
-- provides
+ methods to access catalog's objects' descriptors of exact version and/or
last actual version at
+ given timestamp, which is logical point-in-time.
+*
[CatalogCommand](src/main/java/org/apache/ignite/internal/catalog/CatalogCommand.java)
-- denotes
+ particular modification of the catalog, like creation of particular table,
for example.
+*
[CatalogManager](src/main/java/org/apache/ignite/internal/catalog/CatalogManager.java)
-- provides
+ methods for object manipulation (like creation of new object and/or
modification of existing ones),
+ also takes care of component lifecycle.
+*
[UpdateEntry](src/main/java/org/apache/ignite/internal/catalog/storage/UpdateEntry.java)
--
+ result of applying
[CatalogCommand](src/main/java/org/apache/ignite/internal/catalog/CatalogCommand.java),
+ represents delta required to move current version of catalog to state
defined in the given command.
+*
[UpdateLog](src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLog.java)
-- distributed
+ log of incremental updates.
+
+## How it works
+
+### CatalogManager
+
+Every modification of catalog is split on two phases: accept and apply. During
accept phase, given command
+is validated, then used to generate a list of update entries to save, and,
finally, list is saved to
+distributed update log. Below is a sequence diagram describing accept phase:
+
+
+After update entries are saved to the log, it is the job of the update log to
propagate updates across the
+cluster. On every node, update log notifies catalog manager about new update
entries, and latter applies
+them and stores new version of a catalog in a local cache. Below is a sequence
diagram describing apply phase:
+
+
+### UpdateLog
+
+Current implementation of update log based on a metastorage. Update entries of
version N are stored by
+`catalog.update.{N}` key. Also, the latest known version is stored by
`catalog.version` key. Updates
+are saved on CAS manner with condition `newVersion == value(catalog.version)`.
+
+Over time, the log may grow to a humongous size. To address this, snapshotting
was introduced to UpdateLog.
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
