denis-chudov commented on code in PR #1493: URL: https://github.com/apache/ignite-3/pull/1493#discussion_r1067125861
########## modules/metastorage/README.md: ########## @@ -0,0 +1,45 @@ +# Metastorage + +The module for storing and accessing metadata. This module is linked to another one: + +- metastorage-api - The module contains classes that other components can use to access the metastorage service. + +To avoid data loss, the storage is distributed using the RAFT consensus algorithm. Every cluster node has access to such distributed +storage, but the majority of members are learners, and only a small number are voters in terms of RAFT algorithm. Typically, only a small +number of the cluster's voting nodes make up the raft-group (number of nodes must be odd, either 3 or 5.). The remaining nodes listen to +metadata updates but do not vote (they called learners in terms of RAFT). + +## Threading model + +There is only one thread specifically for notifying watchers. The thread is created in KeyValueStorage from the executor: + +```java +private ExecutorService watchExecutor = Executors.newSingleThreadExecutor(new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-watch-executor"),LOG)); +``` + +Additionally, the storage internally has two thread executors for creating a snapshot: + +```java +private ExecutorService snapshotExecutor = Executors.newFixedThreadPool(2,new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-snapshot-executor"),LOG)); +``` + +To distinguish to which node the specific thread belongs, each executor has a node name in its prefix. + +### Interface methods + +Futures are returned by a number of Metastorage Manager operations, like get(), getAll(), invoke(), etc. Those futures are completed when +the corresponding RAFT command is completed in the Metastorage group. Although the entire replication process took place in the RAFT +threads, this result appears in the RAFT client executor with prefix <NODE_NAME>%Raft-Group-Client. See RAFT module for more information Review Comment: ```suggestion the corresponding RAFT command is completed in the Metastorage group. As the entire replication process takes place in RAFT threads, this result appears in the RAFT client executor with prefix <NODE_NAME>%Raft-Group-Client. See RAFT module for more information ``` ########## modules/metastorage/README.md: ########## @@ -0,0 +1,45 @@ +# Metastorage + +The module for storing and accessing metadata. This module is linked to another one: + +- metastorage-api - The module contains classes that other components can use to access the metastorage service. + +To avoid data loss, the storage is distributed using the RAFT consensus algorithm. Every cluster node has access to such distributed +storage, but the majority of members are learners, and only a small number are voters in terms of RAFT algorithm. Typically, only a small +number of the cluster's voting nodes make up the raft-group (number of nodes must be odd, either 3 or 5.). The remaining nodes listen to +metadata updates but do not vote (they called learners in terms of RAFT). + +## Threading model + +There is only one thread specifically for notifying watchers. The thread is created in KeyValueStorage from the executor: + +```java +private ExecutorService watchExecutor = Executors.newSingleThreadExecutor(new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-watch-executor"),LOG)); +``` + +Additionally, the storage internally has two thread executors for creating a snapshot: + +```java +private ExecutorService snapshotExecutor = Executors.newFixedThreadPool(2,new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-snapshot-executor"),LOG)); +``` + +To distinguish to which node the specific thread belongs, each executor has a node name in its prefix. + +### Interface methods + +Futures are returned by a number of Metastorage Manager operations, like get(), getAll(), invoke(), etc. Those futures are completed when +the corresponding RAFT command is completed in the Metastorage group. Although the entire replication process took place in the RAFT +threads, this result appears in the RAFT client executor with prefix <NODE_NAME>%Raft-Group-Client. See RAFT module for more information +about its threading model. + +Although some methods return futures, they are often run synchronously. Futures are dependent on asynchronous Metastorage initialization, Review Comment: shouldn'we specify these methods? "some methods" sounds too uncertain ########## modules/metastorage/README.md: ########## @@ -0,0 +1,45 @@ +# Metastorage + +The module for storing and accessing metadata. This module is linked to another one: + +- metastorage-api - The module contains classes that other components can use to access the metastorage service. + +To avoid data loss, the storage is distributed using the RAFT consensus algorithm. Every cluster node has access to such distributed +storage, but the majority of members are learners, and only a small number are voters in terms of RAFT algorithm. Typically, only a small +number of the cluster's voting nodes make up the raft-group (number of nodes must be odd, either 3 or 5.). The remaining nodes listen to +metadata updates but do not vote (they called learners in terms of RAFT). + +## Threading model + +There is only one thread specifically for notifying watchers. The thread is created in KeyValueStorage from the executor: + +```java +private ExecutorService watchExecutor = Executors.newSingleThreadExecutor(new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-watch-executor"),LOG)); +``` + +Additionally, the storage internally has two thread executors for creating a snapshot: + +```java +private ExecutorService snapshotExecutor = Executors.newFixedThreadPool(2,new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-snapshot-executor"),LOG)); +``` + +To distinguish to which node the specific thread belongs, each executor has a node name in its prefix. + +### Interface methods + +Futures are returned by a number of Metastorage Manager operations, like get(), getAll(), invoke(), etc. Those futures are completed when +the corresponding RAFT command is completed in the Metastorage group. Although the entire replication process took place in the RAFT +threads, this result appears in the RAFT client executor with prefix <NODE_NAME>%Raft-Group-Client. See RAFT module for more information +about its threading model. + +Although some methods return futures, they are often run synchronously. Futures are dependent on asynchronous Metastorage initialization, +since starting an RAFT group requires time. In other words, we have a chance to complete those features in the RAFT client executor thread ( +prefix NODE NAME>%Raft-Group-Client). + +### Using common pool + +The component uses common ForkJoinPool on start (in fact, it is not necessary, because all components starts asynchronously in the same Review Comment: ```suggestion The component uses common ForkJoinPool on start (in fact, it is not necessary, because all components start asynchronously in the same ``` ########## modules/metastorage/README.md: ########## @@ -0,0 +1,45 @@ +# Metastorage + +The module for storing and accessing metadata. This module is linked to another one: + +- metastorage-api - The module contains classes that other components can use to access the metastorage service. + +To avoid data loss, the storage is distributed using the RAFT consensus algorithm. Every cluster node has access to such distributed +storage, but the majority of members are learners, and only a small number are voters in terms of RAFT algorithm. Typically, only a small +number of the cluster's voting nodes make up the raft-group (number of nodes must be odd, either 3 or 5.). The remaining nodes listen to +metadata updates but do not vote (they called learners in terms of RAFT). + +## Threading model + +There is only one thread specifically for notifying watchers. The thread is created in KeyValueStorage from the executor: + +```java +private ExecutorService watchExecutor = Executors.newSingleThreadExecutor(new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-watch-executor"),LOG)); +``` + +Additionally, the storage internally has two thread executors for creating a snapshot: + +```java +private ExecutorService snapshotExecutor = Executors.newFixedThreadPool(2,new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-snapshot-executor"),LOG)); +``` + +To distinguish to which node the specific thread belongs, each executor has a node name in its prefix. + +### Interface methods + +Futures are returned by a number of Metastorage Manager operations, like get(), getAll(), invoke(), etc. Those futures are completed when +the corresponding RAFT command is completed in the Metastorage group. Although the entire replication process took place in the RAFT +threads, this result appears in the RAFT client executor with prefix <NODE_NAME>%Raft-Group-Client. See RAFT module for more information +about its threading model. + +Although some methods return futures, they are often run synchronously. Futures are dependent on asynchronous Metastorage initialization, +since starting an RAFT group requires time. In other words, we have a chance to complete those features in the RAFT client executor thread ( +prefix NODE NAME>%Raft-Group-Client). + +### Using common pool + +The component uses common ForkJoinPool on start (in fact, it is not necessary, because all components starts asynchronously in the same +ForkJoinPool). The using of the common pool is dangerous, because the pool can be busy by another threads that hosted on the same JVM Review Comment: ```suggestion ForkJoinPool). The using of the common pool is dangerous, because the pool can be occupied by another threads that hosted on the same JVM ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
