alievmirza commented on code in PR #1493: URL: https://github.com/apache/ignite-3/pull/1493#discussion_r1066146675
########## modules/metastorage/README.md: ########## @@ -0,0 +1,46 @@ +# Metastorage + +The module for store and access to metadata. There is module is connected with this one: + +- metastorage-api - the module contains classes to access to metastorage service from another components. + +The storage is distributed to prevent losing data and based of RAFT. That distributed storage persists on top of every cluster node, however +most of such participants are learners and only few of them are voting ones. In typical case, the group consists of only several voted nodes +of the cluster (amount of nodes have to be odd 3 or 5). The rest of nodes listens metadata update but does not participant in voting (RAFT +learners). + +## Threading model + +There is only one dedicate thread to notify watches. This is a thread with prefix metastorage-watch-executor. +The thread is created from executor in KeyValueStorage: + Review Comment: There is only one thread specifically for notifying watchers. The thread is created in KeyValueStorage from the executor: ########## modules/metastorage/README.md: ########## @@ -0,0 +1,46 @@ +# Metastorage + +The module for store and access to metadata. There is module is connected with this one: Review Comment: The module for storing and accessing metadata. This module is linked to another one: ########## modules/metastorage/README.md: ########## @@ -0,0 +1,46 @@ +# Metastorage + +The module for store and access to metadata. There is module is connected with this one: + +- metastorage-api - the module contains classes to access to metastorage service from another components. Review Comment: The module contains classes that other components can use to access the metastorage service. ########## modules/metastorage/README.md: ########## @@ -0,0 +1,46 @@ +# Metastorage + +The module for store and access to metadata. There is module is connected with this one: + +- metastorage-api - the module contains classes to access to metastorage service from another components. + +The storage is distributed to prevent losing data and based of RAFT. That distributed storage persists on top of every cluster node, however +most of such participants are learners and only few of them are voting ones. In typical case, the group consists of only several voted nodes +of the cluster (amount of nodes have to be odd 3 or 5). The rest of nodes listens metadata update but does not participant in voting (RAFT +learners). + +## Threading model + +There is only one dedicate thread to notify watches. This is a thread with prefix metastorage-watch-executor. +The thread is created from executor in KeyValueStorage: + +```java +private ExecutorService watchExecutor = Executors.newSingleThreadExecutor(new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-watch-executor"),LOG)); +``` + +Also, the storage internally contains a two threads executor to create a snapshot: + +```java +private ExecutorService snapshotExecutor = Executors.newFixedThreadPool(2,new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-snapshot-executor"),LOG)); +``` + +Both of the executors have a node name in their prefixes to distinguish to which node the particular thread belongs. + +### Interface methods + +Various operations (*get()*, *getAll()*, *invoke()*) in Metastorage manager return futures. Those futures are completed when the matched +RAFT command completes to the Metastorage group. This result appears in RAFT client executor (prefix <NODE_NAME>%Raft-Group-Client), but the +entire replication procedure is happened tin the RAFT threads (description of that treading model available in +[Raft module](../raft/README.md)). Review Comment: Futures are returned by a number of Metastorage Manager operations, like *get()*, *getAll()*, *invoke()*, etc. Those futures are completed when the corresponding RAFT command is completed in the Metastorage group. Although the entire replication process took place in the RAFT threads, this result appears in the RAFT client executor with prefix <NODE_NAME>%Raft-Group-Client. See RAFT module for more information about its threading model. ########## modules/metastorage/README.md: ########## @@ -0,0 +1,46 @@ +# Metastorage + +The module for store and access to metadata. There is module is connected with this one: + +- metastorage-api - the module contains classes to access to metastorage service from another components. + +The storage is distributed to prevent losing data and based of RAFT. That distributed storage persists on top of every cluster node, however +most of such participants are learners and only few of them are voting ones. In typical case, the group consists of only several voted nodes +of the cluster (amount of nodes have to be odd 3 or 5). The rest of nodes listens metadata update but does not participant in voting (RAFT +learners). + +## Threading model + +There is only one dedicate thread to notify watches. This is a thread with prefix metastorage-watch-executor. +The thread is created from executor in KeyValueStorage: + +```java +private ExecutorService watchExecutor = Executors.newSingleThreadExecutor(new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-watch-executor"),LOG)); +``` + +Also, the storage internally contains a two threads executor to create a snapshot: + Review Comment: Additionally, the storage internally has two thread executors for creating a snapshot: ########## modules/metastorage/README.md: ########## @@ -0,0 +1,46 @@ +# Metastorage + +The module for store and access to metadata. There is module is connected with this one: + +- metastorage-api - the module contains classes to access to metastorage service from another components. + +The storage is distributed to prevent losing data and based of RAFT. That distributed storage persists on top of every cluster node, however +most of such participants are learners and only few of them are voting ones. In typical case, the group consists of only several voted nodes +of the cluster (amount of nodes have to be odd 3 or 5). The rest of nodes listens metadata update but does not participant in voting (RAFT +learners). + +## Threading model + +There is only one dedicate thread to notify watches. This is a thread with prefix metastorage-watch-executor. +The thread is created from executor in KeyValueStorage: + +```java +private ExecutorService watchExecutor = Executors.newSingleThreadExecutor(new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-watch-executor"),LOG)); +``` + +Also, the storage internally contains a two threads executor to create a snapshot: + +```java +private ExecutorService snapshotExecutor = Executors.newFixedThreadPool(2,new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-snapshot-executor"),LOG)); +``` + +Both of the executors have a node name in their prefixes to distinguish to which node the particular thread belongs. + +### Interface methods + +Various operations (*get()*, *getAll()*, *invoke()*) in Metastorage manager return futures. Those futures are completed when the matched +RAFT command completes to the Metastorage group. This result appears in RAFT client executor (prefix <NODE_NAME>%Raft-Group-Client), but the +entire replication procedure is happened tin the RAFT threads (description of that treading model available in +[Raft module](../raft/README.md)). + +Although another methods are returned futures, but in most cases they are executed synchronously. The futures are conditioned by +asynchronous Metastorage initialization, that happens because required a time to start a RAFT group. In other words, we have a chance of +complete those features in thread of RAFT client executor (prefix <NODE_NAME>%Raft-Group-Client). + Review Comment: Although some methods return futures, they are often run synchronously. Futures are dependent on asynchronous Metastorage initialization, since starting an RAFT group requires time. In other words, we have a chance to complete those features in the RAFT client executor thread (prefix NODE NAME>%Raft-Group-Client). ########## modules/metastorage/README.md: ########## @@ -0,0 +1,46 @@ +# Metastorage + +The module for store and access to metadata. There is module is connected with this one: + +- metastorage-api - the module contains classes to access to metastorage service from another components. + +The storage is distributed to prevent losing data and based of RAFT. That distributed storage persists on top of every cluster node, however +most of such participants are learners and only few of them are voting ones. In typical case, the group consists of only several voted nodes +of the cluster (amount of nodes have to be odd 3 or 5). The rest of nodes listens metadata update but does not participant in voting (RAFT +learners). + Review Comment: To avoid data loss, the storage is distributed using the RAFT consensus algorithm. Every cluster node has access to such distributed storage, but the majority of members are learners, and only a small number are voters in terms of RAFT algorithm. Typically, only a small number of the cluster's voting nodes make up the raft-group (number of nodes must be odd, either 3 or 5.). The remaining nodes listen to metadata updates but do not vote (they called learners in terms of RAFT). ########## modules/metastorage/README.md: ########## @@ -0,0 +1,46 @@ +# Metastorage + +The module for store and access to metadata. There is module is connected with this one: + +- metastorage-api - the module contains classes to access to metastorage service from another components. + +The storage is distributed to prevent losing data and based of RAFT. That distributed storage persists on top of every cluster node, however +most of such participants are learners and only few of them are voting ones. In typical case, the group consists of only several voted nodes +of the cluster (amount of nodes have to be odd 3 or 5). The rest of nodes listens metadata update but does not participant in voting (RAFT +learners). + +## Threading model + +There is only one dedicate thread to notify watches. This is a thread with prefix metastorage-watch-executor. +The thread is created from executor in KeyValueStorage: + +```java +private ExecutorService watchExecutor = Executors.newSingleThreadExecutor(new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-watch-executor"),LOG)); +``` + +Also, the storage internally contains a two threads executor to create a snapshot: + +```java +private ExecutorService snapshotExecutor = Executors.newFixedThreadPool(2,new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-snapshot-executor"),LOG)); +``` + +Both of the executors have a node name in their prefixes to distinguish to which node the particular thread belongs. + +### Interface methods + +Various operations (*get()*, *getAll()*, *invoke()*) in Metastorage manager return futures. Those futures are completed when the matched +RAFT command completes to the Metastorage group. This result appears in RAFT client executor (prefix <NODE_NAME>%Raft-Group-Client), but the +entire replication procedure is happened tin the RAFT threads (description of that treading model available in +[Raft module](../raft/README.md)). + +Although another methods are returned futures, but in most cases they are executed synchronously. The futures are conditioned by +asynchronous Metastorage initialization, that happens because required a time to start a RAFT group. In other words, we have a chance of +complete those features in thread of RAFT client executor (prefix <NODE_NAME>%Raft-Group-Client). + +### Using common pool + +The component is used common ForkJoinPool on start (in fact, it is not necessary, because all components started in asynchronously in the +same ForkJoinPool). The using of the common pool is dangerous, because the pool can be busy by another threads that hosted on the same JVM +(TODO: IGNITE-18505 Avoid using common pool on start components). Review Comment: The component uses common ForkJoinPool on start (in fact, it is not necessary, because all components starts asynchronously in the same ForkJoinPool). The using of the common pool is dangerous, because the pool can be busy by another threads that hosted on the same JVM (TODO: IGNITE-18505 Avoid using common pool on start components). ########## modules/metastorage/README.md: ########## @@ -0,0 +1,46 @@ +# Metastorage + +The module for store and access to metadata. There is module is connected with this one: + +- metastorage-api - the module contains classes to access to metastorage service from another components. + +The storage is distributed to prevent losing data and based of RAFT. That distributed storage persists on top of every cluster node, however +most of such participants are learners and only few of them are voting ones. In typical case, the group consists of only several voted nodes +of the cluster (amount of nodes have to be odd 3 or 5). The rest of nodes listens metadata update but does not participant in voting (RAFT +learners). + +## Threading model + +There is only one dedicate thread to notify watches. This is a thread with prefix metastorage-watch-executor. +The thread is created from executor in KeyValueStorage: + +```java +private ExecutorService watchExecutor = Executors.newSingleThreadExecutor(new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-watch-executor"),LOG)); +``` + +Also, the storage internally contains a two threads executor to create a snapshot: + +```java +private ExecutorService snapshotExecutor = Executors.newFixedThreadPool(2,new NamedThreadFactory(NamedThreadFactory + .threadPrefix(nodeName, "metastorage-snapshot-executor"),LOG)); +``` + +Both of the executors have a node name in their prefixes to distinguish to which node the particular thread belongs. + Review Comment: To distinguish to which node the specific thread belongs, each executor has a node name in its prefix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
