[
https://issues.apache.org/jira/browse/IGNITE-18763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Polovtcev updated IGNITE-18763:
-----------------------------------------
Description:
Currently Meta Storage Watches follow the given contract:
# All listeners are processed sequentially in the order that they have been
registered;
# All listeners are processed in the same thread;
# All notifications are grouped by a particular revision. After all events for
a particular revision get processed, all modified keys are persisted into the
Vault.
Some components may rely on this to ensure that dependent components can
observe the most recent component state, because they registered their Meta
Storage listeners after the component they depend on, therefore it is
guaranteed that that component has already processed current Meta Storage
revision.
This approach can be improved in terms of performance by enabling concurrent
Watch processing in multiple threads. In order to do that, we need to check and
fix the following places:
# If a component depends on another component's state, this state must be
extracted into a Versioned Value;
# If a component updates its state, relying on the fact, that this state is
always updated in a single thread, concurrent primitives must be introduced to
ensure thread safety;
# Check if it is possible to persist updated keys in the Vault individually,
thus making it possible not to wait for all Watches for a particular revision
to complete before starting to process updates belonging to the next revision.
was:
Our first approach to design metastorage concentrated on hiding metastorage
revision from other components has shown that it complicates code and creates
unexpected dependencies.
One example of such dependencies is that notification of configuration changes
is delivered to components in the same order as they appear in configuration.
If two components require specific order of notifications it enforces the same
order of their configuration which is absolutely unclear for developer or
end-user (if some user-facing configuration is involved).
In order to fix these complications we want to expose metastorage revision in
the form of VersionedValue. In new API keys in metastorage are exposed to
client components in the form of VersionedValues with revision as their
integral part.
Components will use this additional information to coordinate between each
other and if necessary to allow their dependants to wait for a particular
version of configuration being applied.
In this task we need to define a set of requirements for a new API, define how
it should look and behave.
> Concurrent Meta Storage Watch processing
> ----------------------------------------
>
> Key: IGNITE-18763
> URL: https://issues.apache.org/jira/browse/IGNITE-18763
> Project: Ignite
> Issue Type: Task
> Components: persistence
> Reporter: Sergey Chugunov
> Assignee: Aleksandr Polovtcev
> Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> Currently Meta Storage Watches follow the given contract:
> # All listeners are processed sequentially in the order that they have been
> registered;
> # All listeners are processed in the same thread;
> # All notifications are grouped by a particular revision. After all events
> for a particular revision get processed, all modified keys are persisted into
> the Vault.
> Some components may rely on this to ensure that dependent components can
> observe the most recent component state, because they registered their Meta
> Storage listeners after the component they depend on, therefore it is
> guaranteed that that component has already processed current Meta Storage
> revision.
> This approach can be improved in terms of performance by enabling concurrent
> Watch processing in multiple threads. In order to do that, we need to check
> and fix the following places:
> # If a component depends on another component's state, this state must be
> extracted into a Versioned Value;
> # If a component updates its state, relying on the fact, that this state is
> always updated in a single thread, concurrent primitives must be introduced
> to ensure thread safety;
> # Check if it is possible to persist updated keys in the Vault individually,
> thus making it possible not to wait for all Watches for a particular revision
> to complete before starting to process updates belonging to the next
> revision.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
