[
https://issues.apache.org/jira/browse/IGNITE-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Polovtcev updated IGNITE-19129:
-----------------------------------------
Description:
Current implementation of Parallel Meta Storage Watches works as following:
# Every Watch has an associated unique ID and is processed independently from
other Watches.
# After a Watch has finished processing an event, the processed revision and
the Watch ID gets saved in the Vault.
# During recovery, a minimum processed revision is computed among all Watches
and events are replayed, starting from that revision. If a Watch has already
processed an event, it discards it.
This approach leads to a problem which can happen during recovery when
interacting with the Configuration component:
# Imagine two Watches, one being the Watch registered by the Configuration
component.
# One Watch has processed a revision 5, and the Configuration Watch has
processed a revision 6. After that the node is stopped.
# When the node is started again, the recovery process kicks in and the
Configuration component notifies its listeners about its current state as of
revision 6. It means that components that are dependent on the Configuration
component have indirectly observed the 6th revision and may have completed
their Versioned Values, for example.
# After that, Meta Storage recovery starts and events get replayed. Events will
start from revision 5 as it's the lowest processed revision. Now imagine that,
during the replay, the first Watch needs to access a component's Versioned
Value and will try to do that using revision 5. However, this Versioned Value
will never have a value for this revision, since it has already been completed
by the Configuration component with the revision 6.
Since I don't see viable solution to this problem, it is proposed to disable
fully parallel Watch processing and require so that no Watches can process a
new revision until all Watches have finished processing events from the
previous revision.
was:
Current implementation of Parallel Meta Storage Watches works as following:
# Every Watch has an associated unique ID and is processed independently from
other Watches.
# After a Watch has finished processing an event, the processed revision and
the Watch ID gets saved in the Vault.
# During recovery, a minimum processed revision is computed among all Watches
and events are replayed, starting from that revision. If a Watch has already
processed an event, it discards it.
This approach leads to a problem which can happen during recovery when
interacting with the Configuration component:
WIP
> Remove parallel Watch processing
> --------------------------------
>
> Key: IGNITE-19129
> URL: https://issues.apache.org/jira/browse/IGNITE-19129
> Project: Ignite
> Issue Type: Task
> Reporter: Aleksandr Polovtcev
> Assignee: Aleksandr Polovtcev
> Priority: Major
> Labels: ignite-3
>
> Current implementation of Parallel Meta Storage Watches works as following:
> # Every Watch has an associated unique ID and is processed independently from
> other Watches.
> # After a Watch has finished processing an event, the processed revision and
> the Watch ID gets saved in the Vault.
> # During recovery, a minimum processed revision is computed among all Watches
> and events are replayed, starting from that revision. If a Watch has already
> processed an event, it discards it.
>
> This approach leads to a problem which can happen during recovery when
> interacting with the Configuration component:
> # Imagine two Watches, one being the Watch registered by the Configuration
> component.
> # One Watch has processed a revision 5, and the Configuration Watch has
> processed a revision 6. After that the node is stopped.
> # When the node is started again, the recovery process kicks in and the
> Configuration component notifies its listeners about its current state as of
> revision 6. It means that components that are dependent on the Configuration
> component have indirectly observed the 6th revision and may have completed
> their Versioned Values, for example.
> # After that, Meta Storage recovery starts and events get replayed. Events
> will start from revision 5 as it's the lowest processed revision. Now imagine
> that, during the replay, the first Watch needs to access a component's
> Versioned Value and will try to do that using revision 5. However, this
> Versioned Value will never have a value for this revision, since it has
> already been completed by the Configuration component with the revision 6.
>
> Since I don't see viable solution to this problem, it is proposed to disable
> fully parallel Watch processing and require so that no Watches can process a
> new revision until all Watches have finished processing events from the
> previous revision.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
