[
https://issues.apache.org/jira/browse/IGNITE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-15910:
-------------------------------------
Description:
h3. Context
General concept of processing distributed notifications from meta storage
assumes that such events will be processed synchronously one after another with
sort of finalization step during which watch event projection is committed to
vault together with corresponding applied revision. It's worth to mention that
it's guaranteed that all notification events with same revision will be grouped
to one and only one watch event.
Besides synchronous watch events processing it's guaranteed that
MetaStorageManager also notifies components about related distributed updates
synchronously in the components hierarchy order. In other words, within meta
storage watch notification processing thread, MetaStorageManager funs-out
corresponding sub-notifications to component A, then, when component A
finishes it's notification processing, to component B, then, when B finishes
it's part, to C, etc.
So far so good. Components cannot overtake each other in knowledge of the
distribution state. Also if node stops of fails it's guaranteed, that after
restart, it'll continue to retrieve the notifications from the last seen
applied revision .
h3. Problem
If however a component process it's distributed notification asynchronously
it's possible for node to lost pending updates in case of restart. Let's say
that ConfigurationRegistry (seems that currently it's the only component with
async notifications processing) retrieves it's portion of data from
MetaStorageManager applies it and notifies corresponding listeners with
configuration updates. Because of asynchronous nature of configuration events
it's possible that such notification aren't processed despite the fact that
top-level related meta storage event was already committed to vault.
onWatchEventN [updated keys=[updates keys for component A, updated keys for
component B], applied revision ]
* fun-out to component A
** async processing of some keys for component A
* fun-out to component B
* commit projections for all keys and applied recision to vault. // even if
async processing of keys for component A isn't finished
h3. Possible solution
One of the options that will help to overcome given issue is to convert
in-component notification to pending events that implies committing such
pending events to vault within meta storage watch notification processing
thread.
onWatchEventN [updated keys=[updates keys for component A, updated keys for
component B], applied revision ]
* fun-out to component A
** *commit corresponding sub sets of keys to vault as pending events is
they'll be processed asynchronously.*
** async processing of such keys.
* fun-out to component B
* commit projections for all keys and applied recision to vault. // even if
async processing of keys for component A isn't finished
was:
h3. Context
General concept of processing distributed notifications from meta storage
assumes that such events will be processed synchronously one after another with
sort of finalization step during which watch event projection is committed to
vault together with corresponding applied revision. It's worth to mention that
it's guaranteed that all notification events with same revision will be grouped
to one and only one watch event.
Besides synchronous watch events processing it's guaranteed that
MetaStorageManager also notifies components about related distributed updates
synchronously in the components hierarchy order. In other words, within meta
storage watch notification processing thread, MetaStorageManager funs-out
corresponding sub-notifications to component A, then, when component A
finishes it's notification processing, to component B, then, when B finishes
it's part, to C, etc.
So far so good. Components cannot overtake each other in knowledge of the
distribution state. Also if node stops of fails it's guaranteed, that after
restart, it'll continue to retrieve the notifications from the last seen
applied revision .
h3. Problem
If however a component process it's distributed notification asynchronously
it's possible for node to lost pending updates in case of restart. Let's say
that ConfigurationRegistry (seems that currently it's the only component with
async notifications processing) retrieves it's portion of data from
MetaStorageManager applies it and notifies corresponding listeners with
configuration updates. Because of asynchronous nature of configuration events
it's possible that such notification isn't processed despite the fact that
top-level related meta storage event was already committed to vault.
onWatchEventN [updated keys=[updates keys for component A, updated keys for
component B], applied revision ]
* fun-out to component A
** async processing of some keys for component A
* fun-out to component B
* commit projections for all keys and applied recision to vault. // even if
async processing of keys for component A isn't finished
h3. Possible solution
One of the options that will help to overcome given issue is to convert
in-component notification to pending events that implies committing such
pending events to vault within meta storage watch notification processing
thread.
onWatchEventN [updated keys=[updates keys for component A, updated keys for
component B], applied revision ]
* fun-out to component A
** *commit corresponding sub sets of keys to vault as pending events is
they'll be processed asynchronously.*
** async processing of such keys.
* fun-out to component B
* commit projections for all keys and applied recision to vault. // even if
async processing of keys for component A isn't finished
> Pending actions are lost in case of node stop
> ---------------------------------------------
>
> Key: IGNITE-15910
> URL: https://issues.apache.org/jira/browse/IGNITE-15910
> Project: Ignite
> Issue Type: Bug
> Reporter: Alexander Lapin
> Priority: Major
> Labels: ignite-3
>
> h3. Context
> General concept of processing distributed notifications from meta storage
> assumes that such events will be processed synchronously one after another
> with sort of finalization step during which watch event projection is
> committed to vault together with corresponding applied revision. It's worth
> to mention that it's guaranteed that all notification events with same
> revision will be grouped to one and only one watch event.
> Besides synchronous watch events processing it's guaranteed that
> MetaStorageManager also notifies components about related distributed
> updates synchronously in the components hierarchy order. In other words,
> within meta storage watch notification processing thread, MetaStorageManager
> funs-out corresponding sub-notifications to component A, then, when
> component A finishes it's notification processing, to component B, then, when
> B finishes it's part, to C, etc.
> So far so good. Components cannot overtake each other in knowledge of the
> distribution state. Also if node stops of fails it's guaranteed, that after
> restart, it'll continue to retrieve the notifications from the last seen
> applied revision .
> h3. Problem
> If however a component process it's distributed notification asynchronously
> it's possible for node to lost pending updates in case of restart. Let's say
> that ConfigurationRegistry (seems that currently it's the only component with
> async notifications processing) retrieves it's portion of data from
> MetaStorageManager applies it and notifies corresponding listeners with
> configuration updates. Because of asynchronous nature of configuration events
> it's possible that such notification aren't processed despite the fact that
> top-level related meta storage event was already committed to vault.
> onWatchEventN [updated keys=[updates keys for component A, updated keys for
> component B], applied revision ]
> * fun-out to component A
> ** async processing of some keys for component A
> * fun-out to component B
> * commit projections for all keys and applied recision to vault. // even if
> async processing of keys for component A isn't finished
> h3. Possible solution
> One of the options that will help to overcome given issue is to convert
> in-component notification to pending events that implies committing such
> pending events to vault within meta storage watch notification processing
> thread.
> onWatchEventN [updated keys=[updates keys for component A, updated keys for
> component B], applied revision ]
> * fun-out to component A
> ** *commit corresponding sub sets of keys to vault as pending events is
> they'll be processed asynchronously.*
> ** async processing of such keys.
> * fun-out to component B
> * commit projections for all keys and applied recision to vault. // even if
> async processing of keys for component A isn't finished
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
