[jira] [Updated] (IGNITE-20559) Return metastorage invokes in DistributionZoneManager#createMetastorageTopologyListener
[
https://issues.apache.org/jira/browse/IGNITE-20559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mirza Aliev updated IGNITE-20559:
-
Epic Link: IGNITE-20611 (was: IGNITE-20166)
> Return metastorage invokes in
> DistributionZoneManager#createMetastorageTopologyListener
> ---
>
> Key: IGNITE-20559
> URL: https://issues.apache.org/jira/browse/IGNITE-20559
> Project: Ignite
> Issue Type: Bug
>Reporter: Mirza Aliev
>Assignee: Mirza Aliev
>Priority: Major
> Labels: ignite-3
>
> h3. *Motivation*
> There are meta storage invokes in DistributionZoneManager in zone's
> lifecycle. The futures of these invokes are ignored, so after the lifecycle
> method is completed actually not all its actions are completed. Therefore
> several invokes for example on createZone and alterZone can be reordered.
> Currently it does the meta storage invokes in:
> # LogicalTopologyEventListener to update logical topology.
> Also we need to save {{nodeAttriburtes}} and {{topologyAugmentationMap}} in MS
> h3. *Definition of Done*
> Need to ensure event handling linearization. All immediate data nodes
> recalculation must be returned to the event handler. Also
> {{nodeAttriburtes}} and {{topologyAugmentationMap}} in must be saved in MS,
> so we can use this fields when recovery DZM
> h3. *Implementation notes*
> When topology update is handled (createMetastorageTopologyListener),
> immediately recalculate data nodes within caller handler for all zones, which
> have immediate timer. Also within the caller handler, write nodesAttributes
> and topologyAugmentationMaps to metastore. Only after completion of this ms
> invoke, schedule local timers. All futures of these changes must be returned
> as a result of the watch listener update, so this update could be marked as
> processed only after all above mentioned actions are completed.
> For CAS-ing nodesAttributes and topologyAugmentationMaps, we can reuse
> DistributionZonesUtil#zonesGlobalStateRevision, but change it from vault key
> to MS and make it per zone. Every time we try to save these keys, we will
> take revision of topology update (topRev) and will try to write changes with
> condition topRev > ms.zonesGlobalStateRevision
> To clean up this map, we can remove all augmentations that are less than
> min(scaleUpTriggerKeys, scaleDownTriggerKeys) when we write the new one.
> Further optimization: we can save only one topologyAugmentationMap, the only
> question is how to clean up the map. We can find minimal
> min(scaleUPTriggerKeys, scaleDownTriggerKeys) among all zones and clean up
> all up to this minimum.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20559) Return metastorage invokes in DistributionZoneManager#createMetastorageTopologyListener
[
https://issues.apache.org/jira/browse/IGNITE-20559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mirza Aliev updated IGNITE-20559:
-
Description:
h3. *Motivation*
There are meta storage invokes in DistributionZoneManager in zone's lifecycle.
The futures of these invokes are ignored, so after the lifecycle method is
completed actually not all its actions are completed. Therefore several invokes
for example on createZone and alterZone can be reordered. Currently it does the
meta storage invokes in:
# LogicalTopologyEventListener to update logical topology.
Also we need to save {{nodeAttriburtes}} and {{topologyAugmentationMap}} in MS
h3. *Definition of Done*
Need to ensure event handling linearization. All immediate data nodes
recalculation must be returned to the event handler. Also {{nodeAttriburtes}}
and {{topologyAugmentationMap}} in must be saved in MS, so we can use this
fields when recovery DZM
h3. *Implementation notes*
When topology update is handled (createMetastorageTopologyListener),
immediately recalculate data nodes within caller handler for all zones, which
have immediate timer. Also within the caller handler, write nodesAttributes and
topologyAugmentationMaps to metastore. Only after completion of this ms invoke,
schedule local timers. All futures of these changes must be returned as a
result of the watch listener update, so this update could be marked as
processed only after all above mentioned actions are completed.
For CAS-ing nodesAttributes and topologyAugmentationMaps, we can reuse
DistributionZonesUtil#zonesGlobalStateRevision, but change it from vault key to
MS and make it per zone. Every time we try to save these keys, we will take
revision of topology update (topRev) and will try to write changes with
condition topRev > ms.zonesGlobalStateRevision
To clean up this map, we can remove all augmentations that are less than
min(scaleUpTriggerKeys, scaleDownTriggerKeys) when we write the new one.
Further optimization: we can save only one topologyAugmentationMap, the only
question is how to clean up the map. We can find minimal
min(scaleUPTriggerKeys, scaleDownTriggerKeys) among all zones and clean up all
up to this minimum.
was:
h3. *Motivation*
There are meta storage invokes in DistributionZoneManager in zone's lifecycle.
The futures of these invokes are ignored, so after the lifecycle method is
completed actually not all its actions are completed. Therefore several invokes
for example on createZone and alterZone can be reordered. Currently it does the
meta storage invokes in:
# LogicalTopologyEventListener to update logical topology.
Also we need to save {{nodeAttriburtes}} and {{topologyAugmentationMap}} in MS
h3. *Definition of Done*
Need to ensure event handling linearization. All immediate data nodes
recalculation must be returned to the event handler. Also {{nodeAttriburtes}}
and {{topologyAugmentationMap}} in must be saved in MS, so we can use this
fields when recovery DZM
> Return metastorage invokes in
> DistributionZoneManager#createMetastorageTopologyListener
> ---
>
> Key: IGNITE-20559
> URL: https://issues.apache.org/jira/browse/IGNITE-20559
> Project: Ignite
> Issue Type: Bug
>Reporter: Mirza Aliev
>Assignee: Mirza Aliev
>Priority: Major
> Labels: ignite-3
>
> h3. *Motivation*
> There are meta storage invokes in DistributionZoneManager in zone's
> lifecycle. The futures of these invokes are ignored, so after the lifecycle
> method is completed actually not all its actions are completed. Therefore
> several invokes for example on createZone and alterZone can be reordered.
> Currently it does the meta storage invokes in:
> # LogicalTopologyEventListener to update logical topology.
> Also we need to save {{nodeAttriburtes}} and {{topologyAugmentationMap}} in MS
> h3. *Definition of Done*
> Need to ensure event handling linearization. All immediate data nodes
> recalculation must be returned to the event handler. Also
> {{nodeAttriburtes}} and {{topologyAugmentationMap}} in must be saved in MS,
> so we can use this fields when recovery DZM
> h3. *Implementation notes*
> When topology update is handled (createMetastorageTopologyListener),
> immediately recalculate data nodes within caller handler for all zones, which
> have immediate timer. Also within the caller handler, write nodesAttributes
> and topologyAugmentationMaps to metastore. Only after completion of this ms
> invoke, schedule local timers. All futures of these changes must be returned
> as a result of the watch listener update, so this update could be marked as
> processed only after all above mentioned actions are completed.
> For CAS-ing nodesAttributes and topologyAugmentationMaps, we can reuse
[jira] [Updated] (IGNITE-20559) Return metastorage invokes in DistributionZoneManager#createMetastorageTopologyListener
[
https://issues.apache.org/jira/browse/IGNITE-20559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mirza Aliev updated IGNITE-20559:
-
Epic Link: IGNITE-20166
> Return metastorage invokes in
> DistributionZoneManager#createMetastorageTopologyListener
> ---
>
> Key: IGNITE-20559
> URL: https://issues.apache.org/jira/browse/IGNITE-20559
> Project: Ignite
> Issue Type: Bug
>Reporter: Mirza Aliev
>Priority: Major
> Labels: ignite-3
>
> h3. *Motivation*
> There are meta storage invokes in DistributionZoneManager in zone's
> lifecycle. The futures of these invokes are ignored, so after the lifecycle
> method is completed actually not all its actions are completed. Therefore
> several invokes for example on createZone and alterZone can be reordered.
> Currently it does the meta storage invokes in:
> # LogicalTopologyEventListener to update logical topology.
> Also we need to save {{nodeAttriburtes}} and {{topologyAugmentationMap}} in MS
> h3. *Definition of Done*
> Need to ensure event handling linearization. All immediate data nodes
> recalculation must be returned to the event handler. Also
> {{nodeAttriburtes}} and {{topologyAugmentationMap}} in must be saved in MS,
> so we can use this fields when recovery DZM
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20559) Return metastorage invokes in DistributionZoneManager#createMetastorageTopologyListener
[
https://issues.apache.org/jira/browse/IGNITE-20559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mirza Aliev updated IGNITE-20559:
-
Description:
h3. *Motivation*
There are meta storage invokes in DistributionZoneManager in zone's lifecycle.
The futures of these invokes are ignored, so after the lifecycle method is
completed actually not all its actions are completed. Therefore several invokes
for example on createZone and alterZone can be reordered. Currently it does the
meta storage invokes in:
# LogicalTopologyEventListener to update logical topology.
Also we need to save {{nodeAttriburtes}} and {{topologyAugmentationMap}} in MS
h3. *Definition of Done*
Need to ensure event handling linearization. All immediate data nodes
recalculation must be returned to the event handler. Also {{nodeAttriburtes}}
and {{topologyAugmentationMap}} in must be saved in MS, so we can use this
fields when recovery DZM
was:
h3. *Motivation*
There are meta storage invokes in DistributionZoneManager in zone's lifecycle.
The futures of these invokes are ignored, so after the lifecycle method is
completed actually not all its actions are completed. Therefore several invokes
for example on createZone and alterZone can be reordered. Currently it does the
meta storage invokes in:
# LogicalTopologyEventListener to update logical topology.
Also we need to save {{nodeAttriburtes}} and {{topologyAugmentationMap}} in MS
h3. *Definition of Done*
Need to ensure event handling linearization. All immediate data nodes
recalculation must be returned to the event handler.
> Return metastorage invokes in
> DistributionZoneManager#createMetastorageTopologyListener
> ---
>
> Key: IGNITE-20559
> URL: https://issues.apache.org/jira/browse/IGNITE-20559
> Project: Ignite
> Issue Type: Bug
>Reporter: Mirza Aliev
>Priority: Major
> Labels: ignite-3
>
> h3. *Motivation*
> There are meta storage invokes in DistributionZoneManager in zone's
> lifecycle. The futures of these invokes are ignored, so after the lifecycle
> method is completed actually not all its actions are completed. Therefore
> several invokes for example on createZone and alterZone can be reordered.
> Currently it does the meta storage invokes in:
> # LogicalTopologyEventListener to update logical topology.
> Also we need to save {{nodeAttriburtes}} and {{topologyAugmentationMap}} in MS
> h3. *Definition of Done*
> Need to ensure event handling linearization. All immediate data nodes
> recalculation must be returned to the event handler. Also
> {{nodeAttriburtes}} and {{topologyAugmentationMap}} in must be saved in MS,
> so we can use this fields when recovery DZM
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20559) Return metastorage invokes in DistributionZoneManager#createMetastorageTopologyListener
[
https://issues.apache.org/jira/browse/IGNITE-20559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mirza Aliev updated IGNITE-20559:
-
Description:
h3. *Motivation*
There are meta storage invokes in DistributionZoneManager in zone's lifecycle.
The futures of these invokes are ignored, so after the lifecycle method is
completed actually not all its actions are completed. Therefore several invokes
for example on createZone and alterZone can be reordered. Currently it does the
meta storage invokes in:
# LogicalTopologyEventListener to update logical topology.
Also we need to save {{nodeAttriburtes}} and {{topologyAugmentationMap}} in MS
h3. *Definition of Done*
Need to ensure event handling linearization. All immediate data nodes
recalculation must be returned to the event handler.
> Return metastorage invokes in
> DistributionZoneManager#createMetastorageTopologyListener
> ---
>
> Key: IGNITE-20559
> URL: https://issues.apache.org/jira/browse/IGNITE-20559
> Project: Ignite
> Issue Type: Bug
>Reporter: Mirza Aliev
>Priority: Major
> Labels: ignite-3
>
> h3. *Motivation*
> There are meta storage invokes in DistributionZoneManager in zone's
> lifecycle. The futures of these invokes are ignored, so after the lifecycle
> method is completed actually not all its actions are completed. Therefore
> several invokes for example on createZone and alterZone can be reordered.
> Currently it does the meta storage invokes in:
> # LogicalTopologyEventListener to update logical topology.
> Also we need to save {{nodeAttriburtes}} and {{topologyAugmentationMap}} in MS
> h3. *Definition of Done*
> Need to ensure event handling linearization. All immediate data nodes
> recalculation must be returned to the event handler.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
