[
https://issues.apache.org/jira/browse/IGNITE-10821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16742752#comment-16742752
]
Pavel Kovalenko edited comment on IGNITE-10821 at 1/15/19 5:30 AM:
-------------------------------------------------------------------
The issue is partially completed. What is done at the moment?
1) Introduced IdealAffinityAssignment abstraction. This abstraction remembers
primary partition nodes.
2) Introduced method "calculateWithCachedIdealAffinity". In this method, ideal
affinity assignment for cache groups with similar affinity is calculated once
and cached. This saves time/memory consumption.
3) All affinity recalculation methods are reworked with newly introduced
methods.
4) Non-cooridnator node affinity recalculations are now using
calculateWithCachedIdealAffinity where it's possible.
5) initAffinityOnNodeJoin time consumption is optimized. Now it works in O (Px
* N), where Px - total number of partitions that only has changed the primary
node, instead of whole (P * N).
What should be completed?
1) calculateWithCachedIdealAffinity method should be used where it's possible.
Currently, it's not used in coordinator node affinity manipulations due to
CacheGroupHolder type restrictions. This should be fixed.
2) onServerLeftWithExchangeMergeProtocol shouldn't always use general
(partitions availability) approach as it slow. If Exchange has only server node
left events, we can keep ideal assignment and change assignment only for
partitions where left nodes were primaries using topology owners information
(similar to initAffinityOnNodeJoin approach).
3) If Exchange has the only server left events, affinity message for other
nodes in full message should contain only changed primary nodes for partitions,
instead of whole assignment list. NOTE: there can be problems with backward
compatibility.
4) In the rare case when Exchange has both server left/server join events
onReassignmentEnforced may be used (just for simplification), this should also
be optimized and fixed in nearest future.
5) Introduce tests checking that for similar affinity cache groups
IdealAffinityAssignment object is the same.
was (Author: jokser):
The issue is partially completed. What is done at the moment?
1) Introduced IdealAffinityAssignment abstraction. This abstraction remembers
primary partition nodes.
2) Introduced method "calculateWithCachedIdealAffinity". In this method, ideal
affinity assignment for cache groups with similar affinity is calculated once
and cached. This saves time/memory consumption.
3) All affinity recalculation methods are reworked with newly introduced
methods.
4) Non-cooridnator node affinity recalculations are now using
calculateWithCachedIdealAffinity where it's possible.
5) initAffinityOnNodeJoin time consumption is optimized. Now it works in O (Px
* N), where Px - total number of partitions that changed the primary node,
instead of whole (P * N).
What should be completed?
1) calculateWithCachedIdealAffinity method should be used where it's possible.
Currently, it's not used in coordinator node affinity manipulations due to
CacheGroupHolder type restrictions. This should be fixed.
2) onServerLeftWithExchangeMergeProtocol shouldn't always use general
(partitions availability) approach as it slow. If Exchange has only server node
left events, we can keep ideal assignment and change assignment only for
partitions where left nodes were primaries using topology owners information
(similar to initAffinityOnNodeJoin approach).
3) If Exchange has the only server left events, affinity message for other
nodes in full message should contain only changed primary nodes for partitions,
instead of whole assignment list. NOTE: there can be problems with backward
compatibility.
4) In the rare case when Exchange has both server left/server join events
onReassignmentEnforced may be used (just for simplification), this should also
be optimized and fixed in nearest future.
5) Introduce tests checking that for similar affinity cache groups
IdealAffinityAssignment object is the same.
> Caching affinity with affinity similarity key is broken
> -------------------------------------------------------
>
> Key: IGNITE-10821
> URL: https://issues.apache.org/jira/browse/IGNITE-10821
> Project: Ignite
> Issue Type: Bug
> Components: cache
> Affects Versions: 2.8
> Reporter: Pavel Kovalenko
> Priority: Major
> Fix For: 2.8
>
>
> When some cache groups have the same affinity function, number of partitions,
> backups and the same node filter they can use the same affinity distribution
> without needs for explicit recalculating. These parameters are called as
> "Affinity similarity key".
> In case of affinity recalculation caching affinity using this key may
> speed-up the process.
> However, after https://issues.apache.org/jira/browse/IGNITE-9561 merge this
> mechanishm become broken, because parallell execution of affinity
> recalculation for the similar affinity groups leads to caching affinity
> misses.
> To fix it we should couple together similar affinity groups and run affinity
> recalculation for them in one thread, caching previous results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
