[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313843#comment-15313843
]
ASF GitHub Bot commented on IGNITE-2655:
Github user vldpyatkov closed the pull request at:
https://github.com/apache/ignite/pull/731
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312308#comment-15312308
]
Denis Magda commented on IGNITE-2655:
-
Merged the changes. Vlad thanks for the contribution.
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310311#comment-15310311
]
Denis Magda commented on IGNITE-2655:
-
[~v.pyatkov], I've reviewed your latest changes. Please see my comments in the
pull-request. In short the code can fail or work improperly.
[~yzhdanov], I would use this predicate for backups only, leaving method's
signature as is, considering that the primary is chosen by affinity function
before. As you said if someone needs to control the location of primary he can
write his own affinity function and most likely he will because "partition
number" passed to the filter says nothing in regards to where this partition is
physically located.
>> As I side note I would say that we are trying to force our users to do some
>> programming. Does anyone have any idea on how to do this without code? How
>> about supporting simple string expressions based on node attributes and/or
>> ip addresses?
Presently I would fully rely on the programmable way because it's flexible and
covers all the cases. If there is a demand for the thing you're talking about
we can add an overloaded method that will except a kind of predicate.
[~dsetrakyan], presently the API is left the same for all the supported
affinity function:
{{public void setAffinityBackupFilter(@Nullable IgniteBiPredicate> affinityBackupFilter)}}
where
the first parameter (ClusterNode) is a node being tested and the second list
contains nodes that are already assigned for a partition (primary is always the
first).
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308907#comment-15308907
]
Dmitriy Setrakyan commented on IGNITE-2655:
---
I am not sure I follow the proposed changes, but I still would like to see the
final API here, if it has changed. I also strongly believe that the behavior of
this backup-filter should not be different as we move from one affinity
function to another.
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308862#comment-15308862
]
Yakov Zhdanov commented on IGNITE-2655:
---
[~dmagda] [~v.pyatkov]
Couple of comments:
# Can you please think over this signature for predicate? apply(int n, Node n,
List assigned)
# If predicate is called with empty list then primary node for this partition
is examined.
# Or we can assign primary always on our own and call predicate only for
backups (which I would prefer). If someone needs full control, then he always
have an opportunity to implement affinity function on his own.
As I side node I would say that we are trying to force our users to do some
programming. Does anyone have any idea on how to do this without code? How
about supporting simple string expressions based on node attributes and/or ip
addresses?
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307986#comment-15307986
]
Vladislav Pyatkov commented on IGNITE-2655:
---
I fixed AffinityBackupFilter incorrect behavior (in {{FairAffinictyFunction}}).
Denis, please, review last change.
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307980#comment-15307980
]
Vladislav Pyatkov commented on IGNITE-2655:
---
I fixed AffinityBackupFilter incorrect behavior.
Denis, please, review last change.
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307586#comment-15307586
]
Denis Magda commented on IGNITE-2655:
-
Dmitriy, it means that in case of {{FairAffinityFunction}} the method can check
both the primary and the backup against already assigned nodes. The already
assigned nodes may or may node contains the primary.
Vlad, I think that we can preserve the same semantic and behavior at the level
of {{FairAffinictyFunction}} if do the following at the implementation level:
- if {{tier=0}} is checked (primary) then we prepare new assignments list that
will have the primary, that is being checked, first in the list and the rest of
the nodes will be the nodes that are already assigned (backups);
- after that we're iterating over a sublist calling
{{affinityBackupFilter.apply(...)}} for every backup from the list with
assignments. During the iteration if we get {{false}} for at least backup then
it means that the primary is non assignable.
Such implementation will help us to preserve the same semantic as
{{RendezvousAffinityFunction}} has
n - potential backup to check
assigned - list of current partition holders (first node in the list is primary)
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307495#comment-15307495
]
Dmitriy Setrakyan commented on IGNITE-2655:
---
Then I do not understand. Are you saying that we will have a method called
{{setAffinityBackupFilter(...)}}, but in case of {{FairAffinityFunction}} it
will be checking the primary node and not the backup?
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307442#comment-15307442
]
Vladislav Pyatkov commented on IGNITE-2655:
---
[~dsetrakyan] No, in the case when the function ({{FairAffinityFunction}})
moves the primary partition:
n - candidate for primary
assignd - only backups
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307433#comment-15307433
]
Dmitriy Setrakyan commented on IGNITE-2655:
---
What is the 1st node in the list for the {{FairAffinityFunction}}? Is it not
the primary node as well?
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307412#comment-15307412
]
Vladislav Pyatkov commented on IGNITE-2655:
---
Small description of my changes:
Added AffinityBackupFilter for FairAffinityFunction and
RendezvousAffinityFunction (The method is setAffinityBackupFilter).
Interface for affinity backup filter is
public boolean apply (Node n, List assigned)
Where
n - potential backup to check
assignd - list of current partition holders (for RendezvousAffinityFunction
first node in the list is primary)
result - if can assign true, false another.
Use as follows:
{code}
RendezvousAffinityFunction aff = new RendezvousAffinityFunction(false);
aff.setAffinityBackupFilter(new IgniteBiPredicate>() {
@Override public boolean apply(ClusterNode node, List
nodes) {
return false;
}
});
{code}
Old method setBackupFilter marked as Deprecated.
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301888#comment-15301888
]
Denis Magda commented on IGNITE-2655:
-
Vlad, looks good to me. Please do final minor modifications left in the
pull-request and update the latest.
[~yzhdanov] please do final technical review.
[~dsetrakyan] please review change in the public API at the level of rendezvous
and fair affinity function.
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15300310#comment-15300310
]
Vladislav Pyatkov commented on IGNITE-2655:
---
Thank you for your comment. I added all needed changes.
Could you please review it?
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299873#comment-15299873
]
Denis Magda commented on IGNITE-2655:
-
Vlad, thanks for the contribution.
Please see my comment in the pull-request which are related to:
- documentation modifications;
- improper implementation for {{FairAffinityFunction}}.
Also please do the following:
- add tests that will set two backups for a cache and primary, 1st backup and
2nd backup has to be located in different groups;
- don't add additional test suite, add your tests into existed
{{AffinityFunctionBackupFilterAbstractSelfTest}}.
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290632#comment-15290632
]
Vladislav Pyatkov commented on IGNITE-2655:
---
Added new backup filter in FairAffinityFunction and RendezvouseAffinityFunction.
Could anyone review?
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-2655) AffinityFunction: primary and backup copies in different locations
[
https://issues.apache.org/jira/browse/IGNITE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290612#comment-15290612
]
ASF GitHub Bot commented on IGNITE-2655:
GitHub user vldpyatkov opened a pull request:
https://github.com/apache/ignite/pull/731
IGNITE-2655
AffinityFunction: primary and backup copies in different locations
Add affinity backup filter to FairAffinityFunction and
RendezvousAffinityFunction
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/vldpyatkov/ignite ignite-2655
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/731.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #731
commit a3276ab8210a41495dca1f7f1c50db4036bd1afa
Author: vdpyatkov
Date: 2016-05-19T07:19:05Z
IGNITE-2655
AffinityFunction: primary and backup copies in different locations
Add affinity backup filter to FairAffinityFunction and
RendezvousAffinityFunction
> AffinityFunction: primary and backup copies in different locations
> --
>
> Key: IGNITE-2655
> URL: https://issues.apache.org/jira/browse/IGNITE-2655
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Vladislav Pyatkov
>Priority: Critical
> Labels: important
> Fix For: 1.7
>
>
> There is a use case when primary and backup copies have to be located in
> different racks, building, cities, etc.
> A simple scenario is the following. When nodes are started they will have
> either "rack1" or "rack2" value in their attributes list and we will enforce
> that the backups won't be selected among the nodes with the same attribute.
> It should be possible to filter out backups using IP addresses as well.
> Presently rendezvous and fair affinity function has {{backupFilter}} that
> will work perfectly for the scenario above but only for cases when number of
> backups for a cache is equal to 1.
> In case when the number of backups is bigger than one {{backupFilter}} will
> only guarantee that the primary is located in different location but will NOT
> guarantee that all the backups are spread out across different locations as
> well.
> So we need to provide an API that will allow to spread the primary and ALL
> backups copies across different locations.
> The proposal is to introduce {{AffinityBackupFilter}} with the following
> method
> {{AffinityBackupFilter.isAssignable(Node n, List assigned)}}
> Where n - potential backup to check, assigned - list of current partition
> holders, 1st is primary
> {{AffinityBackupFilter}} will be set using
> {{affinity.setAffinityBackupFilter}}.
> {{Affinity.setBackupFilter}} has to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
