Re: Customized affinity function
Statistically it's very unlikely that all big subscriptions (or many of them) will end up together. So I would check if there is an issue first and go from there. Technically you can implement your own AffinityFunction to customize distribution. But that's not a trivial task as you will have to consider all the cases of nodes going out of topology, scaling out, etc. -Val -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Customized affinity function
Hi Val, Out 1 subscriptions, only 10-20 subscriptions are very big. I just want to make sure that these big subscriptions do not land up in partitions which are located on same node. On Thu, Apr 12, 2018 at 7:44 AM, vkulichenko wrote: > Prasad, > > Are you collocating data by subscription? Do you have any actual issue with > data distribution? With 1 subscriptions, I don't see why would you have > one with default settings. > > -Val > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Customized affinity function
Prasad, Are you collocating data by subscription? Do you have any actual issue with data distribution? With 1 subscriptions, I don't see why would you have one with default settings. -Val -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Customized affinity function
Hi Prasad, Affinity function does not directly map keys to nodes, it maps keys to partitions. So it takes care about entities balance between nodes. To read more please take a look here: https://apacheignite.readme.io/docs/affinity-collocation#affinity-function In case if for every subscriber you are going to create it's own cache then you can try to setup the node filters for every subscriber: https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/CacheConfiguration.html#setNodeFilter-org.apache.ignite.lang.IgnitePredicate- If you have the same cache for all subscribers then try to setup your own AffinityKey (as descibed above): IgniteCache cache = ignite.getOrCreateCache(cfg); for (int i = 0 ; i < 1_000_000; i++) { cache.put(new EntityKey(2*i, 1L), "Value1" + i); //sub id = 1 cache.put(new EntityKey(2*i + 1, 2L), "Value2" + i); //sub id =2 } where: public static class EntityKey { public EntityKey(long id, long subscriberId) { this.id = id; this.subscriberId = subscriberId; } private long id; // Subscriber ID which will be used for affinity. @AffinityKeyMapped private long subscriberId; public long getSubscriberId() { return subscriberId; } public long getId() { return id; } } Thank you, Andrei -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Customized affinity function
Hi! The |@AffinityKeyMapped annotation can be used on a field or a method, so you can have a method returning anything you want (based on size), would that not work ?| |Mikael | Den 2018-04-11 kl. 07:59, skrev Prasad Bhalerao: Hi, I have following case. I have around 1 subscriptions. Each subscription has data varying from 1 to 10 million. Currently the affinity key is set on subscriptionId. e.g: Subscription1: 2 million rows Subscription2: 10 million rows Subscription3: 50 million rows Subscription5: 30 million rows Subscription6: 2 rows Now I want make sure that all big subscription should be evenly distributed across all nodes. I meant Subscription2,Subscription3,Subscription4,Subscription5 should not land on same node based on their affinity id. Is there any way with which I can distribute all big subscriptions across all node evenly or in round robin fashion? Thanks, Prasad