Re: How to avoid data skew in collocate data with data

Hi, One AffinityKey binding to one node always. This is AffinityKey meaning (all entries with one AffinityKey stored into one node). On Thu, Oct 27, 2016 at 9:45 AM, ght230 wrote: > If there are too many data related to one affinity key, even more than the > capacity of one

Re: How to avoid data skew in collocate data with data

If there are too many data related to one affinity key, even more than the capacity of one node. Will Ignite automatically split that data and stored them in several nodes? -- View this message in context:

Re: How to avoid data skew in collocate data with data

No, for that you will have to implement your own affinity function. But there are a lot of cases that you should keep in mind. What if the node fails, how the data will be redistributed? What if new node joins, will it take some of the data (in other words, will you be able to scale)? Etc. -Val

Re: How to avoid data skew in collocate data with data

I would like to pre-distribution of data according to a certain algorithm to the designated node. Is there an API can be used to put data to a specified node? -- View this message in context:

Re: How to avoid data skew in collocate data with data

Hi, If you have such data distribution, then most likely your collocation strategy is incorrect or at least not optimal. If your number of partitions is much bigger than number of nodes and number of unique affinity keys is much bigger than number of partitions, such skew will be minimized. "Much

Re: How to avoid data skew in collocate data with data

Hi, No, it will not help. Because affinity function have not information about data volume into partition. The purpose of the function (RendezvousAffinityFunction or FairAffinityFunction) is distribute evenly partition quantity between nodes (but not data). You can to deeper understand affinity

Re: How to avoid data skew in collocate data with data

For some reason, I can not choose more selective affinty key than current. If I increase number of partition, will it help me to avoid the case that keys 553, 551, 554, 550 hitting into one node? -- View this message in context:

Re: How to avoid data skew in collocate data with data

Hi, You can increase number of partition through affinity function configuration, but default value (1024 partitions) would be enough. ... I do not shure than FairAffinityFunction will help to you, in case if keys 553, 551, 554, 550 will hit into one node. How about choose more

Re: How to avoid data skew in collocate data with data

How to increase number of partitions? And if data skew happened, how can I rebalance it? Using FairAffinityFunction()? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-avoid-data-skew-in-collocate-data-with-data-tp8454p8491.html Sent from the Apache

Re: How to avoid data skew in collocate data with data

Hi, I agree with Alexey, you need to increase number of partitions. In additional, It look like your Affinity key has bad selectivity. Why so mach data bind to 553, 551, but small data bind to 542, 530? I recomend select other key as affinity or accept that disbalance. On Tue, Oct 25, 2016 at

Re: How to avoid data skew in collocate data with data

Hi! How many partitions configured in your cache? As far as I see - 11 partitions? Could you try to configure more (64, 128, 256)? And see how data will be distributed? By default Ignite caches configured with 1024 partitions. On Tue, Oct 25, 2016 at 8:20 PM, ght230 wrote: >

Re: How to avoid data skew in collocate data with data

When data skew happened, what can I do to rebalance all the data to the 3 nodes. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-avoid-data-skew-in-collocate-data-with-data-tp8454p8473.html Sent from the Apache Ignite Users mailing list archive at