Re: Designing Affinity Key for more locality

2021-04-29 Thread William.L
I am using a user centric modeling approach where most of the computations would be on a per-user basis (joins) before aggregation. The idea is to put the data (across different tables/caches) for the same user in the same partition/server. That's the reason why I chose user-id as the affinity

Re: Designing Affinity Key for more locality

2021-04-28 Thread Ilya Kasnacheev
Hello! SQL query planner will not understand the locality if you use surrogate value as affinity key. Maybe you need to define your own affinity function (extends RendezvousAffinityFunction) which will map keys to partitions. I'm not sure that it will help query planner though. Regards, --

Re: Designing Affinity Key for more locality

2021-04-27 Thread Pavel Tupitsyn
Hi William, Can you describe the use case and domain model in more detail? 1. AffinityKey is used to colocate some data with other data. What do you achieve with user-id being the affinity key? 2. If you'd like to put all users for a given tenant/group to the same node for efficiency,

Re: Designing Affinity Key for more locality

2021-04-26 Thread William.L
Came across this statement in the Data Partitioning documents: "The affinity function determines the mapping between keys and partitions. Each partition is identified by a number from a limited set (0 to 1023 by default)." Looks like there is no point for adding another layer of mapping unless I

Designing Affinity Key for more locality

2021-04-26 Thread William.L
Hi, I am currently using user-id as the affinity key and because it is a uuid/string, it will distribute across all partitions in the cluster. However, for my scenario, I am writing and reading users that are for the same tenant/group so it seems to me that it is better to design with more