On Tue, Apr 21, 2015 at 3:22 PM, <[email protected]> wrote:
>
> thanks. I would like to ensure that when new Nodes are added they do not
> take their share of objects from all the other cache machines.
> I guess I would need to implement my own caching algo for this to work.
>
How would that work? Assuming that you have 3 nodes, and you already
decided which objects map to which nodes, then you have a mapping M1 from
all objects to the nodes {N1, N2 N3}. Now if you add a new node, then that
mapping needs to be changed since it was *total*:
- all nodes have objects that are mapped to them (at least ideally)
- all objects map to exactly one node
Now you will need to maintain a new mapping M2 to {N1, N2, N3, N4} *in
addtion*, *and* the difference between M1(object) and M2(object). The
reason is:
- You need mapping M1 to be able to access object that were mapped to N1-3
before the node N4 has been added
- You need mapping M2 to know where to put new objects
- You need to know if certain object have been added already before N4
came online, or later (since in the former case you need M1 to find its
location, and in the latter case you need M2 to find its location)
And this gets worse with any new additon and removal. There is a reason why
consistent hashing has been invented. Btw, this is a problem that cluster
sharding solves, but it also needs to migrate things, and the only way you
can make that happen is using Akka persistence with it.
Akka Sharding uses the problem outlined above in the following way:
- It maintains a location map, but not for individual entries, but buckets
(called shards)
- this map is maintained resiliently in a distributed way (it is a
singleton that can migrate when the host node fails, and clients can find
it again at the new place)
- the hash function maps objects to buckets, i.e. there is a simple
mapping from object to bucket by using a hash function, and there is an
index that is actively maintained that maps buckets to nodes. The number of
buckets (shards) are always fixed, but the number of nodes can change.
- the buckets are distributed among available nodes
- when nodes are added, some buckets are moved to those nodes, and the
singleton updates the index, so all nodes know where the bucket is located
- when nodes are removed or fail, all the buckets hosted by those nodes
will be distributed and restored from persistent storage at the remaining
nodes.
-Endre
>
>
> Am Dienstag, 21. April 2015 12:09:37 UTC+2 schrieb drewhk:
>>
>> Hi John,
>>
>> On Tue, Apr 21, 2015 at 11:14 AM, <[email protected]> wrote:
>>
>>> akka.actor.deployment {
>>> /myRouter {
>>> router = consistent-hashing-group
>>> nr-of-instances = 10000
>>>
>>> routees.paths = ["/user/worker"]
>>> cluster {
>>> max-nr-of-instances-per-node = 1
>>> enabled = on
>>> allow-local-routees = off
>>> use-role = recipient
>>> }
>>> }
>>> }
>>>
>>> If I start with for example 3 worker nodes "A" "B" "C" the
>>> consistent-hashing-group algorithm works as expected:
>>> If a worker on node "B" recievies a message with a consistentHashKey =
>>> "123" this message is always send to "B"
>>>
>>> But if I add a Node D from now on the message consistentHashKey = "123" is
>>> send to D.
>>>
>>> Is this the expected behaviour?
>>>
>>> Yes it is, this is how consistent hashing works:
>> https://en.wikipedia.org/wiki/Consistent_hashing
>>
>> Quoting:
>> "Consistent hashing maps objects to the same cache machine, as far as
>> possible. It means when a cache machine is added, it takes its share of
>> objects from all the other cache machines and when it is removed, its
>> objects are shared between the remaining machines."
>>
>> The goal of consistent caching is to:
>> - not to store a global index of all key->node mappings (that would be
>> needed in your example not to remap "123" to another node) because it is
>> very expensive and fragile
>> - remap only part of the keyspace between nodes, most objects are mapped
>> to the node they were mapped before when a node is added or removed.
>>
>> -Endre
>>
>>
>>>
>>> --
>>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>>> >>>>>>>>>> Check the FAQ:
>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >>>>>>>>>> Search the archives:
>>> https://groups.google.com/group/akka-user
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To post to this group, send email to [email protected].
>>> Visit this group at http://groups.google.com/group/akka-user.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/
> >>>>>>>>>> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at http://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
