Re: [akka-user] Am I Reinventing the Wheel?

[kraythe]

That is definitely on my list of concerns. However there are a couple of 
issues to deal with. The actors are queried by front end users for 
information and routing the traffic of every user to a single node would be 
hazardous to stability. Furthermore, the actor could hold state that is 
anywhere between a couple of KB to several megabytes big (no, that is not 
avoidable unfortunately) so updating it on one node and sending it to other 
nodes is not really a viable solution. I am looking at the updates taking 
up to 3 seconds to process a large batch. I could do the updates with 
direct messaging to the actors using a wildcard pattern in an actor 
selector but I would be concerned about the updates being copied. One 
update could be consumed by any number of actors on the other node and I 
don't want it copied once per actor. 

Yes there is a danger of synch issues with the replicas being out of synch 
with each other but I am not really sure if there is a viable way to handle 
that. Even if I updated all on once node and took the serialization hit to 
send the object to the other nodes for the "read only" actors, there is a 
danger that the object doesn't arrive and one node is out of synch. One 
thing that is a limiting factor is that these actors are updated from up to 
millions of other actors and I wouldn't want them "pulling" state from 
those other actors on update cycles because that would for sure send the 
state of the smaller actors many time amongst other nodes. I can't be more 
specific into the use case because of NDAs but just think the smaller 
actors have some kind of state but the larger ones have to rank the smaller 
actors into a ranking system whereby the ranking is affected by the smaller 
actors. 

Consider the smaller actors in the millions have a state, lets say that 
state includes 4 attributes which cooperate in a ranking order. The actors 
I am replicating maintain the ranking order of the smaller actors and to do 
that they essentially have to sort the list. The users request some portion 
of this ranking order but the smaller actors can change state at any moment 
causing the ranking order to need to be updated. Since the object is 
hammered by users, driving the load to one machine would be unscalable. 
Essentially all of our web traffic would end up on one node. So replicating 
them seemed the right solution, even with the possibility of synch errors. 

I am open to ideas though. Direct messaging vs pub sub ... if I send a 
message from node 1 to node 9 and that message is destined for 10 actors on 
node 9, will it send only once over wire? 

-- Robert

On Thursday, June 2, 2016 at 8:07:25 AM UTC-5, Justin du coeur wrote:
>
> Hmm.  In that case, that's not Cluster Sharding -- but it's also not 
> common, because it can be challenging to get right (without inconsistencies 
> between replicas) and I suspect *usually* will result in worse throughput. 
>  (Since you're introducing a lot of PubSub traffic, and replicating the 
> processing of events.)  I'd recommend sanity-checking whether it's actually 
> cheaper to do it this way when everything is taken into account.
>
> Assuming so, then no, this is just plain uncommon.  But you might want to 
> take a look at the relatively recent CRDT support, which is the closest 
> common cognate I can think of to this sort of approach...
>
> On Wed, Jun 1, 2016 at 9:25 PM, kraythe  
> wrote:
>
>> They are updated from a DistributedPubSub topic. Each of them gets the 
>> update messages and then recalculates the data they need to maintain 
>> locally. So essentially they all function as independent actors, isolated 
>> from each other, knowing nothing about each other. This object is one of 
>> the most hammered in our system so we need it to scale horizontally and the 
>> object can possibly hold a HUGE state (which is unavoidable) in the nature 
>> of 40 megs. So its cheaper to update them all independently. So when an 
>> update comes in it is published to a Topic that the actors listen to, if 
>> they care about that particular update, they deal with it. If its an update 
>> not within their scope, they ignore it. The supervisor just manages the 
>> objects by key so that you find the right one to interrogate for read ops.
>>
>> -- Robert
>>
>> On Wednesday, June 1, 2016 at 5:52:19 PM UTC-5, Ryan Tanner wrote:
>>>
>>> How are you coordinating state between logically-equal actors on 
>>> different physical nodes?
>>>
>>> On Wednesday, June 1, 2016 at 3:24:57 PM UTC-6, kraythe wrote:

 So the reason I didn't think this was cluster sharding is that I 
 actually want these supervisor actors (and their supervised children) to 
 be 
 REPLICATED on every node (they handle user requests). Basically if there 
 is 
 an actor with key 10, I want one actor with key 10 per node. I didn't want 
 the messages getting rerouted to another node. So if I have one of these 
 actors running on 

Re: [akka-user] Am I Reinventing the Wheel?

[Justin du coeur]

Hmm.  In that case, that's not Cluster Sharding -- but it's also not
common, because it can be challenging to get right (without inconsistencies
between replicas) and I suspect *usually* will result in worse throughput.
 (Since you're introducing a lot of PubSub traffic, and replicating the
processing of events.)  I'd recommend sanity-checking whether it's actually
cheaper to do it this way when everything is taken into account.

Assuming so, then no, this is just plain uncommon.  But you might want to
take a look at the relatively recent CRDT support, which is the closest
common cognate I can think of to this sort of approach...

On Wed, Jun 1, 2016 at 9:25 PM, kraythe  wrote:

> They are updated from a DistributedPubSub topic. Each of them gets the
> update messages and then recalculates the data they need to maintain
> locally. So essentially they all function as independent actors, isolated
> from each other, knowing nothing about each other. This object is one of
> the most hammered in our system so we need it to scale horizontally and the
> object can possibly hold a HUGE state (which is unavoidable) in the nature
> of 40 megs. So its cheaper to update them all independently. So when an
> update comes in it is published to a Topic that the actors listen to, if
> they care about that particular update, they deal with it. If its an update
> not within their scope, they ignore it. The supervisor just manages the
> objects by key so that you find the right one to interrogate for read ops.
>
> -- Robert
>
> On Wednesday, June 1, 2016 at 5:52:19 PM UTC-5, Ryan Tanner wrote:
>>
>> How are you coordinating state between logically-equal actors on
>> different physical nodes?
>>
>> On Wednesday, June 1, 2016 at 3:24:57 PM UTC-6, kraythe wrote:
>>>
>>> So the reason I didn't think this was cluster sharding is that I
>>> actually want these supervisor actors (and their supervised children) to be
>>> REPLICATED on every node (they handle user requests). Basically if there is
>>> an actor with key 10, I want one actor with key 10 per node. I didn't want
>>> the messages getting rerouted to another node. So if I have one of these
>>> actors running on every node how can I do that with sharding? Id imagine I
>>> would have to be fancy with the shard id but i have no idea how.
>>>
>>> -- Robert
>>>
>>> On Wednesday, June 1, 2016 at 3:42:16 PM UTC-5, Konrad Malawski wrote:


 I was working on a supervisor that lazy creates actors based on some
 key and then will forward messages to that actor.

 That's Cluster Sharding :-)

 http://doc.akka.io/docs/akka/snapshot/scala/cluster-sharding.html

 Technically you can use it on one node too, yeah.


 Happy hAkking!
 --
 Konrad `ktoso` Malawski
 Akka  @ Lightbend 


 --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Am I Reinventing the Wheel?

[kraythe]

They are updated from a DistributedPubSub topic. Each of them gets the 
update messages and then recalculates the data they need to maintain 
locally. So essentially they all function as independent actors, isolated 
from each other, knowing nothing about each other. This object is one of 
the most hammered in our system so we need it to scale horizontally and the 
object can possibly hold a HUGE state (which is unavoidable) in the nature 
of 40 megs. So its cheaper to update them all independently. So when an 
update comes in it is published to a Topic that the actors listen to, if 
they care about that particular update, they deal with it. If its an update 
not within their scope, they ignore it. The supervisor just manages the 
objects by key so that you find the right one to interrogate for read ops.

-- Robert

On Wednesday, June 1, 2016 at 5:52:19 PM UTC-5, Ryan Tanner wrote:
>
> How are you coordinating state between logically-equal actors on different 
> physical nodes?
>
> On Wednesday, June 1, 2016 at 3:24:57 PM UTC-6, kraythe wrote:
>>
>> So the reason I didn't think this was cluster sharding is that I actually 
>> want these supervisor actors (and their supervised children) to be 
>> REPLICATED on every node (they handle user requests). Basically if there is 
>> an actor with key 10, I want one actor with key 10 per node. I didn't want 
>> the messages getting rerouted to another node. So if I have one of these 
>> actors running on every node how can I do that with sharding? Id imagine I 
>> would have to be fancy with the shard id but i have no idea how. 
>>
>> -- Robert
>>
>> On Wednesday, June 1, 2016 at 3:42:16 PM UTC-5, Konrad Malawski wrote:
>>>
>>>
>>> I was working on a supervisor that lazy creates actors based on some key 
>>> and then will forward messages to that actor. 
>>>
>>> That's Cluster Sharding :-)
>>>
>>> http://doc.akka.io/docs/akka/snapshot/scala/cluster-sharding.html 
>>>
>>> Technically you can use it on one node too, yeah.
>>>
>>>
>>> Happy hAkking!
>>> -- 
>>> Konrad `ktoso` Malawski
>>> Akka  @ Lightbend 
>>>
>>>
>>>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Am I Reinventing the Wheel?

['Ryan Tanner' via Akka User List]

How are you coordinating state between logically-equal actors on different 
physical nodes?

On Wednesday, June 1, 2016 at 3:24:57 PM UTC-6, kraythe wrote:
>
> So the reason I didn't think this was cluster sharding is that I actually 
> want these supervisor actors (and their supervised children) to be 
> REPLICATED on every node (they handle user requests). Basically if there is 
> an actor with key 10, I want one actor with key 10 per node. I didn't want 
> the messages getting rerouted to another node. So if I have one of these 
> actors running on every node how can I do that with sharding? Id imagine I 
> would have to be fancy with the shard id but i have no idea how. 
>
> -- Robert
>
> On Wednesday, June 1, 2016 at 3:42:16 PM UTC-5, Konrad Malawski wrote:
>>
>>
>> I was working on a supervisor that lazy creates actors based on some key 
>> and then will forward messages to that actor. 
>>
>> That's Cluster Sharding :-)
>>
>> http://doc.akka.io/docs/akka/snapshot/scala/cluster-sharding.html 
>>
>> Technically you can use it on one node too, yeah.
>>
>>
>> Happy hAkking!
>> -- 
>> Konrad `ktoso` Malawski
>> Akka  @ Lightbend 
>>
>>
>>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Am I Reinventing the Wheel?

[kraythe]

So the reason I didn't think this was cluster sharding is that I actually 
want these supervisor actors (and their supervised children) to be 
REPLICATED on every node (they handle user requests). Basically if there is 
an actor with key 10, I want one actor with key 10 per node. I didn't want 
the messages getting rerouted to another node. So if I have one of these 
actors running on every node how can I do that with sharding? Id imagine I 
would have to be fancy with the shard id but i have no idea how. 

-- Robert

On Wednesday, June 1, 2016 at 3:42:16 PM UTC-5, Konrad Malawski wrote:
>
>
> I was working on a supervisor that lazy creates actors based on some key 
> and then will forward messages to that actor. 
>
> That's Cluster Sharding :-)
>
> http://doc.akka.io/docs/akka/snapshot/scala/cluster-sharding.html 
>
> Technically you can use it on one node too, yeah.
>
>
> Happy hAkking!
> -- 
> Konrad `ktoso` Malawski
> Akka  @ Lightbend 
>
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Am I Reinventing the Wheel?

[kraythe]

Hmm, on one node? How  Ok I will look at docs ... kind of thought I 
might be reinventing the wheel? Can I send to one router and have the 
message redirected to the actor with the right key ? 

On Wednesday, June 1, 2016 at 3:42:16 PM UTC-5, Konrad Malawski wrote:
>
>
> I was working on a supervisor that lazy creates actors based on some key 
> and then will forward messages to that actor. 
>
> That's Cluster Sharding :-)
>
> http://doc.akka.io/docs/akka/snapshot/scala/cluster-sharding.html 
>
> Technically you can use it on one node too, yeah.
>
>
> Happy hAkking!
> -- 
> Konrad `ktoso` Malawski
> Akka  @ Lightbend 
>
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Am I Reinventing the Wheel?

[Konrad Malawski]

I was working on a supervisor that lazy creates actors based on some key
and then will forward messages to that actor.

That's Cluster Sharding :-)

http://doc.akka.io/docs/akka/snapshot/scala/cluster-sharding.html

Technically you can use it on one node too, yeah.


Happy hAkking!
-- 
Konrad `ktoso` Malawski
Akka  @ Lightbend 

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Am I Reinventing the Wheel?

[kraythe]

I was working on a supervisor that lazy creates actors based on some key 
and then will forward messages to that actor. However, it just seems like I 
am doing something frightfully common. So I was wondering if anyone knows 
something in Akka that already does this and I should use that. I have 
looked but didn't find anything. If not then it seems this might be a good 
thing to put in the core API. A slice of what I am working on is below. Any 
advice is appreciated.

public class LazyActorSupervisor extends UntypedActor {
private final static String LOG_TAG = 
LazyActorSupervisor.class.getSimpleName();
private LoggingAdapter log = Logging.getLogger(getContext().system(), this);
private Map directory = new HashMap<>();
private Class keyType;
private Class childType;

/**
 * @param keyType   The type of key being used in the directory.
 * @param childType The type of child actor.
 */
public LazyActorSupervisor(final Class keyType, final Class 
childType) {
this.keyType = keyType;
this.childType = childType;
}

/**
 * Finds the child actor or creates that actor if the actor does not exist. 
It is possible that the subclass
 * may decline to create the actor, passing back a null actor ref and thus 
causing that to be returned to the
 * caller.
 *
 * @param key The key to look for.
 */
protected ActorRef findOrCreate(final K key) {
ActorRef child = directory.get(key);
if (child == null || child.isTerminated()) {
if (child != null && child.isTerminated()) removeChild(key);
child = createChildActor(key);
if (child != null) {
directory.put(key, child);
context().watch(child);
if (log.isDebugEnabled()) log.debug(String.format("[%s] Created 
Child Actor for key %s", LOG_TAG, key));
return child;
}
}
// FIXME decide if we want to send them to an error actor or something. 
Do we leave that up to subclass?
return null;
}

/**
 * Creates the child actor from the given key. This can be overridden by 
subclasses which have more complex
 * use cases or used as is in which case it will create a props for the 
actor with the key as an argument.
 *
 * @param key The key to use.
 */
protected ActorRef createChildActor(final K key) {
return context().actorOf(Props.create(childType, key));
}

/**
 * Removes a child ref from the directory by key.
 *
 * @param key The key of the ref to remove.
 */
private void removeChild(final K key) {
final ActorRef ref = directory.get(key);
if (ref == null) return; // nothing to do
context().unwatch(ref);
directory.remove(key);
}

/**
 * removes a child ref from the directory and un-watches it.
 *
 * @param ref The ref to remove.
 */
private void removeChild(final ActorRef ref) {
directory.entrySet().stream().filter(e -> e.getValue().equals(ref))
.findAny().map(e -> {
context().unwatch(ref);
directory.remove(e.getKey());
return null;
});
}

@Override
public void onReceive(final Object message) throws Exception {
if (message instanceof FindLazySupervisedActor) {
@SuppressWarnings("unchecked") final FindLazySupervisedActor 
findMsg = (FindLazySupervisedActor) message;
sender().tell(findOrCreate(findMsg.getKey()), getSelf());
} else if (message instanceof Terminated) {
final Terminated terminated = (Terminated) message;
removeChild(terminated.actor());
} else {
unhandled(message);
}
}

/**
 * The type of key being used in the directory.
 */
public Class getKeyType() {
return keyType;
}

/**
 * The type of child actor.
 */
public Class getChildType() {
return childType;
}
}

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.