[akka-user] Persistent query view implementation

[Filippo De Luca]

Hi Guys,
I have been working for a replacement to the deprecated PersistentView. At 
the moment for the newcomers is very complex to build a view that wants to 
replay messages from different persistence ids. I have written a library 
that allows creating a view that is not bound to a specific persistent id 
but is built on top of a persistent query supported by the journal plugin.

The code and documentation are available here:
https://github.com/ovotech/akka-persistence-query-view

The binary library is here:
https://bintray.com/ovotech/maven/akka-persistence-query-view

Please give it a try, any feedback is welcome.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: Patterns for ensuring perpetual existence of sharded actors

[Matthew Howard]

I agree - if you control the client then a client-side heartbeat is a good 
approach. I haven't had quite that requirement for the actors to always be 
instantiated but I have had a couple semi-related requirements. 

In one case we basically did the same heartbeat on the server side (we had 
a pool of stateless worker actors to perform various tasks, and a 
coordinator that monitored the pool to make sure we had enough alive and 
available for work). It was a little different from your situation in that 
our workers were stateless and we added additional scaling logic to the 
mix. 

In the other case we basically used a persisted actor that managed the 
state of the child actors we wanted up and running. So in your Pingdom 
example there would be something like a PingClientManager actor that is 
always instantiated on start of the actor system, and this manager would 
know which clients should be active and would basically manage their 
lifecycle. Depending on the complexity and volume of managed actors I'd 
probably farm much of that work off to child actors of the 
PingClientManager but basically it would be the root that is responsible 
for ensuring the liveness of the sharded actors. Could manage the state 
within a persisted actor or some other datasource if there were a lot of 
them. When you get into all the guarantees and fault handling to really 
ensure every actor is up and running it definitely isn't trivial, but not 
too bad. Basically in your scenario the actors don't have to worry about 
the lifecycle of the "external" clients initiating the heartbeat, but in 
the Pingdom scenario the clients now come under your responsibility and 
have to be managed. 



On Wednesday, April 19, 2017 at 8:57:08 AM UTC-4, Andrew Easter wrote:
>
> Hi group,
>
> I was wondering if anyone has any experiences/patterns to share with 
> regard to ensuring the perpetual existence of sharded actors?
>
> To explain, what I mean...
>
> I'm prototyping some software that pushes data to connected clients over 
> web sockets (currently using a third party service, Pusher). I have the 
> concept of an actor that pushes data to one or many specific clients via 
> Pusher, and, in turn, there can be many such actors, each with their own 
> set of associated clients. These actors are sharded - each actor has its 
> own specific configuration that determines the data it sends to its 
> connected clients via Pusher, and thus I only want one unique instance of 
> each specific actor within the akka cluster (wouldn't make sense to have 
> multiple instances all duplicating data being sent to their associated 
> clients).
>
> As the clients of an actor are essentially running forever, it makes sense 
> for the actor to have perpetual existence - i.e. it always needs to be 
> alive and sending data to its associated clients. At the moment, I've 
> solved this by having each client send a heartbeat at a scheduled interval. 
> In the event the client's associated actor has terminated (for whatever 
> reason), the heartbeat essentially serves the purpose of waking it up again 
> (recovering it) somewhere in the cluster. This will then ensure the actor 
> starts delivering data to all its clients again.
>
> The solution I have in place does work, but I'm wondering whether there's 
> a more 'elegant' solution that means the server side system ensures that 
> all necessary actors remain running, even in the event of crashing, 
> restarting nodes etc. Of course, maybe the solution in place is good enough 
> as it's actually pretty simple. I can imagine a server side solution might 
> be more involved and require a lot of complex engineering to get right. 
> Anyhow, thought I'd ask in case there was some established patterns that 
> people were already using for this type of thing.
>
> I imagine another example of where you might want such behaviour is if you 
> were building a system like Pingdom - i.e. you create an actor per 
> responsiveness test and know that you always need that actor to be awake, 
> triggering the configured endpoint at configured intervals. In that case, 
> you couldn't rely on some client sending a heartbeart to wake the actor up.
>
> Any thoughts on this would be more than well received :-)
>
> Cheers,
> Andrew
>
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Patterns for ensuring perpetual existence of sharded actors

[Andrew Easter]

Thanks, Patrik. The server-side approach is a neat alternative.

Cheers,
Andrew

On Thursday, 20 April 2017 12:50:40 UTC+1, Patrik Nordwall wrote:
>
> That is a simple and good solution. If the entity ids are known you can do 
> the heartbeating from an actor running on each node, or from a Cluster 
> Singleton, instead of from the clients.
>
> I know something similar is used in Lagom for keeping the read side 
> processors alive.
>
> /Patrik
> ons 19 apr. 2017 kl. 14:57 skrev Andrew Easter  >:
>
>> Hi group,
>>
>> I was wondering if anyone has any experiences/patterns to share with 
>> regard to ensuring the perpetual existence of sharded actors?
>>
>> To explain, what I mean...
>>
>> I'm prototyping some software that pushes data to connected clients over 
>> web sockets (currently using a third party service, Pusher). I have the 
>> concept of an actor that pushes data to one or many specific clients via 
>> Pusher, and, in turn, there can be many such actors, each with their own 
>> set of associated clients. These actors are sharded - each actor has its 
>> own specific configuration that determines the data it sends to its 
>> connected clients via Pusher, and thus I only want one unique instance of 
>> each specific actor within the akka cluster (wouldn't make sense to have 
>> multiple instances all duplicating data being sent to their associated 
>> clients).
>>
>> As the clients of an actor are essentially running forever, it makes 
>> sense for the actor to have perpetual existence - i.e. it always needs to 
>> be alive and sending data to its associated clients. At the moment, I've 
>> solved this by having each client send a heartbeat at a scheduled interval. 
>> In the event the client's associated actor has terminated (for whatever 
>> reason), the heartbeat essentially serves the purpose of waking it up again 
>> (recovering it) somewhere in the cluster. This will then ensure the actor 
>> starts delivering data to all its clients again.
>>
>> The solution I have in place does work, but I'm wondering whether there's 
>> a more 'elegant' solution that means the server side system ensures that 
>> all necessary actors remain running, even in the event of crashing, 
>> restarting nodes etc. Of course, maybe the solution in place is good enough 
>> as it's actually pretty simple. I can imagine a server side solution might 
>> be more involved and require a lot of complex engineering to get right. 
>> Anyhow, thought I'd ask in case there was some established patterns that 
>> people were already using for this type of thing.
>>
>> I imagine another example of where you might want such behaviour is if 
>> you were building a system like Pingdom - i.e. you create an actor per 
>> responsiveness test and know that you always need that actor to be awake, 
>> triggering the configured endpoint at configured intervals. In that case, 
>> you couldn't rely on some client sending a heartbeart to wake the actor up.
>>
>> Any thoughts on this would be more than well received :-)
>>
>> Cheers,
>> Andrew
>>
>>
>> -- 
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: Chunked response with akka-http 10.x

[Thibault Meyer]

Thanks for your responses. It working very well !

Le jeudi 20 avril 2017 12:02:53 UTC+2, Johannes Rudolph a écrit :
>
> Hi Thibault,
>
> if you have the body of the response already as a Source, 
> you can create a response with a chunked entity from it like this:
>
> HttpResponse.create()
>   .withEntity(HttpEntities.create(contentType, source))
>
> Johannes
>
> On Thursday, April 20, 2017 at 9:56:47 AM UTC+2, Thibault Meyer wrote:
>>
>> Hi,
>>
>> I trying akka-http (version 10.x) by creating a small API who generate 
>> UUID on demand. For a bulk UUID creation (ex: */uuid1/5000* to get 5000 
>> UUID v1), I would like to return a chunked response to avoir building a 
>> list of  item on memory.
>>
>> From Play 2.5 I usually using final Source source = 
>> Source.fromGraph(...) but I don't find how to do this with akka-http and 
>> akka-stream.
>>
>> Do you have any idea how to do this ?
>>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Patterns for ensuring perpetual existence of sharded actors

[Patrik Nordwall]

That is a simple and good solution. If the entity ids are known you can do
the heartbeating from an actor running on each node, or from a Cluster
Singleton, instead of from the clients.

I know something similar is used in Lagom for keeping the read side
processors alive.

/Patrik
ons 19 apr. 2017 kl. 14:57 skrev Andrew Easter :

> Hi group,
>
> I was wondering if anyone has any experiences/patterns to share with
> regard to ensuring the perpetual existence of sharded actors?
>
> To explain, what I mean...
>
> I'm prototyping some software that pushes data to connected clients over
> web sockets (currently using a third party service, Pusher). I have the
> concept of an actor that pushes data to one or many specific clients via
> Pusher, and, in turn, there can be many such actors, each with their own
> set of associated clients. These actors are sharded - each actor has its
> own specific configuration that determines the data it sends to its
> connected clients via Pusher, and thus I only want one unique instance of
> each specific actor within the akka cluster (wouldn't make sense to have
> multiple instances all duplicating data being sent to their associated
> clients).
>
> As the clients of an actor are essentially running forever, it makes sense
> for the actor to have perpetual existence - i.e. it always needs to be
> alive and sending data to its associated clients. At the moment, I've
> solved this by having each client send a heartbeat at a scheduled interval.
> In the event the client's associated actor has terminated (for whatever
> reason), the heartbeat essentially serves the purpose of waking it up again
> (recovering it) somewhere in the cluster. This will then ensure the actor
> starts delivering data to all its clients again.
>
> The solution I have in place does work, but I'm wondering whether there's
> a more 'elegant' solution that means the server side system ensures that
> all necessary actors remain running, even in the event of crashing,
> restarting nodes etc. Of course, maybe the solution in place is good enough
> as it's actually pretty simple. I can imagine a server side solution might
> be more involved and require a lot of complex engineering to get right.
> Anyhow, thought I'd ask in case there was some established patterns that
> people were already using for this type of thing.
>
> I imagine another example of where you might want such behaviour is if you
> were building a system like Pingdom - i.e. you create an actor per
> responsiveness test and know that you always need that actor to be awake,
> triggering the configured endpoint at configured intervals. In that case,
> you couldn't rely on some client sending a heartbeart to wake the actor up.
>
> Any thoughts on this would be more than well received :-)
>
> Cheers,
> Andrew
>
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: Chunked response with akka-http 10.x

[Thibault Meyer]

Hi,


thanks for your response. It working very well !



Le jeudi 20 avril 2017 12:02:53 UTC+2, Johannes Rudolph a écrit :
>
> Hi Thibault,
>
> if you have the body of the response already as a Source, 
> you can create a response with a chunked entity from it like this:
>
> HttpResponse.create()
>   .withEntity(HttpEntities.create(contentType, source))
>
> Johannes
>
> On Thursday, April 20, 2017 at 9:56:47 AM UTC+2, Thibault Meyer wrote:
>>
>> Hi,
>>
>> I trying akka-http (version 10.x) by creating a small API who generate 
>> UUID on demand. For a bulk UUID creation (ex: */uuid1/5000* to get 5000 
>> UUID v1), I would like to return a chunked response to avoir building a 
>> list of  item on memory.
>>
>> From Play 2.5 I usually using final Source source = 
>> Source.fromGraph(...) but I don't find how to do this with akka-http and 
>> akka-stream.
>>
>> Do you have any idea how to do this ?
>>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: Chunked response with akka-http 10.x

['Johannes Rudolph' via Akka User List]

Hi Thibault,

if you have the body of the response already as a Source, 
you can create a response with a chunked entity from it like this:

HttpResponse.create()
  .withEntity(HttpEntities.create(contentType, source))

Johannes

On Thursday, April 20, 2017 at 9:56:47 AM UTC+2, Thibault Meyer wrote:
>
> Hi,
>
> I trying akka-http (version 10.x) by creating a small API who generate 
> UUID on demand. For a bulk UUID creation (ex: */uuid1/5000* to get 5000 
> UUID v1), I would like to return a chunked response to avoir building a 
> list of  item on memory.
>
> From Play 2.5 I usually using final Source source = 
> Source.fromGraph(...) but I don't find how to do this with akka-http and 
> akka-stream.
>
> Do you have any idea how to do this ?
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Chunked response with akka-http 10.x

['Michal Borowiecki' via Akka User List]

Hi Thibault,

I do this kind of thing with Source.fromIterator(...) passing it an 
iterator obtained from a java stream.


In your case, it would look something like:

Source source = Source.fromIterator( () ->

Stream.generate( ()->

UUID.randomUUID()

).iterator()

).take(n);

then I stream it out as per:

http://doc.akka.io/docs/akka-http/current/java/http/routing-dsl/source-streaming-support.html#simple-csv-streaming-example

The conversion from the domain object (UUID in your case) to ByteString 
happens in the function passed when creating the Marshaller as in the 
CSV example in the doc section above (I just use Jackson.marshaller() in 
my case)


Happy to learn if there's a better way to do this.

Cheers,

Michal


On 20/04/17 08:56, Thibault Meyer wrote:

Hi,

I trying akka-http (version 10.x) by creating a small API who generate 
UUID on demand. For a bulk UUID creation (ex: //uuid1/5000/ to get 
5000 UUID v1), I would like to return a chunked response to avoir 
building a list of  item on memory.


From Play 2.5 I usually using final Source source = 
Source.fromGraph(...) but I don't find how to do this with akka-http 
and akka-stream.


Do you have any idea how to do this ?
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ: 
http://doc.akka.io/docs/akka/current/additional/faq.html

>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google 
Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to akka-user+unsubscr...@googlegroups.com 
.
To post to this group, send email to akka-user@googlegroups.com 
.

Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.


--
Signature
 Michal Borowiecki
Senior Software Engineer L4
T:  +44 208 742 1600


+44 203 249 8448



E:  michal.borowie...@openbet.com
W:  www.openbet.com 


OpenBet Ltd

Chiswick Park Building 9

566 Chiswick High Rd

London

W4 5XT

UK




This message is confidential and intended only for the addressee. If you 
have received this message in error, please immediately notify the 
postmas...@openbet.com  and delete it 
from your system as well as any copies. The content of e-mails as well 
as traffic data may be monitored by OpenBet for employment and security 
purposes. To protect the environment please do not print this e-mail 
unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building 
9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company 
registered in England and Wales. Registered no. 3134634. VAT no. 
GB927523612


--

 Read the docs: http://akka.io/docs/
 Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
 Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Chunked response with akka-http 10.x

[Thibault Meyer]

Hi,

I trying akka-http (version 10.x) by creating a small API who generate UUID 
on demand. For a bulk UUID creation (ex: */uuid1/5000* to get 5000 UUID 
v1), I would like to return a chunked response to avoir building a list of 
 item on memory.

>From Play 2.5 I usually using final Source source = 
Source.fromGraph(...) but I don't find how to do this with akka-http and 
akka-stream.

Do you have any idea how to do this ?

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.