[akka-user] streams: unroll -> sequence for flow -> reroll -- howto?

2017-05-09 Thread ben fleis 
Hi,

I am new to akka streams, and working on a project having both real-time 
and batch needs. In the batch cases, I need to take context (e.g., 
requestID), from the initial request, and apply it at the end of batch-item 
processing. The batch-item subflow is by itself rather simple: (parse, 
mapConcat, transform).  The item transforms comes from the real-time 
context.

It is the folding into an output format that is less obvious. That code 
needs the request ID and other context from the original request to 
generate an output byte stream.

What I want to do is treat the context and input bytes as a tuple, unzip 
them in the beginning, zip them at the fold stage. I need to understand how 
to map each input byte stream into a sequence of stream elements in a 
sub-flow, and fold each subsequence in its natural grouping.

mapConcat flattens, thus dropping the boundaries between adjacent batch 
requests.

I could use map, and materialize a new stream within each batch handler for 
its items, and send the folded result as the output of the flow... but that 
seems strange, and likely to be already available. (Especially since 
GroupBy and SubFlow seem closely related...)

Does this exist?

Thanks,

b

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] [akka-cluster] ShardRegion in proxy mode vs. access ShardRegion via ClusterClient

2017-05-09 Thread Evgeny Shepelyuk 
Hello

We're implementing  microservice system consisting of several services 
representing completely different domains.
Let's consider that we have following 

   - wallet service representing customer balance, supporting deposit, 
   withdraw and get balance commands
   - wallet client service that interacts with a wallet

Because of Docker based infrastructure we have to use ConstructR for Akka 
Cluster,so setup is not quite trivial.
Our intentions were

   - join wallet service instance into akka cluster
   - do not join wallet client service into cluster
   - expose each ShardRegion via ClusterClientReceptionist
   - from wallet client service access wallets in shards via ClusterClient
   
But after digging into akka docs, I've found a possibility to use 
ShardRegion in proxy only mode, so another architecture idea appeared


   - join wallet and wallet client into the same Akka Cluster
   - on wallet service nodes start ShardRegion in normal mode
   - on wallet client nodes start ShardRegion in proxy only mode
   - access wallets from wallet client using local ShardRegion

Dear community, could you provide pros / cons of both solutions ?
What is more AKKA way of achieving the goal.

Thanks in advance

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: [akka-cluster] ClusterClient and ConstructR

2017-05-09 Thread Evgeny Shepelyuk 
Thanks Rafał

Already figured it out :)

понеділок, 8 травня 2017 р. 17:58:51 UTC+3 користувач Rafał Krzewski 
написав:
>
> Hi Evgeny,
>
> In such situation I'd 
> use de.heikoseeberger.constructr.coordination.Coordination (it's subclass 
> specific for the data store you are using) on the nodes outside the cluster 
> that need to look up contact points.
> You can call Coordination.getNodes() and if None comes back, wait some 
> time and retry.
>
> Cheers,
> Rafał
>
>
> W dniu czwartek, 4 maja 2017 14:28:53 UTC+2 użytkownik Evgeny Shepelyuk 
> napisał:
>>
>> Hi All
>>
>> Is there any example of using ConstructR with ClusterClient.
>>
>> I am having AKKA cluster bootstrapped with ConstructR and Zookeeper, 
>> i.e. I have no predefined seed nodes to create ClusterClient.
>>
>> So, I should connect too Zookeeper and retrieve seed nodes set by 
>> ConstructR.
>>
>> What is the proper way of doing this except using low level Zookeeper API
>>
>> P.S. This is not Zookeeper specific question, it's the same for  ETCD / 
>> Consul.
>>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Architectural suggestion on stream mixed Actor/Stream application

2017-05-09 Thread Henrik Larsson 
Im finding the idea of Akka Stream a very nice fit to the application im 
building. Im building a realtime decision system with the following 
functionality.


   1. Make HTTPS POST request to get a *sessionToken* string
   2. Start a keep-alive service to ping session server at 3h intervals
   3. Use *sessionToken* as payload when in connection message when 
   connecting to TCP socket to start receive *DataItems*
   4. Create a stream pipeline from the socket connection as a function 
   from *DataItem => Decision*
   5. Handle session management faults and any faults in the pipeline

What I have so far is an Actor hierarchy with an FSM SessionManagerActor 
which contains the *sessionToken *state, it gets it from a child actor wich 
does the actual getSessionRequest, when SessionManagerActor receives the 
*sessionToken* it spawns a keep-alive actor.
The problem starts when I want to use the *TCP* flow from Akka stream to 
setup the stream. All the ways im trying to connect the sessionToken as 
input the the TCP stream seems like an ugly hack and I cant figure out how 
to do error handling if i for example need to update the session token.

What would be the best solution for a problem like this and should i use 
Akka Stream for all parts, even keep-alive actor etc. Any input is highly 
appreciated since i feel if i only get and understanding of this part of 
the problem I will be able to solve the actual decision support part.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Restart stream with backoff

2017-05-09 Thread Konrad Malawski 
The hosting within an Actor is a perfectly fine and valid solution IMHO.

On Tue, May 9, 2017 at 4:04 PM, 'Michal Borowiecki' via Akka User List <
akka-user@googlegroups.com> wrote:

> Sounds like what could help in your organization is using a higher-level
> abstraction, such as Lagom.
>
> You don't need to be comfortable with actors to use it and it does provide
> a wrapper around reactive-kafka (or akka-stream-kafka as it's currently
> known) with exponential backoff restarts:
>
> https://www.lagomframework.com/documentation/1.3.x/java/KafkaClient.html
> If you decide to explore that route, there's a lagom framework mailing
> list: lagom-framew...@googlegroups.com
>
> Cheers,
> Michał
>
>
> On 08/05/17 20:56, Richard Rodseth wrote:
>
> I was hoping to promote a pattern in my organization, using reactive-kafka
> (source-per-partition). Some of my colleagues are comfortable with actors,
> but it would be great if others could be introduced to the streams APIs
> without learning all about actors.
>
> Am I correct that there is currently no way to restart a stream with
> exponential backoff, other than running the stream in a "host" actor that
> runs the stream and is fed stream elements using alsoTo ?
>
> Backoff supervision is one of the items mentioned here:
> https://github.com/akka/akka/issues/19950
>
> Anyone else come up with a creative solution?
>
>
>
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
>  Michal Borowiecki
> Senior Software Engineer L4
> T: +44 208 742 1600 <+44%2020%208742%201600>
>
>
> +44 203 249 8448 <+44%2020%203249%208448>
>
>
>
> E: michal.borowie...@openbet.com
> W: www.openbet.com
> OpenBet Ltd
>
> Chiswick Park Building 9
>
> 566 Chiswick High Rd
>
> London
>
> W4 5XT
>
> UK
> 
> This message is confidential and intended only for the addressee. If you
> have received this message in error, please immediately notify the
> postmas...@openbet.com and delete it from your system as well as any
> copies. The content of e-mails as well as traffic data may be monitored by
> OpenBet for employment and security purposes. To protect the environment
> please do not print this e-mail unless necessary. OpenBet Ltd. Registered
> Office: Chiswick Park Building 9, 566 Chiswick High Road, London, W4 5XT,
> United Kingdom. A company registered in England and Wales. Registered no.
> 3134634. VAT no. GB927523612
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Cheers,
Konrad 'ktoso' Malawski
Akka  @ Typesafe 

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Restart stream with backoff

2017-05-09 Thread 'Michal Borowiecki' via Akka User List 
Sounds like what could help in your organization is using a higher-level 
abstraction, such as Lagom.


You don't need to be comfortable with actors to use it and it does 
provide a wrapper around reactive-kafka (or akka-stream-kafka as it's 
currently known) with exponential backoff restarts:


https://www.lagomframework.com/documentation/1.3.x/java/KafkaClient.html

If you decide to explore that route, there's a lagom framework mailing 
list: lagom-framew...@googlegroups.com


Cheers,
Michał

On 08/05/17 20:56, Richard Rodseth wrote:
I was hoping to promote a pattern in my organization, using 
reactive-kafka (source-per-partition). Some of my colleagues are 
comfortable with actors, but it would be great if others could be 
introduced to the streams APIs without learning all about actors.


Am I correct that there is currently no way to restart a stream with 
exponential backoff, other than running the stream in a "host" actor 
that runs the stream and is fed stream elements using alsoTo ?


Backoff supervision is one of the items mentioned here:
https://github.com/akka/akka/issues/19950

Anyone else come up with a creative solution?




--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ: 
http://doc.akka.io/docs/akka/current/additional/faq.html

>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google 
Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to akka-user+unsubscr...@googlegroups.com 
.
To post to this group, send email to akka-user@googlegroups.com 
.

Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.


--
Signature
 Michal Borowiecki
Senior Software Engineer L4
T:  +44 208 742 1600


+44 203 249 8448



E:  michal.borowie...@openbet.com
W:  www.openbet.com 


OpenBet Ltd

Chiswick Park Building 9

566 Chiswick High Rd

London

W4 5XT

UK




This message is confidential and intended only for the addressee. If you 
have received this message in error, please immediately notify the 
postmas...@openbet.com  and delete it 
from your system as well as any copies. The content of e-mails as well 
as traffic data may be monitored by OpenBet for employment and security 
purposes. To protect the environment please do not print this e-mail 
unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building 
9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company 
registered in England and Wales. Registered no. 3134634. VAT no. 
GB927523612


--

 Read the docs: http://akka.io/docs/
 Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
 Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.