cheers :)
-Michael
On 10/02/14 11:17, tigerfoot wrote:
Very interesting! I'll be watching this project go forward.
Like the back-flow Ack too! Simple and effective.
Thank you!
On Thursday, October 2, 2014 10:41:16 AM UTC-5, Michael Frank wrote:
sounds to me like a use case for akka-streams:
http://akka.io/news/2014/09/12/akka-streams-0.7-released.html
<http://akka.io/news/2014/09/12/akka-streams-0.7-released.html>
note that akka-streams is still in development preview. if you
don't want to hitch your cart to it yet, why don't you modify the
storage actor to send an Ack message back upstream to the reader
actor, signalling readiness to read more data?
-Michael
On 10/02/14 08:22, tigerfoot wrote:
Hello,
I have a system that does the following: read/parse from a huge
(150+GB) file of data, run sanity/cleaning operations, and store
the cleaned records in my DB.
I've got this configured using actors such that each time I read
an object from the input file I send a message to the cleaner
actor. When that object is cleaned I send a
message to the storage actor. (The real application is more
complex but this example illustrates the problem.)
read --> clean --> store
Simple on the surface. My problem is that each of these
operations takes different amounts of time. Reading takes say 2
time unit, leaning takes 1, and storage takes 4.
That means I've got a bottleneck on storage and even if I use a
round-robin dispatcher I still run the risk of overflow and data
loss on the mailbox on the storage actor.
Other than massive over-provision of storage actors, is there
another way? Can something within the actor ecosystem provide
back-pressure so that I can slow down
the reader just enough to keep the pipeline clean?
Thanks,
Greg
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
<http://doc.akka.io/docs/akka/current/additional/faq.html>
>>>>>>>>>> Search the archives:
https://groups.google.com/group/akka-user
<https://groups.google.com/group/akka-user>
---
You received this message because you are subscribed to the
Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/akka-user
<http://groups.google.com/group/akka-user>.
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google
Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
--
Read the docs: http://akka.io/docs/
Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
