Hi John! I think I can help you, but I have some follow-on questions :)
On Sun, Feb 7, 2016 at 12:52 PM, <[email protected]> wrote: > Hi Victor, > I know you love simple elegant code but > I looked at unfoldAsync and I don't see that it solves the usecase I have > in mind. > > I'll try to explain why because I am not sure if I am overlooking > something obvious? > > An external system writes events into a log table. Since this is a > non-reactive-sql-database the consumer needs to poll the log table (for > example every 2 seconds) for new events. > (Why) does it need to poll the log table if there is no demand for events? > > To convert this scenario into a streaming source using unfoldAsync > I need to implement a polling loop inside the future so that > Future.success gets only called when new events are inserted in the log > table. > This seems much like a technical aspect rather than a requirement. > > If I don't use polling the future would send 0 Events upstream and the > stream would come to an end? > Events go downstream, or did I misunderstand something? > > That's why I like using instead of unfoldAsync an Actor like JobManager > <http://doc.akka.io/docs/akka-stream-and-http-experimental/2.0.3/java/stream-integrations.html>. > as an Source. Within the Actor I can be more fine grained and use the > scheduler to implement the polling logic. > Let's take a step back, what are the actual requirements? 1. You have a source of "events" and the only way you know if there are any events, is if you ask for events 2. You want to consume events from this source 3. You only need to consume events from the source if there is demand from "downstream" 4. Does the source of events ever end, and how do you know? 5. can you ask for a specified number of events or does polling imply reading all available events? > > Many Greetings > John > > > > > > > > > > > > Am Mittwoch, 3. Februar 2016 23:10:15 UTC+1 schrieb √: >> >> >> http://doc.akka.io/api/akka-stream-and-http-experimental/2.0.3/?_ga=1.45749860.1579561034.1353497989#akka.stream.scaladsl.Source$ >> >> On Wed, Feb 3, 2016 at 11:06 PM, <[email protected]> wrote: >> >>> where do I find unfold/unfoldAsync ? I looked at >>> http://doc.akka.io/docs/akka-stream-and-http-experimental/2.0.3/stages-overview.html >>> ? >>> Many Greetings >>> John >>> >>> >>> Am Mittwoch, 3. Februar 2016 20:51:17 UTC+1 schrieb √: >>>> >>>> I don't see why you'd need to write a custom GraphStage for this. >>>> >>>> unfold/unfoldAsync paired with a buffer with an appropriate dropping >>>> policy should work, no? >>>> >>>> On Wed, Feb 3, 2016 at 8:47 PM, clca <[email protected]> wrote: >>>> >>>>> Yes this is close to what I need to do. >>>>> The code in the onPull method is something like >>>>> >>>>> val m = ReadFromService(...) >>>>> if(m != null) >>>>> push(out, m) >>>>> >>>>> in a traditional app reading would be done in a loop >>>>> >>>>> while(true) { >>>>> val m = ReadFromService(...) >>>>> //do something with the new message >>>>> } >>>>> >>>>> So I'll add a mechanism in the Source to keep polling the external >>>>> service. >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> On Wednesday, February 3, 2016 at 12:55:16 AM UTC-8, >>>>> [email protected] wrote: >>>>>> >>>>>> I've done something similar. >>>>>> I adapted this JobManager >>>>>> <http://doc.akka.io/docs/akka-stream-and-http-experimental/2.0.3/java/stream-integrations.html>. >>>>>> When no data is available( for example when it recieves a Request(16) >>>>>> Messag) it starts a" polling Actor" which polls an external Database for >>>>>> more data. >>>>>> Does this help? >>>>>> >>>>>> Am Mittwoch, 3. Februar 2016 08:33:16 UTC+1 schrieb clca: >>>>>>> >>>>>>> I searched through the topics but I could not find any example on >>>>>>> how to pull from a Source in the case where data is coming in bursts. I >>>>>>> built a customized Source that read from an external source. Data is >>>>>>> coming >>>>>>> in bursts, the flow can stop for a while so I need to keep pulling the >>>>>>> Source in such a way I can keep reading data (the actual read process is >>>>>>> done in the old poll fashion). I guess I need something like a KeepAlive >>>>>>> type of mechanism for the data stream. >>>>>>> >>>>>>> BTW: Fantastic job with Akka, Akka Stream & Akka HTTP! >>>>>>> >>>>>>> Thanks >>>>>>> Claudio >>>>>>> >>>>>>> >>>>>> -- >>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>> >>>>>>>>>> Check the FAQ: >>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>> >>>>>>>>>> Search the archives: >>>>> https://groups.google.com/group/akka-user >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Akka User List" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at https://groups.google.com/group/akka-user. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>>> >>>> -- >>>> Cheers, >>>> √ >>>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: >>> http://doc.akka.io/docs/akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: >>> https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Cheers, >> √ >> > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Cheers, √ -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
