Hi, I was wondering how easy it would be to implement a reactive-stream with back-pressure for parsing files. I have some apps that chew on a large number of large files, each file containing many data items. Threads can block on the IO.
There may be a whole down-stream pipeline of processing going on from the data in from the IO, often consuming data from several of these files but also often with data processing per data item. Ideally, there would be some non-blocking code that it processing a file. It would produce a stream of data items. While these are being eaten, it continues to chew on the file. When the downstream processing has backed up, the parsing suspends and that thread is handed off for other work. When it is no longer backed up, it would regain a thread and unsuspend and chew some more on the file. Does this sound sensible, and is it something I can implement today with akka? Cheers, Matthew -- Dr Matthew Pocock Turing ate my hamster LTD mailto: [email protected] Integrative Bioinformatics Group, School of Computing Science, Newcastle University mailto: [email protected] gchat: [email protected] msn: [email protected] irc.freenode.net: drdozer skype: matthew.pocock tel: (0191) 2566550 mob: +447535664143 -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
