Not sure I followed all that, but thanks. I don't need a general solution. I know that my flow is 1 to 1 , say a Flow[SomeEvent, SomeEventRecorded]
Using typeclasses, I would have Flow[SomeEventLike, SomeEventRecordedLike] and the actual messages would be CommitableMessages, but all the stages involved in defining the flow would use XYZLike, with implicit evidence parameters. Was really hoping I don't have to do that and could somehow nest my Flow[SomeEvent, SomeEventRecorded], extracting SomeEvent from a commitable message and putting SomeEventRecorded back into one. On Wed, Oct 26, 2016 at 10:59 AM, Roland Kuhn <goo...@rkuhn.info> wrote: > Hi Richard, > > unfortunately this is not possible in general: you would need to define > exactly how the extra data (i.e. the commit token) is transported together > with the data it belongs to through all possible stages. The answer from > category theory would be a functor, but how do you define one for > conflate/expand/fold/etc.? Another approach could be to limit the nested > flow to only one element at a time so that the metadata stay valid in the > surrounding context, but there you run into the same problem: what if you > have stateful stages or how do you determine that the element has been > dropped? (timeouts are never a real solution, only an approximate > safe-guard or band-aid) > > Notice that none of this is specific to Akka Streams, this is a problem > for all stream processing that has stages that do not emit one value for > each input. > > Having said all that, it is possible to implement what you want for a > subset of operations where it makes sense, and in principle type classes or > extension methods are a suitable vehicle. Unfortunately you’ll have to > implement the type machinery specifically for all four FlowOps subtypes > (Source, Flow, SubSource, SubFlow). I have not yet tried out whether the > partial fix for SI-2712 that will be in Scala 2.12.0 and 2.11.9 is > sufficient to make this generic—I won’t say “easy” in any case, because the > types you’ll encounter will certainly be daunting. > > I’d love to sink my teeth into this problem, but unfortunately I don’t > have time for that right now :-( > > Regards, > > Roland > > 26 okt. 2016 kl. 18:49 skrev Richard Rodseth <rrods...@gmail.com>: > > Poorly chosen title. > By abstract away, I meant I want to reuse an existing Flow rather than > having to rewrite it to use CommitableMessages all the way through. > So I think I may need something like a nested flow (the "job") in a stage > that hangs onto the current CommitableMessage. > > > On Wed, Oct 26, 2016 at 9:23 AM, Viktor Klang <viktor.kl...@gmail.com> > wrote: > >> what would happen if that stage would silently discard the >> CommittableMessage? >> >> -- >> Cheers, >> √ >> >> On Oct 26, 2016 6:09 PM, "Richard Rodseth" <rrods...@gmail.com> wrote: >> >>> I'm planning to use a commitableSource from akka-streams-kafka. >>> >>> Is there a way to re-use an existing Flow that knows nothing about >>> Kafka, extracting the record value and reconstituting the CommitableMessage >>> at the end of the flow? >>> >>> In the past I've experimented with using TypeClasses in my flow >>> definition signatures, but it got a bit messy. >>> >>> >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/c >>> urrent/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups.google.com/grou >>> p/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to akka-user+unsubscr...@googlegroups.com. >>> To post to this group, send email to akka-user@googlegroups.com. >>> Visit this group at https://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/c >> urrent/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to akka-user+unsubscr...@googlegroups.com. >> To post to this group, send email to akka-user@googlegroups.com. >> Visit this group at https://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ > current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ > current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
