[akka-user] Abstracting away CommitableMessage

I'm planning to use a commitableSource from akka-streams-kafka. Is there a way to re-use an existing Flow that knows nothing about Kafka, extracting the record value and reconstituting the CommitableMessage at the end of the flow? In the past I've experimented with using TypeClasses in my flow

Re: [akka-user] Abstracting away CommitableMessage

what would happen if that stage would silently discard the CommittableMessage? -- Cheers, √ On Oct 26, 2016 6:09 PM, "Richard Rodseth" wrote: > I'm planning to use a commitableSource from akka-streams-kafka. > > Is there a way to re-use an existing Flow that knows nothing

Re: [akka-user] Abstracting away CommitableMessage

Poorly chosen title. By abstract away, I meant I want to reuse an existing Flow rather than having to rewrite it to use CommitableMessages all the way through. So I think I may need something like a nested flow (the "job") in a stage that hangs onto the current CommitableMessage. On Wed, Oct 26,

[akka-user] Re: Using akka streams to read from streamed source, format and send multi-part file to S3

I've created one based on the AWS Java SDK here: https://github.com/3drobotics/cloud-s3-wrapper There is another one that is more "pure" in that it uses Akka-HTTP to interact with S3, thus does not require the AWS Java SDK here: https://github.com/bluelabsio/s3-stream There has been some

Re: [akka-user] Abstracting away CommitableMessage

In general that would only work for strictly 1:1 Flows. -- Cheers, √ On Oct 26, 2016 7:00 PM, "Richard Rodseth" wrote: > Poorly chosen title. > By abstract away, I meant I want to reuse an existing Flow rather than > having to rewrite it to use CommitableMessages all the

Re: [akka-user] Abstracting away CommitableMessage

Hi Richard, unfortunately this is not possible in general: you would need to define exactly how the extra data (i.e. the commit token) is transported together with the data it belongs to through all possible stages. The answer from category theory would be a functor, but how do you define one

Re: [akka-user] Abstracting away CommitableMessage

Not sure I followed all that, but thanks. I don't need a general solution. I know that my flow is 1 to 1 , say a Flow[SomeEvent, SomeEventRecorded] Using typeclasses, I would have Flow[SomeEventLike, SomeEventRecordedLike] and the actual messages would be CommitableMessages, but all the stages

Re: [akka-user] Abstracting away CommitableMessage

Broadcast/Zip if your existing flow is 1:1. Here's a sample: https://github.com/iravid/stream-processing-talk/blob/master/GraphDSL.scala#L86 -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html

Re: [akka-user] Re: Using akka streams to read from streamed source, format and send multi-part file to S3

Hi Jason, Does yours support setting custom headers and permissions during the upload? We need to upload CSV files that will then be exposed via pre-authorized urls. I'm playing with https://github.com/MfgLabs/commons-aws because it had some documented examples on how it works and that it would

Re: [akka-user] Abstracting away CommitableMessage

Can't you use unzip and zip to pass the committable on the side? Only works for 1-to-1 flows. ons 26 okt. 2016 kl. 20:22 skrev Richard Rodseth : > Not sure I followed all that, but thanks. > I don't need a general solution. > I know that my flow is 1 to 1 , say a

[akka-user] Re: Akka Remote Development on One Machine

Even if you use loopback (127.0.0.1) for the two actor systems to communicate, communication is done through remoting, with messages being serialized etc. just like if they were on "real remote hosts". If you want to do it to get real world latencies/throughputs like with separate physical

Re: [akka-user] Re: Combining Kafka, Akka Streams and (Persistent) Actors

ask is fine as long as you don't close over something that is not thread-safe in callbacks of the Future. It should be low risk of mistake in mapAsync since you typically only return the Future (after casting/mapTo to the right type). I updated the Streams documentation for integration with

[akka-user] BidiFlow from custom GraphStage join connection pool flow

I'd like to create a Flow[HttpRequest, HttpResponse] by joining a BidiFlow built from a custom GraphStage with a connection pool flow built with Akka Http's request-level client-side API. When running the stream I am getting the below exception. If I replace the BidiFlow built from a custom

Re: [akka-user] Re: Combining Kafka, Akka Streams and (Persistent) Actors

Thanks! > On Oct 26, 2016, at 2:35 AM, Patrik Nordwall > wrote: > > ask is fine as long as you don't close over something that is not thread-safe > in callbacks of the Future. It should be low risk of mistake in mapAsync > since you typically only return the Future

[akka-user] Re: withSupervisionStrategy passed a japi Function but compiler complains that its expecting a scala Function1

Still looking for an answer to this, if anyone can point out whats wrong and how to get it done. I was looking at some HTTP docs and there was a fix in there and wonder if the problem is related

[akka-user] Akka HTTP Source in a Graph

Hi, I am trying to figure out if it is possible to use an HTTP Akka Source as a Source in a Graph DSL. My main problem is how to reference the next element in the graph in the akka route. When you are not using it as a Source you can do the following: private def route(someActor: ActorRef)

Re: [akka-user] Abstracting away CommitableMessage

Yes, indeed: if it is strictly 1:1 and it retains the order of the messages, then this works. Thanks for the sample! Regards, Roland > 26 okt. 2016 kl. 22:12 skrev Itamar Ravid : > > Broadcast/Zip if your existing flow is 1:1. Here's a sample: >

Re: [akka-user] Re: withSupervisionStrategy passed a japi Function but compiler complains that its expecting a scala Function1

Well that's exactly the thing we fixed. As you've found in the docs. We'll release a stable this week, please upgrade then. -- Konrad `ktoso` Malawski Akka @ Lightbend On 26 October 2016 at 13:29:52, murtuza chhil (chil...@gmail.com) wrote: Still

Re: [akka-user] Re: withSupervisionStrategy passed a japi Function but compiler complains that its expecting a scala Function1

Thanks Konrad. -chhil On Wednesday, October 26, 2016 at 6:01:34 PM UTC+5:30, Konrad Malawski wrote: > > Well that's exactly the thing we fixed. > As you've found in the docs. > > We'll release a stable this week, please upgrade then. > > -- > Konrad `ktoso` Malawski > Akka @

Re: [akka-user] Re: withSupervisionStrategy passed a japi Function but compiler complains that its expecting a scala Function1

Though my code is not HTTP related, so whatever is fixed will fix my problem too , right? -chhil On Wednesday, October 26, 2016 at 6:08:21 PM UTC+5:30, murtuza chhil wrote: > > Thanks Konrad. > > -chhil > > > On Wednesday, October 26, 2016 at 6:01:34 PM UTC+5:30, Konrad Malawski > wrote: >> >>

Re: [akka-user] Abstracting away CommitableMessage

I came up with this before realizing my Flow (at least currently) is not 1:1 :( def kafka1to1Flow[In,Out](job: Flow[In, Out, _]): Graph[FlowShape[CommittableMessage[_,In], (Out, CommittableMessage[_,In])], NotUsed] = { val result = GraphDSL.create() { implicit b => import

Re: [akka-user] Abstracting away CommitableMessage

Yes, thanks. I'll explore this. On Wed, Oct 26, 2016 at 2:04 PM, Roland Kuhn wrote: > Yes, indeed: if it is strictly 1:1 and it retains the order of the > messages, then this works. Thanks for the sample! > > Regards, > > Roland > > > 26 okt. 2016 kl. 22:12 skrev Itamar Ravid