Re: [akka-user] Re: [akka-stream] Problems with the shape creation

Hi Rafał, Code that uses my shape is following: private CompletionStage buildAndRunGraph(ArrayList sourceList) { //First function Source source = Source.from(sourceList); Materializer materializer = ActorMaterializer.create(context());

[akka-user] Re: Simple beginner questions: Accessing values in flows

So, rightly or wrongly, I did this, which seems to work: val flow = builder.add(wsl.mapMaterializedValue(f => { f map { u => if (u.response.status == StatusCodes.SwitchingProtocols) { log.info("Switched protocols") } else

Re: [akka-user] Is contents of GraphDSL.create() {..} thread-safe for BidiFlow?

среда, 19 октября 2016 г., 15:47:31 UTC+3 пользователь √ написал: > > Ok, so a sort of "correlator"-stage? > Exactly. > Yes, so the BidiFlow you create has shared state tied to the intance, not > the mateiralization. > I think you'll need to create a custom GraphStage with a BidiShape. >

Re: [akka-user] Re: Fetching remote file via akka-http singleRequest results in extremely high allocation rate

Hi, there's really not too much to add to the gist from the first post: this method is called once, almost right in the main class, to fetch some resource. For now I managed to work this issue around, but I thought that working on rope-like ByteStrings should be faster than on array-backed

[akka-user] Re: Fetching remote file via akka-http singleRequest results in extremely high allocation rate

Using mapConcat instead may even be faster ;) On Wednesday, October 19, 2016 at 5:05:22 PM UTC+2, vladysla...@rtsmunity.com wrote: > > Okay, so the issue was really in Framing performance. Changing Framing > stage for > > flatMapConcat(chunk -> Source.from(Arrays.asList(chunk.split("\n". >

[akka-user] Re: Fetching remote file via akka-http singleRequest results in extremely high allocation rate

Okay, so the issue was really in Framing performance. Changing Framing stage for flatMapConcat(chunk -> Source.from(Arrays.asList(chunk.split("\n". actually made it reasonably fast. On Wednesday, October 19, 2016 at 3:25:50 PM UTC+2, vladysla...@rtsmunity.com wrote: > > Hello, > > I have

[akka-user] Re: Fetching remote file via akka-http singleRequest results in extremely high allocation rate

Hi Vladyslav, this sounds like a worst-case scenario for the Framing stages: 45M lines and each line 2-3 characters long will put a lot of pressure on the streams infrastructure and the framing stage. It might still make sense to benchmark and optimize the Framing stage. One optimization could

Re: [akka-user] Re: Fetching remote file via akka-http singleRequest results in extremely high allocation rate

Hi Vladyslav, firstly, no reason to get desperate :-) Glad you did some profiling, it would be super helpful next time around if you share more details about both your code an profile you're observing. It's hard to give advice what can be improved when given a blank canvas (no idea what your code

[akka-user] Re: Fetching remote file via akka-http singleRequest results in extremely high allocation rate

After some playing with JMC and async boundaries it turned out the hottest methods are related to ByteString splitting and delimiting (for example, scala.collection.IndexedSeqOptimized$class.sameElements(IndexedSeqOptimized, GenIterable) takes 73-75% of processor time). The input file is 45M

[akka-user] Re: Fetching remote file via akka-http singleRequest results in extremely high allocation rate

Akka and Akka-HTTP version is 2.4.11. On Wednesday, October 19, 2016 at 3:25:50 PM UTC+2, vladysla...@rtsmunity.com wrote: > > Hello, > > I have the following problem. I'm trying to fetch a file via HTTP in > streaming fashion, processing it line by line (code: >

[akka-user] Fetching remote file via akka-http singleRequest results in extremely high allocation rate

Hello, I have the following problem. I'm trying to fetch a file via HTTP in streaming fashion, processing it line by line (code: https://gist.github.com/anonymous/c737fa388b55181dcdab9fa6cb8cb2bc), but it is extremely slow (about a minute to fetch a 150MB file to t2.micro instance from S3). I

Re: [akka-user] Is contents of GraphDSL.create() {..} thread-safe for BidiFlow?

On Wed, Oct 19, 2016 at 2:43 PM, Alexey Shuksto wrote: > 1. Flow itself is a bidi-codec from ByteString to our own Request/Response > entities. Each Request has Promise[Response] attribute. Shared state is > more like Map[Request.Id, Promise[Response]] -- because order of

Re: [akka-user] Is contents of GraphDSL.create() {..} thread-safe for BidiFlow?

1. Flow itself is a bidi-codec from ByteString to our own Request/Response entities. Each Request has Promise[Response] attribute. Shared state is more like Map[Request.Id, Promise[Response]] -- because order of Responses are not guarantied. 2. It should have state shared only "inside" this

Re: [akka-user] Is contents of GraphDSL.create() {..} thread-safe for BidiFlow?

On Wed, Oct 19, 2016 at 2:18 PM, Alexey Shuksto wrote: > 2 Konrad: Yep, in original question I meant not 'DSL construction time' > but 'execution time' thread-safety. Thanks for clarification. > > 2 Victor: Use case is simple: outgoing flow need to store `Promise` of > future

Re: [akka-user] Is contents of GraphDSL.create() {..} thread-safe for BidiFlow?

2 Konrad: Yep, in original question I meant not 'DSL construction time' but 'execution time' thread-safety. Thanks for clarification. 2 Victor: Use case is simple: outgoing flow need to store `Promise` of future remote response in some shared state which then would be completed by incoming

Re: [akka-user] Re: [akka-stream] Problems with the shape creation

Sergey, you haven't shown any code related to passing messages yet, so it's impossible to guess what's happening at this point. As I said in previous message, the critical things are the actual components you put *inside* of your graph. The shapes just determine external connectors. Cheers,

Re: [akka-user] Is contents of GraphDSL.create() {..} thread-safe for BidiFlow?

Hi Alexey, Not only is it not thread-safe, but it also actively prevents multiple materializations. Perhaps if you state your use-case we can suggest an alternative? On Wed, Oct 19, 2016 at 1:24 PM, Alexey Shuksto wrote: > Hello hAkkers, > > Simple example: > val zipper =

Re: [akka-user] Is contents of GraphDSL.create() {..} thread-safe for BidiFlow?

This is not safe, outbound and inbound flows could be executing on different threads. It's not a question about the the DSL being safe - that's fine as it's only constructing stuff, but the graph you constructed is accessing shared state from (potentially) different threads - thus it is not safe.

[akka-user] Is contents of GraphDSL.create() {..} thread-safe for BidiFlow?

Hello hAkkers, Simple example: val zipper = BidiFlow.fromGraph(GraphDSL.create() { b => var counter = 0 val outbound = b.add(Flow[String].map { str => counter += 1 str -> counter }) val inbound = b.add(Flow[(String, Int)].map { pair => counter -= 1 pair._1 })

Re: [akka-user] Re: [akka-stream] Problems with the shape creation

Yep, I was trying to make something like that, but my code doesn't work. Messages stuck somewhere and I don't know how to fix it due to lack of Scala knowledge. You can find the code in my initial message here. I took UniformFanOut as example and unsuccessfully tried to replace single Inlet

Re: [akka-user] Re: ANNOUNCE: Akka HTTP 3.0.0-RC1

Thanks for reporting. We spotted this and have fixed the issue already: https://github.com/akka/akka-http/pull/409 The fixed artifacts are right now on their way to maven-central. Happy hakking. -- Konrad 'ktoso’ Malawski Akka @ Lightbend

[akka-user] Re: ANNOUNCE: Akka HTTP 3.0.0-RC1

Hi, I took a quick look at the new Akka HTTP artifacts published on Maven Central, and I've noticed that artifact names don't have Scala version suffix in them. Is this intentional? Cheers, Rafał W dniu wtorek, 18 października 2016 00:22:17 UTC+2 użytkownik Konrad 'ktoso' Malawski napisał: >

Re: [akka-user] Re: [akka-stream] Problems with the shape creation

Shapes don't need separate java or scala api, it's shared. You can just subclass a shape and make a class that directly represents your shape. If you want AmorphousShape then sure, but please note that it's purpose is to "forget about the types of those". If you want a well typed one simply

[akka-user] Re: [akka-stream] Problems with the shape creation

Hi, Yes, but it seems that I need to create Java API for it, because my app is in Java. I used Inkscape app. to draw the diagram. Cheers, Sergey среда, 19 октября 2016 г., 0:46:00 UTC+3 пользователь Rafał Krzewski написал: > > A custom GraphStage [1] using AmorphousShape is probably the way