Re: [akka-user] Re: Akka-http: how to deal with incorrect http headers

2017-05-15 Thread johannes . rudolph
Hi Florian, can you clarify what needs improvement? Is that about client or server side? Johannes On Saturday, May 13, 2017 at 10:47:52 AM UTC+2, Florian Rosenberg wrote: > > I'm seeing a similar problem, but not wit the URI but with the > Strict-Transport-Header, it seems to be invalid, changi

Re: [akka-user] Re: Akka Http how to add charset utf-8 to the content-type header

2017-05-15 Thread johannes . rudolph
Hi Thibault, you are right, there's currently no built-in way to do this. To achieve it, you could e.g. copy the Jackson marshaller from the sources to use a custom media type. See here: https://github.com/akka/akka-http/blob/5932237a86a432d623fafb1e84eeeff56d7485fe/akka-http-marshallers-java/

[akka-user] Re: akka-http ssl setup documentation / examples isn't very clear. Is config-based

2017-05-22 Thread johannes . rudolph
Hi Andrew, your observation is correct. Server side TLS configuration is only possible through code right now. We have tickets to track improving documentation and maybe adding the configuration based approach https://github.com/akka/akka-http/issues/55 https://github.com/akka/akka-http/issues/

[akka-user] Re: has anyone attempted a multi-source authentication?

2017-05-22 Thread johannes . rudolph
Hi Andrew, here's a general idea at how it could work: If you model each authentication method as a `Directive1[Session]` that returns the session (or user, principal, etc.) for that authentication method and all of the directive return the a value of the same type or a type with a common supe

[akka-user] Re: How to map unstructured Json to a case class in Akka-Http

2017-05-22 Thread johannes . rudolph
Hi, try this case class structure instead: case class Customer(name: String,jsonData: JsValue) Johannes On Saturday, May 13, 2017 at 10:47:52 AM UTC+2, vishal...@exadatum.com wrote: > > > I am trying to create the REST service using akka Http and slick . I am > trying to persist the Raw j

[akka-user] Re: Connection retry for TCP stream.

2017-05-23 Thread johannes . rudolph
Hi Ivan, I guess it depends on how you want to use this connection exactly. As TCP is a stream-based protocol there's usually some state associated with a connection that needs to be regarded when a new connection is opened. Therefore, there cannot be a general solution to the problem that woul

[akka-user] Re: ActorSystem Uncaught fatal error shutting down ActorSystem !

2017-06-06 Thread johannes . rudolph
Hi, in cases of fatal errors, the error and stack trace is logged to stderr (not using the logging framework). Note that in some cases, the logging itself may fail (that's why the error is fatal: after it happens, the state of the JVM might be corrupted and operations like logging may fail for

[akka-user] Re: Akka cluster http management

2017-06-06 Thread johannes . rudolph
Hi, the hostname setting is a bit misnamed. It defines the interface the server binds to. So, you can put "0.0.0.0" in there to make sure the management interface is bound on all interfaces (but make sure not to expose it publicly) or put some other interface address in there. I filed https:/

[akka-user] Re: ActorSystem Uncaught fatal error shutting down ActorSystem !

2017-06-06 Thread johannes . rudolph
Hi, my colleague Arnout just found out that the error will only be logged to stderr if you enable the `akka.jvm-exit-on-fatal-error` setting. Can you try enabling this setting and then run again? I also filed an issue to improve the logging of fatal errors: https://github.com/akka/akka/issue

Re: [akka-user] Re: ActorSystem Uncaught fatal error shutting down ActorSystem !

2017-06-07 Thread Johannes Rudolph
The rest of the stack here will tell you where the problem comes from: On Wed, Jun 7, 2017 at 10:40 AM, wrote: > * situation 1 : ( has tell me the stack, and tell me *Shutdown finished * > )* > INFO | jvm 1| 2017/04/07 15:49:52 | java.lang.NoClassDefFoundError: > Lws/protos/EnumsProtos$Har

Re: [akka-user] Re: ActorSystem Uncaught fatal error shutting down ActorSystem !

2017-06-07 Thread Johannes Rudolph
ion 2,3,4 *, din't tell me the reason ( the stack ), id dont > hnow how solve this problem. > > 在 2017年6月7日星期三 UTC+8下午5:00:07,Johannes Rudolph写道: >> >> >> The rest of the stack here will tell you where the problem comes from: >> >> On Wed, Jun 7, 2017 at 10:40

[akka-user] Re: Akka-Http Entity Stream Truncation

2017-07-10 Thread johannes . rudolph
Hi Michael, On Monday, July 10, 2017 at 9:01:00 AM UTC+2, Michael Pisula wrote: > > As far as I saw from the source code, it could point to a problem with > header parsing, but I am not exactly sure what could cause the problem. > The place in the code is actually misleading, as it the error is

[akka-user] Re: [akka-streams] Generic streams and abstract types

2017-07-12 Thread johannes . rudolph
Hi Jeff, your API seems quite complex. I don't know the purpose of it so I cannot suggest anything but I'd try to simplify. :) That said, your problem seems to be that you cannot write a concrete type that would express the dependency between the two components of the tuple `(RequestBuilder, P

[akka-user] Re: [akka-streams] Generic streams and abstract types

2017-07-13 Thread johannes . rudolph
On Wednesday, July 12, 2017 at 9:08:52 PM UTC+2, Jeff wrote: > > As for the issue of complexity, it's actually not as complex as it sounds. > I'm using Http().superPool() to make api requests and I wanted to avoid > having to create a separate stream for every single iteration of api > request w

[akka-user] Re: Akka Cluster Pub/Sub performance for chat-room like applications

2017-07-13 Thread johannes . rudolph
On Thursday, July 13, 2017 at 1:08:19 PM UTC+2, Alexander Lukyanchikov wrote: > > *The only question, is it capable to manage tens of millions of topics? > Would it perform better then our current solution?* > No, most likely it currently won't scale up to 1 million active topics. In Akka's pub

Re: [akka-user] Re: Akka Cluster Pub/Sub performance for chat-room like applications

2017-07-13 Thread johannes . rudolph
On Thursday, July 13, 2017 at 2:56:52 PM UTC+2, Justin du coeur wrote: > > (I should note: I don't use Akka Pub/Sub myself, but I'm wondering whether > Cluster Sharding actually fits your use case well. Depending on the > details, it might.) > Yep, I guess that's true. With cluster sharding eac

[akka-user] Re: akka-http - Setting TLS Server Name Indicator (SNI) explicitly for Client-Side HTTPS

2017-07-13 Thread johannes . rudolph
Hi Shayan, this seems like an uncommon usage for an HTTP client. Basically you want to connect to a server that presents a certificate for the wrong host name. This is unsupported out of the box because it would be an unsafe thing to do in general. The way you tried it does not work because th

[akka-user] Re: Akka http client dispatcher

2017-07-13 Thread johannes . rudolph
Hi Diego, it seems it is an oversight that the dispatcher for at least the actor parts of the pool infrastructure cannot be configured anywhere. I created a ticket to track adding this feature: https://github.com/akka/akka-http/issues/1278 Thanks, Johannes On Thursday, June 22, 2017 at 12:4

[akka-user] Re: Akka-Http Livelock with 100% CPU consumption when creating connection to non-existing host

2017-07-13 Thread johannes . rudolph
Hi Vanger, thanks for the report. Have you changed the value of akka.http.host-connection-pool.min-connections (https://github.com/akka/akka-http/blob/master/akka-http-core/src/main/resources/reference.conf#L233)? That would explain the behavior where the pool tries to keep connections alive

[akka-user] sbt-revolver 0.9.0 released with sbt 1.0.0 support

2017-08-15 Thread johannes . rudolph
Dear fast application restarters, we just released sbt-revolver 0.9.0 which is the first version of sbt-revolver cross-built for sbt 0.13.x and 1.0.x. Thanks go to Olli Helenius / @liff who contributed the sbt 1.0 compatibility changes (#62). We also merged a long-standing PR that allows to custo

[akka-user] Re: Akka HTTP usage of HttpEntity.toStrict

2017-08-15 Thread johannes . rudolph
Hi Yannick, if you want to log the complete request contents, then there is no other way than to collect anything into memory (actually, that's a consequence of logging, not of the API). In that case, you can use toStrict method or the toStrictEntity directive at the root of your routing tree

[akka-user] Re: [akka-http] Http().superPool() and MergeHub.source backpressure

2017-08-15 Thread johannes . rudolph
Hi Jeff, if you don't read the response bodies of all the responses, your pipeline will stall because the super pool connection are still waiting for your code to actually read the responses. In your example, try to add `x.discardEntityBytes` (or actually read the entity) inside of the `Sink.f

[akka-user] Re: [akka-http] adding cookie attributes

2017-08-15 Thread johannes . rudolph
Hi Christophe, yes, that's correct. There seems to be no way to model custom cookie attributes right now. Using RawHeader is the right workaround for now. I filed https://github.com/akka/akka-http/issues/1354 to discuss improvements. Johannes On Monday, August 14, 2017 at 10:31:11 AM UTC+2, Ch

[akka-user] Re: Akka Http & Streams interaction, http requests getting blocked, program is stuck

2017-08-22 Thread johannes . rudolph
Hi Jerry, your explanation is spot-on. You need to be make sure that the entities of all responses are consumed. In your case, that may not happen because of a race-condition: `take(2)` will cancel the stream and responses might get discarded between the first and the second `mapAsyncUnordered`

Re: [akka-user] Re: Akka Http & Streams interaction, http requests getting blocked, program is stuck

2017-08-22 Thread Johannes Rudolph
Hi Jerry, On Tue, Aug 22, 2017 at 12:20 PM, Jerry Tworek wrote: > Do I understand it correctly, that in this case cachedHostConnectionPool > is basically unusable? I assume it will always be executed in a separate > stage from the next stage, that actually consumes the request, and it can > alwa

[akka-user] Re: Akka Http Performance with high-latency route.

2017-09-11 Thread johannes . rudolph
Hi Dominic, it depends on what you mean with "high-latency" API. If you mean that some external service is called which takes a long while, then you need to ensure that executing this external call does not block the thread and the thread can be used for other tasks while waiting for the result

[akka-user] Re: Akka Http: What does the source returned by http://doc.akka.io/japi/akka/2.4.5/akka/http/scaladsl/model/HttpEntity.html#getDataBytes-- materialize to?

2017-09-11 Thread johannes . rudolph
Hi Bwmat, On Saturday, September 9, 2017 at 2:32:13 AM UTC+2, Bwmat wrote: > > The type is just Object, and it's not documented in the linked javadoc. > It is undefined. It needs to have this type so that users can pass in sources with any materialized value. The Akka Http implementation will m

[akka-user] Re: Akka Http memory leak suspect

2017-09-25 Thread johannes . rudolph
Hi Bartosz, I can look into the heap dump. You can send it to me privately. If that's not possible could you post an histogram? It would be great if that could be filtered once for subclasses of `Actor` (which will probably be dominated by `ActorGraphInterpreter`) and once filtered by `GraphSta

[akka-user] Re: Cluster: Loosing messages when rebalancing

2017-09-25 Thread johannes . rudolph
Hi Eduardo, cluster sharding has at-most-once delivery (as most of Akka) so losing some messages is to be expected. Persistent actor can opt-in to at-least-once delivery (see http://doc.akka.io/docs/akka/current/scala/persistence.html#at-least-once-delivery), for other actors, you need to mak

Re: [akka-user] Re: Akka Http memory leak suspect

2017-09-26 Thread Johannes Rudolph
On Tue, Sep 26, 2017 at 7:18 AM, Patrik Nordwall wrote: > If the names are StreamSupervisor- I think it can be that a new > Materializer is created for each request. I don’t know if that is done by > your application or by Akka Http. Does that ring any bells? Do you have any > creation of stream

[akka-user] Re: Materialize future after content negotiation

2017-09-26 Thread johannes . rudolph
Hi Mantis, you are right, `Marshaller.withFixedContentType` is a bit restricted in that regard. Fortunately, it is only a shortcut for simpler cases and the full asynchronous functionality is available for marshallers. Try something like Marshaller[Iterator[Data], HttpResponse] { implicit ec =

[akka-user] Re: Materialize future after content negotiation

2017-09-26 Thread johannes . rudolph
Oops, one should read the whole question before answering... Just saw that you already tried that. Unfortunately, it seems that this is indeed a shortcoming of the current model. I guess with a bit of fiddling you could try making all of those marshallers marshal to `Future[HttpResponse]` inste

Re: [akka-user] Re: Akka Http memory leak suspect

2017-09-27 Thread Johannes Rudolph
ied that but there are 2 places where declare the > materializers. Both are declared as vals. I will verify the number of > materializer instances on my heap-dump to confirm. > > On Tue, 26 Sep 2017 at 13:24 Johannes Rudolph com> wrote: > >> On Tue, Sep 26, 2017 at 7:18 A

Re: [akka-user] Tunning default dispatcher seems to have no effect

2017-09-28 Thread johannes . rudolph
Hi Kilic, Try looking at stack traces during the busy periods (e.g. use `jstack` on the command line to gather some), that should give you a clue what's going on. In the picture you sent in your first email there were actually only 8 regular pool threads. Are there times where more is going on

[akka-user] Re: GraphStageActor and ActorGraphInterpreter

2017-09-28 Thread johannes . rudolph
Hi Unmesh, On Wednesday, September 27, 2017 at 3:01:24 PM UTC+2, Unmesh Joshi wrote: > > I was trying to go through the code to understand how GraphStages receive > actor messages. I see that GraphStageActor is a actor not created like > normal actors. I looks like all the messages to GraphStage

Re: [akka-user] Ask a lot of Actors inside an Iteration in an Akka-Scheduler and wait for reply of everyone results in a stop of actor-system and webserver

2017-09-28 Thread johannes . rudolph
Hi Simon, as Johan said, you shouldn't use `get` to wait for the result of future. This just synchronously blocks the thread from doing any other useful work. Instead, you can asynchronously handle the result of the future once it is available. Because it is so common, we have a pattern for thi

[akka-user] Re: GraphStageActor and ActorGraphInterpreter

2017-09-28 Thread johannes . rudolph
Correct, it will limit parallelism. I usually see the streams infrastructure more as a control channel that makes sure that data flows correctly. These kind of control things shouldn't require much overall CPU share so it should not matter so much. If you want to do CPU-intensive work you need

[akka-user] Re: GraphStageActor and ActorGraphInterpreter

2017-10-04 Thread johannes . rudolph
Cool, thanks for sharing, nice stuff :) We know, btw., that some pieces of the architecture are not completely optimal but are consequences of the history of the projects. E.g. it could make sense to write a streams-only implementation of the TCP layer instead of putting it on top of the actor

Re: [akka-user] Performance of Akka-Http 2.5.4

2017-11-13 Thread johannes . rudolph
I missed this post before. I'd like to add another point. Akka Http hasn't been performance tested on a 40 core machine. The high idle CPU percecntage means that either Akka / Akka Http is not configured correctly for this amount of cores or that there are actual contention issues at these lev

Re: [akka-user] Streaming proxy/tunel on top of akka-http

2017-11-13 Thread johannes . rudolph
I tried your code and it doesn't OOM for me. Have you tried it outside of a test suite? It might be that the test infrastructure is collecting all the data when you use something as `reponse.entity`. If that doesn't help, try capturing a heap dump on OOM and see where the memory is spent. Joh

[akka-user] Re: Akka-HTTP compilation error using custom requireParam() directive

2017-11-13 Thread johannes . rudolph
Hi Evgeny, you discovered one of the reasons for the magnet pattern. If you use `requireParam("param".as[Int]) { abc => ... }` then the `{ abc => }` block is mistaken as the implicit argument of `requireParam`. So, either you are ok with that and require users to use extra parentheses (`(requir

Re: [akka-user] Spray->Akka-Http Migration - seeing high 99th percentile latencies post-migration

2017-11-16 Thread johannes . rudolph
Hi Gary, did you find out what's going on by now? If I understand correctly, you get latency spikes as soon as you use the `entity[as[String]]` directive? Could you narrow down if there's anything special to those requests? I guess you monitor your GC times? Johannes On Wednesday, November 1,

Re: [akka-user] Spray->Akka-Http Migration - seeing high 99th percentile latencies post-migration

2017-11-16 Thread johannes . rudolph
I wonder if you could start a timer when you enter the trace block and then e.g. after 200ms trigger one or multiple stack dumps (using JMX or just by printing out the result of `Thread.getAllStackTraces`). It's not super likely that something will turn up but it seems like a simple enough thing

[akka-user] Akka Http 10.1.0-RC1 Released

2017-12-22 Thread johannes . rudolph
uch! commits added removed 4448414860 Arnout Engelen 321196 613 Johannes Rudolph 22 7202789 Josep Prat 3 122 349 Jonas Fonseca 2 28 17 Pavel Boldyrev 1 6 4 Johan Andrén 1 128 4 Martynas Mickevičius 1 111

Re: [akka-user][deprecated] Re: [akka-user] Spray->Akka-Http Migration - seeing high 99th percentile latencies post-migration

2018-10-09 Thread Johannes Rudolph
That the entity directive is part of the picture could be a hint that indeed streaming requests might be the cause of this. In spray, there was no request streaming enabled by default and the engine just collected the complete stream into a buffer and dispatched it to the app only after everything

[akka-user][deprecated] Re: Akka SSLSession leak when running Akka with native TLS

2018-11-22 Thread johannes . rudolph
Hi Sean, thanks for the comprehensive report. What do you mean with a native vs non-native TLS server? Is the example app for the "native TLS" server? Johannes On Wednesday, November 21, 2018 at 6:26:09 PM UTC+1, Sean Gibbons wrote: > > Hi all, > > I have been working with a native TLS Akka HTT

Re: [akka-user][deprecated] Re: Akka SSLSession leak when running Akka with native TLS

2018-11-22 Thread Johannes Rudolph
I see. Thanks. With the provided code I couldn't reproduce the issue at least in the quick tests I did. Could you run jmap -histo:live on the command line when some memory has accrued and send the output here (or in private)? Johannes On Thu, Nov 22, 2018 at 1:16 PM Sean Gibbons wrote: > An

[akka-user][deprecated] Re: Aggregating millions of records using Akka-Streams

2019-08-08 Thread Johannes Rudolph
Reposted here: https://discuss.lightbend.com/t/aggregating-a-million-records-using-akka-streams/4781 On Thursday, August 8, 2019 at 11:02:13 AM UTC+2, Aditya pavan Kumar wrote: > > I am running a simulation which generates a million records every second. > I’m writing them to Kafka and reading th

[akka-user] Re: [scala-user] Re: Akka Streams & HTTP 1.0-RC1 Announcement

2015-04-27 Thread Johannes Rudolph
Hi Anton, On Sun, Apr 26, 2015 at 9:31 PM, Anton Kulaga wrote: > What about Websockets, as I understood they are supported in RC-1 but is > there anything in documentation that explains how to use them in akka-http? Server-side websocket support is included in RC1 but not yet documented. An exam

[akka-user] Re: [scala-user] Re: Akka Streams & HTTP 1.0-RC1 Announcement

2015-04-27 Thread Johannes Rudolph
; Akka Streams with Java8! >> >> >> Akka Streams with Scala! >> >> >> We’d like to thank all of you for testing and for providing feedback on >> our progress. >> >> >> Happy hAkking! >> >> -- >> >> Patrik N

Re: [akka-user] Akka Http Gzip Json

2015-05-07 Thread Johannes Rudolph
gt;>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>>> Check the FAQ: >>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>>> Search the archive

[akka-user] Re: Akka Streams Merge variance

2015-05-18 Thread Johannes Rudolph
On Saturday, May 16, 2015 at 3:47:51 PM UTC+2, Oliver Winks wrote: > > If you look at the implementation of Merge you find that it is a generic > type, but without any defined variance: > > classMerge[T] extends Graph >

[akka-user] Re: akka-http-RC3 can't set the content type

2015-06-12 Thread Johannes Rudolph
On Friday, June 12, 2015 at 12:30:16 AM UTC+2, ke...@aptoma.com wrote: > > I think ContentType is modeled directly on HttpEntity. Try a combination > of the mapResponseEntity-directive and the .withContentType-method on > HttpEntity. > To expand on this: In spray/akka-http some headers are tr

[akka-user] Re: akka-http-testkit responseAs cannot unmarshal the body

2015-06-12 Thread Johannes Rudolph
On Thursday, June 11, 2015 at 12:04:28 PM UTC+2, yar@gmail.com wrote: > > So if the response has content type text/plain, why the heck does the > testkit try to unmarshal it with an application/json unmarshaller and how > to point it in the right direction? > Good question. Do you use one of

Re: [akka-user] Re: Too many scan calls when profiling my akka application

2015-06-12 Thread Johannes Rudolph
On Sunday, June 7, 2015 at 1:42:52 PM UTC+2, Akka Team wrote: > > The scan method of FJP will very likely be your top method unless you have > very favorable load patterns for FJP. I wouldn't worry about it, unless you > see a performance problem with your application. > > 4% is indeed not much

[akka-user] Re: Akka-http 1.0-RC4 on android 5.0.1 : that rocks !

2015-07-10 Thread Johannes Rudolph
Hi Alain, On Friday, July 3, 2015 at 9:05:50 AM UTC+2, alain marcel wrote: > > Following code is a server realized with akka-http on android that serves > files. > For this to work, just call new ServerForDownloadFile() in an android > AsyncTask. > Thanks for sharing! Any reason you are using a

[akka-user] Re: How to download large file with Akka-http V1.0-RC4 ?

2015-07-10 Thread Johannes Rudolph
Hi Alain, On Thursday, July 2, 2015 at 10:39:27 AM UTC+2, alain marcel wrote: > > If I replace Play server with Akka-http server (see source bellow), then I > can download large file (about 650Mb). > That's probably because you use chunked transfer encoding in your Akka HTTP server for which th

[akka-user] Re: akka-http respond with result of an actor call (java-api)

2015-07-10 Thread Johannes Rudolph
Hi, in the Java API there are currently two ways to deal with Futures. You can use `RequestContext.completeWith` to "transform" a `Future` into a `RouteResult`. Or, if you use the reflective `handleWith` directive you can point it to a method that returns a `Future` instead of a `RouteResult`.

[akka-user] Re: akka-http-testkit route produces "internal service error" but live route works (?!?)

2015-07-12 Thread Johannes Rudolph
Hi Austin, On Saturday, July 11, 2015 at 11:22:44 AM UTC+2, Austin Guest wrote: > > (1) The test for the "hello" route fails because the actual response is an > internal server error. (Despite the fact that this isn't true running the > live code!) And I get the following error messages: > That

[akka-user] Re: [akka-http] Splitting an incoming Source stream to two Sources

2015-07-13 Thread Johannes Rudolph
Hi Michael, leaving aside the API for a moment, the question is how the streams should behave. The actual live streams are not immutable but are running things that need to receive data from the original request and send this data out with another request. (I haven't completely understood in wh

[akka-user] Re: SslTls example

2015-07-14 Thread Johannes Rudolph
Hi Mathias, there may not be any good examples yet. On the server side you need to supply an `HttpsContext` to the `Http.bind()` method with all the SSL settings. On the client side there are either HTTPS variants like `Http.newHostConnectionPoolTls` or if you use the highest-level API you can

[akka-user] Re: akka streams - for comprehension counterpart

2015-07-14 Thread Johannes Rudolph
Hi Leslie, On Tuesday, July 14, 2015 at 1:38:02 PM UTC+2, leslie...@googlemail.com wrote: > > When programming with functions this kind of issue is solved quite > elegantly by using a for comprehension: > > Not a solution but a comment. In a for comprehension with usual types (Future/Option/Eit

[akka-user] Re: Connecting over Https using Akka-Http client

2015-07-15 Thread Johannes Rudolph
Hi Jeroen, is this a virtual host you are connecting against? This may hint towards the client not sending the TLS SNI extension correctly which could be a bug in akka-http or due to an old JDK version on your client. Which JDK version do you use? https://en.wikipedia.org/wiki/Server_Name_Indi

Re: [akka-user] Re: Server attacked. What could have been done?

2015-07-20 Thread Johannes Rudolph
Hi, On Sunday, July 19, 2015 at 10:43:07 PM UTC+2, Adam Shannon wrote: > > I have some logs of this happening to me as well. I'm running on EC2 in > us-east-1. I've got nginx and elb in front of these akka-http instances. > Here's a few stack traces from the instances as well as nginx access log

[akka-user] Re: Implicit value issue for FromRequestUnmarshaller in akka-http with spray-json

2015-07-24 Thread Johannes Rudolph
I agree, this is more confusing than necessary. I created https://github.com/akka/akka/issues/18064 to improve the situation. Johannes On Thursday, July 23, 2015 at 3:27:07 PM UTC+2, Kabir Idris wrote: > > Hi all, Im facing the same error in akka-http 1.0 > > > > On Monday, March 16, 2015 at 12

[akka-user] Re: Akka-http websockets connection fails 95% of the time

2015-07-24 Thread Johannes Rudolph
Hi Luc, I'd like to get to the bottom of this problem to make sure we find the problem if there's one in akka-http. Could you provide an executable reproduction? Johannes On Thursday, July 23, 2015 at 3:05:29 PM UTC+2, Luc Klaassen wrote: > > Hello all, > > I'm having trouble setting up a basi

[akka-user] Re: memory issue in akka application

2015-07-24 Thread Johannes Rudolph
Hi Somak, this seems like an Actor isn't keeping pace with its incoming messages. You can try to follow references from the mail box to the actor instance to find out which actor it is. As you are talking about log message my guess would be that your logging actor is too slow writing logs to di

[akka-user] Re: Akka-http websockets connection fails 95% of the time

2015-07-25 Thread Johannes Rudolph
Thanks Luc. I'll have a look next week. On Saturday, July 25, 2015 at 9:42:42 AM UTC+2, Luc Klaassen wrote: > > I've pushed the repository to github at > https://github.com/Luckl/AkkaHttpWebsockets > > Luc > > Op vrijdag 24 juli 2015 10:07:53 UTC+2 schreef

[akka-user] Re: Can PathMatcher match string as a path parameter?

2015-07-29 Thread Johannes Rudolph
Hi John, yes, this is called PathMachers.segment() (to be consistent with the Scala side). It may make sense to create an alias or have another look at the names in general to see if they can be made more consistent. Johannes On Wednesday, July 29, 2015 at 7:06:16 AM UTC+2, john@gmail.com

[akka-user] Re: How do I create an Unmarshaller / Marshaller in java?

2015-08-04 Thread Johannes Rudolph
Hi John, On Monday, July 27, 2015 at 11:00:57 AM UTC+2, john@gmail.com wrote: > > or am I supposed to use Marshallers.toEntity() but this seems quite a lot > of work > Yes, to create custom marshallers you would need to use one of the methods in `Marshallers`, to create custom unmarshaller

[akka-user] Re: Akka HTTP back-pressured connections

2015-08-10 Thread Johannes Rudolph
Thanks for the reminder Hunor and Chris. The comment is indeed a left-over from previous versions. Since a while the number of concurrently accepted connections for one `bindAndHandle` invocation can now be configured by setting `akka.http.server.max-connections`. I added a commit to PR 18143

[akka-user] Re: akka streams 1.0 and http routing - in java

2015-08-11 Thread Johannes Rudolph
Hi paweł, you are on the right track. You need to provide the data as a `Source`. Then, you can create a response like this: Source data = ... HttpResponse response = HttpResponse.create().withEntity(HttpEntities.createChunked(contentType, data)); If you know the length of the data (i.e. the

[akka-user] Re: Can't find akka.http.javadsl.testkit.JUnitRouteTest?

2015-08-11 Thread Johannes Rudolph
Hi john, it seems that this is a bug that we thought to be fixed but which has reappeared. I filed it as https://github.com/akka/akka/issues/18178 Johannes On Tuesday, August 11, 2015 at 7:08:44 AM UTC+2, john@gmail.com wrote: > > Ok I realized that it is in the release-2.3-dev branch. > >

Re: [akka-user] Re: akka-http 1.0 RC and SSL and Java

2015-08-13 Thread Johannes Rudolph
Hi Rob, here's some recent testing code that creates and uses certificates that are signed by a custom CA. It doesn't require fiddling with Java's `keytool`. The instructions should work similarly for proper certificates in which case you leave out the CA and "Create server certificate" steps a

Re: [akka-user] Re: Looking for Code example to create akka.http.javadsl.testkitTestResponse

2015-08-13 Thread Johannes Rudolph
Yes, good point. https://github.com/akka/akka/pull/18201 On the other hand, I wonder why you need to implement TestResponse, john? You should only ever need to implement it if you want to support another kind of testing framework. Is that what you are trying to achieve? To write tests with JUn

[akka-user] Re: [akka-http][websocket] how to connect two independent WebSocket streams - include an actor as bridge between streams?

2015-08-14 Thread Johannes Rudolph
Hi Michael, here's an idea how it could work without actors: * when the first client connects, create two yet Publisher/Subscriber pairs and wrap them into a Flow[Message, Message] while crossing the links, pass one of the flows as the websocket handler * when the second client connects pass

[akka-user] Re: Specs2RouteTest equivalent for Akka HTTP

2015-08-18 Thread Johannes Rudolph
Hi, specs2 testing support hasn't been ported from spray yet. The basic issue is that akka modules weren't allowed to depend on Scala dependencies because of bootstrapping issues when releasing a new Scala version that includes Akka. I'm not completely sure if this is still the most recent inf

Re: [akka-user] [akka-streams] Http source design question?

2015-08-20 Thread Johannes Rudolph
Hi john, AFAIU your question was not about existing code in akka-http but how to put a queue before a superpool, right? And no, putting a blocking queue inside of an Iterator is not likely the best solution because it will spend one thread permanently that is blocking in `queue.take` almost all

[akka-user] Re: [akka-http-experimental-1.0] More Fine granular logging of accepted connections needed

2015-08-22 Thread Johannes Rudolph
Yes, the solution is not to log at DEBUG level. :) Johannes On Saturday, August 22, 2015 at 1:32:36 PM UTC+2, Simon Schäfer wrote: > > I use akka-http to build a webservice. This works great so far, I just > don't like the noise of logging. The webservice not only answers to > websockets reques

Re: [akka-user] Re: akka-http and selenium

2015-08-24 Thread Johannes Rudolph
Hi Rafał, On Monday, August 24, 2015 at 3:49:57 PM UTC+2, Rafał Krzewski wrote: > > Oh, I see. OneServerPerSuite / OneServerPerTest traits must be really > handy. Providing similar helpers for testing akka-http would be hard, > because Play is a framework that mandates a well defined entry point

Re: [akka-user] [akka-stream-experimental-1.0] How to reuse akka.stream.scaladsl.Tcp connections?

2015-08-25 Thread Johannes Rudolph
Hi Simon, I think there are two conceptual difficulties you need to tackle: The first is the problem which you describe with infinite / finite streams which is actually more one of the "traditional" (= actor based) push-style asynchronous programming versus the "new" [*] pull-style of reactive/

[akka-user] Re: [akka-http] Custom Directives - Scala

2015-08-27 Thread Johannes Rudolph
Hi, use the combinators of the Directive class to make custom directives. E.g. extractClientIp.flatMap { ip => val cond = // something if (cond) pass else reject(...) } or if you want to return a value, i.e. create a Directive1[T], use `provide` instead of `pass`. HTH Johannes On Tuesday,

Re: [akka-user] Akka-http 1.0-RC4 ssl session info/cert info accessible?

2015-08-27 Thread Johannes Rudolph
Hi, this is now tracked as https://github.com/akka/akka/issues/18349. Johannes On Thursday, August 27, 2015 at 4:03:26 PM UTC+2, Mark van Buskirk wrote: > > Hi Dr. Kuhn, > How is this issue progressing, is there a ticket I can watch for updates? > It's hard to use ssl without being able to get

[akka-user] Re: Debugging `jstack PID` on Non-Completing Future?

2015-09-15 Thread Johannes Rudolph
Is the code on the stack of any thread which does the DB query? If not google for thread starvation :) Johannes On Sunday, September 13, 2015 at 6:22:25 PM UTC+2, Kevin Meredith wrote: > > My DB calls, which returns a `Future[...]` is not completing in time for > my remote spray service's time-

[akka-user] Re: Akka-Http handle Multipart formData

2015-09-18 Thread Johannes Rudolph
Hi, the approach that Konrad showed will work, however, the problem with the suggested approach is that it will load the complete file into memory which limits the number of concurrent uploading requests severely. Any multipart entity is a natural stream of parts so your original approach is a

Re: [akka-user] How do I attach multiple subscribers to a stream using fanoutPublisher, actorSubscriber and FlowGraph.partial?

2015-10-05 Thread Johannes Rudolph
Hi Abhijit, from a quick glance to your code it seems that you are broadcasting an HttpResponse and then access its `entity.dataBytes` in several branches. This is not supported. You need to put the broadcast behind the `entity.dataBytes` and not behind the stream of responses. We prepared a s

Re: [akka-user] How do I attach multiple subscribers to a stream using fanoutPublisher, actorSubscriber and FlowGraph.partial?

2015-10-06 Thread Johannes Rudolph
? Please pardon me if this is obvious > as I'm just learning. > > I'm using Source.single(httpRequest), not fanoutPub. The example you > pointed me to use fanout, which I intend to study but not apply in this > case. > > Regards, > Abhijit > > > On Mond

[akka-user] Re: akka-http tcp proxy protocol support

2015-10-06 Thread Johannes Rudolph
Hi Julian, see scaladsl/Http.scala for all the glue code between TCP and HTTP. The hard thing will be getting the metadata from the proxy implementation into the requests but I guess putting it in a @volatile var after reading it and mapping each request would make for a simple solution. In ca

Re: [akka-user] Marshalling FormData to multi-part mime with one field per part

2015-10-15 Thread Johannes Rudolph
On Wednesday, October 14, 2015 at 7:17:14 PM UTC+2, Eric Swenson wrote: > > AWS/S3 doesn’t want the form data parts, nor the whole HTTP POST to be > chunked. So those entities have to be made “strict”. So I had to make each > MIME part be strict, as well as the final Multipart.FormData entity.

Re: [akka-user] Fusing in akka-streams 2.0

2015-12-23 Thread Johannes Rudolph
On Tuesday, December 22, 2015 at 8:37:08 PM UTC+1, Adam Warski wrote: > > Having conflate only makes sense if there's some non-fused component out > there I guess, but you are correct of course that it can be fused with > things before/after it. > I thought a bit about this. Actually, it is not

[akka-user] Re: Memory leak with akka-streams 2.0.1?

2016-01-08 Thread Johannes Rudolph
Hi Leo, when do you want the WS connections to be closed? I probably miss something but I don't see code on either side which would attempt to shutdown the connection. Johannes On Friday, January 8, 2016 at 2:02:12 PM UTC+1, Clelio De Souza wrote: > > > Hi, > > We use Akka Streams for streamin

Re: [akka-user] Akka HTTP File Not Found Exception

2016-01-19 Thread Johannes Rudolph
I see several issues here: * Entity stream failures don't go through the routing layer exception handling code. It also wouldn't make any sense because we are already sending out the response here and we cannot change the response at that time any more. We should document that somewhere. * In

[akka-user] Re: HTTP header 'UpgradeToWebSocketResponseHeader: ' is not allowed in responses

2016-01-28 Thread Johannes Rudolph
Hi Shayan, this seems to be a bug. I filed it here: https://github.com/akka/akka/issues/19639 The functionality shouldn't have changed, though, so you can safely ignore this output. Johannes On Wednesday, January 27, 2016 at 9:19:46 AM UTC+1, sha...@gearzero.com wrote: > > Team, > > I am tr

[akka-user] Re: Dynamically add HTTP Routes

2016-02-01 Thread Johannes Rudolph
Hi Ubaldo, yes, that's certainly possible. A `Route` is just a regular Java object, so you can choose a new `Route` to do the processing for every new request. You just need to make sure that you switch out the Route in a thread-safe way. E.g. val routeHolder = new AtomicReference[Route](init

[akka-user] Re: Dynamically add HTTP Routes

2016-02-01 Thread Johannes Rudolph
On Monday, February 1, 2016 at 11:28:58 AM UTC+1, Johannes Rudolph wrote: > > Http.bindAndHandle(currentRoute(), ...) > Actually, I just figured that this won't work, because currentRoute() will only be evaluated once. You will need a simple wrapper which is evaluated anew for

[akka-user] Re: ('=') sign still not allowed in query string even with Uri.ParsingMode.Relaxed? (akka/akka#18479)

2016-10-17 Thread johannes . rudolph
Great, thanks Richard for tackling this and André for the explanations! On Saturday, October 15, 2016 at 10:05:59 PM UTC+2, Richard Imaoka wrote: > > Sorry I had read the full discussion but I think I didn't correctly > understand what is allowed in Akka, and what is RFC 3986 compliant. > Now it'

Re: [akka-user] How to handle web socket message to upload base64 encoded string of 10mb file

2016-10-17 Thread johannes . rudolph
Thanks Rafał for these explanations. Just a small correction: On Tuesday, October 11, 2016 at 3:58:09 PM UTC+2, Rafał Krzewski wrote: > > An alternative solution would be looking up websocket buffering settings > and jacking it up enough to receive all messages as Strict :) > Unfortunately, no,

[akka-user] Re: [akka-stream] Detect failure in MergeHub

2016-10-18 Thread johannes . rudolph
Hi Victor, good point. I think the Scaladoc is wrong there. Could you raise an issue at akka/akka? Johannes On Tuesday, October 18, 2016 at 2:28:14 PM UTC+2, Victor wrote: > > Hi, > > It's written in the ScalaDoc of the *MergeHub.source* method that: > > If one of the inputs fails the Sink, the

[akka-user] Re: Fetching remote file via akka-http singleRequest results in extremely high allocation rate

2016-10-19 Thread johannes . rudolph
Hi Vladyslav, this sounds like a worst-case scenario for the Framing stages: 45M lines and each line 2-3 characters long will put a lot of pressure on the streams infrastructure and the framing stage. It might still make sense to benchmark and optimize the Framing stage. One optimization could

[akka-user] Re: Fetching remote file via akka-http singleRequest results in extremely high allocation rate

2016-10-19 Thread johannes . rudolph
Using mapConcat instead may even be faster ;) On Wednesday, October 19, 2016 at 5:05:22 PM UTC+2, vladysla...@rtsmunity.com wrote: > > Okay, so the issue was really in Framing performance. Changing Framing > stage for > > flatMapConcat(chunk -> Source.from(Arrays.asList(chunk.split("\n". > >

  1   2   >