Chunk size and entity size are 2 different configuration settings.
You can easily set bigger chunk size for all connections in your
application config see akka.http.parsing.max-chunk-size.
If you want to do it programmatically i think right now it is only possible
via connection:
private final val requestResponseSink: Sink[HttpRequest, Future[HttpResponse]]
= {
val connectionSettings = ClientConnectionSettings(system)
val parserSettings =
connectionSettings.parserSettings.withMaxChunkSize(100000000)
Http().outgoingConnectionHttps(
host = host,
settings = connectionSettings.withParserSettings(parserSettings)
).toMat(Sink.head)(Keep.right)
}
On Thursday, November 9, 2017 at 11:13:45 PM UTC+1, [email protected] wrote:
>
> Hello,
>
> I have API response coming as a JSON string where Server won't support
> chunked response. I'm using below sample code to parse the JSON using
> *Jackson
> parser*.
>
> type Result = Map[String, List[(String, JValue)]]
>
> private def apiResult(resp : HttpResponse): Future[Result] = {
> Unmarshal(resp.entity.withoutSizeLimit).to[String] map {
> jsonResp =>
> getResultMap(jsonResp) // JSON parsing using Jackson
> }
> }
>
>
>
> I'm hitting API for for 10k different results. Some of the JSON results
> are very large, I'm finding below error ultimately (same error encounters
> even after removing *withoutSizeLimit *in response entity) :
>
> [ERROR] [11/09/2017 11:33:58.337] [myactor-threadpool-dispatcher-7]
> [TaskInvocation] HTTP chunk size exceeds the configured limit of 1048576
> bytes
> akka.http.scaladsl.model.EntityStreamException: HTTP chunk size exceeds
> the configured limit of 1048576 bytes
> at
> akka.http.impl.engine.parsing.HttpMessageParser$$anonfun$2.applyOrElse(HttpMessageParser.scala:329)
> at
> akka.http.impl.engine.parsing.HttpMessageParser$$anonfun$2.applyOrElse(HttpMessageParser.scala:327)
> at
> akka.stream.impl.fusing.Collect$$anon$2.$anonfun$wrappedPf$1(Ops.scala:212)
> at
> akka.stream.impl.fusing.SupervisedGraphStageLogic.withSupervision(Ops.scala:173)
> at akka.stream.impl.fusing.Collect$$anon$2.onPush(Ops.scala:214)
> at
> akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:499)
> at
> akka.stream.impl.fusing.GraphInterpreter.processEvent(GraphInterpreter.scala:462)
> at
> akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:368)
> at
> akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:571)
> at
> akka.stream.impl.fusing.GraphInterpreterShell$AsyncInput.execute(ActorGraphInterpreter.scala:457)
> at
> akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:546)
> at
> akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:728)
> at
> akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:743)
> at akka.actor.Actor.aroundReceive(Actor.scala:517)
> at akka.actor.Actor.aroundReceive$(Actor.scala:515)
> at
> akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:653)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
> at akka.actor.ActorCell.invoke(ActorCell.scala:496)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
> akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
> akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
>
> How can i solve my problem?
> Do you think, JSON Streaming can help here for me (Entity size might again
> be an issue as size is restricted to certain limit) ?
>
>
> Regards,
> Aditya Borde
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
