Hi,
there's really not too much to add to the gist from the first post: this
method is called once, almost right in the main class, to fetch some
resource. For now I managed to work this issue around, but I thought that
working on rope-like ByteStrings should be faster than on array-backed Java
Using mapConcat instead may even be faster ;)
On Wednesday, October 19, 2016 at 5:05:22 PM UTC+2,
vladysla...@rtsmunity.com wrote:
>
> Okay, so the issue was really in Framing performance. Changing Framing
> stage for
>
> flatMapConcat(chunk -> Source.from(Arrays.asList(chunk.split("\n".
>
>
Okay, so the issue was really in Framing performance. Changing Framing
stage for
flatMapConcat(chunk -> Source.from(Arrays.asList(chunk.split("\n".
actually made it reasonably fast.
On Wednesday, October 19, 2016 at 3:25:50 PM UTC+2,
vladysla...@rtsmunity.com wrote:
>
> Hello,
>
> I have
Hi Vladyslav,
this sounds like a worst-case scenario for the Framing stages: 45M lines
and each line 2-3 characters long will put a lot of pressure on the streams
infrastructure and the framing stage. It might still make sense to
benchmark and optimize the Framing stage. One optimization could
Hi Vladyslav,
firstly, no reason to get desperate :-)
Glad you did some profiling, it would be super helpful next time around if
you share more details about both your code an profile you're observing.
It's hard to give advice what can be improved when given a blank canvas (no
idea what your code l
After some playing with JMC and async boundaries it turned out the hottest
methods are related to ByteString splitting and delimiting (for example,
scala.collection.IndexedSeqOptimized$class.sameElements(IndexedSeqOptimized,
GenIterable) takes 73-75% of processor time). The input file is 45M li
Akka and Akka-HTTP version is 2.4.11.
On Wednesday, October 19, 2016 at 3:25:50 PM UTC+2,
vladysla...@rtsmunity.com wrote:
>
> Hello,
>
> I have the following problem. I'm trying to fetch a file via HTTP in
> streaming fashion, processing it line by line (code:
> https://gist.github.com/anonymou
