Hi,
I seem to be encountering a reoccurring problem when using Akka Streams and
haven't found the right client API to use to solve it.
The problem usually translates into:
- I have some possibly infinite Source
- I want to split it into multiple inner Sources based on some condition
such as delimiter, count, or whatever
- I then want to create a Sink for each inner Source dynamically and run
each inner Flow
- I want the backpressure, errors, completion, cancellation and stuff
like that to be shared between the outer Flow and the inner Flow
There are a few things come close but all seem to be for a slightly
different use case, such as:
- splitAt / splitWhen
- lazy / lazyInit
- the various hubs
Here is a concrete example:
1. Reading a file in 8KB parts
2. Splitting the first 625 parts into a separate stream as a "chunk"
3. Create a HTTP source that has a URL containing the chunk number
4. Send the 625 parts to that source
5. Take the next chunk from step 2
An attempt using a SubFlow looks like:
val chunkSize = fileResponse.chunkSize
val partsPerChunk = chunkSize / partSize
val counts = Source {
for {
chunk <- 1 to Int.MaxValue
part <- 1 to partsPerChunk
} yield (chunk, part)
}
val source = FileIO.fromPath(filePath, partSize)
.zip(counts)
.splitAfter({ next =>
val (_, (_, part)) = next
part == partsPerChunk
})
This is quite nice but then there doesn't seem to be a way of getting to
the inner Flow even if I were to create a custom Sink.
It would be really awesome if SubFlow had a function like:
def to[M](f: Out => Graph[SinkShape[Out], M]): C
(although probably without the materialised value since there would be
multiple).
Is there something obvious that I am missing?
Cheers,
Jason
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
