If it helps, a simple custom framing stage could look like this:
Flow[ByteString].statefulMapConcat[MyThing] { () =>
var buffer = ByteString.empty
in => {
val (things, remaining) = parseThings(buffer ++ in)
buffer = remaining
things
}
}
def parseThings(bytes: ByteString): (Vector[MyThing], ByteStrings) = { ... }
where parseThings finds as many things as it can from the available bytes,
returning the found things and the unused bytestring.
Other state can also be kept alongside the buffer if you need that as well.
On Wed, Apr 20, 2016 at 2:41 PM Magnus Andersson <[email protected]>
wrote:
> Solved it by doing my own version of something similar to the framing
> classes.
>
> Perhaps a custom stage is what I should use for problems like a protocol
> with variable length payloads, acking on application level, messages
> received in a specific order?
>
> Doing it with flows or graphs seemed to create tons of boilerplate and was
> not really easy to follow along with. I'd really like to do some type of
> state machine for this, mixing Akka FSM and streams. Do you know of any
> idiomatic examples doing something like that?
>
> /Magnus
>
> Den sön 10 apr. 2016 08:51Endre Varga <[email protected]> skrev:
>
>> Hi Magnus,
>>
>>
>>
>> On Sat, Apr 9, 2016 at 11:00 PM, Magnus Andersson <[email protected]>
>> wrote:
>>
>>> Hi
>>>
>>> I'm implementing a TCP protocol (Lumberjack2/Beats) in Akka Streams. It
>>> is a streaming application level protocol where each messages varies data
>>> length. In Akka streams I get ByteStrings fed through the stream but I
>>> can't find any information about how these correlate with TCP packets sent
>>> over wire.
>>>
>>
>> Because TCP is a stream protocol. Hence no sent buffers correlate with
>> TCP frames at all, TCP is free to rechunk them as it sees fit.
>>
>>
>>> Without that knowledge I have to create a buffer for partial message and
>>> handle all theoretical cases that could occur when ByteString does not
>>> correlate 1:1 with TCP payloads. The resulting implementation becomes quite
>>> hairy.
>>>
>>> 1. What are the guarantees around correlation between Akka ByteStrings
>>> and TCP packets?
>>>
>>
>> Nothing. This is the nature of TCP and has nothing to do with Akka.
>>
>>
>>> 2. Examples of TCP protocols implemented with Akka streams that uses
>>> dynamic framing examples?
>>>
>>
>> Check the akka.io Framing classes.
>>
>> -Endre
>>
>>
>>>
>>> /Magnus
>>>
>> --
>>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>>> >>>>>>>>>> Check the FAQ:
>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >>>>>>>>>> Search the archives:
>>> https://groups.google.com/group/akka-user
>>> ---
>>>
>> You received this message because you are subscribed to the Google Groups
>>> "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>
>>
>>> To post to this group, send email to [email protected].
>>> Visit this group at https://groups.google.com/group/akka-user.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> ---
>>
> You received this message because you are subscribed to a topic in the
>> Google Groups "Akka User List" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/akka-user/o8djg3OKV3Q/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> [email protected].
>
>
>> To post to this group, send email to [email protected].
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/
> >>>>>>>>>> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
