What's the layout of the file with protobuf objects? On Sat, Jan 30, 2016 at 2:57 PM, Kyrylo Stokoz <[email protected]> wrote:
> Hi All, > > I`m trying to learn more about akka streams and have on first sight > trivial task. > I have a very large file of protobuf encoded objects i want to stream them > using akka http streams. > > So i would like to create following flow: > > Read file(s) -> parse protobuf into seq(domain objects) -> transform each > object -> render result into json / other format -> send to receiver. > > I start with creating source from each file i needed to render using > FileIO.fromFile(file). > And i`m stuck on 2nd step: to parse protobufs i have 3 options: > 1. Obj.parseFrom(inputstream): Obj > 2. Obj.parseFrom(byte []): Obj > 3. Obj.streamFromDelimitedInput(inputstream): Stream[Obj] > > So far i was not able to find a way how i can properly convert > Source[ByteString, _] to Source[Obj, _]. > > Is there any way to create above mentioned flow? ideally without loading > all data in memory. > > So far i tried to play with: > 1. Framing not success, i was not able to find correct delimiter to parse > protobuf > 2. Create Stream[Obj] using (Obj.streamFromDelimitedInput(input stream): > Stream[Obj]) and later create Source from Stream - but this gives very > poor performance results. > > Thanks in advance for your help, > Kyrylo > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Cheers, √ -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
