[protobuf] Handling streaming in Python

2010-11-30 Thread Dan
Hello,

Does anyone have or can point me to an example of handling a stream of
protobuf messages in python?

Specifically what I'm trying to do is parse a stream that looks like
the following

HeaderSize PB int32
HeaderMessage
MessageSize PB int32
Message
...
MessageSize PB int32
Message

The header just gives the list of field names and their types (int,
double, or string, that's it).  There are no nested messages or
anything like that in the streamed messages.

Ultimately what I would like to do is put the values from a message
into a list and pass it over to some other code that writes the data
into another format (csv, xml, json, etc.)

I've got this handled wonderfully using CodedInputStream in the Java
implementation but now I need to do it in python and I don't see
anything equivalent.

-- 
You received this message because you are subscribed to the Google Groups 
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.



Re: [protobuf] Handling streaming in Python

2010-11-30 Thread Gregory Szorc
You can load the _DecodeVarint and _VarintEncoder modules from 
google.protobuf.internal.{decoder,encoder} if you want to risk using an 
internal interface. See 
https://github.com/indygreg/zippylog/blob/master/lib/py/zippylog/stream.py 
for an example (although I can't guarantee the Python code in that project 
works at this time).


Greg

-Original Message- 
From: Dan

Sent: Tuesday, November 30, 2010 4:01 PM
To: Protocol Buffers
Subject: [protobuf] Handling streaming in Python

Hello,

Does anyone have or can point me to an example of handling a stream of
protobuf messages in python?

Specifically what I'm trying to do is parse a stream that looks like
the following

HeaderSize PB int32
HeaderMessage
MessageSize PB int32
Message
...
MessageSize PB int32
Message

The header just gives the list of field names and their types (int,
double, or string, that's it).  There are no nested messages or
anything like that in the streamed messages.

Ultimately what I would like to do is put the values from a message
into a list and pass it over to some other code that writes the data
into another format (csv, xml, json, etc.)

I've got this handled wonderfully using CodedInputStream in the Java
implementation but now I need to do it in python and I don't see
anything equivalent.

--
You received this message because you are subscribed to the Google Groups 
Protocol Buffers group.

To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.


--
You received this message because you are subscribed to the Google Groups Protocol 
Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.