Attaching python issue: http://bugs.python.org/issue8323
On Apr 28, 11:41 am, vish <vishi...@gmail.com> wrote:
> What I did so far is to store byte content retrieved from wire onto
> Queue itself, and later the consumer of that message is going to
> construct an object out of it.
> On Apr 28, 11:12 am, Petar Petrov <pesho.pet...@gmail.com> wrote:
> > The multiprocessing queue uses pickle to serialize/deserialize the objects
> > before/after sending them between processes.
> > Since the proto message contains a C extension class, it can't be serialized
> > or deserialized with pickle. And it doesn't need to, just use the protobuf
> > serialization/deserialization routines.
> > So serialize and do Q.put(serizlied), then retrieve using Q.get() and
> > deserialize.
> > On Tue, Apr 26, 2011 at 7:37 PM, vish <vishi...@gmail.com> wrote:
> > > So far what I've found is python protocol buffer objects are not
> > > serializable using Pickle.
> > > On Apr 25, 3:55 pm, vish <vishi...@gmail.com> wrote:
> > > > Environment: CentOS 64bit
> > > > Python 2.6
> > > > Protocol Buffers 2.4.0.a
> > > > Python client with Native C++ support
> > > > python protocol buffers installed to ~/protobuf
> > > > LD_LIBRARY_PATH=~/protobuf/lib:~/protobuf/google/protobuf/src/.libs
> > > > PYTHONPATH=:~/protobuf/:.:~/protobuf/google/protobuf/src/.libs:~/
> > > > protobuf/lib
> > > > Hi,
> > > > In a multiprocessing environment, my code is trying to add decoded
> > > > messages to a shared Queue, and then pop them off at a later stage.
> > > > Inserting messages in the Queue is working fine. But while fetching
> > > > those messages, am getting this error:
> > > > Traceback (most recent call last):
> > > > File "/usr/lib64/python2.6/multiprocessing/queues.py", line 242, in
> > > > _feed
> > > > send(obj)
> > > > PicklingError: Can't pickle <type
> > > 'google3.net.google.protobuf.python.internal._net_proto2___python.CMessage'
> > > >:
> > > > import of module
> > > > google3.net.google.protobuf.python.internal._net_proto2___python
> > > > failed
> > > > Verified LD_LIBRARY_PATH, LD_RUN_PATH, PYTHONPATH and they all seem to
> > > > be valid. Anyway help in solving this problem is appreciated.
> > > > Inserting messages to Queue:
> > > > while not self.stopped():
> > > > el = self.rcvr.next_message()
> > > > if el is not None:
> > > > print "%s is the event"%el
> > > > Q.put(el)
> > > > Popping messages off Queue:
> > > > def next_message(self):
> > > > """Returns next event (if available) from Queue"""
> > > > return Q.get()
> > > > where Q is multiprocessing.Queue
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "Protocol Buffers" group.
> > > To post to this group, send email to firstname.lastname@example.org.
> > > To unsubscribe from this group, send email to
> > > protobuf+unsubscr...@googlegroups.com.
> > > For more options, visit this group at
> > >http://groups.google.com/group/protobuf?hl=en.
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to email@example.com.
To unsubscribe from this group, send email to
For more options, visit this group at