Re: Sending binary pickled data through TCP

David Hirschfield wrote: Thanks for the great response. Yeah, by safe I mean that it's all happening on an intranet with no chance of malicious individuals getting access to the stream of data. The chunks are arbitrary collections of python objects. I'm wrapping them up a little, but I

Re: Sending binary pickled data through TCP

Paul Rubin http wrote: As for the network representation, DJB proposes this format: http://cr.yp.to/proto/netstrings.txt Netstrings are cool and you'll find some python implementations if you search. But it is basically number:string,, ie 12:hello world!, Or you could use escaping which is

Re: Sending binary pickled data through TCP

David Hirschfield wrote: I have a pair of programs which trade python data back and forth by pickling up lists of objects on one side (using pickle.HIGHEST_PROTOCOL), and sending that data over a TCP socket connection to the receiver, who unpickles the data and uses it. So far this has been

Re: Sending binary pickled data through TCP

Thanks for the great response. Yeah, by "safe" I mean that it's all happening on an intranet with no chance of malicious individuals getting access to the stream of data. The chunks are arbitrary collections of python objects. I'm wrapping them up a little, but I don't know much about the

Re: Sending binary pickled data through TCP

David Hirschfield wrote: Are there any existing python modules that do the equivalent of pickling on arbitrary python data, but do it a lot faster? I wasn't aware of any that are as easy to use as pickle, or don't require implementing them myself, which is not something I have time for.

Re: Sending binary pickled data through TCP

David Hirschfield wrote: I have a pair of programs which trade python data back and forth by pickling up lists of objects on one side (using pickle.HIGHEST_PROTOCOL), and sending that data over a TCP socket connection to the receiver, who unpickles the data and uses it. So far this has

Re: Sending binary pickled data through TCP

I've looked at pyro, and it is definitely overkill for what I need. If I was requiring some kind of persistent state for objects shared between processes, pyro would be awesome...but I just need to transfer chunks of complex python data back and forth. No method calls or keeping state in

Re: Sending binary pickled data through TCP

I'm using cPickle already. I need to be able to pickle pretty arbitrarily complex python data structures, so I can't use marshal. I'm guessing that cPickle is the best choice, but if someone has a faster pickling-like module, I'd love to know about it. -Dave Fredrik Lundh wrote: David

Sending binary pickled data through TCP

I have a pair of programs which trade python data back and forth by pickling up lists of objects on one side (using pickle.HIGHEST_PROTOCOL), and sending that data over a TCP socket connection to the receiver, who unpickles the data and uses it. So far this has been working fine, but I now

Re: Sending binary pickled data through TCP

David Hirschfield wrote: I have a pair of programs which trade python data back and forth by pickling up lists of objects on one side (using pickle.HIGHEST_PROTOCOL), and sending that data over a TCP socket connection to the receiver, who unpickles the data and uses it. So far this has

Re: Sending binary pickled data through TCP

David Hirschfield [EMAIL PROTECTED] writes: Is there a reliable way to determine the byte count of some pickled binary data? Can I rely on len(pickled data) == bytes? Huh? Yes, of course len gives you the length. As for the network representation, DJB proposes this format: