Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-15 Thread Martin v. Löwis
Fredrik Lundh schrieb: but given that the format *has* been stable for many years, surely it would make more sense to just codify that fact, rather than developing Yet Another Serialization Format instead? There have been minor changes over time, e.g. r26146 (gvanrossum) introduced TYPE_TRUE

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-13 Thread Chetan Pandya
I just got around to reading the messages.When I first saw this, I thought this is so that the processes that need to share and work on shared objects. That is where the locks are required. However, all shread objects are managed by the object manager and thus all such operations are in effect

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Nick Maclaren
M.-A. Lemburg [EMAIL PROTECTED] wrote: This is hard to believe. I've been in that business for a few years and so far have not found an OS/hardware/network combination with the mentioned features. Surely you must have - unless there is another M.-A. Lemburg in IT! Some of the specialist

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Nick Maclaren
Josiah Carlson [EMAIL PROTECTED] wrote: It would be convenient, yes, but the question isn't always 'threads or processes?' In my experience (not to say that it is more or better than anyone else's), when going multi-process, the expense on some platforms is significant enough to want to

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Audun Ostrem Nordal
To: python-dev@python.org Subject: [Python-Dev] Cloning threading.py using proccesses I am not sure how sensible the idea is, but I have had a first stab at writing a module processing.py which is a near clone of threading.py but uses processes and sockets for communication. (It is one way

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Richard Oudkerk
On 10/10/06, Fredrik Lundh [EMAIL PROTECTED] wrote: Josiah Carlson wrote: Presumably with this library you have created, you have also written a fast object encoder/decoder (like marshal or pickle). If it isn't any faster than cPickle or marshal, then users may bypass the module and opt

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Richard Oudkerk
On 10/10/06, Josiah Carlson [EMAIL PROTECTED] wrote: the really interesting thing here is a ready-made threading-style API, I think. reimplementing queues, locks, and semaphores can be a reasonable amount of work; might as well use an existing implementation. Really, it is a matter of

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Greg Ewing
Fredrik Lundh wrote: marshal hasn't changed in many years: Maybe not, but I was given to understand that it's regarded as a private format that's not guaranteed to remain constant across versions. So even if it happens not to change, it wouldn't be wise to rely on that. -- Greg

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Fredrik Lundh
Greg Ewing wrote: Fredrik Lundh wrote: marshal hasn't changed in many years: Maybe not, but I was given to understand that it's regarded as a private format that's not guaranteed to remain constant across versions. So even if it happens not to change, it wouldn't be wise to rely on

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread M.-A. Lemburg
Josiah Carlson wrote: Fredrik Lundh [EMAIL PROTECTED] wrote: Josiah Carlson wrote: Presumably with this library you have created, you have also written a fast object encoder/decoder (like marshal or pickle). If it isn't any faster than cPickle or marshal, then users may bypass the module

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread Josiah Carlson
M.-A. Lemburg [EMAIL PROTECTED] wrote: Josiah Carlson wrote: Fredrik Lundh [EMAIL PROTECTED] wrote: Josiah Carlson wrote: Presumably with this library you have created, you have also written a fast object encoder/decoder (like marshal or pickle). If it isn't any faster than

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread Josiah Carlson
Richard Oudkerk [EMAIL PROTECTED] wrote: On 10/10/06, Josiah Carlson [EMAIL PROTECTED] wrote: the really interesting thing here is a ready-made threading-style API, I think. reimplementing queues, locks, and semaphores can be a reasonable amount of work; might as well use an existing

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread skip
Josiah It would basically be something along the lines of cPickle, but Josiah would only support the basic types of: int, long, float, str, Josiah unicode, tuple, list, dictionary. Isn't that approximately marshal's territory? If you can write a faster encoder/decoder, it might well

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread Josiah Carlson
Fredrik Lundh [EMAIL PROTECTED] wrote: Josiah Carlson wrote: It would basically be something along the lines of cPickle, but would only support the basic types of: int, long, float, str, unicode, tuple, list, dictionary. if you're aware of a way to do that faster than the current

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread Simon Wittber
On 10/12/06, Josiah Carlson [EMAIL PROTECTED] wrote: It would basically be something along the lines of cPickle, but would only support the basic types of: int, long, float, str, unicode, tuple, list, dictionary. Great idea! Check this thread for past efforts:

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread Greg Ewing
Fredrik Lundh wrote: if you're aware of a way to do that faster than the current marshal implementation, maybe you could work on speeding up marshal instead? Even if it weren't faster than marshal, it could still be useful to have something nearly as fast that used a

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread Fredrik Lundh
Greg Ewing wrote: if you're aware of a way to do that faster than the current marshal implementation, maybe you could work on speeding up marshal instead? Even if it weren't faster than marshal, it could still be useful to have something nearly as fast that used a

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-10 Thread Josiah Carlson
Richard Oudkerk [EMAIL PROTECTED] wrote: I am not sure how sensible the idea is, but I have had a first stab at writing a module processing.py which is a near clone of threading.py but uses processes and sockets for communication. (It is one way of avoiding the GIL.) On non-windows

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-10 Thread Fredrik Lundh
Josiah Carlson wrote: Presumably with this library you have created, you have also written a fast object encoder/decoder (like marshal or pickle). If it isn't any faster than cPickle or marshal, then users may bypass the module and opt for fork/etc. + XML-RPC XML-RPC isn't close to marshal

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-10 Thread Josiah Carlson
Fredrik Lundh [EMAIL PROTECTED] wrote: Josiah Carlson wrote: Presumably with this library you have created, you have also written a fast object encoder/decoder (like marshal or pickle). If it isn't any faster than cPickle or marshal, then users may bypass the module and opt for