Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-15 Thread Martin v. Löwis
Fredrik Lundh schrieb: > but given that the format *has* been stable for many years, surely it > would make more sense to just codify that fact, rather than developing > Yet Another Serialization Format instead? There have been minor changes over time, e.g. r26146 (gvanrossum) introduced TYPE_TR

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-13 Thread Chetan Pandya
I just got around to reading the messages.When I first saw this, I thought this is so that the processes that need to share and work on shared objects. That is where the locks are required. However, all shread objects are managed by the object manager and thus all such operations are in effect sequ

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Fredrik Lundh
Greg Ewing wrote: > Fredrik Lundh wrote: > >> marshal hasn't changed in many years: > > Maybe not, but I was given to understand that it's > regarded as a private format that's not guaranteed > to remain constant across versions. So even if > it happens not to change, it wouldn't be wise to > re

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Greg Ewing
Fredrik Lundh wrote: > marshal hasn't changed in many years: Maybe not, but I was given to understand that it's regarded as a private format that's not guaranteed to remain constant across versions. So even if it happens not to change, it wouldn't be wise to rely on that. -- Greg ___

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Richard Oudkerk
On 10/10/06, Josiah Carlson <[EMAIL PROTECTED]> wrote: > > the really interesting thing here is a ready-made threading-style API, I > > think. reimplementing queues, locks, and semaphores can be a reasonable > > amount of work; might as well use an existing implementation. > > Really, it is a matt

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Richard Oudkerk
On 10/10/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > Josiah Carlson wrote: > > > Presumably with this library you have created, you have also written a > > fast object encoder/decoder (like marshal or pickle). If it isn't any > > faster than cPickle or marshal, then users may bypass the module

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Audun Ostrem Nordal
9, 2006 1:59 PM > To: python-dev@python.org > Subject: [Python-Dev] Cloning threading.py using proccesses > > I am not sure how sensible the idea is, but I have had a > first stab at writing a module processing.py which is a near > clone of threading.py but uses processes and soc

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Nick Maclaren
Josiah Carlson <[EMAIL PROTECTED]> wrote: > > It would be convenient, yes, but the question isn't always 'threads or > processes?' In my experience (not to say that it is more or better than > anyone else's), when going multi-process, the expense on some platforms > is significant enough to want

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Nick Maclaren
"M.-A. Lemburg" <[EMAIL PROTECTED]> wrote: > > This is hard to believe. I've been in that business for a few > years and so far have not found an OS/hardware/network combination > with the mentioned features. Surely you must have - unless there is another M.-A. Lemburg in IT! Some of the specialis

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread Fredrik Lundh
Greg Ewing wrote: >> if you're aware of a way to do that faster than the current marshal >> implementation, maybe you could work on speeding up marshal instead? > > Even if it weren't faster than marshal, it could still > be useful to have something nearly as fast that used > a python-version-in

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread Greg Ewing
Fredrik Lundh wrote: > if you're aware of a way to do that faster than the current marshal > implementation, maybe you could work on speeding up marshal instead? Even if it weren't faster than marshal, it could still be useful to have something nearly as fast that used a python-version-independe

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread Simon Wittber
On 10/12/06, Josiah Carlson <[EMAIL PROTECTED]> wrote: > > It would basically be something along the lines of cPickle, but would > only support the basic types of: int, long, float, str, unicode, tuple, > list, dictionary. > Great idea! Check this thread for past efforts: http://mail.python.org/p

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread Josiah Carlson
Fredrik Lundh <[EMAIL PROTECTED]> wrote: > Josiah Carlson wrote: > > > It would basically be something along the lines of cPickle, but would > > only support the basic types of: int, long, float, str, unicode, tuple, > > list, dictionary. > > if you're aware of a way to do that faster than the c

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread skip
Josiah> It would basically be something along the lines of cPickle, but Josiah> would only support the basic types of: int, long, float, str, Josiah> unicode, tuple, list, dictionary. Isn't that approximately marshal's territory? If you can write a faster encoder/decoder, it might we

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread Fredrik Lundh
Josiah Carlson wrote: > It would basically be something along the lines of cPickle, but would > only support the basic types of: int, long, float, str, unicode, tuple, > list, dictionary. if you're aware of a way to do that faster than the current marshal implementation, maybe you could work on

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread Josiah Carlson
"Richard Oudkerk" <[EMAIL PROTECTED]> wrote: > On 10/10/06, Josiah Carlson <[EMAIL PROTECTED]> wrote: > > > the really interesting thing here is a ready-made threading-style API, I > > > think. reimplementing queues, locks, and semaphores can be a reasonable > > > amount of work; might as well us

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread Josiah Carlson
"M.-A. Lemburg" <[EMAIL PROTECTED]> wrote: > > Josiah Carlson wrote: > > Fredrik Lundh <[EMAIL PROTECTED]> wrote: > >> Josiah Carlson wrote: > >> > >>> Presumably with this library you have created, you have also written a > >>> fast object encoder/decoder (like marshal or pickle). If it isn't a

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-11 Thread M.-A. Lemburg
Josiah Carlson wrote: > Fredrik Lundh <[EMAIL PROTECTED]> wrote: >> Josiah Carlson wrote: >> >>> Presumably with this library you have created, you have also written a >>> fast object encoder/decoder (like marshal or pickle). If it isn't any >>> faster than cPickle or marshal, then users may bypas

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-10 Thread Josiah Carlson
Fredrik Lundh <[EMAIL PROTECTED]> wrote: > > Josiah Carlson wrote: > > > Presumably with this library you have created, you have also written a > > fast object encoder/decoder (like marshal or pickle). If it isn't any > > faster than cPickle or marshal, then users may bypass the module and opt

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-10 Thread Fredrik Lundh
Josiah Carlson wrote: > Presumably with this library you have created, you have also written a > fast object encoder/decoder (like marshal or pickle). If it isn't any > faster than cPickle or marshal, then users may bypass the module and opt > for fork/etc. + XML-RPC XML-RPC isn't close to marsh

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-10 Thread Josiah Carlson
"Richard Oudkerk" <[EMAIL PROTECTED]> wrote: > I am not sure how sensible the idea is, but I have had a first stab at > writing a module processing.py which is a near clone of threading.py > but uses processes and sockets for communication. (It is one way of > avoiding the GIL.) On non-windows p