Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Justin Tulloss
What do you think? I'm going to have to agree with Martin here, although I'm not sure I understand what you're saying entirely. Perhaps if you explained where the benefits of this approach come from, it would clear up what you're thinking. After a few days of thought, I'm starting to realize

Re: [Python-Dev] SSL certs

2007-09-13 Thread Martin v. Löwis
My understanding is that the client tells the server which hostname it wants to use; the server should then pass down that information. That's how virtual hosting works in the first place. The only difference with SSL is that the hostname must have a unique IP address, so that when the

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Christian Heimes
Phillip J. Eby wrote: It's not just caches and counters. It's also every built-in type structure, builtin module, builtin function... any Python object that's a built-in, period. That includes things like None, True, and False. Caches would include such things as the pre-created

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread André Malo
* Christian Heimes wrote: Pardon my ignorance but why does Python do reference counting for truly global and static objects like None, True, False, small and cached integers, sys and other builtins? If I understand it correctly these objects are never garbaged collected (at least they

Re: [Python-Dev] Windows package for new SSL package?

2007-09-13 Thread Paul Moore
On 13/09/2007, David Bolen [EMAIL PROTECTED] wrote: Mark Hammond [EMAIL PROTECTED] writes: It might be possible to try and use build_ssl.py to locate the openssl directory, but this will still require that someone building it has Python built from source - I'm fairly sure that someone

Re: [Python-Dev] Windows package for new SSL package?

2007-09-13 Thread Paul Moore
On 13/09/2007, David Bolen [EMAIL PROTECTED] wrote: Bill Janssen [EMAIL PROTECTED] writes: In that case, I think your idea of just hard-coding a path is probably the right thing to do. I'll add a note that this is how you need to do it if you are going to try python setup.py build.

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Jon Ribbens
On Thu, Sep 13, 2007 at 12:19:21PM +0200, André Malo wrote: Pardon my ignorance but why does Python do reference counting for truly global and static objects like None, True, False, small and cached integers, sys and other builtins? If I understand it correctly these objects are never

Re: [Python-Dev] Windows package for new SSL package?

2007-09-13 Thread David Bolen
On 9/13/07, Paul Moore [EMAIL PROTECTED] wrote: On 13/09/2007, David Bolen [EMAIL PROTECTED] wrote: I think where there's probably a small disconnect here is that, there really isn't an OpenSSL installed on the end user's machine. Well, there could be, but Python isn't using it. The

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Martin v. Löwis
To put it another way, would it actually matter if the reference counts for such objects became hopelessly wrong due to non-atomic adjustments? If they drop to zero (which may happen due to non-atomic adjustments), Python will try to release the static memory, which will crash the malloc

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Jon Ribbens
On Thu, Sep 13, 2007 at 01:15:39PM +0200, Martin v. Löwis wrote: To put it another way, would it actually matter if the reference counts for such objects became hopelessly wrong due to non-atomic adjustments? If they drop to zero (which may happen due to non-atomic adjustments), Python

Re: [Python-Dev] Windows package for new SSL package?

2007-09-13 Thread Paul Moore
On 13/09/2007, David Bolen [EMAIL PROTECTED] wrote: That's a fair point - my comments are all related to the standard Python distribution and building extensions with the VS.NET compiler (including the binary installer I had built for Bill). [...] If we're talking about the construction of

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread skip
Jon To put it another way, would it actually matter if the reference Jon counts for such objects became hopelessly wrong due to non-atomic Jon adjustments? I believe this was suggested and tried by someone (within the last few years). It wasn't any benefit. The costs of

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Hrvoje Nikšić
On Thu, 2007-09-13 at 13:15 +0200, Martin v. Löwis wrote: To put it another way, would it actually matter if the reference counts for such objects became hopelessly wrong due to non-atomic adjustments? If they drop to zero (which may happen due to non-atomic adjustments), Python will try

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Prateek Sureka
On Sep 13, 2007, at 10:12 AM, Martin v. Löwis wrote: What do you think? I think what you are describing is the situation of today, except in a less-performant way. The kernel *already* implements such a synchronization server, except that all CPUs can act as such. You write Since we are

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Martin v. Löwis
Since we are guaranteeing that synchronized code is running on a single core, it is the equivalent of a lock at the cost of a context switch. This is precisely what a lock costs today: a context switch. Really? Wouldn't we save some memory allocation overhead (since in my design, the lock

Re: [Python-Dev] Windows package for new SSL package?

2007-09-13 Thread Bill Janssen
In other words, both the standard and your extension module on Windows bring along their own OpenSSL. I see -- thanks. Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

Re: [Python-Dev] Windows package for new SSL package?

2007-09-13 Thread Bill Janssen
Anyway, philosophy aside, I'll try to make some time in the next few days to get a working setup.py for the SSL package using mingw. Hopefully, Bill will then integrate this and we'll have mingw as a supported option. I'll be happy to do that! Bill

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Prateek Sureka
On Sep 13, 2007, at 9:25 PM, Martin v. Löwis wrote: Since we are guaranteeing that synchronized code is running on a single core, it is the equivalent of a lock at the cost of a context switch. This is precisely what a lock costs today: a context switch. Really? Wouldn't we save

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Martin v. Löwis
http://www.artima.com/weblogs/viewpost.jsp?thread=214235) that the slowdown was 2x in a single threaded application (which couldn't be due to lock contention), it must be due to lock overhead (unless the programming was otherwise faulty or there is something else about locks that I don't know

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Adam Olsen
On 9/13/07, Hrvoje Nikšić [EMAIL PROTECTED] wrote: On Thu, 2007-09-13 at 13:15 +0200, Martin v. Löwis wrote: To put it another way, would it actually matter if the reference counts for such objects became hopelessly wrong due to non-atomic adjustments? If they drop to zero (which may

Re: [Python-Dev] SSL certs

2007-09-13 Thread Bill Janssen
However, there is an alternative to using multiple IP addresses: one could also use multiple subject alternative names, and create a certificate that lists them all. Unfortunately, much of the client code that does the hostname verification is wrapped up in gullible Web browsers or Java HTTPS

Re: [Python-Dev] SSL certs

2007-09-13 Thread Martin v. Löwis
However, there is an alternative to using multiple IP addresses: one could also use multiple subject alternative names, and create a certificate that lists them all. Unfortunately, much of the client code that does the hostname verification is wrapped up in gullible Web browsers or Java

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Jason Orendorff
On 9/13/07, Justin Tulloss [EMAIL PROTECTED] wrote: 1. Use message passing and transactions. [...] 2. Do it perl style. [...] 3. Come up with an elegant way of handling multiple python processes. [...] 4. Remove the GIL, use transactions for python objects, [...] The SpiderMonkey JavaScript

[Python-Dev] base64 -- should b64encode introduce line breaks?

2007-09-13 Thread Bill Janssen
I see that base64.b64encode and base64.standard_b64encode no longer introduce line breaks into the output strings, as base64.encodestring does. Shouldn't there be an option on one of them to do this? Bill ___ Python-Dev mailing list

Re: [Python-Dev] base64 -- should b64encode introduce line breaks?

2007-09-13 Thread Bill Janssen
I see that base64.b64encode and base64.standard_b64encode no longer introduce line breaks into the output strings, as base64.encodestring does. Shouldn't there be an option on one of them to do this? See: http://mail.python.org/pipermail/python-bugs-list/2001-October/007856.html section 2.1

Re: [Python-Dev] Python tickets summary

2007-09-13 Thread Facundo Batista
2007/9/10, Facundo Batista [EMAIL PROTECTED]: I modified my tool, whichs makes a summary of all the Python tickets (I moved the source where the info is taken from SF to our Roundup). In result, the summary is now, again, updated daily: Taking an idea from Jeff Rush, now there're separate

Re: [Python-Dev] [Tracker-discuss] Python tickets summary

2007-09-13 Thread Raghuram Devarakonda
On 9/13/07, Facundo Batista [EMAIL PROTECTED] wrote: http://www.taniquetil.com.ar/facundo/py_tickets.html It looks like the column Opened by contains information for Last update by and vice versa. At least, that is the case with issue 1159. Thanks, Raghu

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Adam Olsen
On 9/13/07, Justin Tulloss [EMAIL PROTECTED] wrote: On 9/13/07, Adam Olsen [EMAIL PROTECTED] wrote: Basically though, atomic incref/decref won't work. Once you've got two threads modifying the same location the costs skyrocket. Even without being properly atomic you'll get the same

Re: [Python-Dev] Windows package for new SSL package?

2007-09-13 Thread Paul Moore
On 13/09/2007, Bill Janssen [EMAIL PROTECTED] wrote: Anyway, philosophy aside, I'll try to make some time in the next few days to get a working setup.py for the SSL package using mingw. Hopefully, Bill will then integrate this and we'll have mingw as a supported option. I'll be happy to

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Justin Tulloss
On 9/13/07, Jason Orendorff [EMAIL PROTECTED] wrote: On 9/13/07, Justin Tulloss [EMAIL PROTECTED] wrote: 1. Use message passing and transactions. [...] 2. Do it perl style. [...] 3. Come up with an elegant way of handling multiple python processes. [...] 4. Remove the GIL, use

Re: [Python-Dev] Windows package for new SSL package?

2007-09-13 Thread Bill Janssen
I could build some Windows installers if you want, but I'd need to download and install some extra versions of Python, so you'd have to tell me which you want doing (and I can't offer to commit to doing this on a regular basis...) Thanks, but let's wait till this settles down a bit (say, a

Re: [Python-Dev] Windows package for new SSL package?

2007-09-13 Thread Paul Moore
On 13/09/2007, Bill Janssen [EMAIL PROTECTED] wrote: I could build some Windows installers if you want, but I'd need to download and install some extra versions of Python, so you'd have to tell me which you want doing (and I can't offer to commit to doing this on a regular basis...)

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Justin Tulloss
On 9/13/07, Adam Olsen [EMAIL PROTECTED] wrote: Basically though, atomic incref/decref won't work. Once you've got two threads modifying the same location the costs skyrocket. Even without being properly atomic you'll get the same slowdown on x86 (who's cache coherency is fairly strict.)

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Greg Ewing
Martin v. Löwis wrote: Now we are getting into details: you do NOT have to lock an object to modify its reference count. An atomic increment/decrement operation is enough. I stand corrected. But if it were as simple as that, I think it would have been done by now. I got the impression that

Re: [Python-Dev] Windows package for new SSL package?

2007-09-13 Thread Mark Hammond
You don't need VS and mingw binary installers, though - the mingw ones will work for any Python (ignoring specialised custom builds, and anyone doing one of them is probably capable of building the ssl module!). Why I appreciate your points about building the extension with free tools,

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread skip
Hrvoje More precisely, Python will call the deallocator appropriate for Hrvoje the object type. If that deallocator does nothing, the object Hrvoje continues to live. Such objects could also start out with a Hrvoje refcount of sys.maxint or so to ensure that calls to the no-op

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Jon Ribbens
On Thu, Sep 13, 2007 at 06:38:05PM -0500, [EMAIL PROTECTED] wrote: Hrvoje More precisely, Python will call the deallocator appropriate for Hrvoje the object type. If that deallocator does nothing, the object Hrvoje continues to live. Such objects could also start out with a

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Greg Ewing
Jon Ribbens wrote: To put it another way, would it actually matter if the reference counts for such objects became hopelessly wrong due to non-atomic adjustments? Again, it would cost time to check whether you could get away with doing non-atomic refcounting. If you're thinking that no check

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Greg Ewing
[EMAIL PROTECTED] wrote: what if ... we use atomic test-and-set to handle reference counting (with a lock for those CPU architectures where we haven't written the necessary assembler fragment), then implement a lock for each mutable type and another for global state (thread state, interpreter

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Greg Ewing
Prateek Sureka wrote: Naturally, we need to make the locking more fine-grained to resolve this. Hopefully we can do so in a way that does not increase the lock overhead (hence my suggestion for a lock free approach using an asynch queue and a core as dedicated server). What you don't

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Greg Ewing
Jason Orendorff wrote: The clever bit is that SpiderMonkey's per-object locking does *not* require a context switch or even an atomic instruction, in the usual case where an object is *not* shared among threads. How does it tell whether an object is shared between threads? That sounds like

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Tennessee Leeuwenburg
Pardon me for talking with no experience in such matters, but... Okay, incrementing a reference counter is atomic, therefore the cheapest possible operation. Is it possible to keep reference counting atomic in a multi-thread model? Could you do the following... let's consider two threads, A and

Re: [Python-Dev] Removing the GIL (Me, not you!)

2007-09-13 Thread Justin Tulloss
On 9/13/07, Greg Ewing [EMAIL PROTECTED] wrote: Jason Orendorff wrote: The clever bit is that SpiderMonkey's per-object locking does *not* require a context switch or even an atomic instruction, in the usual case where an object is *not* shared among threads. How does it tell whether