Re: [Python-Dev] Bugs in thread_nt.h
On Thursday 10 March 2011, Sturla Molden wrote: As for InterlockedCompareExchange et al., MSDN says this: The parameters for this function must be aligned on a 32-bit boundary; bit != byte Uli ** Domino Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ** Visit our website at http://www.dominolaser.com/ ** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Domino Laser GmbH ist für diese Folgen nicht verantwortlich. ** ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bugs in thread_nt.h
On Wed, Mar 9, 2011 at 8:42 PM, Martin v. Löwis mar...@v.loewis.de wrote: As for the volatile marker - I believe the code is also correct without it, since the owned field is only accessed through initialization and Interlocked operations. Furthermore, if the code weren't correct, volatile would only be half the solution since it only restricts compiler-induced reorders. Some sort of CPU fence instruction would almost certainly be needed as well. Sturla wrote: releasing the time-slice by Sleep(0) for each iteration, it will certainly fail without a volatile qualifier. In general, your compiler will reload globally-accessible memory after a function call (or after a memory store). In fact, it can be quite a job to give them enough hints that they stop doing so! This behavior (which volatile aggravates) unfortunately makes it even tougher to find race conditions. In my experience, volatile should be avoided. I'd even bet money that some grumpy person has written a volatile considered harmful essay. I Am Not A memory-model expert so consult your local guru, p ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bugs in thread_nt.h
On 3/10/2011 3:07 AM, Paul Du Bois wrote: volatile considered harmful http://www.kernel.org/doc/Documentation/volatile-considered-harmful.txt -- Scott Dial sc...@scottdial.com scod...@cs.indiana.edu ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bugs in thread_nt.h
This behavior (which volatile aggravates) unfortunately makes it even tougher to find race conditions. In my experience, volatile should be avoided. I'd even bet money that some grumpy person has written a volatile considered harmful essay. I guess all this advice doesn't really apply to this case, though. The Microsoft API declares the parameter as a volatile*, indicating that they consider it proper usage of the API to declare the storage volatile. So ISTM that we should comply regardless of whether volatile is considered morally wrong in the general case. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GPL'd python code vs Python2.6 linked against OpenSSL
Note that your interpretation would allow Python to distribute arbitrarily licensed libraries and GPL programs to link with them. That is surely not the intent of the authors of the GPL, and in the past, the FSF has explicitly restricted the interpretation of system library. Note that it is ultimately up to a court to interpret these words of the GPL, not to the FSF lawyer. Specifically, in Major Component, in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. the word essential would refer to running the compiler or the operating system or interpreter, not to a component essential to running the program but in general optional for using the system. Maybe my English is not good enough - I read that to mean that the compiler itself is a major component of the system (and thus doesn't need to be distributed along with the source of the GPL'ed program). Note that it explicitly also says that a object code interpreter used to run it (the executable) is a major component, which would explicitly declare Python a major component (as long as it's actually distributed with the system). If your interpetation was right and Python is *not* a major component, then any GPL'ed Python application would need to distribute the source code of Python along with its own source code, and with that the source of all libraries that are not major components themselves. Also, if your interpretation was right, it wouldn't be possible to license Java programs under the GPL, since the JVM wouldn't be a major component, and hence you would need to distribute JVM's source code under the terms of the GPL (which you may or may not be able to today, but surely couldn't over many years during which many GPL'ed Java programs were written). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bugs in thread_nt.h
Den 10.03.2011 11:06, skrev Scott Dial: http://www.kernel.org/doc/Documentation/volatile-considered-harmful.txt The important part here (forgive me for being a pedant) is that register allocation of the (1) 'owned' field is actually unwanted, and (2) Microsoft specify 'volatile' in calls to the Interlocked* API-functions. I am sorry for misreading 32 bits (4 bytes) as 32 bytes. That is obviously very different. If Microsoft's malloc is sufficient, why does MSDN tell us to use _aligned_malloc instead of malloc? The rest is a comment to the link and a bit OT: Their argument is that (1) volatile supresses optimization; (2) within a critical section, access to shared data is synchronized; and thus (3) volatile is critical there. That is true, to some extent. Volatile is not a memory barrier, nor a mutex. Volatile's main purpose is to prevent the compiler from storing a variable in a register. Volatile might be used incorrectly if we don't understand this. Obvious usecases for volatile are: - Implementation of a spinlock, where register allocation is detrimental. - A buffer that is filled from the outside with some DMA mechanism. - Real-time programs and games where order of execution and and timing is critical, so optimization must be supressed. Even though volatile is not needed for processing within a critical section, we still need the shared data to be re-loaded upon entering and er-written upon leaving. We can use a typecast to non-volatile within a critical section, and achieve both data consisitency and compiler optimization. OpenMP has a better mechanism for this, where a flush-operand (#pragma omp flush) will force a synchronization of shared data among threads (write and reload), and volatile is never needed for consistency of shared data. Sturla ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GPL'd python code vs Python2.6 linked against OpenSSL
On Thu, Mar 10, 2011 at 2:41 AM, Stephen J. Turnbull step...@xemacs.org wrote: Joao S. O. Bueno writes: Any libraries commonly avaliable with a CPython instalation can be considered as system libraries for GPL purposes - and so this would fall in the system library exception as described by the FAQ: Note that your interpretation would allow Python to distribute arbitrarily licensed libraries and GPL programs to link with them. That is surely not the intent of the authors of the GPL, and in the past, the FSF has explicitly restricted the interpretation of system library. Specifically, in Major Component, in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. the word essential would refer to running the compiler or the operating system or interpreter, not to a component essential to running the program but in general optional for using the system. So tellme how the Python interpreter and the libraries linked to it are not essential to a program made in Python? I did not refer to distributing Python itself with GPL librareis linked into it - you had put it this way. The original question is whether a Python program can be GPLed, and the answer is: it can. You can see Antonie Pitrou's first answer to question: A Python program is not a derivative of the Python interpreter any more than a C program is a derivative of gcc (or any other compiler). The fact is that Python programs need more of environment t run than a C program, since besides the system libraries they run oer the Python interpreter and any libraries it links too: all these are intrinsic in order to the proram to run. Otherwise no one could ever have a GPLed program written in notoriously privative language or environments such as Visualbasic, Microsfot's .net, or even Java using Oracle's Java environment. Perhaps this has changed with the advent of GPL v3, but the FSF used this interpretation to block the consideration of (old-Qt-licensed) Qt as a system library on GNU/Linux systems, even where distributed by vendors such as Red Hat. OTOH, for some reason Motif on Sun and HP systems, and Windows and Mac GUIs were considered essential. They where then contradicting their own current FAQs on the GPL. Anyway, the original author has always the right to exempt some library his program links to of complying with the GPL. js -- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bugs in thread_nt.h
Martin v. Löwis: I guess all this advice doesn't really apply to this case, though. The Microsoft API declares the parameter as a volatile*, indicating that they consider it proper usage of the API to declare the storage volatile. The 'volatile' here is a modifier on the parameter and does not require a corresponding agreement in the variable declaration. It indicates that all access through the pointer inside the function will be with volatile semantics. As long as all functions that operate on the variable do so treating access as volatile then everything is fine. You should only need to declare the variable as volatile if there is other code that accesses it directly. If agreement was required then the compiler would print a warning. It is similar to declaring a function to take a const parameter: there is no need for the variable to also be const. Neil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bugs in thread_nt.h
On Thu, 10 Mar 2011 12:18:22 +0100 Sturla Molden stu...@molden.no wrote: Obvious usecases for volatile are: - Implementation of a spinlock, where register allocation is detrimental. - A buffer that is filled from the outside with some DMA mechanism. - Real-time programs and games where order of execution and and timing is critical, so optimization must be supressed. - variables mutated from signal handlers Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bugs in thread_nt.h
I am sorry for misreading 32 bits (4 bytes) as 32 bytes. That is obviously very different. If Microsoft's malloc is sufficient, why does MSDN tell us to use _aligned_malloc instead of malloc? I don't know. Perhaps they assume that people may be using alternative malloc implementations, or (more likely) people in the platform group are unaware of the guarantees the compiler group makes (because of compliance with the C standard). Obvious usecases for volatile are: - Implementation of a spinlock, where register allocation is detrimental. Unfortunately, that's not true. It's not really possible to implement a spinlock in C if the processor can make changes to the write ordering. - A buffer that is filled from the outside with some DMA mechanism. - Real-time programs and games where order of execution and and timing is critical, so optimization must be supressed. Likewise. On many current processors, you have to use fencing instructions to make this kind of stuff work, something that the compiler will not normally generate (except through inline assembly). Even though volatile is not needed for processing within a critical section, we still need the shared data to be re-loaded upon entering and er-written upon leaving. Please see the code. Where exactly does such reloading/rewriting need to happen? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bugs in thread_nt.h
Am 10.03.11 07:55, schrieb Neil Hodgson: Martin v. Löwis: I guess all this advice doesn't really apply to this case, though. The Microsoft API declares the parameter as a volatile*, indicating that they consider it proper usage of the API to declare the storage volatile. The 'volatile' here is a modifier on the parameter and does not require a corresponding agreement in the variable declaration. I see. So then we don't need to make any changes. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bugs in thread_nt.h
I guess all this advice doesn't really apply to this case, though. The Microsoft API declares the parameter as a volatile*, indicating that they consider it proper usage of the API to declare the storage volatile. So ISTM that we should comply regardless of whether volatile is considered morally wrong in the general case. Microsoft compiler implements Microsoft-specific behaviour for volatile variables, making them close to volatiles in Java: http://msdn.microsoft.com/en-us/library/12a04hfd(v=VS.100).aspx That may be the reason they use it in Interlocked* functions. As already been said, the thread_nt doesn't need any changes. Eugene ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] funky buildbot
I'm trying to get a new buildbot in the swim of things, and it keeps getting into this state where the buildslave process seems caught in an endless loop. Perhaps someone here knows why? It's a new Mac Mini running the latest Snow Leopard, with Python 2.6.1 (the /usr/bin/python) and buildslave 0.8.3, using Twisted 8.2.0. Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] constant folding of -0
On Thu, Mar 10, 2011 at 2:17 AM, Eugene Toder elto...@gmail.com wrote: Indeed, see http://bugs.python.org/issue11244 Yes, I've noticed that too. However, if I'm not missing something, your patches do not address folding of -0. Hmm, it seems that way. Could you post a comment on the tracker issue about that, please? I'm not sure why the original changeset went in, but I agree it looks as though it's no longer necessary. Certainly there should be enough -0.0 versus 0.0 tests around to detect any issues, so if all the tests pass with the extra PyObject_IsTrue check disabled then there probably isn't a problem. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] (2.6): Issue #11424: Fix bug in determining child loggers.
Hi, changeset: 68315:b9d76846bb1c branch: 2.6 parent: 68264:50166a4bcfc6 user:Vinay Sajip vinay_sa...@yahoo.co.uk date:Mon Mar 07 15:02:11 2011 + summary: Issue #11424: Fix bug in determining child loggers. This does not look like a security bug, and is therefore not suitable for 2.6. I’d like to know what we should do about this: just ignore it and not import it into Subversion, or revert it in Mercurial too? Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] hg diff
The idea is to pull their remote branch but not merge it, which will create multiple heads locally. “hg pull some-repo-uri” does that. Then find the common ancestor of my regular local head and the new head, and diff the ancestor with the new head. I think Mercurial revsets can do that, but I don’t feel confident enough to try and convert your example to use them. From “hg help revsets” / http://www.selenic.com/mercurial/hg.1#revsets : not x Changesets not in x. Short form is ! x. x::y A DAG range, meaning all changesets that are descendants of x and ancestors of y, including x and y themselves. If the first endpoint is left out, this is equivalent to ancestors(y), if the second is left out it is equivalent to descendants(x). An alternative syntax is x..y. x:y All changesets with revision numbers between x and y, both inclusive. Either endpoint can be left out, they default to 0 and tip. x and y The intersection of changesets in x and y. Short form is x y. x or y The union of changesets in x and y. There are two alternative short forms: x | y and x + y. x - y Changesets in x but not in y. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] (2.6): Issue #11424: Fix bug in determining child loggers.
On Thu, 10 Mar 2011 21:29:59 +0100 Éric Araujo mer...@netwok.org wrote: Hi, changeset: 68315:b9d76846bb1c branch: 2.6 parent: 68264:50166a4bcfc6 user:Vinay Sajip vinay_sa...@yahoo.co.uk date:Mon Mar 07 15:02:11 2011 + summary: Issue #11424: Fix bug in determining child loggers. This does not look like a security bug, and is therefore not suitable for 2.6. I’d like to know what we should do about this: just ignore it and not import it into Subversion, or revert it in Mercurial too? If you revert (backout) it in Mercurial, you should be careful to dummy-merge the backout in 2.7 (so that the backout doesn't get accidentally propagated by someone else doing a merge from 2.6 to 2.7). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] funky buildbot
Bill Janssen jans...@parc.com writes: I'm trying to get a new buildbot in the swim of things, and it keeps getting into this state where the buildslave process seems caught in an endless loop. Perhaps someone here knows why? Do you have any information as to what it is doing while in the loop? I think the only obvious looping I've seen with any of my buildbots is a loop while trying to attach to the master, either because the master was down/unreachable, rejecting connections (bad user/password), not yet configured for my buildbot, or occasionally because it thought the buildbot was still attached. In that latter case I just had to wait long enough for the status page to switch from idle to disconnected and then try again. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] constant folding of -0
I've posted a patch. Eugene On Thu, Mar 10, 2011 at 3:30 PM, Mark Dickinson dicki...@gmail.com wrote: On Thu, Mar 10, 2011 at 2:17 AM, Eugene Toder elto...@gmail.com wrote: Indeed, see http://bugs.python.org/issue11244 Yes, I've noticed that too. However, if I'm not missing something, your patches do not address folding of -0. Hmm, it seems that way. Could you post a comment on the tracker issue about that, please? I'm not sure why the original changeset went in, but I agree it looks as though it's no longer necessary. Certainly there should be enough -0.0 versus 0.0 tests around to detect any issues, so if all the tests pass with the extra PyObject_IsTrue check disabled then there probably isn't a problem. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] (2.6): Issue #11424: Fix bug in determining child loggers.
From what I understand, we're supposed to forward-port in Mercurial, which is why I started with 2.6 (the bugfix wasn't applicable to 2.5). So we don't need to import into Subversion, but I see no point in reverting it in Mercurial. Regards, Vinay Sajip - Original Message From: Éric Araujo mer...@netwok.org To: python-dev@python.org; Friendly Uncle ba...@python.org; Vinay Sajip vinay_sa...@yahoo.co.uk Sent: Thu, 10 March, 2011 20:29:59 Subject: Re: [Python-Dev] [Python-checkins] (2.6): Issue #11424: Fix bug in determining child loggers. Hi, changeset: 68315:b9d76846bb1c branch: 2.6 parent: 68264:50166a4bcfc6 user:Vinay Sajip vinay_sa...@yahoo.co.uk date:Mon Mar 07 15:02:11 2011 + summary: Issue #11424: Fix bug in determining child loggers. This does not look like a security bug, and is therefore not suitable for 2.6. I’d like to know what we should do about this: just ignore it and not import it into Subversion, or revert it in Mercurial too? Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] (2.6): Issue #11424: Fix bug in determining child loggers.
Hi, From what I understand, we're supposed to forward-port in Mercurial, Correct, but only in maintained branches, not security-mode branches. which is why I started with 2.6 (the bugfix wasn't applicable to 2.5). As I said in my first message: This does not look like a security bug, and is therefore not suitable for 2.6. I don’t know if Barry will make the next 2.6 releases from Subversion or Mercurial; anyway, bug fixes should not go into 2.6. Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] (2.6): Issue #11424: Fix bug in determining child loggers.
- Original Message From: Éric Araujo mer...@netwok.org From what I understand, we're supposed to forward-port in Mercurial, Correct, but only in maintained branches, not security-mode branches. Well, I saw this recent mail from Antoine: http://mail.python.org/pipermail/python-dev/2011-March/108766.html where he mentioned - on one hand: 2.5 - 2.6 - 2.7 (if you still want to maintain 2.5) - on the other hand: 3.1 - 3.2 - default and perhaps misunderstood. I was assuming that there wouldn't be any 2.6 releases from Mercurial, perhaps that assumption was unwarranted. Also, the section on forward-porting in the Dev Guide says it should be applied to the oldest branch applicable and this could be interpreted to mean applicable based on the content of the branch, rather than according to release policy. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] constant folding of -0
On Thu, 10 Mar 2011 02:17:34 + (UTC) Eugene Toder elto...@gmail.com wrote: Indeed, see http://bugs.python.org/issue11244 Yes, I've noticed that too. However, if I'm not missing something, your patches do not address folding of -0. Btw, there's an alternative approach to allow recursive constant folding. Instead of keeping a stack of last constants, you can keep a pointer to the start of the last (LOAD_CONSTs + NOPs) run and the number of LOAD_CONSTs in that run (called lastlc in the current version). When you want Nth constant from the end, you start from that pointer and skip lastlc-N constants. You also make a function to get next constant from that point. This approach has worse time complexity for searching in a long run of LOAD_CONSTs, Yes, the stack basically acts as a cache to avoid computing all this again and again. however, there are upsides: - very long runs of constants are rare in real code True, but they might appear in generated code. - it uses less memory and doesn't have arbitrary limits on the size of the run Neither does the latest patch. - it's book-keeping overhead is smaller, so when you don't have long runs of constants (common case, I believe), it should be faster The book-keeping overhead should be quite small really, it's a simple autogrowing array with O(1) access and amortized append time. What's left is the overhead of the initial malloc() (and final free()). - I think it's simpler to implement Feel free to propose an alternate patch, but I'm not sure that it would be significantly simpler (and a stack is a well-known data structure). Also, please present some benchmarks if you do. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] funky buildbot
David Bolen db3l@gmail.com wrote: Bill Janssen jans...@parc.com writes: I'm trying to get a new buildbot in the swim of things, and it keeps getting into this state where the buildslave process seems caught in an endless loop. Perhaps someone here knows why? Do you have any information as to what it is doing while in the loop? Not much. The twistd log shows that it is, in fact, trying to reconnect to the build master. It's just chewing up 100% CPU. Could just be updates to the master that are doing this, but my other (PPC) buildbots seem OK with it. Bill I think the only obvious looping I've seen with any of my buildbots is a loop while trying to attach to the master, either because the master was down/unreachable, rejecting connections (bad user/password), not yet configured for my buildbot, or occasionally because it thought the buildbot was still attached. In that latter case I just had to wait long enough for the status page to switch from idle to disconnected and then try again. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/bill%40janssen.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] funky buildbot
Bill Janssen jans...@parc.com writes: David Bolen db3l@gmail.com wrote: Bill Janssen jans...@parc.com writes: I'm trying to get a new buildbot in the swim of things, and it keeps getting into this state where the buildslave process seems caught in an endless loop. Perhaps someone here knows why? Do you have any information as to what it is doing while in the loop? Not much. The twistd log shows that it is, in fact, trying to reconnect to the build master. It's just chewing up 100% CPU. Could just be updates to the master that are doing this, but my other (PPC) buildbots seem OK with it. When I look at the slave status page[*], it seems to show all of your slaves as idle, so if one of them is looping trying to connect, I suspect you may have run into that last case I've seen (mismatch between slave and master). Try shutting down the build slave for a while and watching the status page until it shows disconnected. Then you should be able to restart things and have it attach successfully. Leaving the slave continually trying to reattach, I believe, prevents the master from ever declaring it disconnected. I'm guessing the master is rejecting the new connection since it thinks the slave is still attached (though the request seems to defeat the status from correcting). I'm not sure how it gets into that state but it's happened a handful of times to me (typically after some network interruptions, so maybe it just needs enough time for buildbot's own keepalives to notice what is otherwise a quiet TCP channel). There used to be a way to request a ping from the master side (I think on the same page you could manually run a build from) that I would used to force it to recognize a slave was really down, but after the web interface got rearranged a while back, I can't seem to find the option any more. -- David [*] http://www.python.org/dev/buildbot/buildslaves ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] funky buildbot
On Thu, 10 Mar 2011 15:46:11 PST Bill Janssen jans...@parc.com wrote: David Bolen db3l@gmail.com wrote: Bill Janssen jans...@parc.com writes: I'm trying to get a new buildbot in the swim of things, and it keeps getting into this state where the buildslave process seems caught in an endless loop. Perhaps someone here knows why? Do you have any information as to what it is doing while in the loop? Not much. The twistd log shows that it is, in fact, trying to reconnect to the build master. It's just chewing up 100% CPU. Could just be updates to the master that are doing this, but my other (PPC) buildbots seem OK with it. What login are you using? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] constant folding of -0
Well, that was just a though. You're right that long runs of constants can appear, and it's better to avoid pathological behaviour in such cases. Your second path looks good. Eugene On Thu, Mar 10, 2011 at 6:30 PM, Antoine Pitrou solip...@pitrou.net wrote: On Thu, 10 Mar 2011 02:17:34 + (UTC) Eugene Toder elto...@gmail.com wrote: Indeed, see http://bugs.python.org/issue11244 Yes, I've noticed that too. However, if I'm not missing something, your patches do not address folding of -0. Btw, there's an alternative approach to allow recursive constant folding. Instead of keeping a stack of last constants, you can keep a pointer to the start of the last (LOAD_CONSTs + NOPs) run and the number of LOAD_CONSTs in that run (called lastlc in the current version). When you want Nth constant from the end, you start from that pointer and skip lastlc-N constants. You also make a function to get next constant from that point. This approach has worse time complexity for searching in a long run of LOAD_CONSTs, Yes, the stack basically acts as a cache to avoid computing all this again and again. however, there are upsides: - very long runs of constants are rare in real code True, but they might appear in generated code. - it uses less memory and doesn't have arbitrary limits on the size of the run Neither does the latest patch. - it's book-keeping overhead is smaller, so when you don't have long runs of constants (common case, I believe), it should be faster The book-keeping overhead should be quite small really, it's a simple autogrowing array with O(1) access and amortized append time. What's left is the overhead of the initial malloc() (and final free()). - I think it's simpler to implement Feel free to propose an alternate patch, but I'm not sure that it would be significantly simpler (and a stack is a well-known data structure). Also, please present some benchmarks if you do. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] funky buildbot
David Bolen db3l@gmail.com wrote: There used to be a way to request a ping from the master side (I think on the same page you could manually run a build from) that I would used to force it to recognize a slave was really down, but after the web interface got rearranged a while back, I can't seem to find the option any more. Looks like the ping option is back now :-). I restarted the buildslave, and it built 3.2 OK. Still don't know what's up, but this runaway buildslave condition has happened three times in the last week or so. Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] funky buildbot
Bill Janssen jans...@parc.com writes: David Bolen db3l@gmail.com wrote: There used to be a way to request a ping from the master side (I think on the same page you could manually run a build from) that I would used to force it to recognize a slave was really down, but after the web interface got rearranged a while back, I can't seem to find the option any more. Looks like the ping option is back now :-). Hey, how about that. (I had taken a quick peek before writing the response to verify it was still unavailable so it's definitely recent) Thanks to whomever restored that. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] (2.6): Issue #11424: Fix bug in determining child loggers.
On Thu, Mar 10, 2011 at 5:41 PM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: - Original Message From: Éric Araujo mer...@netwok.org From what I understand, we're supposed to forward-port in Mercurial, Correct, but only in maintained branches, not security-mode branches. Well, I saw this recent mail from Antoine: http://mail.python.org/pipermail/python-dev/2011-March/108766.html where he mentioned - on one hand: 2.5 - 2.6 - 2.7 (if you still want to maintain 2.5) - on the other hand: 3.1 - 3.2 - default and perhaps misunderstood. I was assuming that there wouldn't be any 2.6 releases from Mercurial, perhaps that assumption was unwarranted. Also, the section on forward-porting in the Dev Guide says it should be applied to the oldest branch applicable and this could be interpreted to mean applicable based on the content of the branch, rather than according to release policy. It's the intersection: if the change applies to the content of that branch and that branch is still maintained, then start there. So security fixes start further back than ordinary bug fixes. This should be reverted in 2.6, with the dummy merge to prevent inadvertent reversion in 2.7. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] funky buildbot
On Mar 10, 2011, at 3:18 PM, Bill Janssen wrote: It's a new Mac Mini running the latest Snow Leopard, with Python 2.6.1 (the /usr/bin/python) and buildslave 0.8.3, using Twisted 8.2.0. I realize that Python 2.6 is pretty old too, but a _lot_ of bugfixes have gone into Twisted since 8.2. I'm not 100% sure this is a Twisted issue but you may want to try upgrading to 10.2.0 and see if that fixes things. (I have a dim memory of similar issues which were eventually fixed by something in our subprocess support...) -glyph ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GPL'd python code vs Python2.6 linked against OpenSSL
Martin v. Löwis writes: Note that it is ultimately up to a court to interpret these words of the GPL, not to the FSF lawyer. True, and in the case of a non-FSF product, any ambiguities would be resolved first by determining the intent of the copyright owner, second (perhaps even overriding the intent of the owner) by the understanding of the user, and only third by asking the author of the license. Nevertheless, in the U.S. at least the court would likely be influenced in its judgments about intent and understanding by public statements of the author, because those would tend to condition the general public's understanding, including that of the parties to the license. Specifically, in Major Component, in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. the word essential would refer to running the compiler or the operating system or interpreter, not to a component essential to running the program but in general optional for using the system. Maybe my English is not good enough Mine is definitely not! :-) - I read that to mean that the compiler itself is a major component of the system (and thus doesn't need to be distributed along with the source of the GPL'ed program). Note that it explicitly also says that a object code interpreter used to run it (the executable) is a major component, which would explicitly declare Python a major component (as long as it's actually distributed with the system). Well, English being what it is, your guess is as good as mine.0.2 wink My understanding is that the compiler or interpreter is not being defined as a major component. It is being defined as something which, like a major component, is exempt from the source distribution and licensing requirements of the GPL. As is typical of the GPL :-(, the English is ambiguous as to whether major essential components of compilers or interpreters are exempt, too. I would think so, though. This simplifies licensing of things like the runtime code distributed with GCC, Java Swing (or whatever the preferred GUI is nowadays), or the implementation of print() in Python 3 dramatically, at little cost in additional leniency that I can see. That is, the reason that Java and Python programs can usefully be GPL is unrelated to the major component part of that clause. Rather, my understanding of the above GPLv3 clause is that it promotes VMs and interpreters to the level of *platform*, just like the kernel. In general using a language *translator* is *not* considered to create a derivative, because a derivative is a (possibly partial or modified) copy of the original. I've asked Richard Stallman about this specifically for GPLv2, and that's what he told me about Emacs Lisp: it is possible to write and distribute non-GPL programs in Emacs Lisp (but difficult, because you have to avoid using Emacs-specific data structures and functions). This makes a fair amount of sense anyway, since the translator works on the program as data, but the translator is not (usually) copied into the program. In other words, the GPL doesn't apply to the output of GCC, not because of the generosity of Richard Stallman, but because in U.S. copyright law, at least, it *can't*.[1] In the case of an interpreter, this is a lot more fuzzy (the interpreter is linked to the implementations of the statements). The English/legalese of GPLv2 doesn't make this clear. I think that's why GPLv3 explicitly specifies interpreters of object code in that clause. If your interpetation was right and Python is *not* a major component, then any GPL'ed Python application would need to distribute the source code of Python along with its own source code, and with that the source of all libraries that are not major components themselves. Yes, for the libraries. That's what James is worried about. Also, if your interpretation was right, it wouldn't be possible to license Java programs under the GPL, since the JVM wouldn't be a major component, No, of course it's possible for the copyright holders to license Java programs under the GPL, and even for you (not the owner) to create derivatives, as long as you are confident that the copyright holders won't sue you. As for what the law and the GPL actually say about these matters, surely it is not news to you that the GPL sucks unless you want all software GPLed, and even then there are regrettable side effects. Footnotes: [1] OTOH, the bison license contains a special clause because the hairy parser is a fairly extensive piece of C code that is copied into a program using bison to compile yacc specifications. This clause says basically the hairy parser is part of bison but as a special exception its presence in your program is OK even if
