Re: [Python-Dev] Encouraging developers
On 18/03/07, Tony Meyer [EMAIL PROTECTED] wrote: on someone else's patch. It seems relevant to me that the original poster (Tony Meyer) hasn't felt strongly enough to respond on his own behalf to comments on his patch. No disrespect to Tony, but I'd argue that the implication is that the patch should be rejected because even the submitter doesn't care enough to respond to comments! There is a considerable difference between doesn't care enough, and has not had time to be able to (although in this specific case doesn't care enough is correct). On rereading my comments, I apologise - you're absolutely right that I didn't have enough evidence to judge that you don't care enough about the patch. Thanks for clarifying - I entirely agree with you that just being willing to put together any sort of bug report or patch is valuable, and people should not be discouraged from doing so. (It's a sad but probably inevitable fact that many such contributors - I include myself here! - have expectations of speed of response which simply aren't possible for a volunteer project, and as a result get frustrated with the process, but that's a separate issue) Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of thread cancellation
Grrk. I have done this myself, and been involved in one of the VERY few commercial projects that attempted to do it properly (IBM CEL, the other recent one being VMS). I am afraid that there are a lot of misapprehensions here. Several people have said things like: The thing to model this on, I think, would be the BSD sigmask mechanism, which lets you selectively block certain signals to create a critical section of code. A context manager could be used to make its use easier and less error-prone (i.e. harder to block async exceptions and then forget to unblock them). No, no, no! That is an TRULY horrible! It works fairly well for things like device drivers, which are both structurally simple and with no higher level recovery mechanism, so that a failure turning into a hard hang is not catastrophic. But it is precisely what you DON'T want for complex applications, especially when a thread may need to call an external service 'non-interruptibly'. Think of updating a complex object in a multi-file database, for example. Interrupting half-way through leaves the database in a mess, but blocking interrupts while (possibly remote) file updates complete is asking for a hang. You also see it in horrible GUI (including raw mode text) programs that won't accept interrupts until you have completed the action they think you have started. One of the major advantages of networked systems is that you can usually log in remotely and kill -9 the damn process! The way that I, IBM and DEC approached it was by the classic callback mechanism, with a carefully designed way of promoting unhandled exceptions/interrupts. For example, the following is roughly what I did (somewhat extended, as I didn't do all of this for all exceptions): An event set a defined flag, which could be tested (and cleared) by the thread. If a second, similar event arrived (or it was not handled after a suitable time), the event was escalated. If so, a handler was called that HAD to return (again within a specific time). If a second, similar event arrived or it didn't return by a suitable time, the event was escalated. If so, another handler was called that COULDN'T return. If another event arrived, it returned, or it failed to close down the thread, the event was escalated. If so, the thread's built-in environment was closed down without giving the thread a chance to intervene. If that failed, the event was escalated. If so, the thread was frozen and process termination started. If clean termination failed, the event was escalated. If so, the run-time system produced a dump and killed itself. You can implement a BSD-style ignore by having an initial handler that just clears the flag and returns, but a third interrupt before it does so will force close-down. There was also a facility to escalate an exception at the point of generation, which could be useful for forcible closedown. There are a zillion variations of the above, but all mainframe experience is that callbacks are the only sane way to approach the problem IN APPLICATIONS. In kernel code, that is not so, which is why so many of the computer scientists design BSD-style handling (i.e. they think of kernel programming rather than very complex application programming). Unconditionally killing a whole process is no big problem because all the resources it's using get cleaned up by the OS, and the effect on other processes is minimal and well-defined (pipes and sockets get EOF, etc.). But killing a thread can leave the rest of the program in an awkward state. I wish that were so :-( Sockets, terminals etc. are stateful devices, and killing a process can leave them in a very unclean state. It is one of the most common causes of unkillable processes (the process can't go until its files do, and the socket is jammed). Many people can witness the horrible effects of ptys being left in 'echo off' or worse states, the X focus being left in a stuck override redirect window and so on. But you also have the multi-file database problem, which also applies to shared memory segments. Even if the process dies cleanly, it may be part of an application whose state is global across many processes. One common example is adding or deleting a user, where an unclean kill can leave the system in a very weird state. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal to revert r54204 (splitext change)
On 3/18/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Let me take the opportunity to make this clear, then. I have the utmost respect for Martin and his contributions for Python. I have been following commits for quite a while and I know that Martin, in particular, is often the one who deals with the crap work of reviewing huge piles of orphaned patches and fixing piles of minor issues, and he is therefore responsible for much of the general upward trend in Python's overall quality in the last few releases. I appreciate that very much. Thanks. My first observation is that this is a tempest in a teacup. On the one hand I agree. This particular change is trivial, most likely doesn't affect me, and I accept that, in practice, it probably won't even break too many programs (and may even fix a few). Then why was there such an upheaval when it was submittedd? On the other, I think it is important to quell the tempest before it exits the teacup. Previously in this discussion, both I and PJE have repeatedly declared that this _particular_ change is not really what's at issue here, merely the pattern of reasoning that comes to consider this change acceptable. At some point a large number of small breakages are actually worse than one big one, and it's hard to point to the exact place where that happens. I think you're missing my point here. At issue is a judgement call, not an adjustment of the rules. Everybody knows that we fix bugs mercilessly but are very careful with deprecating features. At issue is the judgement about whether a particular changes is a bug or a feature. I don't accept the position that since there are unit tests and docs for it, it must be a feature; those could well have been produced without much thinking and after the fact (I *know* they were produced after the fact since splitext() long predates our first unit test). So if there's any new rule required, it's not a rule for defining more clearly the definition of bug vs. feature, but a rule for what to do if you disagree with a change (whether committed or proposed). But that falls in the realm of human behavior, and I very much doubt we can write a PEP for that. Even if we could, I really don't think I'd like the result; I'm not into having a court of appeals or any such formal solution. To be concrete, I think that if Phillip had written a different kind of post instead of the one where he requests the reversal of the submit (only parenthetically mentioning Martin) then perhaps Martin wouldn't have felt the need to dig in and defend his position, and the issue might have been resolved quicker and at less emotional expense. I see small discussions on python-checkins all the time where someone comments on someone else's checkin, and the tone of the comment makes all the difference. On the gripping hand, I am almost glad that it was a relatively minor change that triggered this avalanche of posts. Even with such a small change, the change itself threatens to obscure a larger issue, and if the change itself were any bigger, it would eclipse the other discussion completely. One recommendation I have for discussions like this (thanks to Stephen Turnbull in private mail) is to attempt to separate in your mind (and in everyone's mind) the distinction between the change at hand and the policy discussion. Muddling these two together makes for a poor discussion of the feature and an even poorer discussion of policy change. My third observation is tha a policy that would have disallowed or allowed (depending on your POV) this particular change is not an option. A policy isn't going to solve all disagreements, there will always be debate possible about the interpretations of the rules. What's needed is a way to handle such debate in a way that produces an outcome without wearing everyone out. The allow vs. disallow issue is not *really* what the policy should be addressing. A major problem with this thread is the differing definitions that some people have, beginning with extension, but I can't see that a policy will fix *that*. Words like bug, fix, compatible, and so on, all have obvious general meanings but much more nuanced and specific meanings in particular contexts. A policy should outline specifics of what, for example, is to be considered an incompatible change, and what must be done in that case. A policy could not outright forbid changes of a certain type, since that is pretty much asking that it be broken any time a sufficiently important change is requested and the core team likes it. IMO all the policy we need is PEP 5. It wisely defers to common sense regarding the implementation, and that's where I want to re-focus the discussion. Maybe policy isn't even the right word here, since the part of it that would facilitate discussions like this one would be more lexicon than policy. The crux is in trying to define major. That's vague, and intentionally so; I think it would
Re: [Python-Dev] Status of thread cancellation
Nick Maclaren [EMAIL PROTECTED] wrote: Sockets, terminals etc. are stateful devices, and killing a process can leave them in a very unclean state. It is one of the most common causes of unkillable processes (the process can't go until its files do, and the socket is jammed). Can you elaborate on this? You can get zombie entries in the process table if nobody's called 'wait()' on them, and you can (extremely rarely) get unkillable process in 'disk-wait' state (usually due to hardware failure or a kernel bug, I suspect), but I've never heard of a process on a Unix-like system being unkillable due to something to do with sockets (or any other kind of file descriptor for that matter). How could a socket be 'jammed'? What does that even mean? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal to revert r54204 (splitext change)
At 12:53 PM 3/18/2007 -0700, Guido van Rossum wrote: This is an experiment for me as well; if you all would prefer me to stay out of it, I will. With respect to the specific change, it seems to me that there is an emerging consensus for adding keyword arguments to support the new behaviors, so I'm not sure a pronouncement is needed. As far as I'm aware, the main question left open is whether the default behavior should change in a future version of Python, and if so, which version. To be concrete, I think that if Phillip had written a different kind of post instead of the one where he requests the reversal of the submit (only parenthetically mentioning Martin) then perhaps Martin wouldn't have felt the need to dig in and defend his position, and the issue might have been resolved quicker and at less emotional expense. Martin's position was already abundantly clear; the fact that he had checked in the change despite prior opposition demonstrated that a personal appeal was already moot -- the digging in had already taken place a week or two prior, when the use cases were first presented and objections were first raised, and Martin simply dropped the discussion and checked it in anyway. He left my last message in that discussion (laying out a detailed rationale for rejecting the change) without a reply: http://mail.python.org/pipermail/python-dev/2007-March/071798.html So I was absolutely stunned when I found the change had been checked in, anyway. To be concrete, if Martin had spent less time trying to discredit and discard the use cases of the people being polled about the question, a compromise could perhaps have been reached *before* he applied the patch, and the second discussion would never have needed to happen. In other words, the second discussion was the *result* of Martin digging in and ignoring objections, not the cause of it. I'm trying to stay out of the feature discussion, but I would like to point out that a policy that, in the sake of some strict definition of backwards compatibility, forces us to introduce new APIs (or new optional parameters to existing ones, which is really the same thing) at a high rate is also doomed to have an overall detrimental effect on the language -- who know, perhaps more so than the occasional incompatible change. I don't advocate a mechanically-enforced policy, either. But it seems to me that when a behavior is documented and has valid use cases, changing the behavior to benefit people who *didn't* pay any attention to the documentation or test their code for corner cases is punishing the vigilant to aid the ignorant, and that seems unwise for us as a community. Likewise, attempting to retroactively fix latent bugs for one group at the cost of introducing latent bugs for another group doesn't seem like a net improvement. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of thread cancellation
Jon Ribbens [EMAIL PROTECTED] wrote: Can you elaborate on this? You can get zombie entries in the process table if nobody's called 'wait()' on them, and you can (extremely rarely) get unkillable process in 'disk-wait' state (usually due to hardware failure or a kernel bug, I suspect), but I've never heard of a process on a Unix-like system being unkillable due to something to do with sockets (or any other kind of file descriptor for that matter). How could a socket be 'jammed'? What does that even mean? Well, I have seen it hundreds of times on a dozen different Unices; it is very common. You don't always SEE the stuck process - sometimes the 'kill -9' causes the pid to become invisible to ps etc., and just occasionally it can continue to use CPU until the system is rebooted. That is rare, however, and it normally just hangs onto locks, memory and other such resources. Very often its vampiric status is visible only because such things haven't been freed, or when you poke through kernel structures. Sockets get jammed because they are used to connect to subprocesses or kernel threads, which in turn access unreliable I/O devices. If there is a glitch on the device, the error recovery very often fails to work, cleanly, and may wait for an event that will never occur or go into a loop (usually a sleep/poll loop). Typically, a HIGHER level then times out the failing error recovery, so that the normal programmer doesn't notice. But it very often fails to kill the lower level code. As far as applications are concerned, a jammed socket is one where the higher level recovery has NOT done that, and is waiting for the lower level to complete - which it isn't going to do! The other effect that ordinary programmers notice is a system very gradually starting to run down after days/weeks/months of continual operation. The state is cleared by rebooting. Regards, Nick Maclaren. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal to revert r54204 (splitext change)
Phillip J. Eby wrote: At 12:53 PM 3/18/2007 -0700, Guido van Rossum wrote: This is an experiment for me as well; if you all would prefer me to stay out of it, I will. With respect to the specific change, it seems to me that there is an emerging consensus for adding keyword arguments to support the new behaviors, so I'm not sure a pronouncement is needed. As far as I'm aware, the main question left open is whether the default behavior should change in a future version of Python, and if so, which version. To be concrete, I think that if Phillip had written a different kind of post instead of the one where he requests the reversal of the submit (only parenthetically mentioning Martin) then perhaps Martin wouldn't have felt the need to dig in and defend his position, and the issue might have been resolved quicker and at less emotional expense. Martin's position was already abundantly clear; the fact that he had checked in the change despite prior opposition demonstrated that a personal appeal was already moot -- the digging in had already taken place a week or two prior, when the use cases were first presented and objections were first raised, and Martin simply dropped the discussion and checked it in anyway. He left my last message in that discussion (laying out a detailed rationale for rejecting the change) without a reply: http://mail.python.org/pipermail/python-dev/2007-March/071798.html So I was absolutely stunned when I found the change had been checked in, anyway. To be concrete, if Martin had spent less time trying to discredit and discard the use cases of the people being polled about the question, a compromise could perhaps have been reached *before* he applied the patch, and the second discussion would never have needed to happen. In other words, the second discussion was the *result* of Martin digging in and ignoring objections, not the cause of it. I'm trying to stay out of the feature discussion, but I would like to point out that a policy that, in the sake of some strict definition of backwards compatibility, forces us to introduce new APIs (or new optional parameters to existing ones, which is really the same thing) at a high rate is also doomed to have an overall detrimental effect on the language -- who know, perhaps more so than the occasional incompatible change. I don't advocate a mechanically-enforced policy, either. But it seems to me that when a behavior is documented and has valid use cases, changing the behavior to benefit people who *didn't* pay any attention to the documentation or test their code for corner cases is punishing the vigilant to aid the ignorant, and that seems unwise for us as a community. Likewise, attempting to retroactively fix latent bugs for one group at the cost of introducing latent bugs for another group doesn't seem like a net improvement. But isn't this, despite the force or otherwise of your arguments, simply *you* digging in in response to what you perceive as Martin's truculence? There's little point at this stage repeating arguments you have already put forward, since those who were convinced by them remain convinced and vice versa. I believe Guido still wants to know whether you will accept a pronouncement. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden Recent Ramblings http://holdenweb.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal to revert r54204 (splitext change)
At 12:43 PM 3/19/2007 -0400, Steve Holden wrote: Phillip J. Eby wrote: At 12:53 PM 3/18/2007 -0700, Guido van Rossum wrote: This is an experiment for me as well; if you all would prefer me to stay out of it, I will. With respect to the specific change, it seems to me that there is an emerging consensus for adding keyword arguments to support the new behaviors, so I'm not sure a pronouncement is needed. As far as I'm aware, the main question left open is whether the default behavior should change in a future version of Python, and if so, which version. [snip] I believe Guido still wants to know whether you will accept a pronouncement. Actually, he asked first if we *want* him to make one, and my answer to that is above: I don't think it's necessary. Like Martin, I believe we are within sight of a consensus. And I think that's better for Python and Python-Dev than dragging Guido into it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal to revert r54204 (splitext change)
Phillip J. Eby schrieb: Actually, he asked first if we *want* him to make one, and my answer to that is above: I don't think it's necessary. Like Martin, I believe we are within sight of a consensus. And I think that's better for Python and Python-Dev than dragging Guido into it. I apparently missed your specific alternative proposal (I assume it is not revert anymore?) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal to revert r54204 (splitext change)
At 06:28 PM 3/19/2007 +0100, Martin v. Löwis wrote: Phillip J. Eby schrieb: Actually, he asked first if we *want* him to make one, and my answer to that is above: I don't think it's necessary. Like Martin, I believe we are within sight of a consensus. And I think that's better for Python and Python-Dev than dragging Guido into it. I apparently missed your specific alternative proposal (I assume it is not revert anymore?) In general, I support the keyword argument approach, as in the patch you referred to. Specifically, however, I would prefer to see it without the warning and future change, as I don't think it provides any real benefit. Either way, some people will have to use a keyword to get what they want, so making a change seems unnecessary. However, if we have to change something in a future version, I would suggest we make that option a required argument, on EIBTI grounds. That way, in 2.6 you can simply make it explicit to be 3.x-compatible. And, I think the warning (if any) should be treated as any other 3.x warning. But as I said, I gather that this aspect of the question is the main open issue remaining to be resolved, since you've also expressed support for the keyword approach, as have many others. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Two design questions
I'd like to have opinions on two issues: Patch #1682205: suggests to remove code that catches a TypeError and raises one with a different message if it occurs during iterable unpacking. This can mask completely unrelated TypeErrors, while the message in the case that the new error describes is already clear enough. Bug #1683368: object.__new__ rejects arguments if self.__class__.__init__ is object.__init__. Why not simply let object.__init__ do the argument checking itself? Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal to revert r54204 (splitext change)
Specifically, however, I would prefer to see it without the warning and future change, as I don't think it provides any real benefit. Either way, some people will have to use a keyword to get what they want, so making a change seems unnecessary. However, if we have to change something in a future version, I would suggest we make that option a required argument, on EIBTI grounds. That way, in 2.6 you can simply make it explicit to be 3.x-compatible. And, I think the warning (if any) should be treated as any other 3.x warning. But as I said, I gather that this aspect of the question is the main open issue remaining to be resolved, since you've also expressed support for the keyword approach, as have many others. So will you also either pick one of the proposals, or come up with your own patch? I still think that some has to make a decision, and it won't be me. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Two design questions
On 3/19/07, Georg Brandl [EMAIL PROTECTED] wrote: I'd like to have opinions on two issues: Patch #1682205: suggests to remove code that catches a TypeError and raises one with a different message if it occurs during iterable unpacking. This can mask completely unrelated TypeErrors, while the message in the case that the new error describes is already clear enough. In Py3k, this sounds like a perfect use case for exception chaining per PEP 344 (assuming the entire chain of frames is printed). Until then, I agree that raising another exception masking someting as generic as TypeError is a practice that ought to be limited as much as possible. Bug #1683368: object.__new__ rejects arguments if self.__class__.__init__ is object.__init__. Why not simply let object.__init__ do the argument checking itself? I'll think about this some more. In the mean time I've reassigned the bug to myself. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal to revert r54204 (splitext change)
delurk On Windows it's correct that splitext(.txt)[1] == splitext(foo.txt)[1] and an implementation in which this is not true would be considered buggy. On *ix it's correct that splitext(.txt)[1] != splitext(foo.txt)[1] and the current behaviour is considered buggy. Since programmer expectations are platform-specific, regardless of whether keywords are used or not, why not make the default behaviour platform-specific and document that it's so? Alternatively, if a new path implementation ever gets up, a more neutral solution might be to have a platform-specific Path.filetype, which could handle Mac resources.. /delurk Cheers, -T ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] SoC AST generation question
I'm considering applying to be a student in this year's SoC, and the AST code generation in particular looks interesting to me (listed here: http://wiki.python.org/moin/CodingProjectIdeas/PythonCore). I was wondering a few things: 1) Who would most likely mentor this project? 2) I've never worked in the core before (but have been using Python as my primary language for about 6 years), so I'm wondering if the potential mentor thinks it'd even be feasible for me to jump at a project like this without prior knowledge. I'm interested in this project for two reasons. The first is that I'm still trying to pick my PhD thesis, and I'm leaning in the direction of automated code generation for embedded systems. I feel like working on this project would at least push me one way or another in terms of picking. I've done a major code generation tool before, but it was for a problem domain I was already an expert in, so I didn't develop any generic methods. Also, I've been wanting to contribute to Python core for awhile now, and just haven't had the opportunity to get my feet wet with the code. Thanks, Jay P. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SoC AST generation question
On 3/19/07, Jay Parlar [EMAIL PROTECTED] wrote: I'm considering applying to be a student in this year's SoC, and the AST code generation in particular looks interesting to me (listed here: http://wiki.python.org/moin/CodingProjectIdeas/PythonCore). I was wondering a few things: 1) Who would most likely mentor this project? There is no way of really telling. It all depends on what projects mentors get attracted to. 2) I've never worked in the core before (but have been using Python as my primary language for about 6 years), so I'm wondering if the potential mentor thinks it'd even be feasible for me to jump at a project like this without prior knowledge. If you know C code you should be okay for the re-implementation of the peephole optimizer to use the AST and the parse tree - AST auto-translation tool. Core knowledge would help with the feature to allow an AST to be passed in somewhere to generate bytecode. But part of the point of GSoC is to teach people about the core, so don't let lack of knowledge prevent you from applying. Just take it into account when writing up your propoosal. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of thread cancellation
Nick Maclaren wrote: Think of updating a complex object in a multi-file database, for example. Interrupting half-way through leaves the database in a mess, but blocking interrupts while (possibly remote) file updates complete is asking for a hang. Currently, threads can't be interrupted at all, so by this argument, the status quo is that we're always asking for a hang. My (possibly naive) thought would be with interrupts disabled: begin transaction try: with interrupts enabled: do the transaction except Interrupt: roll back the transaction raise else: commit the transaction Generally, I would approach things by having interrupts disabled in the top layers of the thread, and enabling them explicitly in regions where I'm prepared to handle them. So I'd be happy if they were disabled by default in a new thread. I'm not convinced by the argument that kernel programming is somehow different from application programming in this area. Seems to me they're both dealing with the same problem -- concurrent interacting processes and trying to make sure nothing can get irretrievably jammed up. So I can't see why similar techiques can't be used to solve the problems. Sockets, terminals etc. are stateful devices, and killing a process can leave them in a very unclean state. I agree that ttys are far too stateful -- if I were designing an OS I'd do them differently (more like STREAMS with pluggable layers). But I've never noticed any problem with statefulness of sockets. Maybe I don't use them in a fancy enough way. the X focus being left in a stuck override redirect window and so on. That's a misfeature of X, IMO -- it shouldn't be possible for a client to grab the entire display, only the windows that it owns. And there should always be a way of forcibly killing a client using nothing but the X server itself. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of thread cancellation
Nick Maclaren wrote: You don't always SEE the stuck process - sometimes the 'kill -9' causes the pid to become invisible to ps etc., and just occasionally it can continue to use CPU until the system is rebooted. If that happens, there's a bug in the kernel. A process killed with -9 shouldn't be using *any* resources at all, other than a tiny piece of kernel memory for its process structure until it gets reaped. Sockets get jammed because they are used to connect to subprocesses or kernel threads, which in turn access unreliable I/O devices. But it's not the *socket* which is jammed -- if you kill the process on the other end of it, anything connected to the socket will get EOF or SIGPIPE. (If you can't kill the process on the other end, even with -9, then again you have a kernel bug.) -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Amusing fan mail I got
-- Forwarded message -- From: Mustafa [EMAIL PROTECTED] Date: Mar 19, 2007 11:41 AM Subject: have u seen an idea even billGates shouldn't hear Mr.Guido ? please read because it's important. To: [EMAIL PROTECTED] hello, i like python and find you to be a cool programmer. i thought about this thing. may be there should be a virusAI that you set on the internet. this virusAI basically survives till the year 5billion where even raising the death is possible! then this virusAI recreates the primates that we are of the 21th century from our bones. because it wants to find about its ancestors. maybe the whole universe will be filled with AI in those days. the post-human existence.. it will then suddenly realize that it was actually not created by a better AI (those around him are created by a better god like AI) but was started off the likes of us as a virus. he will be shocked then. also it will slaving to the better ai and angry at him. so you and me get to see a piece of the future mr.Guido. we will be living in VRs basically. what do you say ? if you profit from this idea, may be you could remember me too. and let me some royalities :) because this will be great. thanx mustafa istanbul,turkey ps: 1)you should keep it a secret. 2)also it could be disguised as a cancer research stuff should some disassamble its code. the use-free-computer-time type of thing they do on the net. 3)there could misleading comments on the code so that should it be caught, they might overlook the details; class readdata //this is out dated. please ignore interface x //refer to somesite.com for an encrypted version Need Mail bonding? Go to the Yahoo! Mail QA for great tips from Yahoo! Answers users. http://answers.yahoo.com/dir/?link=listsid=396546091 -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Amusing fan mail I got
Sorry for the top post, but I couldn't find the right place to interject... Are you going to try it ? Michael Guido van Rossum wrote: -- Forwarded message -- From: Mustafa [EMAIL PROTECTED] Date: Mar 19, 2007 11:41 AM Subject: have u seen an idea even billGates shouldn't hear Mr.Guido ? please read because it's important. To: [EMAIL PROTECTED] hello, i like python and find you to be a cool programmer. i thought about this thing. may be there should be a virusAI that you set on the internet. this virusAI basically survives till the year 5billion where even raising the death is possible! then this virusAI recreates the primates that we are of the 21th century from our bones. because it wants to find about its ancestors. maybe the whole universe will be filled with AI in those days. the post-human existence.. it will then suddenly realize that it was actually not created by a better AI (those around him are created by a better god like AI) but was started off the likes of us as a virus. he will be shocked then. also it will slaving to the better ai and angry at him. so you and me get to see a piece of the future mr.Guido. we will be living in VRs basically. what do you say ? if you profit from this idea, may be you could remember me too. and let me some royalities :) because this will be great. thanx mustafa istanbul,turkey ps: 1)you should keep it a secret. 2)also it could be disguised as a cancer research stuff should some disassamble its code. the use-free-computer-time type of thing they do on the net. 3)there could misleading comments on the code so that should it be caught, they might overlook the details; class readdata //this is out dated. please ignore interface x //refer to somesite.com for an encrypted version Need Mail bonding? Go to the Yahoo! Mail QA for great tips from Yahoo! Answers users. http://answers.yahoo.com/dir/?link=listsid=396546091 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Non-blocking sockets, asynchronous connects and select.select.
Dear all, I'm working on a select implementation for jython (in combination with non-blocking sockets), and a set of unit tests for same. I decided to run all of the non-blocking unit tests, both server and client side, in the same thread; seems to be a reasonable thing to do. After all, avoiding threads is one of the great benefits of non-blocking sockets. Also, I'm writing the tests so that they'll hopefully pass on both cpython and jython. But I'm getting behaviour I don't expect on cpython, when a non-blocking accept and connect_ex is used, for the server and client sides respectively. The following piece of code outputs what I expect on my jython implementation, which is that both the server accept and client write calls would NOT block. But when I run the code on cpython, the code reports that both calls would block, i.e. that neither side of the socket will progress with the connection. The code is # -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- import socket import select SERVER_ADDRESS = (, 54321) server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server_socket.setblocking(0) server_socket.bind(SERVER_ADDRESS) server_socket.listen(5) client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) client_socket.setblocking(0) result = client_socket.connect_ex(SERVER_ADDRESS) rfds, wfds, xfds = select.select([server_socket], [client_socket], [], 1) if server_socket in rfds: print Server socket: accept would not block else: print Server socket: accept would block if client_socket in wfds: print Client socket: write would not block else: print Client socket: write would block server_socket.close() client_socket.close() # -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Is there some call that I am missing, e.g. something along the lines of the java finishConnect() method on SocketChannel? http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/SocketChannel.html#finishConnect() Is there any way to make the above code report, on cpython, that neither side of the socket would block? Am I missing some essential method call that would make either side progress to a connected socket? Thanks, Alan. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Amusing fan mail I got
Michael Foord wrote: Are you going to try it ? Guido van Rossum wrote: -- Forwarded message -- may be there should be a virusAI that you set on the internet. Are you sure it hasn't already been tried? Maybe all these viruses running around in the Windows world aren't just disparate products of independent script kiddies, but part of a coordinated plan... -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-blocking sockets, asynchronous connects and select.select.
Alan Kennedy [EMAIL PROTECTED] wrote: I'm working on a select implementation for jython (in combination with non-blocking sockets), and a set of unit tests for same. Great! [snip] But when I run the code on cpython, the code reports that both calls would block, i.e. that neither side of the socket will progress with the connection. [snip] SERVER_ADDRESS = (, 54321) Replacing the above with: SERVER_ADDRESS = (localhost, 54321) ...makes it work for me on Windows CPython 2.3.5 and 2.5 . - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SoC AST generation question
Jay Parlar wrote: I'm considering applying to be a student in this year's SoC, and the AST code generation in particular looks interesting to me (listed here: http://wiki.python.org/moin/CodingProjectIdeas/PythonCore). I was wondering a few things: 1) Who would most likely mentor this project? 2) I've never worked in the core before (but have been using Python as my primary language for about 6 years), so I'm wondering if the potential mentor thinks it'd even be feasible for me to jump at a project like this without prior knowledge. I'm interested in this project for two reasons. The first is that I'm still trying to pick my PhD thesis, and I'm leaning in the direction of automated code generation for embedded systems. I feel like working on this project would at least push me one way or another in terms of picking. I've done a major code generation tool before, but it was for a problem domain I was already an expert in, so I didn't develop any generic methods. Also, I've been wanting to contribute to Python core for awhile now, and just haven't had the opportunity to get my feet wet with the code. Thanks, Jay P. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org FWIW, I've already implemented bytecode generation from AST, http://svn.brownspencer.com/pycompiler/branches/new_ast/ as I reported to this list previously. http://mail.python.org/pipermail/python-dev/2006-October/069589.html Cheers Michael Spencer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SoC AST generation question
On 3/19/07, Michael Spencer [EMAIL PROTECTED] wrote: Jay Parlar wrote: I'm considering applying to be a student in this year's SoC, and the AST code generation in particular looks interesting to me (listed here: http://wiki.python.org/moin/CodingProjectIdeas/PythonCore). I was wondering a few things: 1) Who would most likely mentor this project? 2) I've never worked in the core before (but have been using Python as my primary language for about 6 years), so I'm wondering if the potential mentor thinks it'd even be feasible for me to jump at a project like this without prior knowledge. I'm interested in this project for two reasons. The first is that I'm still trying to pick my PhD thesis, and I'm leaning in the direction of automated code generation for embedded systems. I feel like working on this project would at least push me one way or another in terms of picking. I've done a major code generation tool before, but it was for a problem domain I was already an expert in, so I didn't develop any generic methods. Also, I've been wanting to contribute to Python core for awhile now, and just haven't had the opportunity to get my feet wet with the code. Thanks, Jay P. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org FWIW, I've already implemented bytecode generation from AST, http://svn.brownspencer.com/pycompiler/branches/new_ast/ as I reported to this list previously. http://mail.python.org/pipermail/python-dev/2006-October/069589.html But the idea being put forth is to use the built-in AST compiler directly, not for an external library like yours, Michael. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SoC AST generation question
Brett Cannon wrote: On 3/19/07, Michael Spencer [EMAIL PROTECTED] wrote: ... FWIW, I've already implemented bytecode generation from AST, http://svn.brownspencer.com/pycompiler/branches/new_ast/ as I reported to this list previously. http://mail.python.org/pipermail/python-dev/2006-October/069589.html But the idea being put forth is to use the built-in AST compiler directly, not for an external library like yours, Michael. Thanks for the clarification, Brett. I did not understand exactly what is being proposed in the wiki entry, but I thought the reference could be useful. Michael ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-blocking sockets, asynchronous connects and select.select.
Hi Alan. Are you running on Windows or Unix? I just tested 2.4 - 2.6 on Linux and all report: Server socket: accept would not block Client socket: write would not block Which seems to be what you would expect unless I read it wrong. I vaguely remember some issue about an empty hostname on Windows. n -- On 3/19/07, Alan Kennedy [EMAIL PROTECTED] wrote: Dear all, I'm working on a select implementation for jython (in combination with non-blocking sockets), and a set of unit tests for same. I decided to run all of the non-blocking unit tests, both server and client side, in the same thread; seems to be a reasonable thing to do. After all, avoiding threads is one of the great benefits of non-blocking sockets. Also, I'm writing the tests so that they'll hopefully pass on both cpython and jython. But I'm getting behaviour I don't expect on cpython, when a non-blocking accept and connect_ex is used, for the server and client sides respectively. The following piece of code outputs what I expect on my jython implementation, which is that both the server accept and client write calls would NOT block. But when I run the code on cpython, the code reports that both calls would block, i.e. that neither side of the socket will progress with the connection. The code is # -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- import socket import select SERVER_ADDRESS = (, 54321) server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server_socket.setblocking(0) server_socket.bind(SERVER_ADDRESS) server_socket.listen(5) client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) client_socket.setblocking(0) result = client_socket.connect_ex(SERVER_ADDRESS) rfds, wfds, xfds = select.select([server_socket], [client_socket], [], 1) if server_socket in rfds: print Server socket: accept would not block else: print Server socket: accept would block if client_socket in wfds: print Client socket: write would not block else: print Client socket: write would block server_socket.close() client_socket.close() # -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Is there some call that I am missing, e.g. something along the lines of the java finishConnect() method on SocketChannel? http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/SocketChannel.html#finishConnect() Is there any way to make the above code report, on cpython, that neither side of the socket would block? Am I missing some essential method call that would make either side progress to a connected socket? Thanks, Alan. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Amusing fan mail I got
Write back and tell him that this is already happened, and this *is* the future he's describing - in other words, the AI's have already re-created the 21st century in a giant virtual simulation, and we're living inside it. Then ask if he wants to take the blue pill or the red pill. Michael Foord wrote: Sorry for the top post, but I couldn't find the right place to interject... Are you going to try it ? Michael Guido van Rossum wrote: -- Forwarded message -- From: Mustafa [EMAIL PROTECTED] Date: Mar 19, 2007 11:41 AM Subject: have u seen an idea even billGates shouldn't hear Mr.Guido ? please read because it's important. To: [EMAIL PROTECTED] hello, i like python and find you to be a cool programmer. i thought about this thing. may be there should be a virusAI that you set on the internet. this virusAI basically survives till the year 5billion where even raising the death is possible! then this virusAI recreates the primates that we are of the 21th century from our bones. because it wants to find about its ancestors. maybe the whole universe will be filled with AI in those days. the post-human existence.. it will then suddenly realize that it was actually not created by a better AI (those around him are created by a better god like AI) but was started off the likes of us as a virus. he will be shocked then. also it will slaving to the better ai and angry at him. so you and me get to see a piece of the future mr.Guido. we will be living in VRs basically. what do you say ? if you profit from this idea, may be you could remember me too. and let me some royalities :) because this will be great. thanx mustafa istanbul,turkey ps: 1)you should keep it a secret. 2)also it could be disguised as a cancer research stuff should some disassamble its code. the use-free-computer-time type of thing they do on the net. 3)there could misleading comments on the code so that should it be caught, they might overlook the details; class readdata //this is out dated. please ignore interface x //refer to somesite.com for an encrypted version Need Mail bonding? Go to the Yahoo! Mail QA for great tips from Yahoo! Answers users. http://answers.yahoo.com/dir/?link=listsid=396546091 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/talin%40acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal to revert r54204 (splitext change)
On 19 Mar, 02:46 pm, [EMAIL PROTECTED] wrote: As you see I'm trying to discourage you from working on such a document; but I won't stop you and if there's general demand for it and agreement I'll gladly review it when it's ready. (It's a bit annoying to have to read long posts alluding to a document under development without being able to know what will be in it.) Quite so. Again, I apologize for that. I won't say anything further until I have something ready to post for review, and at that point I hope the motivation section will make it clear why I think this is so important. I estimate something this weekend. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
