Re: [Python-Dev] Small RFEs and the Bug Tracker
On 2/19/08, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > > Well, we have to evaluate the chances of our older tickets to come to > > completion. I'm of the opinion that ticket getting older have very > > small chances of ever being completed. RFE for python 2.4 are likely > > to be irrelevant. > > Do you have any facts to base this theory on? > > Two years for a bug report is *nothing*. Ten years, and I would start > to worry that it might never get implemented. No, I don't, which is why I would find it interesting to run some queries on the roundup database to have completion statistics for low activity tickets. Is is possible to get a copy of that db somehow? Virgil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Different float formatting on Windows and Linux
"Eric Smith" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] | The tests for float.__format__ are breaking on Windows, because of this | issue: http://bugs.python.org/issue1600. Basically, Windows is using 3 | digits for exponents < 100, and Linux (and at least MacOS) are using 2. | | The patch attached to the issue proposes changing all platforms to use | at least 3 digits. It affects both '%' formatting and __format__ | formatting. Altering all float formatting involving exponents is a | pretty big change to make, and I want to get opinions here before making | this change. | | Guido's comments in the issue are supportive, and I agree that the | consistency would be good. I'm just concerned about changing the output | for existing code. | | I suppose another option would be to modify Windows to use 2 digits for | exponents < 100. I guess it just depends on whose output you want to break! | | I think the options are: | 1: Do nothing. Adapt the tests to deal with the differences. | 2: Force 3 characters for exponents < 100. | 3: Force 2 characters for exponents < 100. Since you posted this, Mark Dickensom added a comment to the tracker that 3 conforms to C99. If so, go with that. In any case, consistency would be nice. tjr ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Different float formatting on Windows and Linux
> The tests for float.__format__ are breaking on Windows, because of this > issue: http://bugs.python.org/issue1600. Basically, Windows is using 3 > digits for exponents < 100, and Linux (and at least MacOS) are using 2. Yes, this is very annoying and I once lost of lot of time because of the Windows difference. > I think the options are: > 1: Do nothing. Adapt the tests to deal with the differences. > 2: Force 3 characters for exponents < 100. > 3: Force 2 characters for exponents < 100. I'd go for 3. Rationale: this change will mostly affect scientific code, which is mostly developed and used on Unix systems. Thanks for taking care if this nuisance! Ralf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_CLEAR to avoid crashes
>> Most Py_DECREF calls are probably okay but it's going to be hard >> to find the ones that are not. > > I suppose Py_DECREF is not the only source of trouble. Many calls > to the Python API can end up calling arbitrary user code (via > __getattr__, __getitem__, etc.). Whenever an object does that, it > must be prepared to be accessed from user code. I'm guessing there > are many subtle bugs of this nature lurking. Py_DECREF is perhaps > the most common though. Maybe renaming it to > Py_DECREF_AND_RUN_EVIL_USER_CODE would help. ;-) But that's unrelated to this issue. In those other cases, the refcount won't be zero, so the object is still there. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_CLEAR to avoid crashes
On Feb 18, 2008 4:52 PM, Neil Schemenauer <[EMAIL PROTECTED]> wrote: > That sucks. Most Py_DECREF calls are probably okay but it's going > to be hard to find the ones that are not. I can't think of anything > we can do to make this trap harder to fall into. Even using > Py_CLEAR as a blunt tool is not a total solution. You could still > end up with a null pointer dereference if the code is not written > carefully. > Container types (particularly lists) go through great lengths to postpone object deletion. For example, to delete a slice from a list all of the items must be copied to a temporary array, then the list object's pointers are modified, then all the Py_DECREF's are called just before returning. I have always seen this as a robustness versus efficiency issue. It's theoretically possible to set things up so that reference counter decrements are actually postponed until after the C method/slot returns, but it's slower than doing it immediately. I wonder if adding support for postponed decrements (without making it mandatory) would at least make the trap harder to fall into. For example: - maintain a global array of pending decrefs - before calling into any C method/slot, save the index of the current end-of-array (in a local C variable on the stack) - call the C method, which may call Py_DECREF_LATER(x) to append x to the global array - when the C method returns, decref anything newly appended to the array The array would grow and shrink just as a list does (O(1) amortized time to add/remove a pointer). This would simplify a number of places in listobject.c as well as remove the need for Py_TRASHCAN_*. It would be entirely optional, so anyone who is very careful and wants the speed of Py_DECREF can have it. Also, the deferment is very brief, since the decrefs occur right after the C method returns. The downside is having to store and check the global array length on every C method call (basically 3 machine instructions). The machine instructions aren't so bad, but I'm not sure about the effects on the CPU cache. So, like I said, a robustness versus performance trade-off. :-( -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Py_CLEAR to avoid crashes
> A simple way to do this would be to push objects whose > refcounts had reached 0 onto a list instead of finalizing them > immediately, and have PyEval_EvalFrameEx periodically swap > in a new to-delete list and delete the objects on the old one. Some of the memory management threads discussed something similar to this, and pointed to IBM papers on Java. By adding states like "tenatively finalizable", the cost of using multiple processors was reduced. The down side is that objects which could be released (and recycled) immediately won't be -- which slows down a fair number of real-world programs that are used to the CPython refcount model. If the resource not being immediately released is scarce (such as file handles), it gets even worse. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
> This is still valid? Should we start moving RFEs to this PEP and > closing their issues in the tracker? As other have indicated - PEP 42 was a mistake (IMO). > Or should we try to get more discussion regarding these RFEs? Maybe, > for example, a weekly digest where the latests RFEs added are sent to > python-dev, so we can actually say "no way" and close them, or say > "nice" and *then* moving them to the PEP (this has the risk of nobody > saying anything, and to stay in the same position as before). > > What do you think? Opinions? If you think this could help, sure, but I doubt it would. I personally don't worry about these. If you don't want to see them, filter them out (roundup is very good at custom searches). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
> Well, we have to evaluate the chances of our older tickets to come to > completion. I'm of the opinion that ticket getting older have very > small chances of ever being completed. RFE for python 2.4 are likely > to be irrelevant. Do you have any facts to base this theory on? Two years for a bug report is *nothing*. Ten years, and I would start to worry that it might never get implemented. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_CLEAR to avoid crashes
I wrote: > Most Py_DECREF calls are probably okay but it's going to be hard > to find the ones that are not. I suppose Py_DECREF is not the only source of trouble. Many calls to the Python API can end up calling arbitrary user code (via __getattr__, __getitem__, etc.). Whenever an object does that, it must be prepared to be accessed from user code. I'm guessing there are many subtle bugs of this nature lurking. Py_DECREF is perhaps the most common though. Maybe renaming it to Py_DECREF_AND_RUN_EVIL_USER_CODE would help. ;-) Neil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_CLEAR to avoid crashes
> That sucks. Most Py_DECREF calls are probably okay but it's going > to be hard to find the ones that are not. Methinks that egrep 'DECREF\([a-zA-Z0-9_]->[a-zA-Z0-9_]+\)' */*.c gives a good overview - you should never DECREF a variable on heap. In the trunk, this command finds 36 candidates. Now, some of them are in tp_dealloc methods, so they shouldn't cause problems - but it can't hurt to replace them with Py_CLEAR, either. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use Python 3.0 syntax for except and raise?
I don't know if you're done with this already, but there's a lot of experience suggesting such sweeps are quite dangerous. In the past, whenever a sweep across the entire stdlib was done, it's always caused a few breakages, some of which didn't get caught until the next release. Things to worry about with these two changes: raise X, y -> raise X(y): this is incorrect if y is a tuple; *only* if it's a tuple, the correct translation is raise X(*y). But 2to3 can't know that (unless the tuple is written out in place). except X as y: in 3.0 this has different semantics -- y is explicitly deleted at the end of the except block. I don't know if this is also the semantics implemented in 2.6 (I think it should be), but again this can cause som equite subtle breakages that are hard to catch automatically. And since both of these are about exceptions, there's a high likelihood that some occurrences are not reached by a unittest. IOW, while I'm not dead set against it (I agree with your motivation in principle) I worry that in practice it may destabilize things., and would prefer a different approach where these things are only changed when someone is revising the module anyway. --Guido On Feb 17, 2008 8:57 AM, Christian Heimes <[EMAIL PROTECTED]> wrote: > Good evening everybody! > > I like to run the 2to3 tool with raise and except fixers over the 2.6 > sources. The raise fixer changes "raise Exception, msg" to "raise > Exception(msg)" and the except fixer replaces "except Exception, err" by > "except Exception as err". In my humble opinion the Python stdlib should > give proper examples how write good code. > > During the migration period from the 2.x series to 3.x we have two > obvious ways to write code. Let's stick to the new and preferred way. > > Oh and please use the new syntax for patches, too. It makes my job with > svnmerge a little bit easier. > > Thanks! > > Christian > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_CLEAR to avoid crashes
On Mon, Feb 18, 2008 at 05:48:57PM +0100, Amaury Forgeot d'Arc wrote: > For example, in exception.c, BaseException_init() starts with the instruction: > Py_DECREF(self->args); > this may call __del__ on self->args Ah, I understand now. We are not talking about tp_dealloc methods (the GC takes great pains to avoid this scenario). However, any object that calls Py_DECREF outside of its tp_dealloc method must be prepared for finalizers to access it in arbitrary ways. That sucks. Most Py_DECREF calls are probably okay but it's going to be hard to find the ones that are not. I can't think of anything we can do to make this trap harder to fall into. Even using Py_CLEAR as a blunt tool is not a total solution. You could still end up with a null pointer dereference if the code is not written carefully. Neil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
[Steve Holden] > I appreciate the desire to "keep the issue tracker clean", but I think > human intelligence needs to be applied to the task, not just a > date-based cutoff. I concur. The older bug reports are ones that have survived past human-based clean-up efforts. They were left open as a reminder, as a todo, to await time or decision by a module author, or it wasn't clear how to solve valid report. IIRC, there are some for regexes that would take the time of the one or two people supporting that module and the request would cover a not urgently needed corner cases (like handling empty matches or speeding-up badly designed regexes). Those bug reports and rfes are valid and should be left open unless a module author decides that re's won't support the request. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
On 2/18/08, Steve Holden <[EMAIL PROTECTED]> wrote: > I appreciate the desire to "keep the issue tracker clean", but I think > human intelligence needs to be applied to the task, not just a > date-based cutoff. Personally, the bug count doesn't bother me. I was just responding to Brett's concerns about the high issue count, saying that having the issue tracker keep track of the RFE is not what makes that count so high. I'm ok with the status quo. Virgil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
Virgil Dupras wrote: > On 2/18/08, Steve Holden <[EMAIL PROTECTED]> wrote: >> I'm not sure we should be throwing RFE's away with such casual abandon >> just because nobody had time to pay them any attention in six months - >> nor bugs neither, come to that. > > Well, we have to evaluate the chances of our older tickets to come to > completion. I'm of the opinion that ticket getting older have very > small chances of ever being completed. RFE for python 2.4 are likely > to be irrelevant. old bugs are likely to already be fixed. Maybe we > could run a statistical analysis to compute the chances of a ticket > that have seen no activity for 8 months to ever be successfully > completed? How many successful tickets to we have that had a 8+ months > gap between activity? Or maybe we could just clean out the 400 tickets > that are 2+ years old? What are the chances for a 2 years old ticket > to be completed? > But the decision shouldn't be made on the age of the ticket, rather on the (continued?) validity of the information it contains. I appreciate the desire to "keep the issue tracker clean", but I think human intelligence needs to be applied to the task, not just a date-based cutoff. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
Steve Holden schrieb: > Jeroen Ruigrok van der Werven wrote: >> -On [20080218 21:41], Brett Cannon ([EMAIL PROTECTED]) wrote: >>> My issue with keeping the RFEs in the tracker as they are is that it >>> artificially inflates the open issue count. Python does not have over >>> 1,700 open bugs. >> >> An issue does not necessarily mean the same as bug. :) >> >> Is it a bug tracker you have or an issue tracker? If the former, agreed, if >> the latter then it makes sense to track RFEs in the tracker. >> > Certainly, but since some issues *are* bugs we might need to refine our > analysis somewhat. It would be better to have a bug report which omitted > issues of type "rfe". As far as I can see open issues of all other types > would be properly classified as bugs. > > There there's the Status field. I understand "open" and "closed", but > what's the semantic of "pending". Is it awaiting triage, awaiting status > assignment, or what? It's a leftover from SF.net. There it had the feature that pending items were closed automatically after two weeks if no further activity took place. For the new tracker, we should really decide about a "pending" policy, or implement the feature too. Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
On 2/18/08, Steve Holden <[EMAIL PROTECTED]> wrote: > I'm not sure we should be throwing RFE's away with such casual abandon > just because nobody had time to pay them any attention in six months - > nor bugs neither, come to that. Well, we have to evaluate the chances of our older tickets to come to completion. I'm of the opinion that ticket getting older have very small chances of ever being completed. RFE for python 2.4 are likely to be irrelevant. old bugs are likely to already be fixed. Maybe we could run a statistical analysis to compute the chances of a ticket that have seen no activity for 8 months to ever be successfully completed? How many successful tickets to we have that had a 8+ months gap between activity? Or maybe we could just clean out the 400 tickets that are 2+ years old? What are the chances for a 2 years old ticket to be completed? -- Virgil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
> > Which is why I propose to have a mechanism to close bugs and RFE > > nobody cares about. over *1000* out of those 1700 open issues are 6+ > > months old. > > > I'm not sure we should be throwing RFE's away with such casual abandon > just because nobody had time to pay them any attention in six months - > nor bugs neither, come to that. +1 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
Virgil Dupras wrote: > On 2/18/08, Brett Cannon <[EMAIL PROTECTED]> wrote: [...] >> So I have no issue with keeping the RFEs in the tracker, at some point >> I do want to change how they are represnted so that they are a >> separate things from issues representing bugs and patches. >> >> -Brett > > Which is why I propose to have a mechanism to close bugs and RFE > nobody cares about. over *1000* out of those 1700 open issues are 6+ > months old. > I'm not sure we should be throwing RFE's away with such casual abandon just because nobody had time to pay them any attention in six months - nor bugs neither, come to that. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
Jeroen Ruigrok van der Werven wrote: > -On [20080218 21:41], Brett Cannon ([EMAIL PROTECTED]) wrote: >> My issue with keeping the RFEs in the tracker as they are is that it >> artificially inflates the open issue count. Python does not have over >> 1,700 open bugs. > > An issue does not necessarily mean the same as bug. :) > > Is it a bug tracker you have or an issue tracker? If the former, agreed, if > the latter then it makes sense to track RFEs in the tracker. > Certainly, but since some issues *are* bugs we might need to refine our analysis somewhat. It would be better to have a bug report which omitted issues of type "rfe". As far as I can see open issues of all other types would be properly classified as bugs. There there's the Status field. I understand "open" and "closed", but what's the semantic of "pending". Is it awaiting triage, awaiting status assignment, or what? I quite like Django's "triage stage", see http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&group=stage&order=priority The stages availabele appear to be "Accepted", "Someday/Maybe", "Design decision needed", "Ready for checkin" and "Unreviewed". OK. maybe "triage" wasn't such a good name for for a four-state condition, but it serves a useful purpose and helps people understand what the ultimate fate of issues they raise might be. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
On 2/18/08, Brett Cannon <[EMAIL PROTECTED]> wrote: > On Feb 18, 2008 11:11 AM, Amaury Forgeot d'Arc <[EMAIL PROTECTED]> wrote: > > Jeroen Ruigrok van der Werven wrote: > > > -On [20080218 13:38], Virgil Dupras ([EMAIL PROTECTED]) wrote: > > > >Personally, I think that a bug tracker is a good place to keep RFE, > > > >not a PEP. I think that the PEP would tend to be cluttered with RFE > > > >nobody cares about forever. So the clutter can never be cleaned unless > > > >someone takes the responsibility to mercilessly remove them. > > > > > > A bug tracker is a much better way of registering such information. It > > > also > > > can be easier referenced in the future since even though when it is > > > closed, > > > the debate and other stuff will remain in the tracker's tickets for > > > posterity. :) > > > > > > PEP: -1 > > > tracker: +1 > > > > I agree. Then we can set some status/keyword when the subject of a RFE > > is accepted by core developers, saying "if someone proposes a patch, > > it has a chance to be reviewed and applied". > > It may incite occasional contributors to work on some of these tasks, > > confident that their work will not be thrown away in two seconds. > > My issue with keeping the RFEs in the tracker as they are is that it > artificially inflates the open issue count. Python does not have over > 1,700 open bugs. > > So I have no issue with keeping the RFEs in the tracker, at some point > I do want to change how they are represnted so that they are a > separate things from issues representing bugs and patches. > > -Brett Which is why I propose to have a mechanism to close bugs and RFE nobody cares about. over *1000* out of those 1700 open issues are 6+ months old. Virgil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] trunk-math
There is a post on boost (http://boost.org) about floating point utilities that are being considered for review. This seems to have a lot of overlap with python needs. I haven't reviewed this myself, but boost code is meant to be quite portable. Here is the link: http://tinyurl.com/2gg4z3 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
> > > > > > PEP: -1 > > > tracker: +1 > > > > I agree. Then we can set some status/keyword when the subject of a RFE > > is accepted by core developers, saying "if someone proposes a patch, > > it has a chance to be reviewed and applied". > > It may incite occasional contributors to work on some of these tasks, > > confident that their work will not be thrown away in two seconds. > > My issue with keeping the RFEs in the tracker as they are is that it > artificially inflates the open issue count. Python does not have over > 1,700 open bugs. > > So I have no issue with keeping the RFEs in the tracker, at some point > I do want to change how they are represnted so that they are a > separate things from issues representing bugs and patches. > > -Brett Sure but thats merely a tracker problem. Change your count to bugs not marked as a rfe / feature-request and you've got your real count. Tracker entries for rfes are much better than a languid document. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
-On [20080218 21:41], Brett Cannon ([EMAIL PROTECTED]) wrote: >My issue with keeping the RFEs in the tracker as they are is that it >artificially inflates the open issue count. Python does not have over >1,700 open bugs. An issue does not necessarily mean the same as bug. :) Is it a bug tracker you have or an issue tracker? If the former, agreed, if the latter then it makes sense to track RFEs in the tracker. -- Jeroen Ruigrok van der Werven / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ And every word upon your spiralling cross is but a misled sun, a bitter loss... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
On Feb 18, 2008 11:11 AM, Amaury Forgeot d'Arc <[EMAIL PROTECTED]> wrote: > Jeroen Ruigrok van der Werven wrote: > > -On [20080218 13:38], Virgil Dupras ([EMAIL PROTECTED]) wrote: > > >Personally, I think that a bug tracker is a good place to keep RFE, > > >not a PEP. I think that the PEP would tend to be cluttered with RFE > > >nobody cares about forever. So the clutter can never be cleaned unless > > >someone takes the responsibility to mercilessly remove them. > > > > A bug tracker is a much better way of registering such information. It also > > can be easier referenced in the future since even though when it is closed, > > the debate and other stuff will remain in the tracker's tickets for > > posterity. :) > > > > PEP: -1 > > tracker: +1 > > I agree. Then we can set some status/keyword when the subject of a RFE > is accepted by core developers, saying "if someone proposes a patch, > it has a chance to be reviewed and applied". > It may incite occasional contributors to work on some of these tasks, > confident that their work will not be thrown away in two seconds. My issue with keeping the RFEs in the tracker as they are is that it artificially inflates the open issue count. Python does not have over 1,700 open bugs. So I have no issue with keeping the RFEs in the tracker, at some point I do want to change how they are represnted so that they are a separate things from issues representing bugs and patches. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_CLEAR to avoid crashes
On Feb 17, 2008 12:29 PM, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Jeffrey Yasskin wrote: > > On Feb 16, 2008 3:12 PM, Amaury Forgeot d'Arc <[EMAIL PROTECTED]> wrote: > >> Should we however intensively search and correct all of them? > >> Is there a clever way to prevent these problems globally, for example > >> by delaying finalizers "just a little"? > > > > A simple way to do this would be to push objects whose refcounts had > > reached 0 onto a list instead of finalizing them immediately, and have > > PyEval_EvalFrameEx periodically swap in a new to-delete list and > > delete the objects on the old one. > > -1. This would only soften the problem in a shallow way. It should still > be possible to create cases where the final cleanup of the object comes > "too early", e.g. when some caller of PyEval_EvalFrameEx still carries > a pointer to some object that got deleted, and then still some code can > get hold of the then-deleted object. It will just be more difficult to > trigger such crashes, depending on the period in which objects are > cleaned. Right. Changing introducing these bugs from "easy" to "possible" sounds like an improvement. Making them harder to trigger has both good and bad effects: they're harder to exploit and harder to reproduce. If we delete on every iteration, it only costs a couple memory accesses more, and should eliminate most of the non-determinism. Anyway, I saw an approach like this work well on a server I used to work on, but there were differences. In particular, there was no way to call a function to trigger deletions; you had to return out to the main loop, which made it a little more reliable than it would be in Python. I suspect it would still fix nearly all of the bugs Amaury is finding since Py_DECREF would no longer return control to the interpreter, but I could be wrong. > The only sane way to never touch deleted objects is to have no > references to them on heap anymore, and to not touch stack references > after the DECREF. > > Regards, > Martin > -- Namasté, Jeffrey Yasskin http://jeffrey.yasskin.info/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
Jeroen Ruigrok van der Werven wrote: > -On [20080218 13:38], Virgil Dupras ([EMAIL PROTECTED]) wrote: > >Personally, I think that a bug tracker is a good place to keep RFE, > >not a PEP. I think that the PEP would tend to be cluttered with RFE > >nobody cares about forever. So the clutter can never be cleaned unless > >someone takes the responsibility to mercilessly remove them. > > A bug tracker is a much better way of registering such information. It also > can be easier referenced in the future since even though when it is closed, > the debate and other stuff will remain in the tracker's tickets for > posterity. :) > > PEP: -1 > tracker: +1 I agree. Then we can set some status/keyword when the subject of a RFE is accepted by core developers, saying "if someone proposes a patch, it has a chance to be reviewed and applied". It may incite occasional contributors to work on some of these tasks, confident that their work will not be thrown away in two seconds. -- Amaury Forgeot d'Arc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
-On [20080218 13:38], Virgil Dupras ([EMAIL PROTECTED]) wrote: >Personally, I think that a bug tracker is a good place to keep RFE, >not a PEP. I think that the PEP would tend to be cluttered with RFE >nobody cares about forever. So the clutter can never be cleaned unless >someone takes the responsibility to mercilessly remove them. A bug tracker is a much better way of registering such information. It also can be easier referenced in the future since even though when it is closed, the debate and other stuff will remain in the tracker's tickets for posterity. :) PEP: -1 tracker: +1 -- Jeroen Ruigrok van der Werven / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ But Time, keeps flowing like a river (on and on)... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_CLEAR to avoid crashes
Hello,
Neil Schemenauer wrote:
> Nick Coghlan <[EMAIL PROTECTED]> wrote:
> > The problem is calls to Py_DECREF(self->attr) where some of the code
> > invoked by __del__ manages to find a way back around to reference
> > self->attr and gets access to a half-deleted object.
>
> Don't you mean "__del__ manages to find a way back around to self"?
> If so, how can that happen? If such a reference path exists, the
> reference count of self should not be zero. I don't understand why
> Py_CLEAR is necessary outside of tp_clear functions.
Of course we are speaking of different objects.
For example, in exception.c, BaseException_init() starts with the instruction:
Py_DECREF(self->args);
this may call __del__ on self->args, which can execute arbitrary
python code - including access to the now-invalid "args" member of the
exception.
class S:
def __del__(self):
print e.args
e = BaseException(1, S())
e.__init__("hello") # segfault
--
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_CLEAR to avoid crashes
Nick Coghlan <[EMAIL PROTECTED]> wrote: > The problem is calls to Py_DECREF(self->attr) where some of the code > invoked by __del__ manages to find a way back around to reference > self->attr and gets access to a half-deleted object. Don't you mean "__del__ manages to find a way back around to self"? If so, how can that happen? If such a reference path exists, the reference count of self should not be zero. I don't understand why Py_CLEAR is necessary outside of tp_clear functions. Neil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Small RFEs and the Bug Tracker
On 2/18/08, Facundo Batista <[EMAIL PROTECTED]> wrote: > Hi! > > Don't now if always, or in the last few months where I've been > following the issues more closely, but I found that are appearing a > lot of small RFEs in the tracker. > > These normally are small but not trivial things. In most cases when I > read them I think "Mmm, yes... it won't hurt to have it, but it's not > so important, and definitively not my itch to scratch". See, for > example, this [1] one, that ask for a factorial method in the > integers. > > Normally these RFEs stay there for a long time, unless they're clearly > negative, because they don't raise any discussion. > > OTOH, we have a PEP for feature requests [2]. I quote part of it: > > """ > This PEP was created to allow us to close bug reports that are really > feature requests. Marked as Open, they distract from the list of real > bugs (which should ideally be less than a page). Marked as Closed, they > tend to be forgotten. The procedure now is: if a bug report is really > a feature request, add the feature request to this PEP; mark the bug as > "feature request", "later", and "closed"; and add a comment to the bug > saying that this is the case (mentioning the PEP explicitly). > """ > > This is still valid? Should we start moving RFEs to this PEP and > closing their issues in the tracker? > > Or should we try to get more discussion regarding these RFEs? Maybe, > for example, a weekly digest where the latests RFEs added are sent to > python-dev, so we can actually say "no way" and close them, or say > "nice" and *then* moving them to the PEP (this has the risk of nobody > saying anything, and to stay in the same position as before). > > What do you think? Opinions? > > Thank you very much! > > Regards, > > [1] http://bugs.python.org/issue2138 > [2] http://www.python.org/dev/peps/pep-0042/ > > -- > .Facundo > > Blog: http://www.taniquetil.com.ar/plog/ > PyAr: http://www.python.org/ar/ > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/hsoft%40hardcoded.net > Personally, I think that a bug tracker is a good place to keep RFE, not a PEP. I think that the PEP would tend to be cluttered with RFE nobody cares about forever. So the clutter can never be cleaned unless someone takes the responsibility to mercilessly remove them. What I really think should be done is first to get rid of all these 8+ months old issues, and then have a kind of system that after 8 months, an issue goes back in "dying mode" where it surfaces back with a message "Does anyone have any reason to believe this issue should still be alive?" If there is no answer after a week, the issue is closed. -- Virgil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_CLEAR to avoid crashes
Greg Ewing wrote: > Martin v. Löwis wrote: >> when some caller of PyEval_EvalFrameEx still carries >> a pointer to some object that got deleted, and then still some code can >> get hold of the then-deleted object. > > I seem to have missed the beginning of this discussion. > I don't see what the problem is here. If a caller needs > a pointer to an object, shouldn't it be holding a > counted reference to it? The problem is calls to Py_DECREF(self->attr) where some of the code invoked by __del__ manages to find a way back around to reference self->attr and gets access to a half-deleted object. Amaury fixed a few of these recently by replacing the Py_DECREF calls with Py_CLEAR calls (and added the relevant pathological destructors to the test suite), but was wondering if there was a way to be more systematic about fixing them. About the only idea I have is to grep the source for all calls to Py_DECREF that contain a pointer deference and manually check them to see if they should use Py_CLEAR instead. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Different float formatting on Windows and Linux
The tests for float.__format__ are breaking on Windows, because of this issue: http://bugs.python.org/issue1600. Basically, Windows is using 3 digits for exponents < 100, and Linux (and at least MacOS) are using 2. The patch attached to the issue proposes changing all platforms to use at least 3 digits. It affects both '%' formatting and __format__ formatting. Altering all float formatting involving exponents is a pretty big change to make, and I want to get opinions here before making this change. Guido's comments in the issue are supportive, and I agree that the consistency would be good. I'm just concerned about changing the output for existing code. I suppose another option would be to modify Windows to use 2 digits for exponents < 100. I guess it just depends on whose output you want to break! I think the options are: 1: Do nothing. Adapt the tests to deal with the differences. 2: Force 3 characters for exponents < 100. 3: Force 2 characters for exponents < 100. Eric. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Small RFEs and the Bug Tracker
Hi! Don't now if always, or in the last few months where I've been following the issues more closely, but I found that are appearing a lot of small RFEs in the tracker. These normally are small but not trivial things. In most cases when I read them I think "Mmm, yes... it won't hurt to have it, but it's not so important, and definitively not my itch to scratch". See, for example, this [1] one, that ask for a factorial method in the integers. Normally these RFEs stay there for a long time, unless they're clearly negative, because they don't raise any discussion. OTOH, we have a PEP for feature requests [2]. I quote part of it: """ This PEP was created to allow us to close bug reports that are really feature requests. Marked as Open, they distract from the list of real bugs (which should ideally be less than a page). Marked as Closed, they tend to be forgotten. The procedure now is: if a bug report is really a feature request, add the feature request to this PEP; mark the bug as "feature request", "later", and "closed"; and add a comment to the bug saying that this is the case (mentioning the PEP explicitly). """ This is still valid? Should we start moving RFEs to this PEP and closing their issues in the tracker? Or should we try to get more discussion regarding these RFEs? Maybe, for example, a weekly digest where the latests RFEs added are sent to python-dev, so we can actually say "no way" and close them, or say "nice" and *then* moving them to the PEP (this has the risk of nobody saying anything, and to stay in the same position as before). What do you think? Opinions? Thank you very much! Regards, [1] http://bugs.python.org/issue2138 [2] http://www.python.org/dev/peps/pep-0042/ -- .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
