Re: [Python-Dev] Threading, atexit, and logging
[Martin v. Löwis] > In bug #1566280 somebody reported that he gets an > exception where the logging module tries to write > to closed file descriptor. > > Upon investigation, it turns out that the file descriptor > is closed because the logging atexit handler is invoked. > This is surprising, as the program is far from exiting at > this point. But the main thread is done, right? None of this appears to make sense unless we got into Py_Finalize(), and that doesn't happen until the main thread has nothing left to do. > Investigating further, I found that the logging atexit > handler is registered *after* the threading atexit handler, > so it gets invoked *before* the threading's atexit. Ya, and that sucks. Can't recall details now, but it's not the first time the vagaries of atexit ordering bit a threaded program. IMO, `threading` shouldn't use atexit at all. > Now, threading's atexit is the one that keeps the > application running, by waiting for all non-daemon threads > to shut down. As this application does all its work in > non-daemon threads, it keeps running for quite a while - > except that the logging module gives errors. > > The real problem here is that atexit handlers are > invoked even though the program logically doesn't exit, > yet (it's not just that the threading atexit is invoked > after logging atexit - this could happen to any atexit > handler that gets registered). > > I added a patch to this report which makes the MainThread > __exitfunc a sys.exitfunc, chaining what is there already. > This will work fine for atexit (as atexit is still imported > explicitly to register its sys.exitfunc), but it might break > if other applications still insist on installing a > sys.exitfunc. Well, that's been officially deprecated since 2.4, but who knows? > What do you think about this approach? It's expedient :-) So was using atexit for this to begin with. Probably "good enough". I'd rather, e.g., that `threading` stuff an exit function into a module global, and change Py_Finalize() to look for that and run it (if present) before invoking call_sys_exitfunc(). That is, break all connection between the core's implementation of threading and the user-visible `atexit` machinery. `atexit` is a hack specific to "don't care about order" finalization functions, and it gets increasingly obscure to try to force it to respect a specific ordering sometimes (e.g., now you have a patch to try to fix it by relying on an obscure deprecated feature and hoping users don't screw with that too -- probably "good enough", but still sucky). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] __str__ and unicode
over at my work copy of the python language reference, Adrian Holovaty asked about the exact semantics of the __str__ hook: http://effbot.org/pyref/__str__ "The return value must be a string object." Does this mean it can be a *Unicode* string object? This distinction is ambiguous to me because unicode objects and string objects are both subclasses of basestring. May a __str__() return a Unicode object? I seem to remember earlier discussions on this topic, but don't recall when and what. From what I can tell, __str__ may return a Unicode object, but only if can be converted to an 8-bit string using the default encoding. Is this on purpose or by accident? Do we have a plan for improving the situation in future 2.X releases ? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __str__ and unicode
On 2006-12-06 10:26, Fredrik Lundh wrote: > over at my work copy of the python language reference, Adrian Holovaty > asked about the exact semantics of the __str__ hook: > > http://effbot.org/pyref/__str__ > >"The return value must be a string object." Does this mean it can be a >*Unicode* string object? This distinction is ambiguous to me because > unicode objects and string objects are both subclasses of basestring. > May a __str__() return a Unicode object? > > I seem to remember earlier discussions on this topic, but don't recall when > and what. From what I can tell, __str__ may return a Unicode object, but > only if can be converted to an 8-bit string using the default encoding. Is > this > on purpose or by accident? Do we have a plan for improving the situation > in future 2.X releases ? This was added to make the transition to all Unicode in 3k easier: .__str__() may return a string or Unicode object. .__unicode__() must return a Unicode object. There is no restriction on the content of the Unicode string for .__str__(). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 06 2006) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __str__ and unicode
On 2006-12-06 10:46, M.-A. Lemburg wrote: > On 2006-12-06 10:26, Fredrik Lundh wrote: >> over at my work copy of the python language reference, Adrian Holovaty >> asked about the exact semantics of the __str__ hook: >> >> http://effbot.org/pyref/__str__ >> >>"The return value must be a string object." Does this mean it can be a >>*Unicode* string object? This distinction is ambiguous to me because >> unicode objects and string objects are both subclasses of basestring. >> May a __str__() return a Unicode object? >> >> I seem to remember earlier discussions on this topic, but don't recall when >> and what. From what I can tell, __str__ may return a Unicode object, but >> only if can be converted to an 8-bit string using the default encoding. Is >> this >> on purpose or by accident? Do we have a plan for improving the situation >> in future 2.X releases ? > > This was added to make the transition to all Unicode in 3k easier: > > .__str__() may return a string or Unicode object. > > .__unicode__() must return a Unicode object. > > There is no restriction on the content of the Unicode string > for .__str__(). One more thing, since these two hooks are commonly used with str() and unicode(): * unicode(obj) will first try .__unicode() and then revert to .__str__() (possibly converting the string return value to Unicode) * str(obj) will try .__str__() only (possibly converting the Unicode return value to a string using the default encoding) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 06 2006) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __str__ and unicode
M.-A. Lemburg wrote: > This was added to make the transition to all Unicode in 3k easier: thanks for the clarification. do you recall when this was added? 2.5? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __str__ and unicode
On 2006-12-06 10:56, Fredrik Lundh wrote: > M.-A. Lemburg wrote: > >> This was added to make the transition to all Unicode in 3k easier: > > thanks for the clarification. > > do you recall when this was added? 2.5? Not really, only that it was definitely before 2.5. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 06 2006) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __str__ and unicode
M.-A. Lemburg wrote: > On 2006-12-06 10:26, Fredrik Lundh wrote: >> From what I can tell, __str__ may return a Unicode object, but >> only if can be converted to an 8-bit string using the default encoding. Is >> this >> on purpose or by accident? Do we have a plan for improving the situation >> in future 2.X releases ? It has worked that way since at Python least 2.4 (I just tried returning unicode from __str__ in 2.4.1 and it worked fine). That's the oldest version I have handy, so I don't know if it was possible in earlier versions. > This was added to make the transition to all Unicode in 3k easier: > > .__str__() may return a string or Unicode object. > > .__unicode__() must return a Unicode object. > > There is no restriction on the content of the Unicode string > for .__str__(). It's also the basis for a tweak that was made in 2.5 to permit conversion to a builtin string in a way that is idempotent for both str and unicode instances via: as_builtin_string = '%s' % original To use the terms from the deferred PEP 349, that conversion mechanism is both Unicode-safe (unicode doesn't get coerced to str) and str-stable (str doesn't get coerced to unicode). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [NPERS] Re: a feature i'd like to see in python #2: indexing of match objects
Fredrik Lundh wrote: > the problem with slicing is that people may 1) expect a slice to return > a new object *of the same type* (which opens up a *gigantic* can of > worms, both on the implementation level and on the wtf-is-this-thing- > really level), and 2) expect things like [::-1] to work, which opens up > another can of worms. I prefer the "If the implementation is easy to > explain, it may be a good idea." design principle over "can of worms" > design principle. This is a good point - I know I consider "m[0:0] == type(m)()" to be a property a well-behaved sequence should preserve. Since match objects can't really do that, better not to pretend to be a sequence at all. With slicing out of the equation, that only leaves the question of whether or not len(m) should work. I believe it would be nice for len(m) to be supported, so that reversed(m) works along with iter(m). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __str__ and unicode
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Dec 6, 2006, at 7:07 AM, Nick Coghlan wrote: > M.-A. Lemburg wrote: >> On 2006-12-06 10:26, Fredrik Lundh wrote: >>> From what I can tell, __str__ may return a Unicode object, but >>> only if can be converted to an 8-bit string using the default >>> encoding. Is this >>> on purpose or by accident? Do we have a plan for improving the >>> situation >>> in future 2.X releases ? > > It has worked that way since at Python least 2.4 (I just tried > returning > unicode from __str__ in 2.4.1 and it worked fine). That's the > oldest version I > have handy, so I don't know if it was possible in earlier versions. I don't have anything older than 2.4 laying around either, but IIRC in 2.3 unicode() did not call __unicode__(). - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRXbBIHEjvBPtnXfVAQJC6AQAhPDrd451PYhQHTuFZqFX7oJpuadEONxb UaBEpWs6yzJjLAxC2tfRVT8vOc1bTmF3Wzf1y5HZsXcbklOFm3USl0YJ8206oDBN 2MGGf2e/JuC5oajo5RJqQ/oqaLDSHb8cD6GP2y/+FFaAhwDnlgnOlV0TxAggKv4K a9nCnFRwJ8c= =+mj8 -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __str__ and unicode
> I don't have anything older than 2.4 laying around either, but IIRC > in 2.3 unicode() did not call __unicode__(). It turns out __unicode__() is called on Python 2.3.5. % python2.3 Python 2.3.5 (#2, Oct 18 2006, 23:04:45) [GCC 4.1.2 20061015 (prerelease) (Debian 4.1.1-16.1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class Foo(object): ... def __unicode__(self): ... print "unicode" ... return u"hi" ... def __str__(self): ... print "str" ... return "hello" ... >>> unicode(Foo()) unicode u'hi' -- Michael Urman http://www.tortall.net/mu/blog ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __str__ and unicode
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Dec 6, 2006, at 9:15 AM, Michael Urman wrote: >> I don't have anything older than 2.4 laying around either, but IIRC >> in 2.3 unicode() did not call __unicode__(). > > It turns out __unicode__() is called on Python 2.3.5. Ah cool, thanks. I must be misremembering that about some earlier Python. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRXbcenEjvBPtnXfVAQIZWAP/aRXJ1Rq0eMVWhYOYvP6Wdy2v5DCj0Arl yS3n0RURWJj5i+zYtqQUBIJEOcFSLJ69cb1SWl/KTvedI4y0SBQknX0o8EJaYhSU h1Y2gL2X+QnvJxlCf7PCdm2C1jYQgwAmKuebjCwaMPJYBqW9Z27+oSTsyFFM/mPR 2qx++VsRw68= =ABvs -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Threading, atexit, and logging
Tim Peters schrieb: >> Upon investigation, it turns out that the file descriptor >> is closed because the logging atexit handler is invoked. >> This is surprising, as the program is far from exiting at >> this point. > > But the main thread is done, right? Wrong. main.py (which is the __main__ script in the demo code) is done, yes. However, threading.py has machinery to not terminate the main thread as long as there are non-daemon threads. In that sense, the main thread is not done: it still has to .join() all the other threads (rather, they join the main thread). > Ya, and that sucks. Can't recall details now, but it's not the first > time the vagaries of atexit ordering bit a threaded program. In this case, logging/__init__.py imports threading, so that will register its atexit first - even if the application imports logging before threading. > IMO, `threading` shouldn't use atexit at all. That is (in a way) my proposal (although I suggest to use sys.exitfunc instead). > It's expedient :-) So was using atexit for this to begin with. > Probably "good enough". I'd rather, e.g., that `threading` stuff an > exit function into a module global, and change Py_Finalize() to look > for that and run it (if present) before invoking call_sys_exitfunc(). Ok, that's what I'll do then. Yet another alternative would be to have the "daemonic" thread feature in the thread module itself (along with keeping track of a list of all running non-daemonic thread). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LSB: Binary compatibility
Greg Ewing schrieb: > Could backwards compatibility concerns be addressed > by including more than one version of Python in > the LSB? Python already allows multiple versions > to coexist, so applications targeting the LSB > would just need to be explicit about which version > of the interpreter to launch. Yes. However, for that to work, the Linux distributors would not want to maintain the old Python releases themselves. Given the LSB release cycle, we would have still to support Python 2.0 today if that would have been included at the time of its release. Of course, for those vendors, *only* security patches matter. They can justify carrying outdated software around for compatibility only, but they cannot justify including it if it has known security bugs. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [NPERS] Re: a feature i'd like to see in python #2: indexing of match objects
On 5 Dec 2006, at 15:51, Fredrik Lundh wrote: > Alastair Houghton wrote: > >> What's more, I think it will be confusing for Python newbies because >> they'll see someone doing >> >>m[3] >> >> and assume that m is a list-like object, then complain when things >> like >> >>for match in m: >> print match > > that'll work, of course, which might be confusing for people who think > they understand how for-in works but don't ;) Or (as in my case) guessed at how it works because they can't be bothered to check the code and can't remember from the last time they looked. I don't spend a great deal of time in the guts of Python. But I do use it and have a couple of extensions that I've written for it (one of which I was contemplating releasing publicly and that is impacted by this change---it provides, amongst other things, an alternate implementation of the "re" API, so I'm going to want to implement this too). >> or >> >>m[3:4] >> >> fail to do what they expect. > > the problem with slicing is that people may 1) expect a slice to > return > a new object *of the same type* What I would have expected is that it supported a similar set of sequence methods---that is, that it returned something with a similar signature. I don't see why code would care about it being the exact same type. Anyway, clearly what people will expect here (talking about the match object API) is that m[3:4] would give them a list (or some equivalent sequence object) containing groups 3 and 4. Why do you think someone would expect a match object? > 2) expect things like [::-1] to work, which opens up another can of > worms. As long as they aren't expecting it to return the same type of object, is there a can of worms here? > I prefer the "If the implementation is easy to explain, it may be a > good idea." design principle over "can of worms" design principle. As someone who is primarily a *user* of Python, I prefer the idea that sequence objects should operate consistently to the idea that there might be some that don't. By which I mean that anything that supports indexing using integer values should ideally support slicing (including things like [::-1]). Kind regards, Alastair. -- http://alastairs-place.net ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [NPERS] Re: a feature i'd like to see in python #2: indexing of match objects
Alastair Houghton <[EMAIL PROTECTED]> wrote: > On 5 Dec 2006, at 15:51, Fredrik Lundh wrote: > > Alastair Houghton wrote: > >> or > >> > >>m[3:4] > >> > >> fail to do what they expect. > > > > the problem with slicing is that people may 1) expect a slice to > > return > > a new object *of the same type* > > What I would have expected is that it supported a similar set of > sequence methods---that is, that it returned something with a similar > signature. I don't see why code would care about it being the exact > same type. The problem is that either we return a list (easy), or we return something that is basically another match object (not quite so easy). Either way, we would be confusing one set of users or another. By not including slicing functionality by default, we sidestep the confusion. > Anyway, clearly what people will expect here (talking about the match > object API) is that m[3:4] would give them a list (or some equivalent > sequence object) containing groups 3 and 4. Why do you think someone > would expect a match object? Because that is what all other slicing operations in base Python do. List, tuple, string, unicode, array, buffer, ... Even extension writers preserve the functionality with Numeric, etc. When you slice a sequence, you get back a slice of that sequence, of the same type you started out with. > > I prefer the "If the implementation is easy to explain, it may be a > > good idea." design principle over "can of worms" design principle. > > As someone who is primarily a *user* of Python, I prefer the idea > that sequence objects should operate consistently to the idea that > there might be some that don't. By which I mean that anything that > supports indexing using integer values should ideally support slicing > (including things like [::-1]). You are being inconsistant. You want list, tuple, etc. to be consistant, but you don't want match objects to be consistant. Sorry, but that is silly. Better to not support slices than to confuse the hell out of people by returning a tuple or list from a match slicing. If people want slicing, they can do list(m)[x:y]. If their matches are of sufficient size where that is a "slow" operation, then they can do [m[i] for i in xrange(x,y)] . - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [NPERS] Re: a feature i'd like to see in python #2: indexing of match objects
On 6 Dec 2006, at 20:29, Josiah Carlson wrote: > The problem is that either we return a list (easy), or we return > something that is basically another match object (not quite so easy). > Either way, we would be confusing one set of users or another. By not > including slicing functionality by default, we sidestep the confusion. But I don't believe that *anyone* will find it confusing that it returns a list. It's much more likely to be confusing to people that they have to write list(m)[x:y] or [m[i] for i in xrange(x,y)] when m[x] and m[y] work just fine. >> As someone who is primarily a *user* of Python, I prefer the idea >> that sequence objects should operate consistently to the idea that >> there might be some that don't. By which I mean that anything that >> supports indexing using integer values should ideally support slicing >> (including things like [::-1]). > > You are being inconsistant. You want list, tuple, etc. to be > consistant, > but you don't want match objects to be consistant. Sorry, but that is > silly. Better to not support slices than to confuse the hell out of > people by returning a tuple or list from a match slicing. That's not true *and* I object to your characterisation of the idea as "silly". What I'm saying is that the idea of slicing always returning the same exact type of object is pointless consistency, because nobody will care *provided* the thing that is returned supports a sensible set of operations given the original type. Look, I give in. There's no point trying to convince any of you further, and I don't have the time or energy to press the point. Implement it as you will. If necessary it can be an extension of my "re" replacement that slicing is supported on match objects. Kind regards, Alastair. -- http://alastairs-place.net ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [NPERS] Re: a feature i'd like to see in python #2: indexing of match objects
On 12/6/06, Alastair Houghton <[EMAIL PROTECTED]> wrote: [from previous message]: >> Anyway, clearly what people will expect here (talking about the match >> object API) is that m[3:4] would give them a list (or some equivalent >> sequence object) containing groups 3 and 4. Why do you think someone >> would expect a match object? > It's much more likely to be confusing to people that they have to write > >list(m)[x:y] > or >[m[i] for i in xrange(x,y)] > when m[x] and m[y] work just fine. <> > Look, I give in. There's no point trying to convince any of you > further, and I don't have the time or energy to press the point. > Implement it as you will. If necessary it can be an extension of my > "re" replacement that slicing is supported on match objects. Keep in mind when implementing that m[3:4] should contain only the element at index 3, not both 3 and 4, as you've seemed to imply twice. cheers, -Mike ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [NPERS] Re: a feature i'd like to see in python #2: indexing of match objects
Alastair Houghton <[EMAIL PROTECTED]> wrote: > On 6 Dec 2006, at 20:29, Josiah Carlson wrote: > > > The problem is that either we return a list (easy), or we return > > something that is basically another match object (not quite so easy). > > Either way, we would be confusing one set of users or another. By not > > including slicing functionality by default, we sidestep the confusion. > > But I don't believe that *anyone* will find it confusing that it > returns a list. We'll have to agree to disagree. > >> As someone who is primarily a *user* of Python, I prefer the idea > >> that sequence objects should operate consistently to the idea that > >> there might be some that don't. By which I mean that anything that > >> supports indexing using integer values should ideally support slicing > >> (including things like [::-1]). > > > > You are being inconsistant. You want list, tuple, etc. to be > > consistant, > > but you don't want match objects to be consistant. Sorry, but that is > > silly. Better to not support slices than to confuse the hell out of > > people by returning a tuple or list from a match slicing. > > That's not true *and* I object to your characterisation of the idea > as "silly". What I'm saying is that the idea of slicing always > returning the same exact type of object is pointless consistency, > because nobody will care *provided* the thing that is returned > supports a sensible set of operations given the original type. In Python 2.5: >>> a <_sre.SRE_Match object at 0x008F6020> >>> dir(a) ['__copy__', '__deepcopy__', 'end', 'expand', 'group', 'groupdict', 'groups', 'span', 'start'] Not including end, expand, group, groupdict, groups, span, and start may be confusing to some number of users. Why? Because of the historical invariant already present in the standard library (with the exception of buffer, I was wrong about that one). *We* may not be confused, but it's not about us (I'm personally happy to use the .group() interface); it's about relative newbies who, generally speaking, desire/need consistency (see [1] for a paper showing that certain kinds of inconsistancies are bad - at least in terms of grading - for new computer science students). Being inconsistant because it's *easy*, is what I consider silly. We've got the brains, we've got the time, if we want slicing, lets produce a match object. If we don't want slicing, or if prodicing a slice would produce a semantically questionable state, then lets not do it. I honestly don't care about whether slicing should go in or not, I use (?=) when I don't want a group. What I really don't want is someone coming in days after 2.6 is released complaining about match slicing not supporting things they think they need. Better to just tell them: use list(m)[x:y] or islice(iterable, [start,] stop [, step]) (both of which should work on arbitrary iterables, the latter of which works on *infinite* iterables) or produce a match object. All or nothing. Half-assing it is a waste. > Look, I give in. There's no point trying to convince any of you > further, and I don't have the time or energy to press the point. > Implement it as you will. If necessary it can be an extension of my > "re" replacement that slicing is supported on match objects. I'm sorry to see you give up so easily. One thing to realize/remember is that basically everyone who frequents python-dev has their own "make life easier" function/class library for those things that have been rejected for general inclusion in Python. - Josiah [1] http://www.cs.mdx.ac.uk/research/PhDArea/saeed/paper1.pdf ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [NPERS] Re: a feature i'd like to see in python #2: indexing of match objects
On 7 Dec 2006, at 00:39, Mike Klaas wrote: > Keep in mind when implementing that m[3:4] should contain only the > element at index 3, not both 3 and 4, as you've seemed to imply twice. Yes, you're quite right. I was writing off the top of my head and I'm still a relative newbie to Python coding. Kind regards, Alastair. -- http://alastairs-place.net ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [NPERS] Re: a feature i'd like to see in python #2: indexing of match objects
On 7 Dec 2006, at 01:01, Josiah Carlson wrote: > *We* may not be confused, but it's not about us (I'm personally > happy to > use the .group() interface); it's about relative newbies who, > generally > speaking, desire/need consistency (see [1] for a paper showing that > certain kinds of inconsistancies are bad - at least in terms of > grading > - for new computer science students). Being inconsistant because it's > *easy*, is what I consider silly. We've got the brains, we've got the > time, if we want slicing, lets produce a match object. Oh, it isn't that I don't want to produce a match object; I think you've mistaken my intention in that respect. I'd be equally happy for it to be a match object, *but*... > If we don't want > slicing, or if prodicing a slice would produce a semantically > questionable state, then lets not do it. ...if you return match objects from slicing, you have problems like m [::-1].groups(). *I* don't know what that should return. What led me to think that a tuple or list would be appropriate is the idea that slicing was a useful operation and that I felt it was unlikely that anyone would want to call the match object methods on a slice, coupled with the fact that slices clearly have problems with some of the match object methods. A match object, plus sequence functionality, minus match object methods, is basically just a sequence. If you're worried about types, you could do something like this: generic match object | +--+-+ || real match objectmatch object slice where the "generic match object" perhaps doesn't have all the methods that a "real match object" would have. (In the extreme case, generic match object might basically just be a sequence type.) Then slicing something that was a "generic match object" always gives you a "generic match object", but it might not support all the methods that the original match object supported. > Half-assing it is a waste. Sure. We're agreed there :-) >> Look, I give in. There's no point trying to convince any of you >> further, and I don't have the time or energy to press the point. >> Implement it as you will. If necessary it can be an extension of my >> "re" replacement that slicing is supported on match objects. > > I'm sorry to see you give up so easily. One thing to realize/remember > is that basically everyone who frequents python-dev has their own > "make > life easier" function/class library for those things that have been > rejected for general inclusion in Python. It's just that I'm tired and have lots of other things that need doing as well. Maybe I do have a bit more time to talk about it, we'll see. Kind regards, Alastair. -- http://alastairs-place.net ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [NPERS] Re: a feature i'd like to see in python #2: indexing of match objects
On 12/6/06, Josiah Carlson <[EMAIL PROTECTED]> wrote: > *We* may not be confused, but it's not about us (I'm personally happy to > use the .group() interface); it's about relative newbies who, generally > speaking, desire/need consistency (see [1] for a paper showing that > certain kinds of inconsistancies are bad - at least in terms of grading > - for new computer science students). Being inconsistant because it's > *easy*, is what I consider silly. We've got the brains, we've got the > time, if we want slicing, lets produce a match object. If we don't want > slicing, or if prodicing a slice would produce a semantically > questionable state, then lets not do it. The idea that slicing a match object should produce a match object sounds like a foolish consistency to me. It's a useful invariant of lists that slicing them returns lists. It's not a useful invariant of sequences in general. This is similar to how it's a useful invariant that indexing a string returns a string; indexing a list generally does not return a list. I only found a couple __getslice__ definitions in a quick perusal of stdlib. ElementTree.py's _ElementInterface class returns a slice from a contained list; whereas sre_parse.py's SubPattern returns another SubPattern. UserList and UserString also define __getslice__ but I don't consider them representative of the standards of non-string/list classes. As an aside, if you're trying to show that inconsistencies in a language are bad by referencing a paper showing that people who used consistent (if incorrect) mental models scored better than those who did not, you may have to explain further; I don't see the connection. -- Michael Urman http://www.tortall.net/mu/blog ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [NPERS] Re: a feature i'd like to see in python #2: indexing of match objects
Alastair Houghton <[EMAIL PROTECTED]> wrote: > On 7 Dec 2006, at 01:01, Josiah Carlson wrote: > > *We* may not be confused, but it's not about us (I'm personally > > happy to > > use the .group() interface); it's about relative newbies who, > > generally > > speaking, desire/need consistency (see [1] for a paper showing that > > certain kinds of inconsistancies are bad - at least in terms of > > grading > > - for new computer science students). Being inconsistant because it's > > *easy*, is what I consider silly. We've got the brains, we've got the > > time, if we want slicing, lets produce a match object. > > Oh, it isn't that I don't want to produce a match object; I think > you've mistaken my intention in that respect. I'd be equally happy > for it to be a match object, *but*... > > > If we don't want > > slicing, or if prodicing a slice would produce a semantically > > questionable state, then lets not do it. > > ...if you return match objects from slicing, you have problems like m > [::-1].groups(). *I* don't know what that should return. I would argue that any 'step' != 1 has no semantically correct result for slicing on a match object, so we shouldn't support it. In that sense, buffer also doesn't support step != 1, but that's because it's __getitem__ method doesn't accept slice objects, and uses the (I believe deprecated or removed in Py3k) __getslice__ method. We can easily check for such things in the __getitem__ method (to also support the removal? of __getslice__) and raise an exception. For those who want reversed slices, they can use reversed(m[x:y]), etc. > What led me to think that a tuple or list would be appropriate is the > idea that slicing was a useful operation and that I felt it was > unlikely that anyone would want to call the match object methods on a > slice, coupled with the fact that slices clearly have problems with > some of the match object methods. A match object, plus sequence > functionality, minus match object methods, is basically just a sequence. Explicit is better than implicit. Refuse the temptation to guess. Let us give them the full match object. If they want to do something silly with slicing (reversed, skipping some, etc.), make them use list() or islice(). > If you're worried about types, you could do something like this: > >generic match object > | > +--+-+ > || >real match objectmatch object slice I believe the above is unnecessary. Slicing a match could produce another match. It's all internal data semantics. - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [NPERS] Re: a feature i'd like to see in python #2: indexing of match objects
"Michael Urman" <[EMAIL PROTECTED]> wrote: > On 12/6/06, Josiah Carlson <[EMAIL PROTECTED]> wrote: > > *We* may not be confused, but it's not about us (I'm personally happy to > > use the .group() interface); it's about relative newbies who, generally > > speaking, desire/need consistency (see [1] for a paper showing that > > certain kinds of inconsistancies are bad - at least in terms of grading > > - for new computer science students). Being inconsistant because it's > > *easy*, is what I consider silly. We've got the brains, we've got the > > time, if we want slicing, lets produce a match object. If we don't want > > slicing, or if prodicing a slice would produce a semantically > > questionable state, then lets not do it. > > The idea that slicing a match object should produce a match object > sounds like a foolish consistency to me. It's a useful invariant of > lists that slicing them returns lists. It's not a useful invariant of > sequences in general. This is similar to how it's a useful invariant > that indexing a string returns a string; indexing a list generally > does not return a list. The string and unicode case for S[i] is special. Such has already been discussed ad-nauseum. As for seq[i:j] returning an object of the same type, if it was "foolish consistency", then why is it consistent across literally the entire standard library (except for buffer), and (in my experience) many 3rd party libraries? > I only found a couple __getslice__ definitions in a quick perusal of > stdlib. ElementTree.py's _ElementInterface class returns a slice from > a contained list; whereas sre_parse.py's SubPattern returns another > SubPattern. UserList and UserString also define __getslice__ but I > don't consider them representative of the standards of non-string/list > classes. > > As an aside, if you're trying to show that inconsistencies in a > language are bad by referencing a paper showing that people who used > consistent (if incorrect) mental models scored better than those who > did not, you may have to explain further; I don't see the connection. The idea is that those who were consistant in their behavior, regardless of whether they were incorrect, can be trained to do things the correct way. That is to say, people who understand that X = Y will behave consistently regardless of context tend to do better than those who believe that it will do different things. Introducing inconsistencies because it is *easy* for the writer of an API, makes it more difficult to learn said API. In this context, the assumption that one makes when slicing in Python (as stated by someone else whom I can't remember in this thread): X[0:0] == type(X)(). That works _everywhere_ in Python where slices are allowed (except for buffers, which are generally rarely used except by certain crazies (like myself)). By not making it true here, we would be adding an exception to the rule. Special cases aren't special enough to break the rules. I'm not going to go all gloom and doom on you; maybe no one will ever have a situation where it is necessary. But implementing "slice of match returns a slice" isn't impossible, whether it is done via subclass, or by direct manipulation of the match struct. And not implementing the functionality because we are *lazy* isn't a terribly good excuse to give someone if/when they run into this. - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Threading, atexit, and logging
[Martin v. Löwis] >>> Upon investigation, it turns out that the file descriptor >>> is closed because the logging atexit handler is invoked. >>> This is surprising, as the program is far from exiting at >>> this point. [Tim Peters] >> But the main thread is done, right? [Martin] > Wrong. main.py (which is the __main__ script in the demo > code) is done, yes. Fine, but the main thread /has/ entered Py_Finalize(). That's key here, and wasn't clear originally. > However, threading.py has machinery to not terminate the main > thread as long as there are non-daemon threads. Right. ... >> IMO, `threading` shouldn't use atexit at all. > That is (in a way) my proposal (although I suggest to use > sys.exitfunc instead). Same thing to me. I'd rather thread cleanup, which is part of the Python core, not rely on any of the user-visible (hence also user-screwable) "do something at shutdown" gimmicks. Thread cleanup is only vaguely related to that concept because "cleanup" here implies waiting for an arbitrarily long time until all thread threads decide on their own to quit. That's not something to be cleaned up /at/ shutdown time, it's waiting (potentially forever!) /for/ shutdown time, and that mismatch is really the source of the problem. >> It's expedient :-) So was using atexit for this to begin with. >> Probably "good enough". I'd rather, e.g., that `threading` stuff an >> exit function into a module global, and change Py_Finalize() to look >> for that and run it (if present) before invoking call_sys_exitfunc(). > Ok, that's what I'll do then. > > Yet another alternative would be to have the "daemonic" thread feature > in the thread module itself (along with keeping track of a list of > all running non-daemonic thread). Sorry, I couldn't follow the intent there. Not obvious to me how moving this stuff from `threading` into `thread` would make it easier(?) for the implementation to wait for non-daemon threads to finish. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Threading, atexit, and logging
Tim Peters schrieb: > Sorry, I couldn't follow the intent there. Not obvious to me how > moving this stuff from `threading` into `thread` would make it > easier(?) for the implementation to wait for non-daemon threads to > finish. Currently, if you create a thread through the thread module (rather than threading), it won't participate in the "wait until all threads are done" algorithm - you have to use the threading module. Moving it into the thread module would allow to cover all threads. Also, if the interpreter invokes, say, threading._shutdown(): that's also "user-screwable", as a user may put something else into threading._shutdown. To make it non-visible, it has to be in C, not Python (and even then it might be still visible to extension modules). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Threading, atexit, and logging
[Tim Peters] >> Sorry, I couldn't follow the intent there. Not obvious to me how >> moving this stuff from `threading` into `thread` would make it >> easier(?) for the implementation to wait for non-daemon threads to >> finish. [Martin v. Löwis] > Currently, if you create a thread through the thread module > (rather than threading), it won't participate in the "wait until > all threads are done" algorithm - you have to use the threading > module. Moving it into the thread module would allow to cover all > threads. True, but that doesn't appear to have any bearing on the bug originally discussed. You introduced this as "yet another alternative" in the context of how to address the original complaint, but if that was the intent, I still don't get it. Regardless, I personally view the `thread` module as being "de facto" deprecated. If someone /wants/ the ability to create a non-daemon thread, that the ability is only available via `threading` is an incentive to move to the newer, saner module. Besides, if the daemon distinction were grafted on to `thread` threads too, it would have to default to daemonic (a different default than `threading` threads) else be incompatible with current `thread` thread behavior. I personally don't want to add new features to `thread` threads in any case. > Also, if the interpreter invokes, say, threading._shutdown(): > that's also "user-screwable", as a user may put something else > into threading._shutdown. To make it non-visible, it has to be > in C, not Python (and even then it might be still visible to > extension modules). The leading underscore makes it officially invisible <0.7 wink>, and users would have to both seek it out and go out of their way to screw it. If some user believes they have a need to mess wtih threading._shutdown(), that's fine by me too. The problem with atexit and sys.exitfunc is that users can get in trouble now simply by using them in documented ways, because `threading` also uses them (but inappropriately so, IMO). Indeed, that's all the `logging` module did here. While this next is also irrelevant to the original complaint, I think it was also a minor mistake to build the newer atexit gimmicks on top of sys.exitfunc (same reason: a user can destroy the atexit functionality quite innocently if they happen to use sys.exitfunc after they (or code they build on) happens to import `atexit`. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [NPERS] Re: a feature i'd like to see in python #2: indexing of match objects
Michael Urman wrote: > The idea that slicing a match object should produce a match object > sounds like a foolish consistency to me. well, the idea that adding m[x] as a convenience alias for m.group(x) automatically turns m into a list-style sequence that also has to support full slicing sounds like an utterly foolish consistency to me. the OP's original idea was to make a common use case slightly easier to use. if anyone wants to argue for other additions to the match object API, they should at least come up with use cases based on real existing code. (and while you guys are waiting, I suggest you start a new thread where you discuss some other inconsistency that would be easy to solve with more code in the interpreter, like why "-", "/", and "**" doesn't work for strings, lists don't have a "copy" method, sets and lists have different API:s for adding things, we have hex() and oct() but no bin(), str.translate and unicode.translate take different arguments, etc. get to work!) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Weekly Python Patch/Bug Summary
Patch / Bug Summary ___ Patches : 413 open ( +6) / 3489 closed ( +5) / 3902 total (+11) Bugs: 943 open ( +7) / 6364 closed ( +1) / 7307 total ( +8) RFE : 246 open ( +0) / 244 closed ( +0) / 490 total ( +0) New / Reopened Patches __ popen() slow on AIX due to large FOPEN_MAX value (2006-12-01) http://python.org/sf/1607087 opened by John L. Allen bug# 1607041: Condition.wait timeout fails on clock change (2006-12-01) http://python.org/sf/1607149 opened by BC Optional Argument Syntax (2006-12-02) http://python.org/sf/1607548 opened by Tony Lownds Makefile fix (2006-12-03) http://python.org/sf/1608267 opened by Hasan Diwan Race condition in os.makedirs (2006-12-04) http://python.org/sf/1608579 opened by James Bowes Fix pickle doc typo "date", should be "data" (2006-12-04) CLOSED http://python.org/sf/1608758 opened by Graham Horler Py_FileSystemDefaultEncoding can be non-canonical (2006-12-04) http://python.org/sf/1608805 opened by Stephan R.A. Deibel Docstring support for ctypesgen (2006-12-05) CLOSED http://python.org/sf/1609108 opened by David Remahl #1603424 subprocess.py wrongly claims 2.2 compatibility. (2006-12-05) http://python.org/sf/1609282 opened by Robert Carr traceback on exit if syslog handler fails to initialize (2006-12-05) http://python.org/sf/1609407 opened by Jeremy Katz tarfile.py: Fix for bug #1609958 (2006-12-06) CLOSED http://python.org/sf/1610437 opened by Lars Gustäbel Patches Closed __ Socket module is not thread-safe (2006-08-22) http://python.org/sf/1544279 closed by loewis ConfigParser to accept a custom dict to allow ordering (2005-12-01) http://python.org/sf/1371075 closed by loewis Fix pickle doc typo "date", should be "data" (2006-12-05) http://python.org/sf/1608758 closed by quiver Docstring support for ctypesgen (2006-12-05) http://python.org/sf/1609108 closed by theller tarfile.py: Fix for bug #1609958 (2006-12-06) http://python.org/sf/1610437 closed by gbrandl New / Reopened Bugs ___ Condition.wait timeout fails when system clock changes (2006-12-01) http://python.org/sf/1607041 opened by BC Incorrect use of 'int' and 'long' descriptions (2006-12-01) CLOSED http://python.org/sf/1607061 opened by John logging %(module)s reporting wrong modules (2006-11-29) http://python.org/sf/1605110 reopened by mad-marty segfault in python24.dll (2006-12-03) http://python.org/sf/1607759 opened by joe mailbox.Maildir re-reads directory too often (2006-12-03) http://python.org/sf/1607951 opened by Matthias Klose Sloppy error checking in listdir() for Posix (2006-12-04) http://python.org/sf/1608818 opened by Stephan R.A. Deibel PyThread_release_lock with pthreads munges errno (2006-12-04) http://python.org/sf/1608921 opened by Stephan R.A. Deibel tarfile archive paths limited to less than 100 chars (2006-12-06) CLOSED http://python.org/sf/1609958 opened by Frank Rehberger GUI for Python 2.3, 2.4, and 2.5 is very sluggish (2006-12-06) http://python.org/sf/1610485 opened by g4rlik Bugs Closed ___ Incorrect use of 'int' and 'long' descriptions (2006-12-01) http://python.org/sf/1607061 closed by gbrandl not configured for tk (2006-10-18) http://python.org/sf/1579931 closed by sf-robot tarfile archive paths limited to less than 100 chars (2006-12-06) http://python.org/sf/1609958 closed by gbrandl ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
