Re: [Python-Dev] Let's just *keep* lambda
On 2/6/06, Christopher Armstrong [EMAIL PROTECTED] wrote: On 2/7/06, Martin v. Löwis [EMAIL PROTECTED] wrote: Brett Cannon wrote: But I know that everyone and their email client is against me on this one, so I am not going to really try to tear into this. But I do think that lambda needs a renaming. Speaking as someone who still forgets that Python's lambda is not the same as those found in functional languages Can you elaborate on that point? I feel that Python's lambda is exactly the same as the one in Lisp. Sure, the Lisp lambda supports multiple sequential expressions (the progn feature), but I understand that this is just an extension (although one that has been around several decades). Of course, Python's expressions are much more limited as Lisp's (where you really can have macros and special forms in as the expression in a lambda), but the lambda construct itself seems to be the very same one. If we phrase it somewhat differently, we can see that lambdas are different in Python and Lisp, in a very practical way. First: Everything in Lisp is an expression. There's no statement, in Lisp, that isn't also an expression. Lambdas in Lisp can contain arbitrary expressions; therefore you can put any language construct inside a lambda. In Python, you cannot put any language construct inside a lambda. Python's and Lisp's lambdas are effectively totally different. +1 on keeping Lambda, +1 on making it more useful. After lambda being made more useful, can I hope that I will be able to use lambda with multiple statements? :) Lambdas in Lisp and Python are different, but in the usability perspective they don't need to differ too much. -Jiwon ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's just *keep* lambda
On 2/7/06, Brett Cannon [EMAIL PROTECTED] wrote: On 2/5/06, Guido van Rossum [EMAIL PROTECTED] wrote: After so many attempts to come up with an alternative for lambda, perhaps we should admit defeat. I've not had the time to follow the most recent rounds, but I propose that we keep lambda, so as to stop wasting everybody's talent and time on an impossible quest. I have been thinking about this, and I have to say I am a little disappointed (-0 disappointed, not -1 disappointed). I honestly bought the argument for removing lambda. And I think that a deferred object would help with one of lambda's biggest uses and made its loss totally reasonable. I'm not 100% sure what you mean here, but as far as my understanding goes, current lambda *is* a deferred object (or at least a deferred expression, which may not be quite what you mean...) But I know that everyone and their email client is against me on this one, so I am not going to really try to tear into this. But I do think that lambda needs a renaming. I agree with this. The *name* lambda is a wart, even if the deferred expression feature isn't. My preference is to simply replace the keyword lambda with a keyword expr (or if that's not acceptable because there's too much prior use of expr as a variable name, then maybe expression - but that's starting to get a bit long). Speaking as someone who still forgets that Python's lambda is not the same as those found in functional languages, Well, only in the sense that Python's *expressions* are not the same as those found in functional languages (ie, Python has statements which are not expressions). But I see your point - and I strongly object to going the other way and extending lambda/expr to allow statements or suites. I would much rather have it named 'expr' or 'expression' or something that is more inline with its abilities then with a name taken for CS historical reasons. This ain't for father's lambda and thus shouldn't be named so. Agreed. But if expr isn't acceptable, I don't like the other common suggestion of reusing def. It's not a definition, nor is it like an anonymous function (the lack of support for statements/suites being the key difference). Then again, Guido did say he should, not that he did admit defeat. =) OTOH, he was trying to stop endless the discussion... :-) Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Any interest in tail call optimization as a decorator?
Maybe someone has already brought this up, but my searching hasn't revealed it. Is there any interest in something like this for the functional module? #!/usr/bin/env python2.4 # This program shows off a python decorator which implements # tail call optimization. It does this by throwing an exception # if it is it's own grandparent, and catching such exceptions # to recall the stack. import sys class TailRecurseException: def __init__(self, args, kwargs): self.args = args self.kwargs = kwargs def tail_call_optimized(g): def func(*args, **kwargs): try: raise ZeroDivisionError except ZeroDivisionError: f = sys.exc_info()[2].tb_frame if f.f_back and f.f_back.f_back \ and f.f_back.f_back.f_code == f.f_code: raise TailRecurseException(args, kwargs) else: while 1: try: return g(*args, **kwargs) except TailRecurseException, e: args = e.args kwargs = e.kwargs func.__doc__ = g.__doc__ return func @tail_call_optimized def factorial(n, acc=1): calculate a factorial if n == 0: return acc return factorial(n-1, n*acc) print factorial(1) # prints a big, big number, # but doesn't hit the recursion limit. @tail_call_optimized def fib(i, current = 0, next = 1): if i == 0: return current else: return fib(i - 1, next, current + next) print fib(1) # also prints a big number, # but doesn't hit the recursion limit. -- Crutcher Dunnavant [EMAIL PROTECTED] littlelanguages.com monket.samedi-studios.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] cProfile module
Hi all, As promized two months ago, I eventually finished the integration of the 'lsprof' profiler. It's now in an internal '_lsprof' module that is exposed via a 'cProfile' module with the same interface as 'profile', producing compatible dump stats that can be inspected with 'pstats'. See previous discussion here: * http://mail.python.org/pipermail/python-dev/2005-November/058212.html The code is currently in the following repository, from where I'll merge it into CPython if nobody objects: * http://codespeak.net/svn/user/arigo/hack/misc/lsprof/Doc * http://codespeak.net/svn/user/arigo/hack/misc/lsprof/Lib * http://codespeak.net/svn/user/arigo/hack/misc/lsprof/Modules with tests and docs, including new tests and doc refinements for profile itself. The docs mark hotshot as reversed for specialized usage. They probably need a bit of bad-English-hunting... And yes, I do promize to maintain this code in the future. A bientot, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's just *keep* lambda
On 2/6/06, Brett Cannon [EMAIL PROTECTED] wrote: And I think that a deferred object would help with one of lambda's biggest uses and made its loss totally reasonable. The ambiguity inherent from the perspective of a deferred object makes a general one impractical. Both map(Deferred().attribute, seq) and map(Deferred().method(arg), seq) look the same - how does the object know that the first case it should return the attribute of the first element of seq when called, but in the second it should wait for the next call when it will call method(arg) on the first element of seq? Since there's also no way to spell lambda y: foo(x, y, z) on a simple deferred object, it's strictly less powerful. If the current Python lambda's functionality is desired, there is no better pythonic way to spell it. There are plenty of new syntactic options that help highlight its expression nature, but are they worth the change? MIchael -- Michael Urman http://www.tortall.net/mu/blog/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's just *keep* lambda
Jiwon Seo wrote: After lambda being made more useful, can I hope that I will be able to use lambda with multiple statements? :) Lambdas in Lisp and Python are different, but in the usability perspective they don't need to differ too much. To my knowledge, nobody proposed to make it more useful, or to allow statements in the body of a lambda expression (neither single nor multiple). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Help with Unicode arrays in NumPy
This is a design question which is why I'm posting here. Recently the NumPy developers have become more aware of the difference between UCS2 and UCS4 builds of Python. NumPy arrays can be of Unicode type. In other words a NumPy array can be made of up fixed-data-length unicode strings. Currently that means that they are unicode strings of basic size UCS2 or UCS4 depending on the platform. It is this duality that has some people concerned. For all other data-types, NumPy allows the user to explicitly request a bit-width for the data-type. So, we are thinking of introducing another data-type to NumPy to differentiate between UCS2 and UCS4 unicode strings. (This also means a unicode scalar object, i.e. string of each of these, exactly one of which will inherit from the Python type). Before embarking on this journey, however, we are seeking advice from individuals wiser to the way of Unicode on this list. Perhaps all we need to do is be more careful on input and output of Unicode data-types so that transfer of unicode can be handled correctly on each platform. Any thoughts? -Travis Oliphant ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help with Unicode arrays in NumPy
Travis E. Oliphant wrote: Currently that means that they are unicode strings of basic size UCS2 or UCS4 depending on the platform. It is this duality that has some people concerned. For all other data-types, NumPy allows the user to explicitly request a bit-width for the data-type. Why is that a desirable property? Also: Why does have NumPy support for Unicode arrays in the first place? Before embarking on this journey, however, we are seeking advice from individuals wiser to the way of Unicode on this list. My initial reaction is: use whatever Python uses in NumPy Unicode. Upon closer inspection, it is not all that clear what operations are supported on a Unicode array, and how these operations relate to the Python Unicode type. In any case, I think NumPy should have only a single Unicode array type (please do explain why having zero of them is insufficient). If the purpose of the type is to interoperate with a Python unicode object, it should use the same width (as this will allow for mempcy). If the purpose is to support arbitrary Unicode characters, it should use 4 bytes (as two bytes are insufficient to represent arbitrary Unicode characters). If the purpose is something else, please explain what the purpose is. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes patch (was: (libffi) Re: Copyright issue)
Hye-Shik Chang [EMAIL PROTECTED] writes: I did some work to make ctypes+libffi compacter and liberal. http://openlook.org/svnpublic/ctypes-compactffi/ (svn) Here goes patches for the integration: [1] http://people.freebsd.org/~perky/ctypesinteg-f1.diff.bz2 [2] http://people.freebsd.org/~perky/ctypesinteg-f2.diff.bz2 I implemented it in two flavors. [1] runs libffi's configure along with Python's and setup.py just builds it. And [2] has no change to Python's configure and setup.py runs libffi configure and builds it. And both patches don't have things for documentations yet. [Thomas Heller] My plan is to make separate ctypes releases for 2.3 and 2.4, even after it is integrated into Python 2.5, so it seems [2] would be better - it must be possible to build ctypes without Python. As I said before, docs need still to be written. I think content is more important than markup, so I'm writing in rest, it can be converted to latex later. I expect that writing the docs will show quite some edges that need to be cleaned up - that should certainly be done before the first 2.5 release. Also I want to make a few releases before declaring the 1.0 version. This does not mean that I'm against integrating it right now. Martin v. Löwis [EMAIL PROTECTED] writes: Not sure whether you think you need further approval: if you are ready to check this into the Python trunk, just go ahead. As I said, I would prefer if what is checked in is a literal copy of the ctypes CVS (as far as reasonable). I was not looking for further approval, I wanted to explain why I prefer the patch [2] that Hye-Shik posted above. I'll do at least one separate ctypes release before checking this into the Python trunk. Thanks, Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help with Unicode arrays in NumPy
Travis E. Oliphant wrote: Numpy supports arrays of arbitrary fixed-length records. It is much more than numeric-only data now. One of the fields that a record can contain is a string. If strings are supported, it makes sense to support unicode strings as well. Hmm. How do you support strings in fixed-length records? Strings are variable-sized, after all. On common application is that you have a C struct in some API which has a fixed-size array for string data (either with a length field, or null-terminated), in this case, it is moderately useful to model such a struct in Python. However, transferring this to Unicode is pointless - there aren't any similar Unicode structs that need support. This allows NumPy to memory-map arbitrary data-files on disk. Ok, so this is the C struct case. Then why do you need Unicode support there? Which common file format has embedded fixed-size Unicode data? Perhaps you should explain why you think NumPy shouldn't support Unicode I think I said Unicode arrays, not Unicode. Unicode arrays are a pointless data type, IMO. Unicode always comes in strings (i.e. variable sized, either null-terminated or with an introducing length). On disk/on the wire Unicode comes as UTF-8 more often than not. Using UCS-2/UCS-2 as an on-disk represenationis also questionable practice (although admittedly Microsoft uses that a lot). That is currently what is done. The current unicode data-type is exactly what Python uses. Then I wonder how this goes along with the use case allow to map arbitrary files. The chararray subclass gives to unicode and string arrays all the methods of unicode and strings (operating on an element-by-element basis). For strings, I can see use cases (although I wonder how you deal with data formats that also support variable-sized strings, as most data formats supporting strings do). Please explain why having zero of them is *sufficient*. Because I (still) cannot imagine any specific application that might need such a feature (IOWYAGNI). If the purpose is to support arbitrary Unicode characters, it should use 4 bytes (as two bytes are insufficient to represent arbitrary Unicode characters). And Python does not support arbitrary Unicode characters on narrow builds? Then how is \U0010 represented? It's represented using UTF-16. Try this for yourself: py len(u\U0010) 2 py u\U0010[0] u'\udbff' py u\U0010[1] u'\udfff' This has all kinds of non-obvious implications. The purpose is to represent bytes as they might exist in a file or data-stream according to the users specification. See, and this is precisely the statement that I challenge. Sure, they might exist - but I'd rather expect that they don't. If they exist, Unicode might come as variable-sized UTF-8, UTF-16, or UTF-32. In either case, NumPy should already support that by mapping a string object onto the encoded bytes, to which you then can apply .decode() should you need to process the actual Unicode data. The purpose is whatever the user wants them for. It's the same purpose as having an unsigned 64-bit data-type --- because users may need it to represent data as it exists in a file. No. I would expect you have 64-bit longs because users *do* need them, and because there wouldn't be an easy work-around if users wouldn't have them. For Unicode, it's different: users don't directly need them (atleast not many users), and if they do, there is an easy work-around for their absence. Say I want to process NTFS run lists. In NTFS run lists, there are 24-bit integers, 40-bit integers, and 4-bit integers (i.e. nibbles). Can I represent them all in NumPy? Can I have NumPy transparently map a sequence of run list records (which are variable-sized) map as an array of run list records? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's just *keep* lambda
On 2/7/06, Paul Moore [EMAIL PROTECTED] wrote: On 2/7/06, Brett Cannon [EMAIL PROTECTED] wrote: On 2/5/06, Guido van Rossum [EMAIL PROTECTED] wrote: After so many attempts to come up with an alternative for lambda, perhaps we should admit defeat. I've not had the time to follow the most recent rounds, but I propose that we keep lambda, so as to stop wasting everybody's talent and time on an impossible quest. I have been thinking about this, and I have to say I am a little disappointed (-0 disappointed, not -1 disappointed). I honestly bought the argument for removing lambda. And I think that a deferred object would help with one of lambda's biggest uses and made its loss totally reasonable. I'm not 100% sure what you mean here, but as far as my understanding goes, current lambda *is* a deferred object (or at least a deferred expression, which may not be quite what you mean...) Yes, lambda is deferred. What I mean is using lambda for things like ``lambda x: x.attr`` and such; specifically for deferred execution, and not for stuff like ``lambda x: func(1, 2, x, 3, 4)`` stuff. But I know that everyone and their email client is against me on this one, so I am not going to really try to tear into this. But I do think that lambda needs a renaming. I agree with this. The *name* lambda is a wart, even if the deferred expression feature isn't. My preference is to simply replace the keyword lambda with a keyword expr (or if that's not acceptable because there's too much prior use of expr as a variable name, then maybe expression - but that's starting to get a bit long). Speaking as someone who still forgets that Python's lambda is not the same as those found in functional languages, Well, only in the sense that Python's *expressions* are not the same as those found in functional languages (ie, Python has statements which are not expressions). But I see your point - and I strongly object to going the other way and extending lambda/expr to allow statements or suites. I would much rather have it named 'expr' or 'expression' or something that is more inline with its abilities then with a name taken for CS historical reasons. This ain't for father's lambda and thus shouldn't be named so. Agreed. But if expr isn't acceptable, I don't like the other common suggestion of reusing def. It's not a definition, nor is it like an anonymous function (the lack of support for statements/suites being the key difference). Yeah, reusing def is taking back into the functional world too much. It makes our current use of def seem more like syntactic sugar for assigning a lambda to a name for function definition and that is not what is happening here. Then again, Guido did say he should, not that he did admit defeat. =) OTOH, he was trying to stop endless the discussion... :-) =) Well, it should when Python 3 comes out, so there is some extra incentive for that to happen sooner than later. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's just *keep* lambda
On 2/7/06, Michael Urman [EMAIL PROTECTED] wrote: On 2/6/06, Brett Cannon [EMAIL PROTECTED] wrote: And I think that a deferred object would help with one of lambda's biggest uses and made its loss totally reasonable. The ambiguity inherent from the perspective of a deferred object makes a general one impractical. Both map(Deferred().attribute, seq) and map(Deferred().method(arg), seq) look the same - how does the object know that the first case it should return the attribute of the first element of seq when called, but in the second it should wait for the next call when it will call method(arg) on the first element of seq? Magic. =) Honestly, I don't know, but I bet there is some evil, black magic way to pull it off. Otherwise, worst case, Deferred takes an argument that flags that it has a method being called on it for it to defer against and not to treat it as an attribute access only. And that is within reason in terms of interace requirement for the object, in my opinion. Since there's also no way to spell lambda y: foo(x, y, z) on a simple deferred object, it's strictly less powerful. If the current Python lambda's functionality is desired, there is no better pythonic way to spell it. There are plenty of new syntactic options that help highlight its expression nature, but are they worth the change? I never claimed that a deferred object would replace all uses of lambda, just that it would make it reasonable. For the above suggestion I would go to a named function. Or, if the argument was on the end or everything named, use functional.partial(). -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Linking with mscvrt
I just came up with an idea how to resolve the VC versioning problems for good: Python should link with mscvrt.dll (which is part of the operating system), not with the CRT that the compiler provides. To do that, we would need to compile and link with the SDK header files and import libraries, not with the ones that visual studio provides. For that to work, everyone building Python or Python extensions (*) would have to install the Platform SDK (which is available for free, but contains quite a number of bits). Would that be acceptable? Disclaimer: I haven't tried yet whether this would actually work. Regards, Martin (*) For Python extensions, it should be possible to use mingw instead, and configure it for linking against msvcrt. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threadsafe patch for asynchat
IMO asynchat and asyncore are braindead. The should really be removed from the standard library. The code is 10 years old and represents at least 10-year-old thinking about how to do this. The amount of hackery in Zope related to asyncore was outrageous -- basically most of asyncore's guts were replaced with more advanced Zope code, but the API was maintained for compatibility reasons. A nightmare. --Guido On 2/6/06, Martin v. Löwis [EMAIL PROTECTED] wrote: Mark Edgington wrote: Does anyone have any comments about applying the following patch to asynchat? That patch looks wrong. What does it mean to run in a thread? All code runs in a thread, all the time: sometime, that thread is the main thread. Furthermore, I can't see any presumed thread-unsafety in asynchat. Sure, there is a lot of member variables in asynchat which aren't specifically protected against mutual access from different threads. So you shouldn't be accessing the same async_chat object from multiple threads. I cannot see why using a creating and using an async_chat object in a thread that is not the main thread could cause any problems. I also cannot see how this patch could have significant effect on asyn_chat's behaviour when used in multiple threads. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threadsafe patch for asynchat
Guido van Rossum wrote: IMO asynchat and asyncore are braindead. The should really be removed from the standard library. The code is 10 years old and represents at least 10-year-old thinking about how to do this. The amount of hackery in Zope related to asyncore was outrageous -- basically most of asyncore's guts were replaced with more advanced Zope code, but the API was maintained for compatibility reasons. A nightmare. Perhaps, but please keep in mind that the smtpd module uses both, currently, and would have to be rewritten if either is removed. Robert Brewer System Architect Amor Ministries [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threadsafe patch for asynchat
On 2/7/06, Fredrik Lundh [EMAIL PROTECTED] wrote: ... what other reactive socket framework is there that would fit well into the standard library ? is twisted really simple enough ? Twisted is wonderful, powerful, rich, and very large. Perhaps a small subset could be carefully extracted that (given suitable volunteers to maintain it in the future) might fit in the standard library, but [a] that extraction is not going to be a simple or fast job, and [b] I suspect that the minimum sensible subset would still be much larger (and richer / more powerful) than asyncore. Alex ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threadsafe patch for asynchat
Guido van Rossum [EMAIL PROTECTED] wrote: IMO asynchat and asyncore are braindead. The should really be removed from the standard library. The code is 10 years old and represents at least 10-year-old thinking about how to do this. The amount of hackery in Zope related to asyncore was outrageous -- basically most of asyncore's guts were replaced with more advanced Zope code, but the API was maintained for compatibility reasons. A nightmare. I'm going to go ahead and disagree with Guido on this one. Before removing asyncore (and asynchat) from the standard library, I believe that there would necessarily need to be a viable replacement already in place. The SocketServer module and its derivatives are wholly unscalable for server-oriented applications once you get past a few dozen threads (where properly designed asyncore derivatives will do quite well all the way to your platform file handle limit). Every once and a while I hear about people pushing for Twisted to be included with Python, but at 2 megs for the base bz2 package, it seems a little...hefty. I'm not aware of any other community-accepted package for asynchronous socket clients and servers, but I'm always looking. Now, don't get me wrong, writing servers and clients using asyncore or asynchat can be a beast, but it does get one into the callback/reactor method of programming, which seems to have invaded other parts of Python and 3rd party libraries (xml.sax, tk, Twisted, wxPython, ...). Back to the topic that Guido was really complaining about: Zope + asyncore. I don't doubt that getting Zope to play nicely with asyncore was difficult, but it begs the questions: what would have been done if asyncore didn't exist, and why wasn't that done instead of trying to play nicely with asyncore? - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threadsafe patch for asynchat
On Tue, 2006-02-07 at 16:01 -0800, Robert Brewer wrote: Perhaps, but please keep in mind that the smtpd module uses both, currently, and would have to be rewritten if either is removed. Would that really be a huge loss? -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threadsafe patch for asynchat
On 2/8/06, Alex Martelli [EMAIL PROTECTED] wrote: On 2/7/06, Fredrik Lundh [EMAIL PROTECTED] wrote: ... what other reactive socket framework is there that would fit well into the standard library ? is twisted really simple enough ? Twisted is wonderful, powerful, rich, and very large. Perhaps a small subset could be carefully extracted that (given suitable volunteers to maintain it in the future) might fit in the standard library, but [a] that extraction is not going to be a simple or fast job, and [b] I suspect that the minimum sensible subset would still be much larger (and richer / more powerful) than asyncore. The subject of putting (parts of) Twisted into the standard library comes up once every 6 months or so, at least on our mailing list. For all that I think asyncore is worthless, I'm still against copying Twisted into the stdlib. Or at least I'm not willing to maintain the necessary fork, and I fear the nightmares about versioning that can easily occur when you've got both standard library and third party versions of a project. But, for the record, to the people who argue not to put Twisted into the stdlib because of its size: The parts of it that would actually be applicable (i.e. those that obselete async* in the stdlib) are only a few kilobytes of code. At a quick run of wc, the parts that support event loops, accurate timed calls, SSL, Unix sockets, TCP, UDP, arbitrary file descriptors, processes, and threads sums up to about 5300 lines of code. asynchat and asyncore are about 1200. -- Twisted | Christopher Armstrong: International Man of Twistery Radix|-- http://radix.twistedmatrix.com | Release Manager, Twisted Project \\\V/// |-- http://twistedmatrix.com |o O|| wvw-+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] release plan for 2.5 ?
On 2/7/06, Fredrik Lundh [EMAIL PROTECTED] wrote: what's the current release plan for Python 2.5, btw? I cannot find a relevant PEP, and the what's new says late 2005: but I don't think that anyone followed up on this. what's the current status ? Guido and I had a brief discussion about this. IIRC, he was thinking alpha around March and release around summer. I think this is aggressive with all the things still to do. We really need to get the ssize_t branch integrated. There are a bunch of PEPs that have been accepted (or close), but not implemented. I think these include (please correct me, so we can get a good list): http://www.python.org/peps/ SA 308 Conditional Expressions SA 328 Imports: Multi-Line and Absolute/Relative SA 342 Coroutines via Enhanced Generators S 343 The with Statement S 353 Using ssize_t as the index type This one should be marked as final I believe: SA 341 Unifying try-except and try-finally n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] release plan for 2.5 ?
It looks like we need a Python 2.5 Release Schedule PEP. Jeremy On 2/7/06, Neal Norwitz [EMAIL PROTECTED] wrote: On 2/7/06, Fredrik Lundh [EMAIL PROTECTED] wrote: what's the current release plan for Python 2.5, btw? I cannot find a relevant PEP, and the what's new says late 2005: but I don't think that anyone followed up on this. what's the current status ? Guido and I had a brief discussion about this. IIRC, he was thinking alpha around March and release around summer. I think this is aggressive with all the things still to do. We really need to get the ssize_t branch integrated. There are a bunch of PEPs that have been accepted (or close), but not implemented. I think these include (please correct me, so we can get a good list): http://www.python.org/peps/ SA 308 Conditional Expressions SA 328 Imports: Multi-Line and Absolute/Relative SA 342 Coroutines via Enhanced Generators S 343 The with Statement S 353 Using ssize_t as the index type This one should be marked as final I believe: SA 341 Unifying try-except and try-finally n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threadsafe patch for asynchat
On 2/7/06, Christopher Armstrong [EMAIL PROTECTED] wrote: Twisted is wonderful, powerful, rich, and very large. Perhaps a small subset could be carefully extracted The subject of putting (parts of) Twisted into the standard library comes up once every 6 months or so, at least on our mailing list. For all that I think asyncore is worthless, I'm still against copying Twisted into the stdlib. Or at least I'm not willing to maintain the necessary fork, and I fear the nightmares about versioning that can easily occur when you've got both standard library and third party versions of a project. I wouldn't be enthusiastic about putting all of Twisted in the stdlib either. Twisted is on a different release schedule than Python. However, isn't there a relatively small core subset like Alex mentioned that isn't changing much? Could we split up those components and have those live in the core, but the vast majority of Twisted live outside as it does now? n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threadsafe patch for asynchat
what other reactive socket framework is there that would fit well into the standard library ? is twisted really simple enough ? I've been very happy with Medusa, which is asyncore-based. Perhaps the right idea is to fix the various problems of asyncore. We might lift the similar code from the kernel of ILU, for example, which carefully addresses the various issues around this style of action loop. Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threadsafe patch for asynchat
[Josiah Carlson] ... Back to the topic that Guido was really complaining about: Zope + asyncore. I don't doubt that getting Zope to play nicely with asyncore was difficult, It's more that mixing asyncore with threads is a bloody nightmare, and ZEO and Zope both do that. Zope (but not ZEO) goes on to mix threads with asynchat too. In addition, ZEO makes life much harder than should be necessary by running in two different modes and auto-switching between them, depending on whether the app is or is not running an asyncore mainloop itself. In order to _detect_ when the app fires up an asyncore mainloop, ZEO monkey-patches asyncore's loop() function and physically replaces it with its own loop() function. It goes downhill from there. Guido's memories are partly out of date now: ZEO used to replace a lot more of asyncore than it does now, because of bugs in the asyncore distributed with older Python versions. The _needs_ for that went away little by little over the years, but the code in ZEO stuck around much longer. ZEO's ThreadedAsync/LoopCallback.py is much smaller now (ZODB 3.6) than Guido remembers. For a brief while, I even ripped out ZEO's monkey-patching of Python's asyncore loop(), but it turned out that newer code in Zope3 (but not Zope2) relied on, in turn, poking values into ZEO's module globals to cause ZEO's loop() replacement to shut down (that's the kind of expedient joy you get when mixing asyncore with threads). Every piece of it remains underdocumented and, IMO, highly obscure. but it begs the questions: what would have been done if asyncore didn't exist, Who knows? What would python-dev be like if you didn't exist :-)? and why wasn't that done instead of trying to play nicely with asyncore? Bugs and missing features in asyncore. For ZEO's purposes, if I had designed it, I expect it would have used threads (without asyncore). However, bits of code still sitting around suggest that it was at least the _intent_ at one time that ZEO be able to run without threads at all. That's certainly not possible now. If you look at asyncore's revision history, you'll note that Jeremy and Guido made many changes when they worked at Zope Corp. Those largely reflect the history of moving ZEO's asyncore monkey-patches into the Python core. BTW, if you don't use ZEO, I believe it's possible to run Zope3 without asyncore (you can use Twisted in Zope3 instead). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] release plan for 2.5 ?
On 2/7/06, Neal Norwitz [EMAIL PROTECTED] wrote: On 2/7/06, Fredrik Lundh [EMAIL PROTECTED] wrote: what's the current release plan for Python 2.5, btw? I cannot find a relevant PEP, and the what's new says late 2005: but I don't think that anyone followed up on this. what's the current status ? Guido and I had a brief discussion about this. IIRC, he was thinking alpha around March and release around summer. I think this is aggressive with all the things still to do. We really need to get the ssize_t branch integrated. There are a bunch of PEPs that have been accepted (or close), but not implemented. I think these include (please correct me, so we can get a good list): http://www.python.org/peps/ SA 308 Conditional Expressions SA 328 Imports: Multi-Line and Absolute/Relative SA 342 Coroutines via Enhanced Generators S 343 The with Statement S 353 Using ssize_t as the index type This one should be marked as final I believe: SA 341 Unifying try-except and try-finally Supposedly Guido is close on pronouncing on PEP 352 (Required Superclass for Exceptions), or so he said last time that thread came about. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] release plan for 2.5 ?
On 2/7/06, Jeremy Hylton [EMAIL PROTECTED] wrote: It looks like we need a Python 2.5 Release Schedule PEP. Very draft: http://www.python.org/peps/pep-0356.html Needs lots of work and release managers. Anthony, Martin, Fred, Sean are all mentioned with TBDs and question marks. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] math.areclose ...?
Raymond Hettinger wrote: | [Chris Smith] || Does it help to spell it like this? || || def areclose(x, y, relative_err = 1.e-5, absolute_err=1.e-8): || diff = abs(x - y) || ave = (abs(x) + abs(y))/2 || return diff absolute_err or diff/ave relative_err | | There is a certain beauty and clarity to this presentation; however, | it is problematic numerically: | | * the division by either absolute_err and relative_err can overflow or | trigger a ZeroDivisionError I'm not dividing by either of these values so that shouldn't be a problem. As long as absolute_err is not 0 then the first test would catch the possiblity that x==y==ave==0. (see below) As for the overflow, does your version of python overflow? Mine (2.4) just returns 1.#INF which still computes as a number: ### 1.79769313486e+308+1.79769313486e+308 1.#INF inf=_ inf1 True inf1 False 2./inf 0.0 inf/inf -1.#IND ### There is a problem with dividing by 'ave' if the x and y are at the floating point limits, but the symmetric behaving form (presented by Scott Daniels) will have the same problem. The following format for close() has the same semantic meaning but avoids the overflow possibility and avoids extra work for the case when abs_tol=0 and x==y: ### def close(x, y, abs_tol=1.e-8, rel_tol=1.e-5): '''Return True if |x-y| abs_tol or |x-y|/ave(|x|,|y|) rel_tol. The average is not computed directly so as to avoid overflow for numbers close to the floating point upper limit.''' if x==y: return True diff = abs(x - y) if diff abs_tol: return True f = rel_tol/2. if diff f*abs(x) + f*abs(y): return True return False ### | | * the 'or' part of the expression can introduce an unnecessary | discontinuity in the first derivative. | If a value other than boolean were being returned, I could see the desire for continuity in derivative. Since the original form presents a boolean result, however, I'm having a hard time thinking of how the continuity issue comes to play. | The original Numeric definition is likely to be better for people who | know what they're doing; however, I still question whether it is an | appropriate remedy for the beginner issue | of why 1.1 + 1.1 + 1.1 doesn't equal 3.3. | I'm in total agreement. Being able to see that math.areclose(1.1*3,3.3) is True but 1.1*3==3.3 is False is not going to make them feel much better. They are going to have to face the floating point issue. As for the experienced user, perhaps such a function would be helpful. Maybe it would be better to require that the tolerances be given rather than defaulting so as to make clear which test is being used if only one test was going to be used: close(x,y,rel_tol=1e-5) close(x,y,abs_tol=1e-8) /c ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] small floating point number problem
I just ran into a curious behavior with small floating points, trying to find the limits of them on my machine (XP). Does anyone know why the '0.0' is showing up for one case below but not for the other? According to my tests, the smallest representable float on my machine is much smaller than 1e-308: it is 2.470328229206234e-325 but I can only create it as a product of two numbers, not directly. Here is an attempt to create the much larger 1e-308: a=1e-308 a 0.0 a==0 True-- it really is 0; this is not a repr issue b=.1*1e-307 b 9.9991e-309 a==b False--they really are different Also, I see that there is some graininess in the numbers at the low end, but I'm guessing that there is some issue with floating points that I would need to read up on again. The above dilemma is a little more troublesome. m=2.470328229206234e-017 s=1e-307 m*s 4.9406564584124654e-324 #2x too large 2*m*s 4.9406564584124654e-324 3*m*s==4*m*s True /c ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threadsafe patch for asynchat
Tim Peters wrote: Bugs and missing features in asyncore. For ZEO's purposes, if I had designed it, I expect it would have used threads (without asyncore). However, bits of code still sitting around suggest that it was at least the _intent_ at one time that ZEO be able to run without threads at all. That's certainly not possible now. What is the reason that people want to use threads when they can have poll/select-style message processing? Why does Zope require threads? IOW, why would anybody *want* a threadsafe patch for asynchat? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help with Unicode arrays in NumPy
Travis == Travis E Oliphant [EMAIL PROTECTED] writes: Travis Numpy supports arrays of arbitrary fixed-length records. Travis It is much more than numeric-only data now. One of the Travis fields that a record can contain is a string. If strings Travis are supported, it makes sense to support unicode strings Travis as well. That is not obvious. A string is really an array of bytes, which for historical reasons in some places (primarily the U.S. of A.) can be used to represent text. Unicode, on the other hand, is intended to represent text streams robustly and does so in a universal but flexible way ... but all of the different Unicode transformation formats are considered to represent the *identical* text stream. Some applications may specify a transformation format, others will not. In any case, internally Python is only going to support *one*; all the others must be read in through codecs anyway. See below. Travis This allows NumPy to memory-map arbitrary data-files on Travis disk. In the case where a transformation format *is* specified, I don't see why you can't use a byte array field (ie, ordinary string) of appropriate size for this purpose, and read it through a codec when it needs to be treated as text. This is going to be necessary in essentially all of the cases I encounter, because the files are UTF-8 and sane internal representations are either UTF-16 or UTF-32. In particular, Python's internal representation is 16 or 32 bits wide. Travis Perhaps you should explain why you think NumPy shouldn't Travis support Unicode Because it can't, not in the way you would like to, if I understand you correctly. Python chooses *one* of the many standard representations for internal use, and because of the way the standard is specified, it doesn't matter which one! And none of the others can be represented directly, all must be decoded for internal use and encoded when written back to external media. So any memory mapping application is inherently nonportable, even across Python implementations. Travis And Python does not support arbitrary Unicode characters Travis on narrow builds? Then how is \U0010 represented? In a way incompatible with the concept of character array. Now what do you do? The point is that Unicode is intentionally designed in such a way that a plethora of representations is possible, but all are easily and reliably interconverted. Implementations are then free to choose an appropriate internal representation, knowing that conversion from external representations is cheap and standardized. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can do free software business; ask what your business can do for free software. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threadsafe patch for asynchat
Martin v. Löwis wrote: Tim Peters wrote: Bugs and missing features in asyncore. For ZEO's purposes, if I had designed it, I expect it would have used threads (without asyncore). However, bits of code still sitting around suggest that it was at least the _intent_ at one time that ZEO be able to run without threads at all. That's certainly not possible now. What is the reason that people want to use threads when they can have poll/select-style message processing? Why does Zope require threads? IOW, why would anybody *want* a threadsafe patch for asynchat? In case the processing of events needed to block? If I'm processing web requests in an async* dispatch loop and a request needs the results of a (probably lengthy) database query in order to generate its output, how do I give the dispatcher control again to process the next asynchronous network event? The usual answer is process the request in a thread. That way the dispatcher can spring to life for each event as quickly as needed. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threadsafe patch for asynchat
Steve Holden wrote: What is the reason that people want to use threads when they can have poll/select-style message processing? Why does Zope require threads? IOW, why would anybody *want* a threadsafe patch for asynchat? In case the processing of events needed to block? If I'm processing web requests in an async* dispatch loop and a request needs the results of a (probably lengthy) database query in order to generate its output, how do I give the dispatcher control again to process the next asynchronous network event? The usual answer is process the request in a thread. That way the dispatcher can spring to life for each event as quickly as needed. but why do such threads have to talk to asyncore directly ? /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threadsafe patch for asynchat
Fredrik Lundh [EMAIL PROTECTED] wrote: Steve Holden wrote: What is the reason that people want to use threads when they can have poll/select-style message processing? Why does Zope require threads? IOW, why would anybody *want* a threadsafe patch for asynchat? In case the processing of events needed to block? If I'm processing web requests in an async* dispatch loop and a request needs the results of a (probably lengthy) database query in order to generate its output, how do I give the dispatcher control again to process the next asynchronous network event? The usual answer is process the request in a thread. That way the dispatcher can spring to life for each event as quickly as needed. but why do such threads have to talk to asyncore directly ? Indeed. I seem to remember a discussion a few months ago about easy thread programming, which invariably directed people off to use the simplest abstractions necessary: Queues. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
