Re: [Python-Dev] Slides from today's parallel/async Python talk
Just a quick implementation question (didn't have time to read through all your emails :-) async.submit_work(func, args, kwds, callback=None, errback=None) How do you implement arguments passing and return value? e.g. let's say I pass a list as argument: how do you iterate on the list from the worker thread without modifying the backing objects for refcounts (IIUC you use a per-thread heap and don't do any refcounting). Same thing for return value, how do you pass it to the callback? cf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
Eric Snow: On Wed, Apr 3, 2013 at 6:47 AM, Hrvoje Niksic wrote: It seems like a good feature that an __int__ implementation can choose to return an int subclass with additional (and optional) information. After all, int subclass instances should be usable everywhere where ints are, including in C code. Unless you want to try to use the concrete C-API in CPython. In my experience the concrete API is not very subclass friendly. Nick: > Using it with subclasses is an outright bug (except as part of > a subclass implementation). This is true for mutable objects like dicts and lists where calling things like PyDict_SetItem will happily circumvent the object. But for ints and floats, all that the C code really cares about is the object's intrinsic value as returned by PyLong_AS_LONG and friends, which is constant and unchangeable by subclasses. The typical reason to subclass int is to add more information or new methods on the instance, not to break basic arithmetic. Doing anything else breaks subtitutability and is well outside the realm of "consenting adults". Someone who wants to change basic arithmetic is free to implement an independent int type. Hrvoje ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
> -Original Message- > From: Eric Snow [mailto:ericsnowcurren...@gmail.com] > Sent: 4. apríl 2013 04:57 > > imported by both of the original modules. At that point, the code is > > cleaner and more decoupled, and the uneven circular import support > ceases to be a problem for that application. > > +1 I tried to make the point in an earlier mail that I don't think that we ought to let our personal opinions de-jour on software architecture put a constraint on our language features. There can be good and valid reasons to put things that depend on each other in separate modules, for example when trying to separate modules by functionality or simply when splitting a long file into two. Notice that cyclic dependencies are allowed _within_ a module file. Imagine if we decided that we could only refer to objects _previously_ declared within a .py file, because it encouraged the good design practice of factoring out common dependencies. Dependency graphs of software entities can sadly not always be reduced into a DAG, and we should, IMHO, by no means force people to keep each cygle in the dependency graph within a single .py module. K ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8
On Wed, Apr 3, 2013 at 9:21 PM, Alfredo Solano Martínez wrote: > Hi, > > Are you planning to cover the code quality of the interpreter itself > too? I've been recently reading through the cert.org secure coding > practice recommendations and was wondering if there has is any ongoing > effort to perform static analysis on the cpython codebase. Hey Alfredo, We do not currently have any tools to do that, but it would definitely be something interesting to discuss and maybe design on the list. I'm sure there are static analysis tools for the C part and I'm sure we as a community could come up with a "super tool" to check both the C and Python parts of CPython. -- Ian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How to fix the incorrect shared library extension on linux for 3.2 and newer?
The values on macos for these variables still look wrong in 3.3.1rc1: ./configure --prefix=/Users/jtaylor/tmp/py3.3.1 --enable-shared on macosx-10.8-x86_64 sys.version_info(major=3, minor=3, micro=1, releaselevel='candidate', serial=1) SO .so EXT_SUFFIX .so SHLIB_SUFFIX 0 the only correct one here is EXT_SUFFIX, SHLIB_SUFFIX should be .dylib (libpython is a .dylib) and .SO possibly too given for what it was used in the past. 3.3.0 also returns wrong values SO .so EXT_SUFFIX None SHLIB_SUFFIX "" ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
On 4 Apr 2013 19:37, "Kristján Valur Jónsson" wrote: > > > > > -Original Message- > > From: Eric Snow [mailto:ericsnowcurren...@gmail.com] > > Sent: 4. apríl 2013 04:57 > > > imported by both of the original modules. At that point, the code is > > > cleaner and more decoupled, and the uneven circular import support > > ceases to be a problem for that application. > > > > +1 > > I tried to make the point in an earlier mail that I don't think that we ought to let our personal opinions de-jour on software architecture put a constraint on our language features. > There can be good and valid reasons to put things that depend on each other in separate modules, for example when trying to separate modules by functionality or simply when splitting a long file into two. > Notice that cyclic dependencies are allowed _within_ a module file. Imagine if we decided that we could only refer to objects _previously_ declared within a .py file, because it encouraged the good design practice of factoring out common dependencies. > Dependency graphs of software entities can sadly not always be reduced into a DAG, and we should, IMHO, by no means force people to keep each cygle in the dependency graph within a single .py module. That's why it's just a reason none of the current import system devs are inclined to work on it, rather than a reason for rejecting proposals and tested patches that give improved behaviour. Cheers, Nick. > > K > > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On 4 April 2013 10:39, Hrvoje Niksic wrote: > >> On Wed, Apr 3, 2013 at 6:47 AM, Hrvoje Niksic >> wrote: >>> >>> It seems like a good feature that an __int__ implementation can choose to >>> return an int subclass with additional (and optional) information. After >>> all, int subclass instances should be usable everywhere where ints are, >>> including in C code. [SNIP] > > The typical reason to subclass int is to add more information or new methods > on the instance, not to break basic arithmetic. Doing anything else breaks > subtitutability and is well outside the realm of "consenting adults". > Someone who wants to change basic arithmetic is free to implement an > independent int type. The reason for calling int(obj) is to get an object that is precisely of type int. When I call this I do not want any modified or additional methods or data attached to the resulting object. The example given at the start of the thread makes minimal modifications in an int subclass but still allows the result of int(obj) to be unpickleable: >>> class Int1(int): ... def __int__(self): ... return self ... >>> n = Int1(4) >>> n 4 >>> import pickle >>> ni = int(n) >>> pickle.dumps(ni) ... snip ... File "q:\tools\Python27\lib\pickle.py", line 562, in save_tuple save(element) File "q:\tools\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "q:\tools\Python27\lib\pickle.py", line 401, in save_reduce save(args) File "q:\tools\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "q:\tools\Python27\lib\pickle.py", line 562, in save_tuple save(element) File "q:\tools\Python27\lib\pickle.py", line 291, in save issc = issubclass(t, TypeType) RuntimeError: maximum recursion depth exceeded while calling a Python object I don't know whether that's a bug in pickle, but I think it's fair to say that there are times when someone wants an object that is precisely of type int. They should be able to rely on int(obj) returning an int or raising an error. This is true similarly for __index__. The entire purpose of __index__ is to permit APIs like list.__getitem__ to work extensibly with any int-like object. This is achieved by allowing any object to advertise its convertibility from an int-like object to an int with the same numeric value. Anyone who calls operator.index(obj) is explicitly stating that they do not want any property of obj apart from its integer value and that they want that as an int. That it should be an object of type int is explicitly stated in PEP 357 (http://www.python.org/dev/peps/pep-0357/): """ 2) The __index__ special method will have the signature def __index__(self): return obj where obj must be either an int or a long. """ Oscar ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8
On Thu, Apr 4, 2013 at 2:45 PM, Ian Cordasco wrote: > Hey Alfredo, > > We do not currently have any tools to do that, but it would definitely > be something interesting to discuss and maybe design on the list. I'm > sure there are static analysis tools for the C part and I'm sure we as > a community could come up with a "super tool" to check both the C and > Python parts of CPython. As cf said it seems they're using Coverity for the C part, but the idea of checking the cpython interpreter with python itself is intriguing. Suscribed. Alfredo ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8
Le Thu, 4 Apr 2013 06:57:14 +0200, Charles-François Natali a écrit : > > Are you planning to cover the code quality of the interpreter itself > > too? I've been recently reading through the cert.org secure coding > > practice recommendations and was wondering if there has is any > > ongoing effort to perform static analysis on the cpython codebase. > > AFAICT CPython already benefits from Coverity scans (I guess the > Python-security guys receive those notifications). Note that this only > covers the C codebase. Correction: the security@ address doesn't receive any coverity notifications. Perhaps someone checks the (private) coverity builds from time to time, but I don't think there's anything automatic. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On Fri, Apr 5, 2013 at 1:23 AM, Oscar Benjamin wrote: > The reason for calling int(obj) is to get an object that is precisely > of type int. When I call this I do not want any modified or additional > methods or data attached to the resulting object. There's something I'm fundamentally not understanding about this debate, and that is: How is it that calling a class can logically return anything other than an instance of that class? Taking it to a user-defined type: class Foo: pass class Bar(Foo): pass Is there any argument that I can pass to Foo() to get back a Bar()? Would anyone expect there to be one? Sure, I could override __new__ to do stupid things, but in terms of logical expectations, I'd expect that Foo(x) will return a Foo object, not a Bar object. Why should int be any different? What have I missed here? ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
Am 04.04.2013 16:47, schrieb Chris Angelico: > On Fri, Apr 5, 2013 at 1:23 AM, Oscar Benjamin > wrote: >> The reason for calling int(obj) is to get an object that is precisely >> of type int. When I call this I do not want any modified or additional >> methods or data attached to the resulting object. > > There's something I'm fundamentally not understanding about this > debate, and that is: How is it that calling a class can logically > return anything other than an instance of that class? Taking it to a > user-defined type: > > class Foo: > pass > > class Bar(Foo): > pass > > Is there any argument that I can pass to Foo() to get back a Bar()? > Would anyone expect there to be one? Sure, I could override __new__ to > do stupid things, but in terms of logical expectations, I'd expect > that Foo(x) will return a Foo object, not a Bar object. Why should int > be any different? What have I missed here? I think the issue that the constructors for basic classes like int() are often seen as built-in functions that "cast" values to the respective type (or alternatively, functions that call the respective __method__ on the argument, like len()), not class constructors. FWIW, I agree with you that the "class constructor" view is the right one and would prefer exact int/str/... instances returned. Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: > On Fri, Apr 5, 2013 at 1:23 AM, Oscar Benjamin > wrote: > > The reason for calling int(obj) is to get an object that is precisely > > of type int. When I call this I do not want any modified or additional > > methods or data attached to the resulting object. > > There's something I'm fundamentally not understanding about this > debate, and that is: How is it that calling a class can logically > return anything other than an instance of that class? Taking it to a > user-defined type: > > class Foo: > pass > > class Bar(Foo): > pass > > Is there any argument that I can pass to Foo() to get back a Bar()? > Would anyone expect there to be one? Sure, I could override __new__ to > do stupid things, but in terms of logical expectations, I'd expect > that Foo(x) will return a Foo object, not a Bar object. Why should int > be any different? What have I missed here? > A class can define a __new__ method that returns a different object. E.g. (python 3): >>> class C: ... def __new__(cls): return 42 ... >>> C() 42 >>> -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On 2013-04-04, at 16:47 , Chris Angelico wrote: > Sure, I could override __new__ to do stupid things Or to do perfectly logical and sensible things, such as implementing "cluster classes" or using the base class as a factory of sorts. > in terms of logical expectations, I'd expect > that Foo(x) will return a Foo object, not a Bar object. The problem is the expectation of what "a Foo object" is: type-wise, any Bar object is also a Foo object. I would not expect Foo() to return an object of a completely unrelated type, but returning an object of a subtype? That does not seem outlandish. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On Fri, Apr 5, 2013 at 1:59 AM, Guido van Rossum wrote: > On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: >> Is there any argument that I can pass to Foo() to get back a Bar()? >> Would anyone expect there to be one? Sure, I could override __new__ to >> do stupid things, but in terms of logical expectations, I'd expect >> that Foo(x) will return a Foo object, not a Bar object. Why should int >> be any different? What have I missed here? > > > A class can define a __new__ method that returns a different object. E.g. > (python 3): > Right, I'm aware it's possible. But who would expect it of a class? ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On 2013-04-04, at 17:01 , Chris Angelico wrote: > On Fri, Apr 5, 2013 at 1:59 AM, Guido van Rossum wrote: >> On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: >>> Is there any argument that I can pass to Foo() to get back a Bar()? >>> Would anyone expect there to be one? Sure, I could override __new__ to >>> do stupid things, but in terms of logical expectations, I'd expect >>> that Foo(x) will return a Foo object, not a Bar object. Why should int >>> be any different? What have I missed here? >> >> >> A class can define a __new__ method that returns a different object. E.g. >> (python 3): >> > > Right, I'm aware it's possible. But who would expect it of a class? Given it's one of the use cases for __new__ and immutable types have to be initialized through __new__ anyway, why would it be unexpected? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
On Thu, Apr 4, 2013 at 2:36 AM, Kristján Valur Jónsson < krist...@ccpgames.com> wrote: [...] > There can be good and valid reasons to put things that depend on each > other in separate modules, for example when trying to separate modules by > functionality or simply when splitting a long file into two. > Notice that cyclic dependencies are allowed _within_ a module file. > Imagine if we decided that we could only refer to objects _previously_ > declared within a .py file, because it encouraged the good design practice > of factoring out common dependencies. > Dependency graphs of software entities can sadly not always be reduced > into a DAG, and we should, IMHO, by no means force people to keep each > cygle in the dependency graph within a single .py module. > IMO in both cases (cyclic dependencies within modules and between modules) Python's approach is consistently derived from a specific implementation strategy: names are available after they are defined. This works (at least it doesn't raise NameError :-): def f(): return g() def g(): return f() f() Because by the time f() is called, g() is defined. Note that this doesn't work: def f(): return g() f() def g(): return f() IOW there is no magic. If you think of putting objects in a dictionary, the semantics are clear. (Except for the shenanigans that the compiler carries out for determining whether something is a local variable reference or not, but that's also explainable.) This does not work: a = b b = a because the first line references b which is not yet defined. Similarly, if module A imports module B and B imports A, that works because imports are first satisfied from sys.modules, and when A is loaded, it is entered into sys.modules before its code is executed. But it may be that if A is loaded and starts importing B, B is then loaded and imports A, which finds A in the state it reached when it decided to import B. The rules here are more complicated because of what "from A import X" means (it depends on whether X is a submodule or not) but there are rules, and they are totally well-defined. The final piece of this puzzle is that, given A and B mutually importing each other, the top-level app code could start by importing A or B, and the execution order will be different then. (You can force this by putting A and B in a package whose __init__.py imports them in the desired order.) I don't really see what we could change to avoid breaking code in any particular case -- the burden is up to the library to do it right. I don't see a reason to forbid any of this either. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On Thu, Apr 4, 2013 at 8:01 AM, Chris Angelico wrote: > On Fri, Apr 5, 2013 at 1:59 AM, Guido van Rossum wrote: > > On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: > >> Is there any argument that I can pass to Foo() to get back a Bar()? > >> Would anyone expect there to be one? Sure, I could override __new__ to > >> do stupid things, but in terms of logical expectations, I'd expect > >> that Foo(x) will return a Foo object, not a Bar object. Why should int > >> be any different? What have I missed here? > > > > > > A class can define a __new__ method that returns a different object. E.g. > > (python 3): > > > > Right, I'm aware it's possible. But who would expect it of a class? > If it's documented you could expect it. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
Le Fri, 5 Apr 2013 01:47:45 +1100, Chris Angelico a écrit : > > class Foo: > pass > > class Bar(Foo): > pass > > Is there any argument that I can pass to Foo() to get back a Bar()? > Would anyone expect there to be one? Sure, I could override __new__ to > do stupid things, but in terms of logical expectations, I'd expect > that Foo(x) will return a Foo object, not a Bar object. >>> OSError(errno.ENOENT, "couldn't find that file") FileNotFoundError(2, "couldn't find that file") Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On 04/04/2013 08:01 AM, Chris Angelico wrote: On Fri, Apr 5, 2013 at 1:59 AM, Guido van Rossum wrote: On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: Is there any argument that I can pass to Foo() to get back a Bar()? Would anyone expect there to be one? Sure, I could override __new__ to do stupid things, but in terms of logical expectations, I'd expect that Foo(x) will return a Foo object, not a Bar object. Why should int be any different? What have I missed here? A class can define a __new__ method that returns a different object. E.g. (python 3): Right, I'm aware it's possible. But who would expect it of a class? FTR I'm in the int() should return an int camp, but to answer your question: my dbf module has a Table class, but it returns either a Db3Table, FpTable, VfpTable, or ClpTable depending on arguments (if creating a new one) or the type of the table in the existing dbf file. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8
On Thu, Apr 4, 2013 at 10:42 AM, Antoine Pitrou wrote: > Le Thu, 4 Apr 2013 06:57:14 +0200, > Charles-François Natali a écrit : > > > Are you planning to cover the code quality of the interpreter itself > > > too? I've been recently reading through the cert.org secure coding > > > practice recommendations and was wondering if there has is any > > > ongoing effort to perform static analysis on the cpython codebase. > > > > AFAICT CPython already benefits from Coverity scans (I guess the > > Python-security guys receive those notifications). Note that this only > > covers the C codebase. > > Correction: the security@ address doesn't receive any coverity > notifications. Perhaps someone checks the (private) coverity builds from > time to time, but I don't think there's anything automatic. > Christian Heimes has a daily build set up and checks the results periodically. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How to fix the incorrect shared library extension on linux for 3.2 and newer?
In article <515d760d.50...@googlemail.com>, Julian Taylor wrote: > The values on macos for these variables still look wrong in 3.3.1rc1: > > ./configure --prefix=/Users/jtaylor/tmp/py3.3.1 --enable-shared > on macosx-10.8-x86_64 > > sys.version_info(major=3, minor=3, micro=1, releaselevel='candidate', > serial=1) > SO .so > EXT_SUFFIX .so > SHLIB_SUFFIX 0 > > > the only correct one here is EXT_SUFFIX, SHLIB_SUFFIX should be .dylib > (libpython is a .dylib) and .SO possibly too given for what it was used > in the past. > > 3.3.0 also returns wrong values > SO .so > EXT_SUFFIX None > SHLIB_SUFFIX "" I get a SHLIB_SUFFIX of '.so'. --enable-shared configurations are particularly error-prone; for instance, you either need to test using an installed version or be very careful to override the dynamic loading path. Please see the discussion at http://bugs.python.org/issue16754 and either report your concerns there or open an new issue. -- Ned Deily, n...@acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
On 04/04/2013 4:17pm, Guido van Rossum wrote:
I don't really see what we could change to avoid breaking code in any
particular case -- the burden is up to the library to do it right. I
don't see a reason to forbid any of this either.
How about having a form of relative import which only works for
submodules. For instance, instead of
from . import moduleX
write
import .moduleX
which is currently a SyntaxError. I think this could be implemented as
moduleX = importlib.import_module('.moduleX', __package__)
--
Richard
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
Redirecting to python-ideas.
On Thu, Apr 4, 2013 at 9:26 AM, Richard Oudkerk wrote:
> On 04/04/2013 4:17pm, Guido van Rossum wrote:
>>
>> I don't really see what we could change to avoid breaking code in any
>> particular case -- the burden is up to the library to do it right. I
>> don't see a reason to forbid any of this either.
>
>
> How about having a form of relative import which only works for submodules.
> For instance, instead of
>
> from . import moduleX
>
> write
>
> import .moduleX
>
> which is currently a SyntaxError. I think this could be implemented as
>
> moduleX = importlib.import_module('.moduleX', __package__)
We considered that when relative import was designed and rejected it,
because it violates the expectation that after "import "
you can use exactly "" in your code.
--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Slides from today's parallel/async Python talk
Hi Charles-François,
On Thu, Apr 04, 2013 at 01:18:58AM -0700, Charles-François Natali wrote:
> Just a quick implementation question (didn't have time to read through
> all your emails :-)
>
> async.submit_work(func, args, kwds, callback=None, errback=None)
>
> How do you implement arguments passing and return value?
>
> e.g. let's say I pass a list as argument: how do you iterate on the
> list from the worker thread without modifying the backing objects for
> refcounts (IIUC you use a per-thread heap and don't do any
> refcounting).
Correct, nothing special is done for the arguments (apart from
incref'ing them in the main thread before kicking off the parallel
thread (then decref'ing them in the main thread once we're sure the
parallel thread has finished)).
> Same thing for return value, how do you pass it to the
> callback?
For submit_work(), you can't :-) In fact, an exception is raised if
the func() or callback() or errback() attempts to return a non-None
value.
It's worth noting that I eventually plan to have the map/reduce-type
functionality (similar to what multiprocessing offers) available via
a separate 'parallel' façade. This will be geared towards programs
that are predominantly single-threaded, but have lots of data that
can be processed in parallel at various points.
Now, with that being said, there are a few options available at the
moment if you want to communicate stuff from parallel threads back
to the main thread. Originally, you could do something like this:
d = async.dict()
def foo():
d['foo'] = async.rdtsc()
def bar():
d['bar'] = async.rdtsc()
async.submit_work(foo)
async.submit_work(bar)
But I recently identified a few memory-management flaws with that
approach (I'm still on the fence with this issue... initially I was
going to drop all support, but I've since had ideas to address the
memory issues, so, we'll see).
There's also this option:
d = dict()
@async.call_from_main_thread_and_wait
def store(k, v):
d[str(k)] = str(v)
def foo():
store('foo', async.rdtsc())
def bar():
store('bar', async.rdtsc())
async.submit_work(foo)
async.submit_work(bar)
(Not a particularly performant option though; the main-thread
instantly becomes the bottleneck.)
Post-PyCon, I've been working on providing new interlocked data
types that are specifically designed to bridge the parallel/main-
thread divide:
xl = async.xlist()
def foo():
xl.push(async.rdtsc())
def bar():
xl.push(async.rdtsc())
async.submit_work(foo)
async.submit_work(bar)
while True:
x = xl.pop()
if not x:
break
process(x)
What's interesting about xlist() is that it takes ownership of the
parallel objects being pushed onto it. That is, it basically clones
them, using memory allocated from its own internal heap (allowing
the parallel-thread's context heap to be freed, which is desirable).
The push/pop operations are interlocked at the C level, which
obviates the need for any explicit locking.
I've put that work on hold for now though; I want to finish the
async client/server stuff (it's about 60-70% done) first. Once
that's done, I'll tackle the parallel.*-type façade.
Trent.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
On Thu, Apr 4, 2013 at 11:17 AM, Guido van Rossum wrote: > I don't really see what we could change to avoid breaking code in any > particular case Actually, the problem has *nothing* to do with circularity per se; it's that "import a.b" and "from a import b" behave differently in terms of how they obtain the module 'a.b'... And "from a import b" will *always* fail if 'a.b' is part of a cycle with the current module, whereas "import a.b" will *always* succeed. The workaround is to tell people to always use "import a.b" in the case of circular imports; it's practically a FAQ, at least to me. ;-) The problem with "from import" is that it always tries to getattr(a,'b'), even if 'a.b' is in sys.modules. In contrast, a plain import will simply fetch a.b from sys.modules first. In the case of a normal import, this is no problem, because a.b is set to sys.modules['a.b'] at the end of the module loading process. But if the import is circular, then the module is in sys.modules['a.b'], but *not* yet bound to a.b. So the "from import" fails. So, this is actually an implementation quirk that could be fixed in a (somewhat) straightforward manner: by making "from a import b" succeed if 'a.b' is in sys.modules, the same way "import a.b" does. It would require a little bit of discussion to hash out the exact way to do it, but it could be done. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
Thanks for the insight. That could indeed be done and I would encourage someone to come up with a fix. FWIW there is another difference between the two forms -- "from a import b" allows b to be any attribute of a, whereas "import a.b" requires b to be a submodule of a. I don't want to compromise on the latter. I do think it would be fine if "from a import b" returned the attribute 'b' of module 'a' if it exists, and otherwise look for module 'a.b' in sys.modules. On Thu, Apr 4, 2013 at 1:28 PM, PJ Eby wrote: > On Thu, Apr 4, 2013 at 11:17 AM, Guido van Rossum wrote: >> I don't really see what we could change to avoid breaking code in any >> particular case > > Actually, the problem has *nothing* to do with circularity per se; > it's that "import a.b" and "from a import b" behave differently in > terms of how they obtain the module 'a.b'... > > And "from a import b" will *always* fail if 'a.b' is part of a cycle > with the current module, whereas "import a.b" will *always* succeed. > > The workaround is to tell people to always use "import a.b" in the > case of circular imports; it's practically a FAQ, at least to me. ;-) > > The problem with "from import" is that it always tries to > getattr(a,'b'), even if 'a.b' is in sys.modules. In contrast, a plain > import will simply fetch a.b from sys.modules first. > > In the case of a normal import, this is no problem, because a.b is set > to sys.modules['a.b'] at the end of the module loading process. But > if the import is circular, then the module is in sys.modules['a.b'], > but *not* yet bound to a.b. So the "from import" fails. > > So, this is actually an implementation quirk that could be fixed in a > (somewhat) straightforward manner: by making "from a import b" succeed > if 'a.b' is in sys.modules, the same way "import a.b" does. It would > require a little bit of discussion to hash out the exact way to do it, > but it could be done. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum wrote: > I do think it would be fine if "from a import b" returned the > attribute 'b' of module 'a' if it exists, and otherwise look for > module 'a.b' in sys.modules. Technically, it already does that -- but inside of __import__, not in the IMPORT_FROM opcode. But then *after* doing that check-and-fallback, __import__ doesn't assign a.b, because it assumes the recursive import it called has already done this... which means that when __import__ returns, the IMPORT_FROM opcode tries and fails to do the getattr. This could be fixed in one of two ways. Either: 1. Change importlib._bootstrap._handle_fromlist() to set a.b if it successfully imports 'a.b' (inside its duplicate handling for what IMPORT_FROM does), or 2. Change the IMPORT_FROM opcode to handle the fallback itself While the latter involves a bit of C coding, it has fewer potential side-effects on the import system as a whole, and simply ensures that if "import" would succeed, then so would "from...import" targeting the same module. (There might be other fixes I haven't thought of, but really, changing IMPORT_FROM to fallback to a sys.modules check is probably by far the least-invasive way to handle it.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On 5 April 2013 02:16, Ethan Furman wrote: > On 04/04/2013 08:01 AM, Chris Angelico wrote: > >> On Fri, Apr 5, 2013 at 1:59 AM, Guido van Rossum >> wrote: >> >>> On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: >>> Is there any argument that I can pass to Foo() to get back a Bar()? Would anyone expect there to be one? Sure, I could override __new__ to do stupid things, but in terms of logical expectations, I'd expect that Foo(x) will return a Foo object, not a Bar object. Why should int be any different? What have I missed here? >>> >>> >>> A class can define a __new__ method that returns a different object. E.g. >>> (python 3): >>> >> >> Right, I'm aware it's possible. But who would expect it of a class? >> > > FTR I'm in the int() should return an int camp, but to answer your > question: my dbf module has a Table class, but it returns either a > Db3Table, FpTable, VfpTable, or ClpTable depending on arguments (if > creating a new one) or the type of the table in the existing dbf file. > I fall into: 1. int(), float(), str() etc should return that exact class (and operator.index() should return exactly an int). 2. It could sometimes be useful for __int__() and __index__() to return a subclass of int. So, for the int constructor, I would have the following logic (assume appropriate try/catch): def __new__(cls, obj): i = obj.__int__() if type(i) is int: return i return i._internal_value Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On Thu, Apr 4, 2013 at 1:50 PM, Tim Delaney wrote: > I fall into: > > 1. int(), float(), str() etc should return that exact class (and > operator.index() should return exactly an int). > > 2. It could sometimes be useful for __int__() and __index__() to return a > subclass of int. > > So, for the int constructor, I would have the following logic (assume > appropriate try/catch): > > def __new__(cls, obj): > i = obj.__int__() > > if type(i) is int: > return i > > return i._internal_value CPython can solve this in C using an unsafe cast, and the code that checks for allowable subclasses of int actually ensures such a cast will work. But it still feels wrong; __int__ should be expected to do the work. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
On 03Apr2013 12:12, Greg Ewing wrote: | Kristján Valur Jónsson wrote: | >However, relative imports can _only_ be performed using the "from X import Y syntax" | | This seems like a legitimate complaint on its own, [...] | There are a couple of ways that this could be resolved. One | would be to use the name resulting from stripping off the | leading dots, so that |import .foo | would bind the module to the name 'foo'. +0 from me (I'd have been +0.5 but for Guido's explaination about "import X" generally letting you use "X" exactly as phrased in the import). | Another would be | to always require an 'as' clause in this case, so that you | would have to write' |import .foo as foo And a big +1 for this. Cheers, -- Cameron Simpson The Force. It surrounds us; It enfolds us; It gets us dates on Saturday Nights. - Obi Wan Kenobi, Famous Jedi Knight and Party Animal. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
On Thu, Apr 4, 2013 at 5:00 PM, PJ Eby wrote: > On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum wrote: > > I do think it would be fine if "from a import b" returned the > > attribute 'b' of module 'a' if it exists, and otherwise look for > > module 'a.b' in sys.modules. > > Technically, it already does that -- but inside of __import__, not in > the IMPORT_FROM opcode. > > But then *after* doing that check-and-fallback, __import__ doesn't > assign a.b, because it assumes the recursive import it called has > already done this... It's an unfortunate side-effect of having loaders set sys.modules for new modules not also set them as an attribute on their parent package immediately as well (or you could argue it's a side-effect of not passing in a module instead of a name to load_module() but that's another discussion). > which means that when __import__ returns, the > IMPORT_FROM opcode tries and fails to do the getattr. > > This could be fixed in one of two ways. Either: > > 1. Change importlib._bootstrap._handle_fromlist() to set a.b if it > successfully imports 'a.b' (inside its duplicate handling for what > IMPORT_FROM does), or > It's three lines, one of which is 'else:'. Just did it. > 2. Change the IMPORT_FROM opcode to handle the fallback itself > While the latter involves a bit of C coding, it has fewer potential > side-effects on the import system as a whole, and simply ensures that > if "import" would succeed, then so would "from...import" targeting the > same module. > > (There might be other fixes I haven't thought of, but really, changing > IMPORT_FROM to fallback to a sys.modules check is probably by far the > least-invasive way to handle it.) > This is my preference as well. The change would be small: I think all you need to do is if the getattr() fails then fall back to sys.modules. Although if it were me and I was casting backwards-compatibility to the wind I would rip out the whole fromlist part of __import__() and let the bytecode worry about the fromlist, basically making the import opcode call importlib.import_module(). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
+1 on Brett and PJE just doing this. On Thu, Apr 4, 2013 at 3:38 PM, Brett Cannon wrote: > > > > On Thu, Apr 4, 2013 at 5:00 PM, PJ Eby wrote: >> >> On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum wrote: >> > I do think it would be fine if "from a import b" returned the >> > attribute 'b' of module 'a' if it exists, and otherwise look for >> > module 'a.b' in sys.modules. >> >> Technically, it already does that -- but inside of __import__, not in >> the IMPORT_FROM opcode. >> >> But then *after* doing that check-and-fallback, __import__ doesn't >> assign a.b, because it assumes the recursive import it called has >> already done this... > > > It's an unfortunate side-effect of having loaders set sys.modules for new > modules not also set them as an attribute on their parent package > immediately as well (or you could argue it's a side-effect of not passing in > a module instead of a name to load_module() but that's another discussion). > >> >> which means that when __import__ returns, the >> IMPORT_FROM opcode tries and fails to do the getattr. >> >> This could be fixed in one of two ways. Either: >> >> 1. Change importlib._bootstrap._handle_fromlist() to set a.b if it >> successfully imports 'a.b' (inside its duplicate handling for what >> IMPORT_FROM does), or > > > It's three lines, one of which is 'else:'. Just did it. > >> >> 2. Change the IMPORT_FROM opcode to handle the fallback itself >> >> >> While the latter involves a bit of C coding, it has fewer potential >> side-effects on the import system as a whole, and simply ensures that >> if "import" would succeed, then so would "from...import" targeting the >> same module. >> >> (There might be other fixes I haven't thought of, but really, changing >> IMPORT_FROM to fallback to a sys.modules check is probably by far the >> least-invasive way to handle it.) > > > This is my preference as well. The change would be small: I think all you > need to do is if the getattr() fails then fall back to sys.modules. Although > if it were me and I was casting backwards-compatibility to the wind I would > rip out the whole fromlist part of __import__() and let the bytecode worry > about the fromlist, basically making the import opcode call > importlib.import_module(). > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
On Apr 4, 2013 6:47 PM, "Guido van Rossum" wrote: > > +1 on Brett and PJE just doing this. I'll file a bug when I get home. -brett > > On Thu, Apr 4, 2013 at 3:38 PM, Brett Cannon wrote: > > > > > > > > On Thu, Apr 4, 2013 at 5:00 PM, PJ Eby wrote: > >> > >> On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum wrote: > >> > I do think it would be fine if "from a import b" returned the > >> > attribute 'b' of module 'a' if it exists, and otherwise look for > >> > module 'a.b' in sys.modules. > >> > >> Technically, it already does that -- but inside of __import__, not in > >> the IMPORT_FROM opcode. > >> > >> But then *after* doing that check-and-fallback, __import__ doesn't > >> assign a.b, because it assumes the recursive import it called has > >> already done this... > > > > > > It's an unfortunate side-effect of having loaders set sys.modules for new > > modules not also set them as an attribute on their parent package > > immediately as well (or you could argue it's a side-effect of not passing in > > a module instead of a name to load_module() but that's another discussion). > > > >> > >> which means that when __import__ returns, the > >> IMPORT_FROM opcode tries and fails to do the getattr. > >> > >> This could be fixed in one of two ways. Either: > >> > >> 1. Change importlib._bootstrap._handle_fromlist() to set a.b if it > >> successfully imports 'a.b' (inside its duplicate handling for what > >> IMPORT_FROM does), or > > > > > > It's three lines, one of which is 'else:'. Just did it. > > > >> > >> 2. Change the IMPORT_FROM opcode to handle the fallback itself > >> > >> > >> While the latter involves a bit of C coding, it has fewer potential > >> side-effects on the import system as a whole, and simply ensures that > >> if "import" would succeed, then so would "from...import" targeting the > >> same module. > >> > >> (There might be other fixes I haven't thought of, but really, changing > >> IMPORT_FROM to fallback to a sys.modules check is probably by far the > >> least-invasive way to handle it.) > > > > > > This is my preference as well. The change would be small: I think all you > > need to do is if the getattr() fails then fall back to sys.modules. Although > > if it were me and I was casting backwards-compatibility to the wind I would > > rip out the whole fromlist part of __import__() and let the bytecode worry > > about the fromlist, basically making the import opcode call > > importlib.import_module(). > > > > > > -- > --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On 5 Apr 2013 01:07, "Chris Angelico" wrote: > > On Fri, Apr 5, 2013 at 1:59 AM, Guido van Rossum wrote: > > On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: > >> Is there any argument that I can pass to Foo() to get back a Bar()? > >> Would anyone expect there to be one? Sure, I could override __new__ to > >> do stupid things, but in terms of logical expectations, I'd expect > >> that Foo(x) will return a Foo object, not a Bar object. Why should int > >> be any different? What have I missed here? > > > > > > A class can define a __new__ method that returns a different object. E.g. > > (python 3): > > > > Right, I'm aware it's possible. But who would expect it of a class? Python 3.3 does it for OSError to map errno values to the appropriate subclasses. That's mainly to aid migration to the new exception structure, though (see PEP 3151). For a clean slate API design you would use a separate factory function or class method to do the conversion. Cheers, Nick. > > ChrisA > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
http://bugs.python.org/issue17636 On Thu, Apr 4, 2013 at 8:03 PM, Brett Cannon wrote: > > On Apr 4, 2013 6:47 PM, "Guido van Rossum" wrote: > > > > +1 on Brett and PJE just doing this. > > I'll file a bug when I get home. > > -brett > > > > > On Thu, Apr 4, 2013 at 3:38 PM, Brett Cannon wrote: > > > > > > > > > > > > On Thu, Apr 4, 2013 at 5:00 PM, PJ Eby wrote: > > >> > > >> On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum > wrote: > > >> > I do think it would be fine if "from a import b" returned the > > >> > attribute 'b' of module 'a' if it exists, and otherwise look for > > >> > module 'a.b' in sys.modules. > > >> > > >> Technically, it already does that -- but inside of __import__, not in > > >> the IMPORT_FROM opcode. > > >> > > >> But then *after* doing that check-and-fallback, __import__ doesn't > > >> assign a.b, because it assumes the recursive import it called has > > >> already done this... > > > > > > > > > It's an unfortunate side-effect of having loaders set sys.modules for > new > > > modules not also set them as an attribute on their parent package > > > immediately as well (or you could argue it's a side-effect of not > passing in > > > a module instead of a name to load_module() but that's another > discussion). > > > > > >> > > >> which means that when __import__ returns, the > > >> IMPORT_FROM opcode tries and fails to do the getattr. > > >> > > >> This could be fixed in one of two ways. Either: > > >> > > >> 1. Change importlib._bootstrap._handle_fromlist() to set a.b if it > > >> successfully imports 'a.b' (inside its duplicate handling for what > > >> IMPORT_FROM does), or > > > > > > > > > It's three lines, one of which is 'else:'. Just did it. > > > > > >> > > >> 2. Change the IMPORT_FROM opcode to handle the fallback itself > > >> > > >> > > >> While the latter involves a bit of C coding, it has fewer potential > > >> side-effects on the import system as a whole, and simply ensures that > > >> if "import" would succeed, then so would "from...import" targeting the > > >> same module. > > >> > > >> (There might be other fixes I haven't thought of, but really, changing > > >> IMPORT_FROM to fallback to a sys.modules check is probably by far the > > >> least-invasive way to handle it.) > > > > > > > > > This is my preference as well. The change would be small: I think all > you > > > need to do is if the getattr() fails then fall back to sys.modules. > Although > > > if it were me and I was casting backwards-compatibility to the wind I > would > > > rip out the whole fromlist part of __import__() and let the bytecode > worry > > > about the fromlist, basically making the import opcode call > > > importlib.import_module(). > > > > > > > > > > > -- > > --Guido van Rossum (python.org/~guido) > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On 05/04/13 01:23, Oscar Benjamin wrote: The reason for calling int(obj) is to get an object that is precisely of type int. When I call this I do not want any modified or additional methods or data attached to the resulting object. When I call int(), I'm expecting an int. That includes well-behaved subclasses of int that continue to behave like ints in all the ways that matter. It's curious to see the (d)evolution of thinking on type checking in Python circles. Once upon a time, type checking was discouraged, duck-typing was encouraged, and the philosophy was that an int is anything that behaves like an int. Now type-checking is tolerated or even encouraged, provided that you use isinstance, and an int is anything that inherits from the builtin (or the ABC). And this proposed change in behaviour continues the move away from Python's former emphasis on duck-typing. Now an int is only the builtin. Oscar, a question: what harm does it do to you if the int you receive has additional methods or data attached? I can appreciate Guido's concern that (say) a subclass might mess with the repr of the number, and he doesn't want that. (I don't find that argument compelling, but it does make sense.) But I don't understand why you would care if the int you receive happens to have an additional method or data that you don't expect. [...] I think it's fair to say that there are times when someone wants an object that is precisely of type int. They should be able to rely on int(obj) returning an int or raising an error. The first part of that is correct, but I disagree on the second. The 90-10 rule applies here. 90% of the time, we shouldn't care whether we receive an actual builtin int, only that we receive something that quacks like an int. In practice, that generally means something that inherits from int. IMO, int() and __int__ should support the common case, and not enforce the uncommon case This is true similarly for __index__. I have no argument with the idea that __index__ should guarantee to return a builtin int. I thought it already did make that guarantee. -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On 04/03/2013 09:04 AM, Steven D'Aprano wrote: On 04/04/13 01:16, Barry Warsaw wrote: It's analogous to all the other built-in types-as-functions, so int() calls __int__() which must return a concrete integer. Why must it? I think that's the claim which must be justified, not just taken as a given. Principle Of Least Surprise. My observation of the thread is that the majority of people feel "of course __int__ needs to return a real int". People are surprised that you can return subclasses. Nick thought there was an explicit check to prevent it! Why is this so surprising? I think it's because int, str, and float are all classes. Therefore, as callables they are constructors. When you call a constructor, you expect to get an instance of that specific class. Yes, it's always been possible with new-style classes to return a subclass from the constructor. But calling a constructor and getting a subclass is always surprising behavior. Also, permitting subclasses means the interface becomes conceptually far more complicated. We would need to restrict it to subclasses that don't hijack the representation. It's plausible, for example, to write a subclass of str that uses a novel approach for storing the string data, say as a rope. It could overload all the magic methods and behave identically to a str in every important way. But if its __str__ returned self instead of a real str object, that means that PyUnicode_AS_UNICODE would blissfully return the bypassed internal string value, whatever it is, and Python breaks. So we can't enforce it programmatically, we'd need to document it as convention. But explaining that is complicated, and If The Implementation Is Hard To Explain, It's A Bad Idea. When we call n = int(something), what's the use-case for caring that n is an instance of built-in int but not of a subclass, and is that use-case so compelling that it must be enforced for all uses of int() etc.? I'm much more interested in your counter use-case. When is it appealing or necessary to you to return a subclass of int from __int__()? In your postings so far you've said that this makes sense to you, but you haven't said why you need it. In lieu of a compelling use case, my vote is firmly against surprise and complexity. //arry/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
Guido van Rossum, 04.04.2013 23:14: > On Thu, Apr 4, 2013 at 1:50 PM, Tim Delaney wrote: >> I fall into: >> >> 1. int(), float(), str() etc should return that exact class (and >> operator.index() should return exactly an int). >> >> 2. It could sometimes be useful for __int__() and __index__() to return a >> subclass of int. >> >> So, for the int constructor, I would have the following logic (assume >> appropriate try/catch): >> >> def __new__(cls, obj): >> i = obj.__int__() >> >> if type(i) is int: >> return i >> >> return i._internal_value > > CPython can solve this in C using an unsafe cast, and the code that > checks for allowable subclasses of int actually ensures such a cast > will work. But it still feels wrong; __int__ should be expected to do > the work. +1, that's why it's called "__int__" (even if it returns a PyLong in Py3 ;) Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Slides from today's parallel/async Python talk
Hello, >> async.submit_work(func, args, kwds, callback=None, errback=None) >> >> How do you implement arguments passing and return value? >> >> e.g. let's say I pass a list as argument: how do you iterate on the >> list from the worker thread without modifying the backing objects for >> refcounts (IIUC you use a per-thread heap and don't do any >> refcounting). > > Correct, nothing special is done for the arguments (apart from > incref'ing them in the main thread before kicking off the parallel > thread (then decref'ing them in the main thread once we're sure the > parallel thread has finished)). IIUC you incref the argument from the main thread before publishing it to the worker thread: but what about containers like list? How do you make sure the refcounts of the elements don't get deallocated while the worker thread iterates? More generally, how do you deal with non-local objects? BTW I don't know if you did, but you could probably have a look at Go's goroutines and Erlang processes. cf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
