Re: [Python-Dev] boxing and unboxing data types
On Mon, Mar 9, 2015 at 2:07 AM, Serhiy Storchaka storch...@gmail.com
wrote:
On 09.03.15 06:33, Ethan Furman wrote:
I guess it could boil down to: if IntEnum was not based on 'int', but
instead had the __int__ and __index__ methods
(plus all the other __xxx__ methods that int has), would it still be a
drop-in replacement for actual ints? Even when
being used to talk to non-Python libs?
If you don't call isinstance(x, int) (PyLong_Check* in C).
Most conversions from Python to C implicitly call __index__ or __int__,
but unfortunately not all.
float(Thin(42))
42.0
float(Wrap(42))
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: float() argument must be a string or a number, not 'Wrap'
'%*s' % (Thin(5), 'x')
'x'
'%*s' % (Wrap(5), 'x')
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: * wants int
OSError(Thin(2), 'No such file or directory')
FileNotFoundError(2, 'No such file or directory')
OSError(Wrap(2), 'No such file or directory')
OSError(__main__.Wrap object at 0xb6fe81ac, 'No such file or directory')
re.match('(x)', 'x').group(Thin(1))
'x'
re.match('(x)', 'x').group(Wrap(1))
Traceback (most recent call last):
File stdin, line 1, in module
IndexError: no such group
And to be ideal drop-in replacement IntEnum should override such methods
as __eq__ and __hash__ (so it could be used as mapping key). If all methods
should be overridden to quack as int, why not take an int?
You're absolutely right that if *all the methods should be overrriden to
quack as int, then you should subclass int (the Liskov substitution
principle). But all methods should not be overridden — mainly the methods
you overrode in your patch should be exposed. Here is a list of methods on
int that should not be on IntFlags in my opinion (give or take a couple):
__abs__, __add__, __delattr__, __divmod__, __float__, __floor__,
__floordiv__, __index__, __lshift__, __mod__, __mul__, __pos__, __pow__,
__radd__, __rdivmod__, __rfloordiv__, __rlshift__, __rmod__, __rmul__,
__round__, __rpow__, __rrshift__, __rshift__, __rsub__, __rtruediv__,
__sub__, __truediv__, __trunc__, conjugate, denominator, imag, numerator,
real.
I don't think __index__ should be exposed either since are you really going
to slice a list using IntFlags? Really?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/
mistersheik%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
понеділок, 09-бер-2015 10:18:50 ви написали: On Mon, Mar 9, 2015 at 10:10 AM, Serhiy Storchaka storch...@gmail.com wrote: понеділок, 09-бер-2015 09:52:01 ви написали: On Mon, Mar 9, 2015 at 2:07 AM, Serhiy Storchaka storch...@gmail.com And to be ideal drop-in replacement IntEnum should override such methods as __eq__ and __hash__ (so it could be used as mapping key). If all methods should be overridden to quack as int, why not take an int? You're absolutely right that if *all the methods should be overrriden to quack as int, then you should subclass int (the Liskov substitution principle). But all methods should not be overridden — mainly the methods you overrode in your patch should be exposed. Here is a list of methods on int that should not be on IntFlags in my opinion (give or take a couple): __abs__, __add__, __delattr__, __divmod__, __float__, __floor__, __floordiv__, __index__, __lshift__, __mod__, __mul__, __pos__, __pow__, __radd__, __rdivmod__, __rfloordiv__, __rlshift__, __rmod__, __rmul__, __round__, __rpow__, __rrshift__, __rshift__, __rsub__, __rtruediv__, __sub__, __truediv__, __trunc__, conjugate, denominator, imag, numerator, real. I don't think __index__ should be exposed either since are you really going to slice a list using IntFlags? Really? Definitely __index__ should be exposed. __int__ is for lossy conversion to int (as in float). __index__ is for lossless conversion. Is it? __index__ promises lossless conversion, but __index__ is *for* indexing. I spite of its name it is for any lossless conversion. __add__ should be exposed because some code can use + instead of | for combining flags. But it shouldn't preserve the type, because this is not recommended way. I think it should be blocked because it can lead to all kinds of weird bugs. If the flag is already set and you add it a copy, it silently spills over into other flags. This is a mistake that a good interface prevents. I think this is a case when backward compatibility has larger weight. For the same reason I think __lshift__, __rshift__, __sub__, __mul__, __divmod__, __floordiv__, __mod__, etc should be exposed too. So the majority of the methods should be exposed, and there is a risk that we loss something. I totally disagree with all of those. For good compatibility with Python code IntFlags should expose also __subclasscheck__ or __subclasshook__. And when we are at this point, why not use int subclass? Here's another reason. What if someone wants to use an IntFlags object, but wants to use a fixed width type for storage, say numpy.int32? Why shouldn't they be able to do that? By using composition, you can easily provide such an option. You can design abstract interface Flags that can be combined with int or other type. But why you want to use numpy.int32 as storage? This doesn't save much memory, because with composition the IntFlags class weighs more than int subclass. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On Mon, Mar 09, 2015 at 09:52:01AM -0400, Neil Girdhar wrote: Here is a list of methods on int that should not be on IntFlags in my opinion (give or take a couple): __abs__, __add__, __delattr__, __divmod__, __float__, __floor__, __floordiv__, __index__, __lshift__, __mod__, __mul__, __pos__, __pow__, __radd__, __rdivmod__, __rfloordiv__, __rlshift__, __rmod__, __rmul__, __round__, __rpow__, __rrshift__, __rshift__, __rsub__, __rtruediv__, __sub__, __truediv__, __trunc__, conjugate, denominator, imag, numerator, real. I don't think __index__ should be exposed either since are you really going to slice a list using IntFlags? Really? In what way is this an *Int*Flags object if it is nothing like an int? It sounds like what you want is a bunch of Enum inside a set with a custom __str__, not IntFlags. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On Mon, Mar 9, 2015 at 11:46 AM, Steven D'Aprano st...@pearwood.info wrote: On Mon, Mar 09, 2015 at 09:52:01AM -0400, Neil Girdhar wrote: Here is a list of methods on int that should not be on IntFlags in my opinion (give or take a couple): __abs__, __add__, __delattr__, __divmod__, __float__, __floor__, __floordiv__, __index__, __lshift__, __mod__, __mul__, __pos__, __pow__, __radd__, __rdivmod__, __rfloordiv__, __rlshift__, __rmod__, __rmul__, __round__, __rpow__, __rrshift__, __rshift__, __rsub__, __rtruediv__, __sub__, __truediv__, __trunc__, conjugate, denominator, imag, numerator, real. I don't think __index__ should be exposed either since are you really going to slice a list using IntFlags? Really? In what way is this an *Int*Flags object if it is nothing like an int? It sounds like what you want is a bunch of Enum inside a set with a custom __str__, not IntFlags. It doesn't matter what you call it. I believe the goal of this is to have a flags object with flags operations and pretty-printing. It makes more sense to me to decide the interface and then the implementation. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/mistersheik%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On Mon, Mar 9, 2015 at 11:11 AM, Serhiy Storchaka storch...@gmail.com wrote: понеділок, 09-бер-2015 10:18:50 ви написали: On Mon, Mar 9, 2015 at 10:10 AM, Serhiy Storchaka storch...@gmail.com wrote: понеділок, 09-бер-2015 09:52:01 ви написали: On Mon, Mar 9, 2015 at 2:07 AM, Serhiy Storchaka storch...@gmail.com And to be ideal drop-in replacement IntEnum should override such methods as __eq__ and __hash__ (so it could be used as mapping key). If all methods should be overridden to quack as int, why not take an int? You're absolutely right that if *all the methods should be overrriden to quack as int, then you should subclass int (the Liskov substitution principle). But all methods should not be overridden — mainly the methods you overrode in your patch should be exposed. Here is a list of methods on int that should not be on IntFlags in my opinion (give or take a couple): __abs__, __add__, __delattr__, __divmod__, __float__, __floor__, __floordiv__, __index__, __lshift__, __mod__, __mul__, __pos__, __pow__, __radd__, __rdivmod__, __rfloordiv__, __rlshift__, __rmod__, __rmul__, __round__, __rpow__, __rrshift__, __rshift__, __rsub__, __rtruediv__, __sub__, __truediv__, __trunc__, conjugate, denominator, imag, numerator, real. I don't think __index__ should be exposed either since are you really going to slice a list using IntFlags? Really? Definitely __index__ should be exposed. __int__ is for lossy conversion to int (as in float). __index__ is for lossless conversion. Is it? __index__ promises lossless conversion, but __index__ is *for* indexing. I spite of its name it is for any lossless conversion. You're right. __add__ should be exposed because some code can use + instead of | for combining flags. But it shouldn't preserve the type, because this is not recommended way. I think it should be blocked because it can lead to all kinds of weird bugs. If the flag is already set and you add it a copy, it silently spills over into other flags. This is a mistake that a good interface prevents. I think this is a case when backward compatibility has larger weight. So you agree that the ideal solution is composition, but you prefer inheritance in order to not break code? Then,I think the big question is how much code would actually break if you presented the ideal interface. I imagine that 99% of the code using flags only uses __or__ to compose and __and__, __invert__ to erase flags. For the same reason I think __lshift__, __rshift__, __sub__, __mul__, __divmod__, __floordiv__, __mod__, etc should be exposed too. So the majority of the methods should be exposed, and there is a risk that we loss something. I totally disagree with all of those. For good compatibility with Python code IntFlags should expose also __subclasscheck__ or __subclasshook__. And when we are at this point, why not use int subclass? Here's another reason. What if someone wants to use an IntFlags object, but wants to use a fixed width type for storage, say numpy.int32? Why shouldn't they be able to do that? By using composition, you can easily provide such an option. You can design abstract interface Flags that can be combined with int or other type. But why you want to use numpy.int32 as storage? This doesn't save much memory, because with composition the IntFlags class weighs more than int subclass. Maybe you're storing a bunch of flags in a numpy array having dtype np.int32? It's contrived, I agree. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On 09.03.15 17:48, Neil Girdhar wrote: So you agree that the ideal solution is composition, but you prefer inheritance in order to not break code? Yes, I agree. There is two advantages in the inheritance: larger backward compatibility and simpler implementation. Then,I think the big question is how much code would actually break if you presented the ideal interface. I imagine that 99% of the code using flags only uses __or__ to compose and __and__, __invert__ to erase flags. I don't know and don't want to guess. Let just follow the way of bool and IntEnum. When users will be encouraged to use IntEnum and IntFlags instead of plain ints we could consider the idea of dropping inheritance of bool, IntEnum and IntFlags from int. This is not near future. Here's another reason. What if someone wants to use an IntFlags object, but wants to use a fixed width type for storage, say numpy.int32? Why shouldn't they be able to do that? By using composition, you can easily provide such an option. You can design abstract interface Flags that can be combined with int or other type. But why you want to use numpy.int32 as storage? This doesn't save much memory, because with composition the IntFlags class weighs more than int subclass. Maybe you're storing a bunch of flags in a numpy array having dtype np.int32? It's contrived, I agree. I afraid that composition will not help you with this. Can numpy array pack int-like objects into fixed-width integer array and then restore original type on unboxing? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On Mon, Mar 9, 2015 at 12:54 PM, Serhiy Storchaka storch...@gmail.com wrote: On 09.03.15 17:48, Neil Girdhar wrote: So you agree that the ideal solution is composition, but you prefer inheritance in order to not break code? Yes, I agree. There is two advantages in the inheritance: larger backward compatibility and simpler implementation. Inheritance might be more backwards compatible, but I believe that you should check how much code is genuine not restricted to the idealized flags interface. It's not worth talking about simpler implementation since the two solutions differ by only a couple dozen lines. On the other hand, composition is better design. It prevents you from making mistakes like adding to flags and having carries, or using flags in an unintended way. Then,I think the big question is how much code would actually break if you presented the ideal interface. I imagine that 99% of the code using flags only uses __or__ to compose and __and__, __invert__ to erase flags. I don't know and don't want to guess. Let just follow the way of bool and IntEnum. When users will be encouraged to use IntEnum and IntFlags instead of plain ints we could consider the idea of dropping inheritance of bool, IntEnum and IntFlags from int. This is not near future. I think it's the other way around. You should typically start with the modest interface and add methods as you need. If you start with full blown inheritance, you will find it only increasingly more difficult to remove methods in changing your solution. Using inheritance instead of composition is one of the most common errors in objected oriented programming, and I get the impression from your other paragraph that you're seduced by the slightly shorter code. I don't think it's worth giving in to that without proof that composition will actually break a significant amount of code. Regarding IntEnum — that should inherit from int since they are truly just integer constants. It's too late for bool; that ship has sailed unfortunately. Here's another reason. What if someone wants to use an IntFlags object, but wants to use a fixed width type for storage, say numpy.int32? Why shouldn't they be able to do that? By using composition, you can easily provide such an option. You can design abstract interface Flags that can be combined with int or other type. But why you want to use numpy.int32 as storage? This doesn't save much memory, because with composition the IntFlags class weighs more than int subclass. Maybe you're storing a bunch of flags in a numpy array having dtype np.int32? It's contrived, I agree. I afraid that composition will not help you with this. Can numpy array pack int-like objects into fixed-width integer array and then restore original type on unboxing? You're right. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ mistersheik%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
Totally agree On 9 Mar 2015 19:22, Nick Coghlan ncogh...@gmail.com wrote: On 10 Mar 2015 06:51, Neil Girdhar mistersh...@gmail.com wrote: On Mon, Mar 9, 2015 at 12:54 PM, Serhiy Storchaka storch...@gmail.com wrote: On 09.03.15 17:48, Neil Girdhar wrote: So you agree that the ideal solution is composition, but you prefer inheritance in order to not break code? Yes, I agree. There is two advantages in the inheritance: larger backward compatibility and simpler implementation. Inheritance might be more backwards compatible, but I believe that you should check how much code is genuine not restricted to the idealized flags interface. It's not worth talking about simpler implementation since the two solutions differ by only a couple dozen lines. We literally can't do this, as the vast majority of Python code in the world is locked up behind institutional firewalls or has otherwise never been published. The open source stuff is merely the tip of a truly enormous iceberg. If we want to *use* IntFlags in the standard library (and that's the only pay-off significant enough to justify having it in the standard library), then it needs to inherit from int. However, cloning the full enum module architecture to create flags.FlagsMeta, flags.Flags and flags.IntFlags would make sense to me. It would also make sense to try that idea out on PyPI for a while before incorporating it into the stdlib. Regards, Nick. On the other hand, composition is better design. It prevents you from making mistakes like adding to flags and having carries, or using flags in an unintended way. Then,I think the big question is how much code would actually break if you presented the ideal interface. I imagine that 99% of the code using flags only uses __or__ to compose and __and__, __invert__ to erase flags. I don't know and don't want to guess. Let just follow the way of bool and IntEnum. When users will be encouraged to use IntEnum and IntFlags instead of plain ints we could consider the idea of dropping inheritance of bool, IntEnum and IntFlags from int. This is not near future. I think it's the other way around. You should typically start with the modest interface and add methods as you need. If you start with full blown inheritance, you will find it only increasingly more difficult to remove methods in changing your solution. Using inheritance instead of composition is one of the most common errors in objected oriented programming, and I get the impression from your other paragraph that you're seduced by the slightly shorter code. I don't think it's worth giving in to that without proof that composition will actually break a significant amount of code. Regarding IntEnum — that should inherit from int since they are truly just integer constants. It's too late for bool; that ship has sailed unfortunately. Here's another reason. What if someone wants to use an IntFlags object, but wants to use a fixed width type for storage, say numpy.int32? Why shouldn't they be able to do that? By using composition, you can easily provide such an option. You can design abstract interface Flags that can be combined with int or other type. But why you want to use numpy.int32 as storage? This doesn't save much memory, because with composition the IntFlags class weighs more than int subclass. Maybe you're storing a bunch of flags in a numpy array having dtype np.int32? It's contrived, I agree. I afraid that composition will not help you with this. Can numpy array pack int-like objects into fixed-width integer array and then restore original type on unboxing? You're right. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/mistersheik%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On 10 Mar 2015 06:51, Neil Girdhar mistersh...@gmail.com wrote: On Mon, Mar 9, 2015 at 12:54 PM, Serhiy Storchaka storch...@gmail.com wrote: On 09.03.15 17:48, Neil Girdhar wrote: So you agree that the ideal solution is composition, but you prefer inheritance in order to not break code? Yes, I agree. There is two advantages in the inheritance: larger backward compatibility and simpler implementation. Inheritance might be more backwards compatible, but I believe that you should check how much code is genuine not restricted to the idealized flags interface. It's not worth talking about simpler implementation since the two solutions differ by only a couple dozen lines. We literally can't do this, as the vast majority of Python code in the world is locked up behind institutional firewalls or has otherwise never been published. The open source stuff is merely the tip of a truly enormous iceberg. If we want to *use* IntFlags in the standard library (and that's the only pay-off significant enough to justify having it in the standard library), then it needs to inherit from int. However, cloning the full enum module architecture to create flags.FlagsMeta, flags.Flags and flags.IntFlags would make sense to me. It would also make sense to try that idea out on PyPI for a while before incorporating it into the stdlib. Regards, Nick. On the other hand, composition is better design. It prevents you from making mistakes like adding to flags and having carries, or using flags in an unintended way. Then,I think the big question is how much code would actually break if you presented the ideal interface. I imagine that 99% of the code using flags only uses __or__ to compose and __and__, __invert__ to erase flags. I don't know and don't want to guess. Let just follow the way of bool and IntEnum. When users will be encouraged to use IntEnum and IntFlags instead of plain ints we could consider the idea of dropping inheritance of bool, IntEnum and IntFlags from int. This is not near future. I think it's the other way around. You should typically start with the modest interface and add methods as you need. If you start with full blown inheritance, you will find it only increasingly more difficult to remove methods in changing your solution. Using inheritance instead of composition is one of the most common errors in objected oriented programming, and I get the impression from your other paragraph that you're seduced by the slightly shorter code. I don't think it's worth giving in to that without proof that composition will actually break a significant amount of code. Regarding IntEnum — that should inherit from int since they are truly just integer constants. It's too late for bool; that ship has sailed unfortunately. Here's another reason. What if someone wants to use an IntFlags object, but wants to use a fixed width type for storage, say numpy.int32? Why shouldn't they be able to do that? By using composition, you can easily provide such an option. You can design abstract interface Flags that can be combined with int or other type. But why you want to use numpy.int32 as storage? This doesn't save much memory, because with composition the IntFlags class weighs more than int subclass. Maybe you're storing a bunch of flags in a numpy array having dtype np.int32? It's contrived, I agree. I afraid that composition will not help you with this. Can numpy array pack int-like objects into fixed-width integer array and then restore original type on unboxing? You're right. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/mistersheik%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
Not all your examples are good.
* float(x) calls __float__ (not __int__)
* re.group requires __eq__ (and __hash__)
* I'm unsure about OSError
* the % thing at the very least works on pypy
On Mon, Mar 9, 2015 at 8:07 AM, Serhiy Storchaka storch...@gmail.com wrote:
On 09.03.15 06:33, Ethan Furman wrote:
I guess it could boil down to: if IntEnum was not based on 'int', but
instead had the __int__ and __index__ methods
(plus all the other __xxx__ methods that int has), would it still be a
drop-in replacement for actual ints? Even when
being used to talk to non-Python libs?
If you don't call isinstance(x, int) (PyLong_Check* in C).
Most conversions from Python to C implicitly call __index__ or __int__, but
unfortunately not all.
float(Thin(42))
42.0
float(Wrap(42))
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: float() argument must be a string or a number, not 'Wrap'
'%*s' % (Thin(5), 'x')
'x'
'%*s' % (Wrap(5), 'x')
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: * wants int
OSError(Thin(2), 'No such file or directory')
FileNotFoundError(2, 'No such file or directory')
OSError(Wrap(2), 'No such file or directory')
OSError(__main__.Wrap object at 0xb6fe81ac, 'No such file or directory')
re.match('(x)', 'x').group(Thin(1))
'x'
re.match('(x)', 'x').group(Wrap(1))
Traceback (most recent call last):
File stdin, line 1, in module
IndexError: no such group
And to be ideal drop-in replacement IntEnum should override such methods as
__eq__ and __hash__ (so it could be used as mapping key). If all methods
should be overridden to quack as int, why not take an int?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On Mon, 9 Mar 2015 15:12:44 +1100 Steven D'Aprano st...@pearwood.info wrote: My summary is as follows: __int__ is used as the special method for int(), and it should coerce the object to an integer. This may be lossy e.g. int(2.999) -- 2 or may involve a conversion from a non-numeric type to integer e.g. int(2). Your example is misleading. Strings don't have an __int__: s = 3 s.__int__() Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'str' object has no attribute '__int__' Only int-compatible or int-coercible types (e.g. float, Decimal) should have an __int__ method. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On 09.03.15 10:19, Maciej Fijalkowski wrote: Not all your examples are good. * float(x) calls __float__ (not __int__) * re.group requires __eq__ (and __hash__) * I'm unsure about OSError * the % thing at the very least works on pypy Yes, all these examples are implementation defined and can differ between CPython and PyPy. There is about a dozen of similar examples only in C part of CPython. Most of them have in common is that the behavior of the function depends on the argument type. For example in case of re.group an argument is either integer index or string group name. OSError constructor can produce OSError subtype if first argument is known integer errno. float either convert a number to float or parse a string (or bytes). Python functions can be more lenient (if they allows ducktyping) or more strict (if they explicitly check the type). They rarely call __index__ or __int__. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On 09.03.15 08:12, Ethan Furman wrote: On 03/08/2015 11:07 PM, Serhiy Storchaka wrote: If you don't call isinstance(x, int) (PyLong_Check* in C). Most conversions from Python to C implicitly call __index__ or __int__, but unfortunately not all. [snip examples] Thanks, Serhiy, that's what I was looking for. May be most if not all of these examples can be considered as bugs and slowly fixed, but we can't control third-party code. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On 09.03.15 06:33, Ethan Furman wrote:
I guess it could boil down to: if IntEnum was not based on 'int', but instead
had the __int__ and __index__ methods
(plus all the other __xxx__ methods that int has), would it still be a drop-in
replacement for actual ints? Even when
being used to talk to non-Python libs?
If you don't call isinstance(x, int) (PyLong_Check* in C).
Most conversions from Python to C implicitly call __index__ or __int__,
but unfortunately not all.
float(Thin(42))
42.0
float(Wrap(42))
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: float() argument must be a string or a number, not 'Wrap'
'%*s' % (Thin(5), 'x')
'x'
'%*s' % (Wrap(5), 'x')
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: * wants int
OSError(Thin(2), 'No such file or directory')
FileNotFoundError(2, 'No such file or directory')
OSError(Wrap(2), 'No such file or directory')
OSError(__main__.Wrap object at 0xb6fe81ac, 'No such file or directory')
re.match('(x)', 'x').group(Thin(1))
'x'
re.match('(x)', 'x').group(Wrap(1))
Traceback (most recent call last):
File stdin, line 1, in module
IndexError: no such group
And to be ideal drop-in replacement IntEnum should override such methods
as __eq__ and __hash__ (so it could be used as mapping key). If all
methods should be overridden to quack as int, why not take an int?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On 03/08/2015 11:07 PM, Serhiy Storchaka wrote: If you don't call isinstance(x, int) (PyLong_Check* in C). Most conversions from Python to C implicitly call __index__ or __int__, but unfortunately not all. [snip examples] Thanks, Serhiy, that's what I was looking for. -- ~Ethan~ signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] boxing and unboxing data types
When data is passed from Python to a native library (such as in an O/S call),
how does the unboxing of data types occur?
For a specific instance, os.open allows an integer whose various bits express
desired behavior as `flags` -- if flags is
1, for example, the file is open write-only.
If I pass an int-like object to os.open, __int__ is called; if I pass a thin
wrapper over int to os.open, __int__ is not
called.
So the real question: anywhere in Python where an int is expected (for
lower-level API work), but not directly
received, should __int__ (or __index__) be called? and failure to do so is a
bug?
Here's my simple test code:
class Wrap:
def __init__(self, value):
self.value = value
def __int__(self):
print('__int__')
return self.value
def __index__(self):
print('__index__')
return self.value
class Thin(int):
def __int__(self):
print('__int__')
return super().__int__()
def __index__(self):
print('__index__')
return super().__index__()
two = Wrap(2)
[0, 1, 2][two]
# __index__
# 2
import struct
struct.pack('i', two)
# __index__
# b'\x02\x00\x00\x00'
t = Thin(1)
huh = os.open('blah.txt', t)
# Traceback (most recent call last):
# File stdin, line 1, in module
# FileNotFoundError: [Errno 2] No such file or directory: 'blah.txt'
--
~Ethan~
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On Sun, Mar 08, 2015 at 08:31:30PM -0700, Ethan Furman wrote: When data is passed from Python to a native library (such as in an O/S call), how does the unboxing of data types occur? [...] So the real question: anywhere in Python where an int is expected (for lower-level API work), but not directly received, should __int__ (or __index__) be called? and failure to do so is a bug? I think the answer is in the docs: https://docs.python.org/3/reference/datamodel.html#object.__int__ Immediately below that __index__ is described, with this note: In order to have a coherent integer type class, when __index__() is defined __int__() should also be defined, and both should return the same value. The PEP adding __index__ is also useful: https://www.python.org/dev/peps/pep-0357/ My summary is as follows: __int__ is used as the special method for int(), and it should coerce the object to an integer. This may be lossy e.g. int(2.999) -- 2 or may involve a conversion from a non-numeric type to integer e.g. int(2). __index__ is used when the object in question actually represents an integer of some kind, e.g. a fixed-with integer. Conversion should be lossless and conceptually may be thought of a way of telling Python this value actually is an int, even though it doesn't inherit from int (for some definition of is an int). There's no built-in way of calling __index__ that I know of (no equivalent to int(obj)), but slicing at the very least will call it, e.g. seq[a:] will call type(a).__index__. If you define __index__ for your class, you should also define __int__ and have the two return the same value. I would expect that an IntFlags object should inherit from int, and if that is not possible, practical or desirable for some reason, then it should define __index__ and __int__. Failure to call __index__ is not necessarily a bug. I think it is allowed for functions to insist on an actual int, as slicing did for many years, but it is an obvious enhancement to allow such functions to accept arbitrary int-like objects. Does that answer your questions? -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On Mar 8, 2015 9:13 PM, Steven D'Aprano st...@pearwood.info wrote: There's no built-in way of calling __index__ that I know of (no equivalent to int(obj)), There's operator.index(obj), at least. but slicing at the very least will call it, e.g. seq[a:] will call type(a).__index__. -n ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] boxing and unboxing data types
On 03/08/2015 09:12 PM, Steven D'Aprano wrote: Does that answer your questions? No, unfortunately. You correctly guessed my question is motivated by the IntFlag discussion. I guess it could boil down to: if IntEnum was not based on 'int', but instead had the __int__ and __index__ methods (plus all the other __xxx__ methods that int has), would it still be a drop-in replacement for actual ints? Even when being used to talk to non-Python libs? -- ~Ethan~ signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
