Not those. On Nov 3, 2014 8:56 AM, "Antoine Pitrou" <solip...@pitrou.net> wrote:
> On Mon, 3 Nov 2014 08:48:07 -0800 > Guido van Rossum <gu...@python.org> wrote: > > Gotta be brief, but NotImplemented is for all binary ops. > > Even in-place ops? > > Regards > > Antoine. > > > > Power may be an > > exception because it's ternary? > > On Nov 3, 2014 8:08 AM, "Brett Cannon" <br...@python.org> wrote: > > > > > > > > > > > On Mon Nov 03 2014 at 5:31:21 AM Ethan Furman <et...@stoneleaf.us> > wrote: > > > > > >> Just to be clear, this is about NotImplemented, not > NotImplementedError. > > >> > > >> tl;dr When a binary operation fails, should an exception be raised or > > >> NotImplemented returned? > > >> > > > > > > The docs for NotImplemented suggest it's only for rich comparison > methods > > > and not all binary operators: > > > https://docs.python.org/3/library/constants.html#NotImplemented . But > > > then had I not read that I would have said all binary operator methods > > > should return NotImplemented when the types are incompatible. > > > > > > -Brett > > > > > > > > >> > > >> > > >> When a binary operation in Python is attempted, there are two > > >> possibilities: > > >> > > >> - it can work > > >> - it can't work > > >> > > >> The main reason [1] that it can't work is that the two operands are of > > >> different types, and the first type does not know > > >> how to deal with the second type. > > >> > > >> The question then becomes: how does the first type tell Python that it > > >> cannot perform the requested operation? The most > > >> obvious answer is to raise an exception, and TypeError is a good > > >> candidate. The problem with the exception raising > > >> approach is that once an exception is raised, Python doesn't try > anything > > >> else to make the operation work. > > >> > > >> What's wrong with that? Well, the second type might know how to > perform > > >> the operation, and in fact that is why we have > > >> the reflected special methods, such as __radd__ and __rmod__ -- but if > > >> the first type raises an exception the __rxxx__ > > >> methods will not be tried. > > >> > > >> Okay, how can the first type tell Python that it cannot do what is > > >> requested, but to go ahead and check with the second > > >> type to see if it does? That is where NotImplemented comes in -- if a > > >> special method (and only a special method) > > >> returns NotImplemented then Python will check to see if there is > anything > > >> else it can do to make the operation succeed; > > >> if all attempts return NotImplemented, then Python itself will raise > an > > >> appropriate exception [2]. > > >> > > >> In an effort to see how often NotImplemented is currently being > returned > > >> I crafted a test script [3] to test the types > > >> bytes, bytearray, str, dict, list, tuple, Enum, Counter, defaultdict, > > >> deque, and OrderedDict with the operations for > > >> __add__, __and__, __floordiv__, __iadd__, __iand__, __ifloordiv__, > > >> __ilshift__, __imod__, __imul__, __ior__, __ipow__, > > >> __irshift__, __isub__, __itruediv__, __ixor__, __lshift__, __mod__, > > >> __mul__, __or__, __pow__, __rshift__, __sub__, > > >> __truediv__, and __xor__. > > >> > > >> Here are the results of the 275 tests: > > >> ------------------------------------------------------------ > > >> -------------------- > > >> testing control... > > >> > > >> ipow -- Exception <unsupported operand type(s) for ** or pow(): > 'Control' > > >> and 'subtype'> raised > > >> errors in Control -- misunderstanding or bug? > > >> > > >> testing types against a foreign class > > >> > > >> iadd(Counter()) -- Exception <'SomeOtherClass' object has no attribute > > >> 'items'> raised instead of TypeError > > >> iand(Counter()) -- NotImplemented not returned, TypeError not raised > > >> ior(Counter()) -- Exception <'SomeOtherClass' object has no attribute > > >> 'items'> raised instead of TypeError > > >> isub(Counter()) -- Exception <'SomeOtherClass' object has no attribute > > >> 'items'> raised instead of TypeError > > >> > > >> > > >> testing types against a subclass > > >> > > >> mod(str()) -- NotImplemented not returned, TypeError not raised > > >> > > >> iadd(Counter()) -- Exception <'subtype' object has no attribute > 'items'> > > >> raised (should have worked) > > >> iand(Counter()) -- NotImplemented not returned, TypeError not raised > > >> ior(Counter()) -- Exception <'subtype' object has no attribute > 'items'> > > >> raised (should have worked) > > >> isub(Counter()) -- Exception <'subtype' object has no attribute > 'items'> > > >> raised (should have worked) > > >> ------------------------------------------------------------ > > >> -------------------- > > >> > > >> Two observations: > > >> > > >> - __ipow__ doesn't seem to behave properly in the 3.x line (that > error > > >> doesn't show up when testing against 2.7) > > >> > > >> - Counter should be returning NotImplemented instead of raising an > > >> AttributeError, for three reasons [4]: > > >> - a TypeError is more appropriate > > >> - subclasses /cannot/ work with the current implementation > > >> - __iand__ is currently a silent failure if the Counter is empty, > > >> and the other operand should trigger a failure > > >> > > >> Back to the main point... > > >> > > >> So, if my understanding is correct: > > >> > > >> - NotImplemented is used to signal Python that the requested > operation > > >> could not be performed > > >> - it should be used by the binary special methods to signal type > > >> mismatch failure, so any subclass gets a chance to work. > > >> > > >> Is my understanding correct? Is this already in the docs somewhere, > and > > >> I just missed it? > > >> > > >> -- > > >> ~Ethan~ > > >> > > >> [1] at least, it's the main reason in my code > > >> [2] usually a TypeError, stating either that the operation is not > > >> supported, or the types are unorderable > > >> [3] test script at the end > > >> [4] https://bugs.python.org/issue22766 [returning NotImplemented was > > >> rejected] > > >> > > >> -- 8< ------------------------------------------------------------ > > >> ---------------- > > >> from collections import Counter, defaultdict, deque, OrderedDict > > >> from fractions import Fraction > > >> from decimal import Decimal > > >> from enum import Enum > > >> import operator > > >> import sys > > >> > > >> py_ver = sys.version_info[:2] > > >> > > >> types = ( > > >> bytes, bytearray, str, dict, list, tuple, > > >> Enum, Counter, defaultdict, deque, OrderedDict, > > >> ) > > >> numeric_types = int, float, Decimal, Fraction > > >> > > >> operators = ( > > >> '__add__', '__and__', '__floordiv__', > > >> '__iadd__', '__iand__', '__ifloordiv__', '__ilshift__', > > >> '__imod__', '__imul__', '__ior__', '__ipow__', > > >> '__irshift__', '__isub__', '__itruediv__', '__ixor__', > > >> '__lshift__', '__mod__', '__mul__', > > >> '__or__', '__pow__', '__rshift__', '__sub__', '__truediv__', > > >> '__xor__', > > >> ) > > >> > > >> if py_ver >= (3, 0): > > >> operators += ('__gt__', '__ge__', '__le__','__lt__') > > >> > > >> ordered_reflections = { > > >> '__le__': '__ge__', > > >> '__lt__': '__gt__', > > >> '__ge__': '__le__', > > >> '__gt__': '__lt__', > > >> } > > >> > > >> > > >> # helpers > > >> > > >> class SomeOtherClass: > > >> """" > > >> used to test behavior when a different type is passed in to the > > >> special methods > > >> """ > > >> def __repr__(self): > > >> return 'SomeOtherClass' > > >> some_other_class = SomeOtherClass() > > >> > > >> class MainClassHandled(Exception): > > >> """ > > >> called by base class if both operands are of type base class > > >> """ > > >> > > >> class SubClassCalled(Exception): > > >> """ > > >> called by reflected operations for testing > > >> """ > > >> > > >> def create_control(test_op): > > >> def _any(self, other): > > >> if not type(other) is self.__class__: > > >> return NotImplemented > > >> raise MainClassHandled > > >> class Control: > > >> "returns NotImplemented when other object is not supported" > > >> _any.__name__ = op > > >> setattr(Control, test_op, _any) > > >> return Control() > > >> > > >> def create_subtype(test_op, base_class=object): > > >> def _any(*a): > > >> global subclass_called > > >> subclass_called = True > > >> raise SubClassCalled > > >> class subtype(base_class): > > >> __add__ = __sub__ = __mul__ = __truediv__ = __floordiv__ = > _any > > >> __mod__ = __divmod__ = __pow__ = __lshift__ = __rshift__ = > _any > > >> __and__ = __xor__ = __or__ = _any > > >> __radd__ = __rsub__ = __rmul__ = __rtruediv__ = > __rfloordiv__ = > > >> _any > > >> __rmod__ = __rdivmod__ = __rpow__ = __rlshift__ = > __rrshift__ = > > >> _any > > >> __rand__ = __rxor__ = __ror__ = _any > > >> __le__ = __lt__ = __gt__ = __ge__ = _any > > >> if issubclass(subtype, (bytes, bytearray)): > > >> value = b'hello' > > >> elif issubclass(subtype, str): > > >> value = 'goodbye' > > >> elif issubclass(subtype, (list, tuple)): > > >> value = (1, 2, 3) > > >> elif issubclass(subtype, (int, float, Decimal, Fraction)): > > >> value = 42 > > >> else: > > >> # ignore value > > >> return subtype() > > >> return subtype(value) > > >> > > >> > > >> # test exceptions > > >> > > >> # control against some other class > > >> print('testing control...\n') > > >> errors = False > > >> for op in operators: > > >> control = create_control(op) > > >> op = getattr(operator, op) > > >> try: > > >> op(control, some_other_class) > > >> except TypeError: > > >> # the end result of no method existing, or each method called > > >> returning > > >> # NotImplemented because it does not know how to perform the > > >> requested > > >> # operation between the two types > > >> pass > > >> except Exception as exc: > > >> errors = True > > >> print('%s(%s()) -- Exception <%s> raised instead of > TypeError' % > > >> (op.__name__, test_type.__name__, exc)) > > >> else: > > >> errors = True > > >> print('Control -- TypeError not raised for op %r' % op) > > >> if errors: > > >> print('errors in Control -- misunderstanding or bug?\n') > > >> > > >> # control against a subclass > > >> errors = False > > >> for op in operators: > > >> subclass_called = False > > >> control = create_control(op) > > >> subtype = create_subtype(op, control.__class__) > > >> op = getattr(operator, op) > > >> try: > > >> op(control, subtype) > > >> except SubClassCalled: > > >> # if the control class properly signals that it doesn't know > how > > >> to > > >> # perform the operation, of if Python notices that a > reflected > > >> # operation exists, we get here (which is good) > > >> pass > > >> except MainClassHandled: > > >> errors = True > > >> print('Control did not yield to subclass for op %r' % op) > > >> except Exception as exc: > > >> if subclass_called: > > >> # exception was subverted to something more appropriate > (like > > >> # unorderable types) > > >> pass > > >> errors = True > > >> print('%s -- Exception <%s> raised' % > > >> (op.__name__, exc)) > > >> else: > > >> errors = True > > >> print('Control -- op %r appears to have succeeded (it should > not > > >> have)' % op) > > >> if errors: > > >> print('errors in Control -- misunderstanding or bug?\n') > > >> > > >> > > >> # tests > > >> print('testing types against a foreign class\n') > > >> for test_type in types + numeric_types: > > >> errors = False > > >> for op in operators: > > >> op = getattr(operator, op) > > >> try: > > >> op(test_type(), some_other_class) > > >> except TypeError: > > >> pass > > >> except Exception as exc: > > >> errors = True > > >> print('%s(%s()) -- Exception <%s> raised instead of > > >> TypeError' % > > >> (op.__name__, test_type.__name__, exc)) > > >> else: > > >> print('%s(%s()) -- NotImplemented not returned, TypeError > > >> not raised' % > > >> (op.__name__, test_type.__name__)) > > >> if errors: > > >> print() > > >> > > >> print() > > >> > > >> # test subclasses > > >> print('testing types against a subclass\n') > > >> for test_type in types: > > >> errors = False > > >> for op in operators: > > >> subclass_called = False > > >> if not test_type.__dict__.get(op): > > >> continue > > >> subclass = create_subtype(op, test_type) > > >> op = getattr(operator, op) > > >> try: > > >> if test_type is str: > > >> op('%s', subtype) > > >> else: > > >> op(test_type(), subtype) > > >> except SubClassCalled: > > >> # expected, ignore > > >> pass > > >> except Exception as exc: > > >> if subclass_called: > > >> # exception raised by subclass was changed > > >> pass > > >> errors = True > > >> print('%s(%s()) -- Exception <%s> raised (should have > > >> worked)' % > > >> (op.__name__, test_type.__name__, exc)) > > >> else: > > >> errors = True > > >> print('%s(%s()) -- NotImplemented not returned, TypeError > > >> not raised' % > > >> (op.__name__, test_type.__name__)) > > >> if errors: > > >> print() > > >> for test_type in numeric_types: > > >> errors = False > > >> for op in operators: > > >> subclass_called = False > > >> if not test_type.__dict__.get(op): > > >> continue > > >> subtype = create_subtype(op, test_type) > > >> op = getattr(operator, op) > > >> try: > > >> op(test_type(), subtype) > > >> except SubClassCalled: > > >> # expected, ignore > > >> pass > > >> except Exception as exc: > > >> if subclass_called: > > >> # exception raised by subclass was changed > > >> pass > > >> errors = True > > >> print('%s(%s()) -- Exception <%s> raised (should have > > >> worked)' % > > >> (op.__name__, test_type.__name__, exc)) > > >> else: > > >> errors = True > > >> print('%s(%s)) -- NotImplemented not returned' % > > >> (op.__name__, test_type.__name__)) > > >> if errors: > > >> print() > > >> -- 8< ------------------------------------------------------------ > > >> ---------------- > > >> _______________________________________________ > > >> Python-Dev mailing list > > >> Python-Dev@python.org > > >> https://mail.python.org/mailman/listinfo/python-dev > > >> Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > > >> brett%40python.org > > >> > > > > > > _______________________________________________ > > > Python-Dev mailing list > > > Python-Dev@python.org > > > https://mail.python.org/mailman/listinfo/python-dev > > > Unsubscribe: > > > https://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > > > > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
