Gotta be brief, but NotImplemented is for all binary ops. Power may be an exception because it's ternary? On Nov 3, 2014 8:08 AM, "Brett Cannon" <br...@python.org> wrote:
> > > On Mon Nov 03 2014 at 5:31:21 AM Ethan Furman <et...@stoneleaf.us> wrote: > >> Just to be clear, this is about NotImplemented, not NotImplementedError. >> >> tl;dr When a binary operation fails, should an exception be raised or >> NotImplemented returned? >> > > The docs for NotImplemented suggest it's only for rich comparison methods > and not all binary operators: > https://docs.python.org/3/library/constants.html#NotImplemented . But > then had I not read that I would have said all binary operator methods > should return NotImplemented when the types are incompatible. > > -Brett > > >> >> >> When a binary operation in Python is attempted, there are two >> possibilities: >> >> - it can work >> - it can't work >> >> The main reason [1] that it can't work is that the two operands are of >> different types, and the first type does not know >> how to deal with the second type. >> >> The question then becomes: how does the first type tell Python that it >> cannot perform the requested operation? The most >> obvious answer is to raise an exception, and TypeError is a good >> candidate. The problem with the exception raising >> approach is that once an exception is raised, Python doesn't try anything >> else to make the operation work. >> >> What's wrong with that? Well, the second type might know how to perform >> the operation, and in fact that is why we have >> the reflected special methods, such as __radd__ and __rmod__ -- but if >> the first type raises an exception the __rxxx__ >> methods will not be tried. >> >> Okay, how can the first type tell Python that it cannot do what is >> requested, but to go ahead and check with the second >> type to see if it does? That is where NotImplemented comes in -- if a >> special method (and only a special method) >> returns NotImplemented then Python will check to see if there is anything >> else it can do to make the operation succeed; >> if all attempts return NotImplemented, then Python itself will raise an >> appropriate exception [2]. >> >> In an effort to see how often NotImplemented is currently being returned >> I crafted a test script [3] to test the types >> bytes, bytearray, str, dict, list, tuple, Enum, Counter, defaultdict, >> deque, and OrderedDict with the operations for >> __add__, __and__, __floordiv__, __iadd__, __iand__, __ifloordiv__, >> __ilshift__, __imod__, __imul__, __ior__, __ipow__, >> __irshift__, __isub__, __itruediv__, __ixor__, __lshift__, __mod__, >> __mul__, __or__, __pow__, __rshift__, __sub__, >> __truediv__, and __xor__. >> >> Here are the results of the 275 tests: >> ------------------------------------------------------------ >> -------------------- >> testing control... >> >> ipow -- Exception <unsupported operand type(s) for ** or pow(): 'Control' >> and 'subtype'> raised >> errors in Control -- misunderstanding or bug? >> >> testing types against a foreign class >> >> iadd(Counter()) -- Exception <'SomeOtherClass' object has no attribute >> 'items'> raised instead of TypeError >> iand(Counter()) -- NotImplemented not returned, TypeError not raised >> ior(Counter()) -- Exception <'SomeOtherClass' object has no attribute >> 'items'> raised instead of TypeError >> isub(Counter()) -- Exception <'SomeOtherClass' object has no attribute >> 'items'> raised instead of TypeError >> >> >> testing types against a subclass >> >> mod(str()) -- NotImplemented not returned, TypeError not raised >> >> iadd(Counter()) -- Exception <'subtype' object has no attribute 'items'> >> raised (should have worked) >> iand(Counter()) -- NotImplemented not returned, TypeError not raised >> ior(Counter()) -- Exception <'subtype' object has no attribute 'items'> >> raised (should have worked) >> isub(Counter()) -- Exception <'subtype' object has no attribute 'items'> >> raised (should have worked) >> ------------------------------------------------------------ >> -------------------- >> >> Two observations: >> >> - __ipow__ doesn't seem to behave properly in the 3.x line (that error >> doesn't show up when testing against 2.7) >> >> - Counter should be returning NotImplemented instead of raising an >> AttributeError, for three reasons [4]: >> - a TypeError is more appropriate >> - subclasses /cannot/ work with the current implementation >> - __iand__ is currently a silent failure if the Counter is empty, >> and the other operand should trigger a failure >> >> Back to the main point... >> >> So, if my understanding is correct: >> >> - NotImplemented is used to signal Python that the requested operation >> could not be performed >> - it should be used by the binary special methods to signal type >> mismatch failure, so any subclass gets a chance to work. >> >> Is my understanding correct? Is this already in the docs somewhere, and >> I just missed it? >> >> -- >> ~Ethan~ >> >> [1] at least, it's the main reason in my code >> [2] usually a TypeError, stating either that the operation is not >> supported, or the types are unorderable >> [3] test script at the end >> [4] https://bugs.python.org/issue22766 [returning NotImplemented was >> rejected] >> >> -- 8< ------------------------------------------------------------ >> ---------------- >> from collections import Counter, defaultdict, deque, OrderedDict >> from fractions import Fraction >> from decimal import Decimal >> from enum import Enum >> import operator >> import sys >> >> py_ver = sys.version_info[:2] >> >> types = ( >> bytes, bytearray, str, dict, list, tuple, >> Enum, Counter, defaultdict, deque, OrderedDict, >> ) >> numeric_types = int, float, Decimal, Fraction >> >> operators = ( >> '__add__', '__and__', '__floordiv__', >> '__iadd__', '__iand__', '__ifloordiv__', '__ilshift__', >> '__imod__', '__imul__', '__ior__', '__ipow__', >> '__irshift__', '__isub__', '__itruediv__', '__ixor__', >> '__lshift__', '__mod__', '__mul__', >> '__or__', '__pow__', '__rshift__', '__sub__', '__truediv__', >> '__xor__', >> ) >> >> if py_ver >= (3, 0): >> operators += ('__gt__', '__ge__', '__le__','__lt__') >> >> ordered_reflections = { >> '__le__': '__ge__', >> '__lt__': '__gt__', >> '__ge__': '__le__', >> '__gt__': '__lt__', >> } >> >> >> # helpers >> >> class SomeOtherClass: >> """" >> used to test behavior when a different type is passed in to the >> special methods >> """ >> def __repr__(self): >> return 'SomeOtherClass' >> some_other_class = SomeOtherClass() >> >> class MainClassHandled(Exception): >> """ >> called by base class if both operands are of type base class >> """ >> >> class SubClassCalled(Exception): >> """ >> called by reflected operations for testing >> """ >> >> def create_control(test_op): >> def _any(self, other): >> if not type(other) is self.__class__: >> return NotImplemented >> raise MainClassHandled >> class Control: >> "returns NotImplemented when other object is not supported" >> _any.__name__ = op >> setattr(Control, test_op, _any) >> return Control() >> >> def create_subtype(test_op, base_class=object): >> def _any(*a): >> global subclass_called >> subclass_called = True >> raise SubClassCalled >> class subtype(base_class): >> __add__ = __sub__ = __mul__ = __truediv__ = __floordiv__ = _any >> __mod__ = __divmod__ = __pow__ = __lshift__ = __rshift__ = _any >> __and__ = __xor__ = __or__ = _any >> __radd__ = __rsub__ = __rmul__ = __rtruediv__ = __rfloordiv__ = >> _any >> __rmod__ = __rdivmod__ = __rpow__ = __rlshift__ = __rrshift__ = >> _any >> __rand__ = __rxor__ = __ror__ = _any >> __le__ = __lt__ = __gt__ = __ge__ = _any >> if issubclass(subtype, (bytes, bytearray)): >> value = b'hello' >> elif issubclass(subtype, str): >> value = 'goodbye' >> elif issubclass(subtype, (list, tuple)): >> value = (1, 2, 3) >> elif issubclass(subtype, (int, float, Decimal, Fraction)): >> value = 42 >> else: >> # ignore value >> return subtype() >> return subtype(value) >> >> >> # test exceptions >> >> # control against some other class >> print('testing control...\n') >> errors = False >> for op in operators: >> control = create_control(op) >> op = getattr(operator, op) >> try: >> op(control, some_other_class) >> except TypeError: >> # the end result of no method existing, or each method called >> returning >> # NotImplemented because it does not know how to perform the >> requested >> # operation between the two types >> pass >> except Exception as exc: >> errors = True >> print('%s(%s()) -- Exception <%s> raised instead of TypeError' % >> (op.__name__, test_type.__name__, exc)) >> else: >> errors = True >> print('Control -- TypeError not raised for op %r' % op) >> if errors: >> print('errors in Control -- misunderstanding or bug?\n') >> >> # control against a subclass >> errors = False >> for op in operators: >> subclass_called = False >> control = create_control(op) >> subtype = create_subtype(op, control.__class__) >> op = getattr(operator, op) >> try: >> op(control, subtype) >> except SubClassCalled: >> # if the control class properly signals that it doesn't know how >> to >> # perform the operation, of if Python notices that a reflected >> # operation exists, we get here (which is good) >> pass >> except MainClassHandled: >> errors = True >> print('Control did not yield to subclass for op %r' % op) >> except Exception as exc: >> if subclass_called: >> # exception was subverted to something more appropriate (like >> # unorderable types) >> pass >> errors = True >> print('%s -- Exception <%s> raised' % >> (op.__name__, exc)) >> else: >> errors = True >> print('Control -- op %r appears to have succeeded (it should not >> have)' % op) >> if errors: >> print('errors in Control -- misunderstanding or bug?\n') >> >> >> # tests >> print('testing types against a foreign class\n') >> for test_type in types + numeric_types: >> errors = False >> for op in operators: >> op = getattr(operator, op) >> try: >> op(test_type(), some_other_class) >> except TypeError: >> pass >> except Exception as exc: >> errors = True >> print('%s(%s()) -- Exception <%s> raised instead of >> TypeError' % >> (op.__name__, test_type.__name__, exc)) >> else: >> print('%s(%s()) -- NotImplemented not returned, TypeError >> not raised' % >> (op.__name__, test_type.__name__)) >> if errors: >> print() >> >> print() >> >> # test subclasses >> print('testing types against a subclass\n') >> for test_type in types: >> errors = False >> for op in operators: >> subclass_called = False >> if not test_type.__dict__.get(op): >> continue >> subclass = create_subtype(op, test_type) >> op = getattr(operator, op) >> try: >> if test_type is str: >> op('%s', subtype) >> else: >> op(test_type(), subtype) >> except SubClassCalled: >> # expected, ignore >> pass >> except Exception as exc: >> if subclass_called: >> # exception raised by subclass was changed >> pass >> errors = True >> print('%s(%s()) -- Exception <%s> raised (should have >> worked)' % >> (op.__name__, test_type.__name__, exc)) >> else: >> errors = True >> print('%s(%s()) -- NotImplemented not returned, TypeError >> not raised' % >> (op.__name__, test_type.__name__)) >> if errors: >> print() >> for test_type in numeric_types: >> errors = False >> for op in operators: >> subclass_called = False >> if not test_type.__dict__.get(op): >> continue >> subtype = create_subtype(op, test_type) >> op = getattr(operator, op) >> try: >> op(test_type(), subtype) >> except SubClassCalled: >> # expected, ignore >> pass >> except Exception as exc: >> if subclass_called: >> # exception raised by subclass was changed >> pass >> errors = True >> print('%s(%s()) -- Exception <%s> raised (should have >> worked)' % >> (op.__name__, test_type.__name__, exc)) >> else: >> errors = True >> print('%s(%s)) -- NotImplemented not returned' % >> (op.__name__, test_type.__name__)) >> if errors: >> print() >> -- 8< ------------------------------------------------------------ >> ---------------- >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: https://mail.python.org/mailman/options/python-dev/ >> brett%40python.org >> > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
