[issue26680] Incorporating float.is_integer into Decimal
Robert Smallshire added the comment: Yes Raymond, I can prepare a new PR as soon as the faulty PR has been rolled back. -- ___ Python tracker <https://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire added the comment: First, I would like to apologise for the confusion I have inadvertently caused. I didn't see Raymond's question and Guido's clear response here about not modifying the numeric tower, as it came _long_ after activity had moved to GitHub, and code had been reviewed over there. Now Raymond has linked back to here and I've caught up with the situation, I offer to make a compensating PR with reverts the changes to the numeric tower. I should be able to do this in the next day or two. Please let me know whether you would like me to proceed, or whether you intend to handle the a partial or complete rollback among yourselves. -- ___ Python tracker <https://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire added the comment: > * no forward movement for a year and half For most of that year and a half my pull-request was sitting in Github <https://github.com/python/cpython/pull/6121> awaiting review by two core devs, of which you Raymond, were one. I don't like to hassle people who contribute their free time to Python and assumed you or the other reviewer would get around to it good time. Mark Dickinson kindly reviewed the my changes shortly before the 3.8 window closed, but I didn't have capacity to follow up before it was too late. The changes Mark wanted were largely to do with in-code documentation rather than the the substance of the change. Assuming it really is too late for 3.8, even if I made the changes Mark wanted immediately, I would like to see this change in 3.9. > * opposition from multiple core devs Guido approved the change in a pronouncement on python-dev. <https://mail.python.org/pipermail/python-dev/2018-March/152437.html> -- status: closed -> open ___ Python tracker <https://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire <r...@sixty-north.com> added the comment: Serhiy, you asked for use cases, not uses. The former can exist without the latter. Use cases for is_integer() include all existing uses of x == int(x), or other less obvious means of detecting integer values. Folks try to use x.is_integer(), discover it fails if x is an int, and go in search of a replacement, which may well be x == int(x), which goes on to fail in more complex and awkward to handle ways with NaN or Inf. I've seen is_integer() used in guard clauses for functions which accept numbers which require an integral value (Recall that explicit type checking is usually called out as unPythonic, and rightly so). The Python factorial is example of such a function which makes an equivalent check. StackOverflow is awash with folks using isinstance(x, int) where that is not what they actually mean. Many of these are the same use-case, but don't use is_integer(). I've also seen it used in cases where different algorithms are more optimal depending on whether the arguments are integral or not (imagine some combination of factorial() and gamma()). I've seen it used in conversion from float to custom number types used to simulate the unusual number types in embedded hardware. I'm not going to engage with this discussion further. It's already consumed too much of my time, left me with a somewhat negative view of how Python development proceeds, and a confused view of the values that are most important to the Python language. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire <r...@sixty-north.com> added the comment: Python in effect contains an example itself. The math.factorial(x) function contains what is in-effect an is_integer() check as a guard. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Change by Robert Smallshire <r...@sixty-north.com>: -- pull_requests: +5885 stage: needs patch -> patch review ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire <r...@sixty-north.com> added the comment: Thank you Raymond. I'll work up a PR shortly. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire <r...@sixty-north.com> added the comment: Apologies for the email splurge. That's the first and last time I'll use the email interface to bugs.python.org. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire <r...@sixty-north.com> added the comment: To respond to Raymond's points: 1) Nobody is suggesting that every float method should also be available on int. Clearly some methods on float are not applicable to int. 2) Taken narrowly, you're right that is_integer() does nothing useful for int. Neither do imag, denominator, __floor__, or __trunc__. They're all present so we can successfully use integers in duck-typing situations along with the other number types. My claim is that int.is_integer() is useful for the same reasons. The problem isn't that you or I don't know that we should write a == int(a) to be portable, the problem is that code that *already* uses x.is_integer() fails hard when x happens to be an int. As I've demonstrated, some built-in operators and functions can return either int or float, even when the operand types are all int, depending only on the *values* of the operands. This wouldn't matter if nobody ever wrote f.is_integer(), and instead used the trivially portable solution, but they do, and at your behest: In 2011 you published "f.is_integer() is the new way to test whether a float is integral. The old way, x==int(x), is history." The reality is that folks often write Python functions which accept *numbers*, without giving too much thought to whether calling my_func(42.0) will work, but my_func(42) will cause an unhandled exception that nobody is expecting. Indeed, one of the joys of Python is that we often don't need to overthink this. The trivial portable solution is also nearly three times slower than float.is_integer() and int.is_integer(). Moreso if you package it up in a function so it can be applied to more complex expressions in, say, a comprehension, where an intermediate assignment is not possible. I'm not the only person to be thrown by this. See this: is_integer() not working: https://stackoverflow.com/ questions/17170226/is-integer-not-working/17170511 and this: https://www.reddit.com/r/learnpython/comments/4tp4hy/ need_help_with_identify_number_as_integer/ and this https://wiesmann.codiferes.net/wordpress/?p=13366 Furthermore, once the is_integer() method is discovered, it leads to folks writing odd code in order to leverage it, such as always converting user integer input to float in order to check that it's really is an integer (which of course subtly limits the precision of allowable integers). There's an example of this on page 14 of the book *Doing Math With Python*. https://www.amazon.com/Doing-Math-Python-Programming-Statistics/dp/1593276400 Other prolific and widely respected Python programmers have described this behaviour as "kind of nuts" (though I'm not going to involve them here). The behaviour has also invited unfortunate comparisons with similar non-obvious behaviour in Javascript. 3) I'd be very surprised if the presence of this method on int caused any more confusion, or impediment to learning than do the presence of int.imag or int.denominator. 4) I'm less bothered about the numeric tower aspect than I am about duck-type compatibility between the built-in types int and float. That said, a key part of what I think is your concern about creating work for subclass implementers can be addressed by providing a default implementation Real.is_integer() in terms of int(x) == x. 5) The decimal spec doesn't require an is_decimal function, but it doesn't forbid it either. In any case, the Decimal implementation already implements is_integer internally as `cpx_mpd_isinteger` – and uses it a great deal, which demonstrates its utility. My patch simply exposes it to Python. There's no danger of violating any specification, unless that specification says that you must not implement a method called is_integer, which it doesn't, especially as we would be using a definition which is already de facto compatible with the standard. I don't care very much about Decimal either for my own work, especially as it already stands apart from the numeric tower. I implemented it to be consistent with my argument about duck typed numbers (which still largely holds for Decimal, except for floor division and modulus I believe). Solutions for which use `a == int(a)` or `a == a.to_integral_value()` fail for NaN and infinities, whereas float.is_integer() is more robust. It turns out the trivial portable solution isn't so trivial, or so portable, if implemented robustly, performantly and with duck-typing in mind. Ultimately, my argument is one about duck typing across numbers types. If if that abstraction isn't valued, I have nowhere to go. *Robert Smallshire | *Managing Director *Sixty North* | Applications | Consulting | Training r...@sixty-north.com | T +47 63 01 04 44 | M +47 924 30 350 http://sixty-north.com On 11 March 2018 at 05:58, Raymond Hettinger <rep...@bugs.python.org> wrote: > > Raymond Hettinger <raymond.hettin...@gmail.com> added the comment: > > Sorry Robert
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire <r...@sixty-north.com> added the comment: I've recently run into this issue impeding duck-typing between int and float again, when used in conjunction the int.__pow__, which may variously return an int or float depending on the value - not the type - of the arguments. This is succinctly demonstrated by this example: >>> (10 ** -2).is_integer() False >>> (10 ** 2).is_integer() Traceback (most recent call last): File "", line 1, in AttributeError: 'int' object has no attribute 'is_integer' I hear the argument about Python being harder to learn if more methods are supported on the built-in types - and perhaps float.is_integer should never have been added, but now its there, but I think Python is harder to learn and teach in the presence of these differences. Is is harder to learn "Real numbers support an is_integer() method", than it is "float supports an is_integer() method"? I'm happy to put in the work bring my original patches up-to-date, or create a PR depending on what current process is. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26701] Documentation for int constructor mentions __int__ but not __trunc__
New submission from Robert Smallshire: The documentation for the int(x) constructor explains that if possible, it delegates to x.__int__(). The documentation does not explain that there is a fallback to x.__trunc__() if x.__int__() is not available. The only mention of __trunc__ in the Python documentation is in the entry for math.trunc; the documentation for the numbers module does not describe the underlying special methods. Given that all Real numbers are required to implement __trunc__ but only Integral subclasses are required to implement __int__ this could be important to implementers of other Real types, although in practice I imagine that most Real types will implement __int__ as float does. -- assignee: docs@python components: Documentation messages: 262941 nosy: Robert Smallshire2, docs@python priority: normal severity: normal status: open title: Documentation for int constructor mentions __int__ but not __trunc__ type: enhancement versions: Python 3.5 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26701> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire added the comment: Thanks Stefan for the illuminating example. I knew I shouldn't have strayed off-topic. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire added the comment: >> Were float to inherit from Rational, rather than Real ... > This would break the Liskov substitution principle. How so? Rational extends Real with only numerator, denominator and __float__. Isn't the existence of float.as_integer_ratio demonstration that numerator and denominator could be implemented? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire added the comment: Java makes no claim to have a numeric tower. Amongst the dynamic languages I surveyed Matlab (isinteger), Javascript ES6 (isInteger), PHP (is_integer), R (is.integer), TCL (is entier), and as we have seen Scheme (integer?) all have methods for testing for integer values. Python has a numeric tower modelled on Scheme. In the Scheme documentation we find this: "...the integer 5 may have several representations. Scheme's numerical operations treat number objects as abstract data, as independent of their representation as possible. Although an implementation of Scheme may use many different representations for numbers, this should not be apparent to a casual programmer writing simple programs." This is what I'm advocating. There isn't a single mathematical (as opposed to representational) method on int that isn't 'inherited' from the numeric tower. There are exactly two methods on float which aren't inherited from the tower: is_integer and as_integer_ratio. So I think it's would be a stretch to claim that "Most of the [numerical] ABCs have only a subset of the [numerical] methods in the [numerical] concrete types." Rather than looking at the numeric tower as a construct which forces proliferation of methods, it would be better to look on it as a device to prevent bloat. I risk straying off topic here, but I want to give an example of why the numeric tower is important: Were float to inherit from Rational, rather than Real (all finite floats are rationals with a power-of-two denominator, all Decimals are rationals with a power-of-ten denominator, so this is reasonable) then the as_integer_ratio method which was added to float and latterly Decimal (http://bugs.python.org/issue25928), arguably cluttering their interfaces, may have been deemed unnecessary. The numerator and denominator attributes present in Rational could have been used instead. I think this is an example of lack of adherence to the numeric tower (at least in spirit in the case of Decimal) resulting in interface clutter or bloat. The consequent control-flow complexity required handle numeric objects as 'abstract data' is surprising: statistics._exact_ratio is a good example of this. I count five API tests just to be able to treat numbers as, well, just numbers. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire added the comment: To be clear, I'm not arguing that is_integer is in the same category as hex and fromhex; is_integer is a mathematical property of the number, whereas hex and from hex are representational. Nobody expects interoperability of string representations of the different number types. Neither do I take issue with the general argument against enlarging the API. In fact, if float.is_integer did not exist, I would not be campaigning for it to be invented. What I do take issue with is already having a method on float which makes sense for all Real numbers, but then not supporting it for those other Real types. This just gets in the way of programming generally in terms of *numbers* rather than concrete types, especially when the is_integer method is being advocated as the *right way* to do such a test. This is can lead to other problems as people unthinkingly convert other number types to floats, with loss of information, just to get access to this convenience method. I'm (really) surprised that structural typing and polymorphism over numbers carries so little weight – a cursory look at StackOverflow* shows that there are already too many people being encouraged to answer the 'is integral' question with isinstance(x, int) when that is not what they mean or need. As a precedent we already have int.numerator, int.denominator, int.real and int.imag which are presumably using Raymond's argument are also 'total waste', but the reality is that the presence of these attributes causes no impediment to learning and makes many tasks easier with fewer special cases. When working with rationals, I frequently rely on the fact that ints implement the Rational interface. When working with complex numbers, I frequently rely on the fact that both int and float implement the Complex interface. For example. I have used use these attributes in the constructor for a fixed-point number type, and was thankful for their existence. I'm also happy to set the Decimal aspect of my proposal to one side as Decimal is explicitly outside the numeric tower. This isn't about me avoiding writing trivial one-line functions or "the OPs use case". I'm trying to help you make Python more predictable and easier to use. I'm far from the first person to be surprised by this. I'd be happy to trade adding is_integer to the numeric tower for a deprecation notice on float.is_integer - the outcome is the same - polymorphic number types with fewer special cases. <http://stackoverflow.com/questions/17170226/is-integer-not-working> <http://stackoverflow.com/questions/6239967/determining-whether-an-value-is-a-whole-number-in-python/6239987#6239987> <http://stackoverflow.com/questions/33002747/try-except-float-but-not-integer/33002796#33002796> <http://stackoverflow.com/a/22053804/107907> <http://stackoverflow.com/questions/36209324/trouble-using-is-integer-in-python> -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire added the comment: As for whether the shortcut float.is_integer(x) was needed, it has different behavior to x==int(x) when x is either NaN or an infinity. We must even deal with two different exception types OverflowError or ValueError respectively for these two values on conversion to int. That float.is_integer() simply returns False for these values makes it more straightforward to use robustly. The same would go for Decimal, which has the same behavior with respect to NaNs and infinities as float. I agree that is_integral may have been a better name, although is_integer has the advantage that it avoids conflating numeric values with either of the types 'int' or 'Integral'. The motivation for my patches is to converge the interfaces of the various number types so that we can simply, and robustly, check for integer values (as opposed to integer types) without needing to be concerned about the concrete number type, so long as it is Real. Indeed, this is largely the point of having a numeric tower at all. I am more motivated by usability and concision and correctness than efficiency concerns: I believe that where possible we should allow one number type to be substituted for another, and in particular `int` for any other Real type where purely mathematical - rather than representational operations - are in play. Use of the existing float.is_integer is compromised by the fact that people have an entirely reasonably habit of passing integers (particularly literals) to functions which accept floats which then fail if they use float.is_integer. Adding this method would reduce the educational load as the various number types would be more similar, not less. I work in industrial fields where computational geometry, and hence rationals, floats, infinities and large integers are a day-to-day occurrence. Ultimately, I care more about consistency within the numeric tower types (Real, float, int, Rational, Integral, Fraction) than I do about Decimal, which is why I separated my changes to Decimal into a separate patch. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
Robert Smallshire added the comment: Adding the second patch file. -- Added file: http://bugs.python.org/file42336/is_integer_decimal.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26680] Incorporating float.is_integer into the numeric tower and Decimal
New submission from Robert Smallshire: When the useful float.is_integer method was added the opportunity was missed to incorporate this method into the numeric tower defined in numbers.py. This increased the API distance between different number types, making them less substitutable than previously, leading to what might be considered to be absurd behaviour: >>> a = 5.0 >>> b = 5 >>> a.is_integer() True >>> b.is_integer() Traceback (most recent call last): File "", line 1, in AttributeError: 'int' object has no attribute 'is_integer' The first attached patch modifies Python to: 1) Implement int.is_integer() to always return True 2) Add Real.is_integer() as an abstract method in numbers.py 3) Provide a default implementation in Rational.is_integer() in numbers.py 4) Adds tests for is_integer() for int and Fraction. 5) Documentation changes commensurate with above. Although the Decimal type deliberately lies outside the numeric tower for reasons not relevant here, the principle of least surprise suggests that it too should support is_integer(). In fact, the implementation already contains just such a function, although it is not exposed to Python. The second patch implements is_integer() for both the pure Python and C implementations of Decimal, again with commensurate tests and documentation changes. I hope these changes can be implemented to reduce the degree of surprise encountered when working with different number types in Python. -- components: Library (Lib) files: is_integer_numeric_tower.patch keywords: patch messages: 262702 nosy: Robert Smallshire2 priority: normal severity: normal status: open title: Incorporating float.is_integer into the numeric tower and Decimal type: enhancement Added file: http://bugs.python.org/file42335/is_integer_numeric_tower.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26663] asyncio _UnixWritePipeTransport._close abandons unflushed writes
Changes by Robert Smallshire <robert.smallsh...@gmail.com>: -- type: -> behavior ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26663> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26663] asyncio _UnixWritePipeTransport._close abandons unflushed writes
New submission from Robert Smallshire: Note: This report is based on a close reading of the asyncio code, rather than having experienced defective behaviour in test or production. The documentation for BaseTransport.close() states: "Close the transport. If the transport has a buffer for outgoing data, buffered data will be flushed asynchronously." The _UnixWritePipeTransport._close method, which is called by close() contains this code in Python 3.5.1 def _close(self, exc=None): self._closing = True if self._buffer: self._loop.remove_writer(self._fileno) self._buffer.clear() self._loop.remove_reader(self._fileno) self._loop.call_soon(self._call_connection_lost, exc) In this context, _buffer is a list of bytes objects comprising yet-to-be-written data. Note that close() removes the writer if _buffer is *not* empty, so the buffered data will never be written. I believe this conditional should be inverted, so the writer is removed only if the buffer is *empty*. So: if not self._buffer: self._loop.remove_writer(self._fileno) Arguably though, a more robust and easier to read test would be to call get_write_buffer_size(), like this: if self.get_write_buffer_size() == 0: self._loop.remove_writer(self._fileno) Note that in the event that the writer is not removed by _close(), the writer will remove itself when the buffer does eventually become empty. Assuming my analysis is correct, and a fix is made to close(), then abort() will also need to be modified to *not* flush buffered writes, as the documentation for abort() states "Close the transport immediately, without waiting for pending operations to complete. Buffered data will be lost." -- components: asyncio messages: 262601 nosy: Robert Smallshire, gvanrossum, haypo, yselivanov priority: normal severity: normal status: open title: asyncio _UnixWritePipeTransport._close abandons unflushed writes versions: Python 3.5 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26663> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
