[issue46639] Ceil division with math.ceildiv
Vladimir Feinberg added the comment: Mark, I will say I'm pretty sympathetic to the feature-bloat avoidance perspective here, and if the outcome here is to improve docs, that's still a win, I think. That said, since this thread will become precedent, and I think `math.ceildiv` is the exactly-appropriate amount of commitment Lib should make to the function (not __ceildiv__ and not "just" a doc note), let me try to give `ceildiv` the strongest legs I can think of before we make a decision. 1. Not needing an import - I don't find importing such a standard library as `math` that onerous. We're not adding a new package here, just a function. This skepticism could be applied to any existing library function. Even `sys.stdout` needs an import. 2. Natural duck typing - I'll admit, this is pretty nice. But if that's the argument, I'd expect this to work to its fullest extent. Namely, I'd expect this to "naturally" work for any ring, and it doesn't. Z/nZ is a common one and np.uint is a more common one where the identity -(-x // y) = ceildiv(x, y) does not hold. The benefit of `math.ceildiv` is it'd either support it, or say it doesn't, but at least it's explicit. 3. Thin end of wedge - A priori, I would put ceildiv as special because of the "resource coverage" use case I described in my initial bug message. A posteriori, there's a clear "kink" in the graph of usage here: ceildiv (3033) <https://github.com/search?l==ceildiv+language%3APython=code>, rounddiv (25) <https://github.com/search?q=rounddiv+language%3APython=code>, roundmod (7) <https://github.com/search?q=roundmod+language%3APython=code>, ceilrem (0) <https://github.com/search?q=ceilrem+language%3APython=code>, ceildivrem (0) <https://github.com/search?q=ceildivrem+language%3APython=code>. But most importantly, let me detail what motivated me to post this. I was working on unit tests for linear algebra code which blocked its operations. But to not involve a lot of context, I'll provide a similarly-structured use case. Say we're making a controller for a game engine GUI and need to figure out how to paint sprites. ``` # sprite_A.py class A: def get_covering_rectangle(): return self.x, self.y, self.x - (-self.width // GRID_WIDTH), self.y - (-self.height // GRID_HEIGHT) ``` Especially if I also use `-(-x//y)` elsewhere, this is just asking too much of the reader. I could leave a comment to the tune of `# Note below is equivalent to + (-(-x//y)), the ceildiv operator, and this works because x isn't a uint`. Should I do this at all usage sites? I'd end up factoring into my own `ceildiv` for clarity, especially if I use this elsewhere, like a test. Where should this hand-rolled ceildiv live, if not recreated in everyone's code? It seems too light to wrap as its own dependency, and we probably don't want to go down the leftpad <https://www.theregister.com/2016/03/23/npm_left_pad_chaos/> path. `math` seems most apt. On Sat, Feb 5, 2022 at 8:31 AM Mark Dickinson wrote: > > Mark Dickinson added the comment: > > I'm not convinced that this deserves to be a math module function. I agree > that `-(-x // y)`, while simple to write, isn't necessarily obvious. But it > does have some advantages, like not needing an import, and being naturally > duck-typed, so that it automatically does the right thing for floats, or > `fractions.Fraction` objects, or `numpy.int64` objects, or SymPy integers. > (Not for `Decimal` instances, but that's another story.) Unless we were to > add a whole __ceildiv__ mechanism, a math module implementation would > necessarily be limited to integers. (Or perhaps integers and floats.) > > There's also the "thin end of the wedge" argument: if ceildiv, why not > also ceilrem, ceildivrem, rounddiv, roundmod, etc. > > The main issue with the `-(-x // y)` spelling seems to be discoverability: > if everyone knew that this was the right way to spell ceiling division, > then there wouldn't be a problem. And I'm not convinced that a math.ceildiv > function would necessarily solve the discoverability problem, either. > > So maybe the solution is to advertise the `-(-x // y)` pattern better in > documentation, for example at the point where floor division is introduced > in the library reference? > > -- > > ___ > Python tracker > <https://bugs.python.org/issue46639> > ___ > -- ___ Python tracker <https://bugs.python.org/issue46639> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46639] Ceil division with math.ceildiv
New submission from Vladimir Feinberg : I have a request related to the rejected proposal (https://bugs.python.org/issue43255) to introduce a ceildiv operator. I frequently find myself wishing for a ceildiv function which computes `ceil(x/y)` for integers `x,y`. This comes up all the time when "batching" some resource and finding total consumption, be it for memory allocation or GUI manipulation or time bucketing or whatnot. It is easy enough to implement this inline, but `math.ceildiv` would express intent clearly. ``` # x, y, out: int # (A) import math out = math.ceil(x / y) # clear intent but subtly changes type, and also incorrect for big ints # (B) out = int(math.ceil(x / y)) # wordy, especially if using this multiple times, still technically wrong # (C) out = (x + y - 1) // y # too clever if you haven't seen it before, does it have desirable semantics for negatives? # (D) out = -(-x//y) def ceildiv(a: int, b: int) -> int: # Clearest and correct, but should my library code really invent this wheel? """Returns ceil(a/b).""" return -(-x//y) out = ceildiv(x, y) ``` Even though these are all "one-liners", as you can see leaving people to complex manually-implemented `ceildiv`s might result in bugs or unclear handling of negatives. ------ components: Library (Lib) messages: 412527 nosy: Vladimir Feinberg priority: normal severity: normal status: open title: Ceil division with math.ceildiv type: enhancement versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue46639> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43048] Printing RecursionError results in RecursionError
Change by Vladimir Feinberg : -- keywords: +patch pull_requests: +23256 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/24460 ___ Python tracker <https://bugs.python.org/issue43048> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43048] Printing RecursionError results in RecursionError
Vladimir Feinberg added the comment: Yep, you're right. I'd be happy to (but I've never done it before, so please give me some time). On Tue, Feb 2, 2021 at 12:35 AM Irit Katriel wrote: > > Irit Katriel added the comment: > > It should truncate at the call that raised the recursion error, not the > first one. Do you want to create a patch? > > -- > > ___ > Python tracker > <https://bugs.python.org/issue43048> > ___ > -- ___ Python tracker <https://bugs.python.org/issue43048> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43048] Printing RecursionError results in RecursionError
Vladimir Feinberg added the comment: Oh, yes, I suppose, that'll truncate to just the first TracebackException. On Mon, Feb 1, 2021 at 4:38 PM Irit Katriel wrote: > > Irit Katriel added the comment: > > I meant to catch the exception in the constructor’s recursive call, and if > necessary then the same again in format (if there are more function calls > per exception in format, it will be necessary. The unit test from the 3.10 > patch will tell). > > Would that not work? > > -- > > ___ > Python tracker > <https://bugs.python.org/issue43048> > ___ > -- ___ Python tracker <https://bugs.python.org/issue43048> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43048] Printing RecursionError results in RecursionError
Vladimir Feinberg added the comment: A simple catch may not work (the very first TracebackException is the one that gets the RecursionError during initialization of its __context__), but one thing I was thinking about was walking the __context__ pointers and nulling out anything past the sys.getrecursionlimit() with a warning. On Sat, Jan 30, 2021 at 10:00 AM Irit Katriel wrote: > > Change by Irit Katriel : > > > -- > keywords: +easy > > ___ > Python tracker > <https://bugs.python.org/issue43048> > ___ > -- ___ Python tracker <https://bugs.python.org/issue43048> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43048] Printing RecursionError results in RecursionError
Vladimir Feinberg added the comment: I agree with both the duplicate classification and am glad the fix works in 3.10. Thanks all for the responses. Given the issue can be triggered with a fairly benign setup (pandas triggers such an error, and logger.exception is idiomatic), I do think backport should be worth consideration, but maybe I'm overindexing on the frequency of such exceptions in the wild. -- ___ Python tracker <https://bugs.python.org/issue43048> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43048] Printing RecursionError results in RecursionError
New submission from Vladimir Feinberg :
Python's standard logger provides an exception() method, which is to be called
from except blocks to log out exception traces by pulling from sys.exc_info().
Per https://github.com/python/cpython/blob/3.9/Lib/logging/__init__.py#L617 ,
logger.exception('my message') eventually boils down to something like
traceback.print_exception(ei[0], ei[1], ei[2], None, some_buffer) where ei is
the sys.exc_info().
However, the traceback code generates that printout by constructing a
TracebackException recursively for every `except` context.
In turn, if a RecursionError was generated from an exception thrown across many
except blocks, the TracebackException construction itself will have a
RecursionError. This is particularly bad in cases where you'd want to capture
this failure information, such as when you're printing the exception, since
you'll never find out about the originating error.
Certain (well-used) libraries rely on try/except for control flow, and
occasionally do recurse in their except blocks, so such exceptions are not
hypothetical.
A solution to this might be to avoid constructing a TracebackException, and
instead unwind traceback context using a while loop.
--
components: Library (Lib)
files: exc.py
messages: 385828
nosy: vlad2
priority: normal
severity: normal
status: open
title: Printing RecursionError results in RecursionError
versions: Python 3.6, Python 3.7, Python 3.9
Added file: https://bugs.python.org/file49769/exc.py
___
Python tracker
<https://bugs.python.org/issue43048>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19675] Pool dies with excessive workers, but does not cleanup
Vladimir Feinberg added the comment: I'm still hitting this issue in Python 3.6.0 :: Anaconda 4.3.0 (64-bit). Is there a reason this patch has been ignored? This is on CentOS release 6.5 -- nosy: +Vladimir Feinberg versions: +Python 3.6 -Python 2.7, Python 3.3 Added file: http://bugs.python.org/file46560/trace.txt ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue19675> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
