[Numpy-discussion] Re: Managing integer overflow
On Tue, May 2, 2023 at 8:03 AM wrote: > > I think this example shows that you don't need any special infrastructure > > from numpy. I don't think there is going to be much appetite to expand > our > > API in this direction. > > But I do! I'm looking for something that implements the > multiply_check_ov(X,Y) and similar functionality for addition and > subtraction, for large arrays; I don't know how to do that efficiently. (Or > even correctly for 64-bit operands --- for 8/16/32 I suppose as a baseline > I could do the math in 64-bit arithmetic and if the answer doesn't match, > then use the sign bit of the 64-bit result for determining overflow.) > Some C compilers provide some builtin functions that do this for you, and some cross-platform headers around that show how to do this. https://gcc.gnu.org/onlinedocs/gcc/Integer-Overflow-Builtins.html https://github.com/pytorch/pytorch/blob/main/c10/util/safe_numerics.h https://github.com/dcleblanc/SafeInt -- Robert Kern ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Managing integer overflow
On Mon, 2023-05-01 at 17:50 +, jmsa...@gmail.com wrote: > > I think this example shows that you don't need any special > > infrastructure > > from numpy. I don't think there is going to be much appetite to > > expand our > > API in this direction. > > But I do! I'm looking for something that implements the > multiply_check_ov(X,Y) and similar functionality for addition and > subtraction, for large arrays; I don't know how to do that > efficiently. (Or even correctly for 64-bit operands --- for 8/16/32 I > suppose as a baseline I could do the math in 64-bit arithmetic and if > the answer doesn't match, then use the sign bit of the 64-bit result > for determining overflow.) There are two things I could imagine. First, improving the warnings and maybe making some overflow errors (or optional). This never happened since it should come with performance impact and that should be low. Second, there could be a `BigInt` dtype that has full precision like Python integers, but I would prefer such development to start outside of NumPy first. But, neither is even what you want probably. So I agree, NumPy is probably not in a position to help you. You can implement the functions that you described yourself (i.e. with overflow return indication). And they will be fully compatible with NumPy (see for example https://github.com/WarrenWeckesser/ufunclab or maybe via jitters like numba). - Sebastian > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: sebast...@sipsolutions.net > ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Managing integer overflow
> I think this example shows that you don't need any special infrastructure > from numpy. I don't think there is going to be much appetite to expand our > API in this direction. But I do! I'm looking for something that implements the multiply_check_ov(X,Y) and similar functionality for addition and subtraction, for large arrays; I don't know how to do that efficiently. (Or even correctly for 64-bit operands --- for 8/16/32 I suppose as a baseline I could do the math in 64-bit arithmetic and if the answer doesn't match, then use the sign bit of the 64-bit result for determining overflow.) ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Managing integer overflow
Two boolean arrays are not more compact than one signed `int8` array. But since you'd have to make bool masks from such an array anyways to do anything useful, you might as well pass the bool masks. I think this example shows that you don't need any special infrastructure from numpy. I don't think there is going to be much appetite to expand our API in this direction. On Thu, Apr 27, 2023 at 9:13 AM wrote: > > Ideally if there is an overflow in an array operation I'd like to > produce an "overflow array" that's the same size as the result array, but > with the values +1 if it's a positive overflow or -1 if it's a negative > overflow. > > Alternatively, if boolean arrays are a much more compact memory footprint, > then return two overflow arrays, one positive and one negative. > > Use case: > > X = ... something ... > Y = ... something else ... > Z, ovpos, ovneg = multiply_check_ov(X,Y) > if ovpos.any(): >np.iinfo(Z.dtype).max > if ovneg.any(): ># uh oh, something we didn't expect, so raise an error > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: robert.k...@gmail.com > -- Robert Kern ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Managing integer overflow
> Ideally if there is an overflow in an array operation I'd like to produce an > "overflow array" that's the same size as the result array, but with the > values +1 if it's a positive overflow or -1 if it's a negative overflow. Alternatively, if boolean arrays are a much more compact memory footprint, then return two overflow arrays, one positive and one negative. Use case: X = ... something ... Y = ... something else ... Z, ovpos, ovneg = multiply_check_ov(X,Y) if ovpos.any(): np.iinfo(Z.dtype).max if ovneg.any(): # uh oh, something we didn't expect, so raise an error ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
