On 9/7/23 23:34, glaserj--- via NumPy-Discussion wrote:
Reviving this old thread - I note that numpy.dot supports in-place
computation for performance reasons like this
c = np.empty_like(a, order='C')
np.dot(a, b, out=c)
However, the data type of the pre-allocated c array must match the result
datatype of a times b. Now, with some accelerator hardware (i.e. tensor cores
or matrix multiplication engines in GPUs), mixed precision arithmetics with
relaxed floating point precision (i.e.., which are not necessarily IEEE754
conformant) but with faster performance are possible, which could be supported
in downstream libraries such as cupy.
Case in point, a mixed precision calculation may take half precision inputs,
but accumulate in and return full precision outputs. Due to the above mentioned
type consistency, the outputs would be unnecessarily demoted (truncated) to
half precision again. The current API of numpy does not expose mixed precision
concepts. Therefore, it would be nice if it was possible to build in support
for hardware accelerated linear algebra, even if that may not be available on
the standard (CPU) platforms numpy is typically compiled for.
I'd be happy to flesh out some API concepts, but would be curious to first get
an opinion from others. It may be necessary to weigh the complexity of adding
such support explicitly against providing minimal hooks for add-on libraries in
the style of JMP (for jax.numpy), or AMP (for torch).
Jens
If your goal is "accumulate in and return full precision outputs", then
you can allocate C as the full precision type, and NumPy should do the
right thing. Note it may convert the entire input array to the final
dtype, rather than doing it "on the fly" which could be expensive in
terms of memory.
Matti
_______________________________________________
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com
