What processor you are running this on? np.sort uses AVX-512 accelerated
sorting for np.int32, so just wondering if you that is the reason for this
difference.
Raghuveer
> -----Original Message-----
> From: sal...@caltech.edu <sal...@caltech.edu>
> Sent: Wednesday, September 13, 2023 6:14 PM
> To: numpy-discussion@python.org
> Subject: [Numpy-discussion] Curious performance different with np.unique on
> arrays of characters
>
> Hello -
>
> In the course of some genomics simulations, I seem to have come across a
> curious
> (to me at least) performance difference in np.unique that I wanted to share.
> (If
> this is not the right forum for this, please let me know!)
>
> With a np.array of characters (U1), np.unique seems to be much faster when
> doing np.view as int -> np.unique -> np.view as U1 for arrays of decent size.
> I
> would not have expected this since np.unique knows what's coming in as S1 and
> could handle the view-stuff internally. I've played with this a number of
> ways (e.g.
> S1 vs U1; int32 vs int64; return_counts = True vs False; 100, 1000, or 10k
> elements) and seem to notice the same pattern. A short illustration below with
> U1, int32, return_counts = False, 10 vs 10k.
>
> I wonder if this is actually intended behavior, i.e. the view-stuff is
> actually a good
> idea for the user to think about and implement if appropriate for their
> usecase (as
> it is for me).
>
> Best regards,
> Shyam
>
>
> import numpy as np
>
> charlist_10 = np.array(list('ASDFGHJKLZ'), dtype='U1') charlist_10k =
> np.array(list('ASDFGHJKLZ' * 1000), dtype='U1')
>
> def unique_basic(x):
> return np.unique(x)
>
> def unique_view(x):
> return np.unique(x.view(np.int32)).view(x.dtype)
>
> In [27]: %timeit unique_basic(charlist_10)
> 2.17 µs ± 40.7 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)
>
> In [28]: %timeit unique_view(charlist_10)
> 2.53 µs ± 38.4 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)
>
> In [29]: %timeit unique_basic(charlist_10k)
> 204 µs ± 4.61 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
>
> In [30]: %timeit unique_view(charlist_10k)
> 66.7 µs ± 2.91 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)
>
> In [31]: np.__version__
> Out[31]: '1.25.2'
>
>
>
> --
> Shyam Saladi
> https://shyam.saladi.org
> _______________________________________________
> NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe
> send an email to numpy-discussion-le...@python.org
> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> Member address: raghuveer.devulapa...@intel.com
_______________________________________________
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com
