On 13 Mar 2024, at 6:01 PM, Dom Grigonis <dom.grigo...@gmail.com> wrote:
So my array sizes in this case are 3e8. Thus, 32bit ints would be needed. So it is not a solution for this case. Nevertheless, such concept would still be worthwhile for cases where integers are say max 256bits (or unlimited), then even if memory addresses or offsets are 64bit. This would both: a) save memory if many of values in array are much smaller than 256bits b) provide a standard for dynamically unlimited size values In principle one could encode individual offsets in a smarter way, using just the minimal number of bits required, but again that would make random access impossible or very expensive – probably more or less amounting to what smart compression algorithms are already doing. Another approach might be to to use the mask approach after all (or just flag all you uint8 data valued 2**8 as overflows) and store the correct (uint64 or whatever) values and their indices in a second array. May still not vectorise very efficiently with just numpy if your typical operations are non-local. Derek
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
