On Sun, Oct 10, 2021 at 7:25 AM Facundo Batista <[email protected]>
wrote:
> Hello everyone!
>
> I need to pack a long list of numbers into shared memory, so I thought
> about using `struct.pack_into`.
>
> Its signature is
>
> struct.pack_into(format, buffer, offset, v1, v2, ...)
>
> I have a long list of nums (several millions), ended up doing the
> following:
>
> struct.pack_into(f'{len(nums)}Q', buf, 0, *nums)
>
> However, passing all nums as `*args` is very inefficient [0]. So I
> started wondering why we don't have something like:
>
> struct.pack_into(format, buffer, offset, values=values)
>
> which would receive the list of values directly.
>
> Is that because my particular case is very uncommon? Or maybe we *do*
> want this but we don't have it yet? Or do we already have a better way
> of doing this?
>
> Thanks!
>
> [0] https://linkode.org/#95ZZtVCIVtBbx72dURK7a4
My first reaction on seeing things like this is "Why not use a numpy.array?"
Does what you have really need to be a long list? If so, that's already a
huge amount of Python object storage as it is. Is it possible for your
application to have kept that in a numpy array for the entirety of the data
lifetime?
https://numpy.org/doc/stable/reference/routines.array-creation.html
I'm not saying the stdlib shouldn't have a better way to do this by not
abusing *args as an API, just that other libraries solve the larger problem
of data-memory-inefficiency in their own way already.
*(neat tricks from others regarding stdlib array, shm, & memoryview even
if... not ideal)*
-gps
_______________________________________________
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/YD7WP6F3LEXFSEITVPZUNONVUBK3AKUW/
Code of Conduct: http://python.org/psf/codeofconduct/
