[issue30977] reduce uuid.UUID() memory footprint

[Tal Einat]

Tal Einat  added the comment:


New changeset 54752533b2ed1c898ffe5ec2e795c6910ee46a39 by Tal Einat in branch 
'master':
bpo-30977: rework code changes according to post-merge code review (GH-9106)
https://github.com/python/cpython/commit/54752533b2ed1c898ffe5ec2e795c6910ee46a39


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[Tal Einat]

Change by Tal Einat :


--
pull_requests: +8586

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[Tal Einat]

Change by Tal Einat :


--
pull_requests: +8560

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[Tal Einat]

Tal Einat  added the comment:

Thanks for the suggestion and the original patch, Wouter!

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[Tal Einat]

Tal Einat  added the comment:


New changeset 3e2b29dccc3ca9fbc418bfa312ad655782e250f2 by Tal Einat in branch 
'master':
bpo-30977: make uuid.UUID use __slots__ (GH-9078)
https://github.com/python/cpython/commit/3e2b29dccc3ca9fbc418bfa312ad655782e250f2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[STINNER Victor]

STINNER Victor  added the comment:

Oops. I missed the fact that Tal created PR 9078. Sorry, I reopen the issue ;-)

--
resolution: out of date -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[STINNER Victor]

STINNER Victor  added the comment:

I close the issue because of the pickle issue that hasn't been addressed by the 
wouter bolsterlee (the author) didn't reply for 1 month 1/2.

@wouter bolsterlee: if you still want to work on that issue, you should try to 
address the pickle issue first, then reopen this issue or maybe create a new 
issue pointing to this one.

--
resolution:  -> out of date
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[Tal Einat]

Tal Einat  added the comment:

See new PR which addresses pickle forward and backward compatibility.

--
nosy: +taleinat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[Tal Einat]

Change by Tal Einat :


--
keywords: +patch
pull_requests: +8536
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
versions: +Python 3.8 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

This change breaks pickle.

You should preserve forward and backward pickle compatibility.

1. Pickle data produced by old Python versions should be unpickleable with a 
new implementation. Implement __setstate__ for satisfying this.

2. Pickle data produced by a new implementation should be unpickleable in old 
Python versions. There are many ways to satisfy this, you should choose the 
most efficient.

--
components: +Library (Lib)
nosy: +serhiy.storchaka
type:  -> resource usage
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[wouter bolsterlee]

wouter bolsterlee added the comment:

i consider uuids as low level data types, not as fancy containers, similar to 
how i view datetime objects. given the native support in e.g. postgresql and 
many other systems, it's common to deal with uuids.

of course you can convert to/from strings or numbers, but that is cumbersome in 
many cases. for comparison, one would typically not convert unicode text 
from/into utf-8 encoded byte strings either, even though the latter will save 
memory in many cases.

from experience: converting can lead to nasty bugs, e.g. because you forgot 
about a conversion, and then a uuid string does not compare equal to a 
uuid.UUID instance, leaving you puzzled.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[Nir Soffer]

Nir Soffer added the comment:

This saves memory, but using str(uuid.uuid4()) requires even less memory.
If you really want to save memory, you can keep the uuid.uuid4().int.

Can you explain someone would like to have 100 uuid objects, instead of 
100 strings? What is the advantage of keeping UUID objects around?

--
nosy: +Nir Soffer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[Wouter Bolsterlee]

Wouter Bolsterlee added the comment:

as a follow-up note, i also experimented with keeping the actual value as a 
bytes object instead of an integer, but that does not lead to less memory being 
used: a 128-bit integer uses less memory than a 16 byte bytes object 
(presumably because PyBytesObject has a cached hash() field and a trailing null 
byte).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[Wouter Bolsterlee]

New submission from Wouter Bolsterlee:

memory usage for uuid.UUID seems larger than it has to be. it seems that using 
__slots__ will save around ~100 bytes per instance, which is very significant, 
e.g. when dealing with large sets of uuids (which are often used as "primary 
keys" into external data stores).

uuid.UUID has a __setattr__ that prevents any extra attributes to be
set:

def __setattr__(self, name, value):
raise TypeError('UUID objects are immutable')

...so it seems to me not having __dict__ should not cause any problems?


before (RES column):

>>> import uuid
>>> y = {uuid.uuid4() for _ in range(100)}

  PID USER  PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command
23020 wbolster   20   0  315M  253M  7256 S  0.0  1.6  0:04.98 python

with slots:

>>> import uuid
>>> y = {uuid.uuid4() for _ in range(100)}

  PID USER  PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command
21722 wbolster   20   0  206M  145M  7348 S  0.0  0.9  0:05.03 python

i will open a pr on github shortly.

--
messages: 298738
nosy: wbolster
priority: normal
severity: normal
status: open
title: reduce uuid.UUID() memory footprint

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30977] reduce uuid.UUID() memory footprint

[Wouter Bolsterlee]

Changes by Wouter Bolsterlee :


--
pull_requests: +2835

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com