[issue12414] getsizeof() on code objects is wrong

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset df5df13fdc3a71bcf2295acc2cba7f22cfe2d669 by Serhiy Storchaka 
(Dong-hee Na) in branch '3.6':
[3.6] bpo-12414: Update code_sizeof() to take in account co_extra memory. 
(#1168) (#1198)
https://github.com/python/cpython/commit/df5df13fdc3a71bcf2295acc2cba7f22cfe2d669


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset b4dc6af7a7862a8996cffed30d39d6add5ee58a3 by Serhiy Storchaka 
(Dong-hee Na) in branch 'master':
bpo-12414: Update code_sizeof() to take in account co_extra memory. (#1168)
https://github.com/python/cpython/commit/b4dc6af7a7862a8996cffed30d39d6add5ee58a3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[Dong-hee Na]

Dong-hee Na added the comment:

I create PR 1168 and PR 1198 for master branch and the 3.6 branch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[Dong-hee Na]

Changes by Dong-hee Na :


--
pull_requests: +1323

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[Dong-hee Na]

Changes by Dong-hee Na :


--
pull_requests: +1298

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

This is easy issue. The only tricky part is testing. AFAIK we have no control 
on co_extra from Python. Therefore we can't create a new test. But existing 
test perhaps should be weaken for the case when it is ran on the interpreter 
that sets co_extra.

--
keywords: +easy (C)
stage:  -> needs patch
versions: +Python 3.6, Python 3.7 -Python 3.2, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[Dong-hee Na]

Dong-hee Na added the comment:

I am a newcomer who is interesting with contributing CPython project.
This issue could be the first challenging issue for me.
It looks like we need to multiply the number of co_extra and return the result.

This approach is right and can I proceed this issue?

--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[STINNER Victor]

STINNER Victor added the comment:

code_sizeof() must be updated to take in account co_extra memory: 
co_extra.ce_size * sizeof(co_extra.ce_extras[0]) bytes.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See gettotalsizeof.py attached to issue19048. It contains two functions that 
calculates the total size of the object with subobjects recursively. The 
problem is that virtually all objects are linked, two functions use slightly 
different criteria for stopping.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[Raymond Hettinger]

Raymond Hettinger added the comment:

Not including the Python accessible referred-to objects is consistent with how 
sys.getsizeof() works elsewhere (i.e. for object instances, the size of 
__dict__ is not included).

>>> import sys
>>> class A:
pass

>>> a = A()
>>> sys.getsizeof(a)
56
>>> sys.getsizeof(a.__dict__)
112

The result is easily misleading but this seems to have been an early design 
decision about the semanatics __sizeof__.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I concur with Martin. sys.getsizeof() should only count the memory that is not 
exposed as separate Python objects. In case of a code object this is the memory 
of the PyCodeObject structure and the memory of dynamic array co_cellvars 
(issue15456). Other subobjects are exposed as code object attributes and by 
gc.get_referents(). For counting the summary size you should recursively call 
sys.getsizeof() for objects returned by gc.get_referents(). But be aware that 
some subobjects (for example interned strings) can be shared between different 
code objects, so the average memory consumption is less than the simple sum.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--
nosy: +rhettinger

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12414
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[Martin v . Löwis]

Martin v. Löwis mar...@v.loewis.de added the comment:

For composite objects, getsizeof should only return the memory of the object 
itself, and not that of other objects it refers to. the object itself 
definitely includes the struct of the object, and also definitely includes 
non-PyObject blocks uniquely referred to by the object. It definitely should 
not return objects it reports in gc.get_referents. It probably should include 
PyObjects not shared with any other object, and not accessible from outside of 
the object.

There are boundary cases, such as memory blocks which are not PyObject, but may 
be shared across objects, and PyObjects not reported in get_referents.

It seems this case is the latter: the PyObjects are not returned from 
get_referents, but are certainly available to Python, e.g. through co_code and 
co_consts.

I don't think there sizes should be added to the size of the PyObject, since 
otherwise accounting algorithms may account for it twice.

What's your use case for including it in the total size?

--
nosy: +loewis

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12414
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12414] getsizeof() on code objects is wrong

[Benjamin Peterson]

New submission from Benjamin Peterson benja...@python.org:

sys.getsizeof() on a code object returns on the size of the code struct, not 
the arrays and tuples which it references.

--
components: Interpreter Core
messages: 139142
nosy: benjamin.peterson
priority: normal
severity: normal
status: open
title: getsizeof() on code objects is wrong
type: behavior
versions: Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12414
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com