[issue19533] Unloading docstrings from memory if -OO is given

2020-03-06 Thread Brett Cannon 


Brett Cannon  added the comment:

Do note that .pyc files now encode their optimization levels, so the only thing 
to potentially do here is change the compiler to toss docstrings out and make 
sure they are freed when they are parsed to avoid holding on to them.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2020-03-06 Thread Brett Cannon 


Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2014-05-22 Thread Stefan Krah 

Changes by Stefan Krah stefan-use...@bytereef.org:


--
nosy:  -skrah

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19533
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2014-04-23 Thread Phil Connell 

Changes by Phil Connell pconn...@gmail.com:


--
nosy: +pconnell

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19533
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2013-11-10 Thread Stefan Krah 

Stefan Krah added the comment:

It looks like the memory management is based directly on Py_Arenas:

def f():
squeamish ossifrage
pass

Breakpoint 1, PyArena_Free (arena=0x9a5120) at Python/pyarena.c:159
159 assert(arena);
(gdb) p arena-a_objects
$1 = ['f', 'squeamish ossifrage']
(gdb) bt
#0  PyArena_Free (arena=0x9a5120) at Python/pyarena.c:159
#1  0x00425af5 in PyRun_FileExFlags (fp=0xa1b780, 
filename_str=0x77f37eb0 docstr.py, start=257, globals=
{'f': function at remote 0x77f04058, '__builtins__': module at 
remote 0x77f6a358, '__name__': '__main__', '__file__': 'docstr.py', 
'__package__': None, '__loader__': SourceFileLoader(name='__main__', 
path='docstr.py') at remote 0x77ede608, '__cached__': None, '__doc__': 
None}, locals=
{'f': function at remote 0x77f04058, '__builtins__': module at 
remote 0x77f6a358, '__name__': '__main__', '__file__': 'docstr.py', 
'__package__': None, '__loader__': SourceFileLoader(name='__main__', 
path='docstr.py') at remote 0x77ede608, '__cached__': None, '__doc__': 
None}, closeit=1, flags=0x7fffe490) at Python/pythonrun.c:2114
#2  0x00423a0c in PyRun_SimpleFileExFlags (fp=0xa1b780, 
filename=0x77f37eb0 docstr.py, closeit=1, flags=
0x7fffe490) at Python/pythonrun.c:1589
#3  0x0042289c in PyRun_AnyFileExFlags (fp=0xa1b780, 
filename=0x77f37eb0 docstr.py, closeit=1, flags=0x7fffe490)
at Python/pythonrun.c:1276
#4  0x0043bc83 in run_file (fp=0xa1b780, filename=0x9669b0 
Ldocstr.py, p_cf=0x7fffe490) at Modules/main.c:336
#5  0x0043c8c5 in Py_Main (argc=3, argv=0x964020) at Modules/main.c:780
#6  0x0041cdb5 in main (argc=3, argv=0x7fffe688) at 
./Modules/python.c:69

So the string 'squeamish ossifrage' is still in arena-a_objects right until
end of PyRun_FileExFlags(), even with -OO.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19533
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2013-11-09 Thread Serhiy Storchaka 

Serhiy Storchaka added the comment:

Some tests fail when ran with -OO and then with -O. Short example (there are 
more examples):

$ rm -rf Lib/test/__pycache__
$ ./python -OO -m test.regrtest test_property
[1/1] test_property
1 test OK.
$ ./python -O -m test.regrtest test_property
[1/1] test_property
test test_property failed -- multiple errors occurred; run in verbose mode for 
details
1 test failed:
test_property

--
components: +Tests
nosy: +serhiy.storchaka, skrah
type: enhancement - behavior
versions: +Python 2.7, Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19533
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2013-11-09 Thread Serhiy Storchaka 

Changes by Serhiy Storchaka storch...@gmail.com:


--
Removed message: http://bugs.python.org/msg202474

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19533
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2013-11-09 Thread Serhiy Storchaka 

Changes by Serhiy Storchaka storch...@gmail.com:


--
components:  -Tests
type: behavior - enhancement
versions:  -Python 3.3, Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19533
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2013-11-09 Thread Serhiy Storchaka 

Changes by Serhiy Storchaka storch...@gmail.com:


--
versions: +Python 3.3 -Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19533
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2013-11-09 Thread Brett Cannon 

Brett Cannon added the comment:

Do realize this is a one-time memory cost, though, because next execution will 
load from the .pyo and thus will never load the docstring into memory. If you 
pre-compile all bytecode with -OO this will never even occur.

--
nosy: +brett.cannon

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19533
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2013-11-09 Thread Sworddragon 

Sworddragon added the comment:

 Do realize this is a one-time memory cost, though, because next execution 
 will load from the .pyo and thus will never load the docstring into memory.

Except in 2 cases:

- The bytecode was previously generated with -O.
- The bytecode couldn't be written (for example permission issues or Python was 
invoked with -B).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19533
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2013-11-09 Thread R. David Murray 

R. David Murray added the comment:

So the question is, if there is no longer a reference to the docstring, why 
isn't it garbage collected?  (I tested adding a gc.collect(), and it didn't 
make any difference.)

--
nosy: +r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19533
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2013-11-09 Thread Stefan Krah 

Stefan Krah added the comment:

R. David Murray rep...@bugs.python.org wrote:
 So the question is, if there is no longer a reference to the docstring, why 
 isn't it garbage collected?  (I tested adding a gc.collect(), and it didn't 
 make any difference.)

I think it probably is garbage collected but the freed memory is not returned 
to the OS
by the memory allocator.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19533
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2013-11-09 Thread R. David Murray 

R. David Murray added the comment:

Hmm.  If I turn on gc debugging before the def, I don't see anything get 
collected.  If I allocate a series of new 10K strings, the memory keeps 
growing.  Of course, that could still be down to the vagaries of OS memory 
management.  Time to break out Victor's tracemalloc, but I probably don't have 
that much ambition today :)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19533
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19533] Unloading docstrings from memory if -OO is given

2013-11-08 Thread Sworddragon 

New submission from Sworddragon:

Using -OO on a script will remove the __doc__ attributes but the docstrings 
will still be in the process memory. In the attachments is an example script 
which demonstrates this with a docstring of ~10 MiB (opening the file in an 
editor can need some time). Calling python3 -OO test.py will result in a 
memory usage of ~16 MiB on my system (Linux 64 Bit) while test.__doc__ is None.

--
components: Interpreter Core
files: test.py
messages: 202465
nosy: Sworddragon
priority: normal
severity: normal
status: open
title: Unloading docstrings from memory if -OO is given
type: enhancement
versions: Python 3.3
Added file: http://bugs.python.org/file32550/test.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19533
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com