On 27 Mar 2017, at 18:42, Chris Angelico wrote:
Are you able to share the program? I could try it on my system and see
if the same thing happens.
Yes, it is on GitHub (use the fixes branch):
https://github.com/ctn-archive/gosmann-frontiers2017/tree/fixes
Installation instructions are in the readme.
The command I'm running is `python scripts/log_reduction.py spaun`
I was looking at the output of htop to see the memory consumption. The
`time` built-in of the zsh might report a lower memory usage (I'm
running it right now to make sure), which would also be surprising. But
I'm certain that Python 3.6 uses more memory because the run time
increases due to using the swap partition.
If you run the program, it will take an initial 10–15 minutes of
processing during which the memory usage should increase to roughly
10GB. Then the output “[INFO] Optimizing model...” will be printed
after which the memory usage continues to increase to around 30GB (maybe
a bit more with Python 3.6). This constitutes a first “optimization
pass” that will be ended with the message “[INFO] Pass 1 [views]:
Reduced 996917 to 913666 operators in 62.519006s.” Then in the second
optimization pass the memory consumption will continue to increase with
Python 3.6 (up to ~60GB being consumed, where less than 5GB should be
due to other processes, though the virtual and resident memory reported
for the Python process are only ~42GB and ~30GB respectively) and in
later passes it will start decreasing again. With Python 3.5 the memory
consumption stays about 30GB and decreases in later passes.
Let me know if you have more questions.
--
https://mail.python.org/mailman/listinfo/python-list
