On 27 Mar 2017, at 20:12, Chris Angelico wrote:
On Tue, Mar 28, 2017 at 11:00 AM, Chris Angelico <ros...@gmail.com>
wrote:
In any case, I've installed nvidia-opencl-dev and it seems to be
happy. How long should I expect this to run for?
Now testing under CPython 3.7.
It ran for about 14 minutes, then memory usage spiked and went into
the page file. Used roughly 8GB of RAM prior to that point, I think?
Unfortunately, I don't think my hard disk is fast enough for this to
run through the page file in any reasonable time.
Yeah, it will be problematic with less than 32GB.
My latest findings: The zsh `time` built-in reports a maximum of only
30GB used which seems to correspond to the resident memory size, while
the virtual memory size exceeds 40GB for the process and the total
memory allocated over all processes exceeds 60GB (with the main
contribution from the process that supposedly only uses ~40GB).
My best idea about what's going on at the moment is that memory
fragmentation is worse in Python 3.6 for some reason. The virtual memory
size indicates that a large address space is acquired, but the resident
memory size is smaller indicating that not all of that address space is
actually used. In fact, the code might be especially bad to
fragmentation because it takes a lot of small NumPy arrays and
concatenates them into larger arrays. But I'm still surprised that this
is only a problem with Python 3.6 (if this hypothesis is correct).
Jan
--
https://mail.python.org/mailman/listinfo/python-list
