Hi Matthias, Matthias Lee <[email protected]> writes: > I noticed something interesting today. > I am working on an image processing tool which loops several times over > each of a series of images. Everything is done in place and I should not be > growing my memory footprint between iterations. > > Now when I tracked the actual GPU memory consumption I found that I would > ultimately I would run out of GPU memory (just a short excerpt): > http://i.imgur.com/AjmmpEk.png > > I double and triple checked that everything is happening in place, started > trying to delete GPU objects as soon as I'm finished with them to try to > trigger the GC, but that only had limited success. I would expect the GC to > kick in before the GPU runs out of memory.. > > I then started manually calling gc.collect() every few iteration and > suddenly everything started behaving and is now relatively stable. See here > (note the scale difference): http://i.imgur.com/Zzq5YdC.png > > Is this normal? Is this a bug?
First off, you can force-free GPU memory using this, if all else fails: http://documen.tician.de/pycuda/driver.html#pycuda.driver.DeviceAllocation.free Next, the behavior you're seeing means that a reference cycle of some sort must exist within the object that's holding on to you GPU memory. (Could be PyCUDA's GPUArray--it has happened before, but I'd consider it a bug. I'll go poke around if it is. Let me know.) A reference cycle means that Python will only free these objects upon a GC run (since the refcount will never return to zero on its own). Unless told explicitly otherwise (see above), PyCUDA will only free GPU memory once the associated Python handle objects have been pronounced unused by the Python runtime. PyCUDA is smart enough to force a GC run before declaring defeat on a memory allocation, so if you're the only one using a GPU, this shouldn't pose an issue. If you're using other libraries that also (try to) allocate GPU memory, then this might pose an issue, because they *won't* know to try GC'ing. Hope that helps, Andreas
pgpnKV4tXCcuC.pgp
Description: PGP signature
_______________________________________________ PyCUDA mailing list [email protected] http://lists.tiker.net/listinfo/pycuda
