[issue14597] Cannot unload dll in ctypes until script exits
New submission from Pat Lynch plync...@gmail.com: If I load a dll in ctypes, then delete that loaded DLL instance, the DLL is not unloaded until the script finishes and exits. I'm trying to write some unit tests in python to exercise that DLL where each test case loads a DLL, does some work, then unloads the DLL. Unfortunately the DLL only gets unloaded when the unit tests finish. I've tried forcing the garbage collector to run to get the DLL to unload. It did nothing... # load the DLL parser_dll = CDLL(dllpath) # do some work here # 'unload' the dll (or as close as I can get it to it) if (parser_dll): del parser_dll -- components: ctypes messages: 158433 nosy: plynch76 priority: normal severity: normal status: open title: Cannot unload dll in ctypes until script exits type: enhancement versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14597 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14597] Cannot unload dll in ctypes until script exits
Pat Lynch plync...@gmail.com added the comment: I should mention also, that this is mostly an issue for me on Win7 x64. It does behave 'slightly' better on WinXP x86. (I have the 64-bit version of python installed on Win7 x64 the 32-bit version installed on WinXP) thanks, Pat. On 16 April 2012 14:09, Pat Lynch rep...@bugs.python.org wrote: New submission from Pat Lynch plync...@gmail.com: If I load a dll in ctypes, then delete that loaded DLL instance, the DLL is not unloaded until the script finishes and exits. I'm trying to write some unit tests in python to exercise that DLL where each test case loads a DLL, does some work, then unloads the DLL. Unfortunately the DLL only gets unloaded when the unit tests finish. I've tried forcing the garbage collector to run to get the DLL to unload. It did nothing... # load the DLL parser_dll = CDLL(dllpath) # do some work here # 'unload' the dll (or as close as I can get it to it) if (parser_dll): del parser_dll -- components: ctypes messages: 158433 nosy: plynch76 priority: normal severity: normal status: open title: Cannot unload dll in ctypes until script exits type: enhancement versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14597 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14597 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14597] Cannot unload dll in ctypes until script exits
Pat Lynch plync...@gmail.com added the comment: thanks for the very quick response. Since LoadLibrary is called in the constructor, why can't FreeLibrary be called in the destructor? or at least expose a function to unload that calls FreeLibrary? http://msdn.microsoft.com/en-us/library/windows/desktop/ms683152%28v=vs.85%29.aspx thanks again, Pat. On 16 April 2012 14:30, R. David Murray rep...@bugs.python.org wrote: R. David Murray rdmur...@bitdance.com added the comment: In general it is difficult to impossible to get Python2 to unload modules before the interpreter shuts down. See issue 9072. I'm not savvy enough with the C stuff to know if the fact that you loaded it via ctypes changes anything, but I doubt it. Note that the implication of that issue is that if you could move to Python3 there might be a way to do it, but that would indeed be an enhancement as there is no direct support for it yet. -- nosy: +r.david.murray versions: +Python 3.3 -Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14597 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14597 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14597] Cannot unload dll in ctypes until script exits
Pat Lynch plync...@gmail.com added the comment: ok, that's fair enough if most usage of ctypes is from people accessing system libraries :) I wouldn't have thought my usage was that weird though (given the strength of using python for unit testing). In local tests, adding a function CDLL::ForceUnloadDll (which just calls FreeLibrary(self._handle)) seems to work well. I haven't done intensive testing though at this point. I could be missing something though. thanks, Pat. On 16 April 2012 14:45, Martin v. Löwis rep...@bugs.python.org wrote: Martin v. Löwis mar...@v.loewis.de added the comment: In principle, it should be possible (but perhaps not desirable, see below) to call FreeLibrary in a CDLL's __del__. However, since this would be a new feature, it can't go into 2.7. Patches are welcome; make sure to support both FreeLIbrary and dlclose. There is a general issue with closing/freeing DLLs: if they are still referenced somewhere (e.g. in an atexit function, a C++ virtual method table, or on the call stack of another thread), then a later access to the code will crash the interpreter. In a typical DLL today (including all Python extension modules), the likelihood of crashes is close to 100%. For that reason, it's probably not a good idea to have ctypes auto-close DLLs; instead, it should be an opt-in mechanism. For most ctypes uses, closing is irrelevant, since people typically access system libraries that are independently loaded anyway, so closing them would not have any effect. -- nosy: +loewis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14597 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14597 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14597] Cannot unload dll in ctypes until script exits
Pat Lynch plync...@gmail.com added the comment: Just to update:- I've run this pretty extensively on multiple systems (XP x86 Win7 64-bit) and it appears to behave as expected (haven't checked it on Linux). I have that code being called in 100s of unit tests. For python 3.1, would it make sense to add it as a ForceUnload function?? - for safety bail out if handle was not None when passed into the constructor? i.e. if somebody has accessed an independently loaded DLL, they will pass in the handle when constructing the CDLL object. Disallow ForceUnload in that case. ForceUnload will only be allowed in cases where we created that type by passing in the path to the DLL. I'll be using this code as a local patch, so no rush to put it into 3.1 etc. thanks for all the info - much appreciated :) Pat. On 16 April 2012 15:04, Pat Lynch rep...@bugs.python.org wrote: Pat Lynch plync...@gmail.com added the comment: ok, that's fair enough if most usage of ctypes is from people accessing system libraries :) I wouldn't have thought my usage was that weird though (given the strength of using python for unit testing). In local tests, adding a function CDLL::ForceUnloadDll (which just calls FreeLibrary(self._handle)) seems to work well. I haven't done intensive testing though at this point. I could be missing something though. thanks, Pat. On 16 April 2012 14:45, Martin v. Löwis rep...@bugs.python.org wrote: Martin v. Löwis mar...@v.loewis.de added the comment: In principle, it should be possible (but perhaps not desirable, see below) to call FreeLibrary in a CDLL's __del__. However, since this would be a new feature, it can't go into 2.7. Patches are welcome; make sure to support both FreeLIbrary and dlclose. There is a general issue with closing/freeing DLLs: if they are still referenced somewhere (e.g. in an atexit function, a C++ virtual method table, or on the call stack of another thread), then a later access to the code will crash the interpreter. In a typical DLL today (including all Python extension modules), the likelihood of crashes is close to 100%. For that reason, it's probably not a good idea to have ctypes auto-close DLLs; instead, it should be an opt-in mechanism. For most ctypes uses, closing is irrelevant, since people typically access system libraries that are independently loaded anyway, so closing them would not have any effect. -- nosy: +loewis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14597 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14597 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14597 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
