[Python-Dev] tp_dealloc
Hi, when embedding python 3.1, I have my own free-method in tp_dealloc. The allocated memory is in host-memory, not in python (dll). Now, the problem is, Python appears to read-access the deallocated memory still after tp_dealloc. After tp_dealloc, I get an access violation if the pyobject-header fields have been modified inside tp_dealloc. If I leave the header unmodified, then no access violation occurs. Accessing deallocated memory sounds like a bug, or is this intended design? Thank you Marvin -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] tp_dealloc
My tp_dealloc method (of non-subtypable type) calls the freeMem-method of a memory manager (this manager was also used for the corresponding allocation). This freeMem-method deallocates and modifies the memory, which is a valid action, because after free, the memory-manager has ownership of the freed memory. Several memory managers do this (for example the Memory Manager in Delphi during debug mode, in order to track invalid memory access after free). The python31.dll calls tp_alloc and later (after return of tp-alloc) the python31.dll is still awaiting valid content in the deallocated memory. I don't know where this happens, I'm not a developer of CPython, but at this point the python31.dll causes an access violation. IMO the python31.dll assumes that freeMem never modifies the memory (pyobject header), this is valid for many memory managers, but not for all. And from my perspective, this assumption a bug, which can cause access violations in many applications (for example, applications which use the PythonForDelphi-package; PyScripter is one of them, but also many others) Please, could some CPython-developer take a look, thank you! -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] tp_dealloc
Sorry, I wrote tp_alloc in last post, it should be always tp_dealloc: My tp_dealloc method (of non-subtypable type) calls the freeMem-method of a memory manager (this manager was also used for the corresponding allocation). This freeMem-method deallocates and modifies the memory, which is a valid action, because after free, the memory-manager has ownership of the freed memory. Several memory managers do this (for example the Memory Manager in Delphi during debug mode, in order to track invalid memory access after free). The python31.dll calls tp_dealloc and later (after return of tp_dealloc) the python31.dll is still awaiting valid content in the deallocated memory. I don't know where this happens, I'm not a developer of CPython, but at this point the python31.dll causes an access violation. IMO the python31.dll assumes that freeMem never modifies the memory (pyobject header), this is valid for many memory managers, but not for all. And from my perspective, this assumption a bug, which can cause access violations in many applications (for example, applications which use the PythonForDelphi-package; PyScripter is one of them, but also many others) Please, could some CPython-developer take a look, thank you! -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] tp_dealloc
> This said, there may be a bug somewhere, but what do you want us to look > at? > Do you have a case that we could reproduce and investigate? > > -- > Amaury Forgeot d'Arc Thank you, I'm not a C-Developer, but still I have one more detail: I call py_decRef( pyObj) of dll (version 3.1.1), ( which calls tp_dealloc, which calls my freeMem() method)) No problem is reported here. Now, the freed memory should not be accessed anymore by python31.dll. You may fill the freed pyObjectHead with invalid values, in my case it's: ob_refcnt= 7851148, ob_type = $80808080 But later, when I call Py_Finalize, there inside is some access to the same freed memory; this causes an AV, more precisely, when the value $80808080 is checked. My Delphi-Debugger shows the following byte-sequence inside python31.dll: 5EC3568B7424088B4604F7405400407504 5E - pop esi C3 - ret 56 - push esi 8B742408- mov esi, [esp+$08] 8B4604 - mov eax, [esi+$04] // eax = $80808080 // F740540040 - test [eax+$54], $4000 // AV exception by read of address $808080D4 // 7504- jnz $1e03681b Maybe this can help someone, thank you! -- Marvin GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] tp_dealloc
> Without further information, I cannot consider this as a problem in > Python. > I know other extension modules that manage memory in their own way, and > work. > It's more probably an issue in the code of your type. > > -- > Amaury Forgeot d'Arc Ok, thank you, but I'm still hoping, someone could test this. I'm very sure, my app is not the cause; only the python31.dll (py_finalize) is accessing the freed memory. Inside py_finalize there is really no call to my hosting app (or reverse), I even tested this in my debugger. In most applications this python-problem remains hidden, because their freeMem() leaves the freed memory unmodified. (And that's why very good debuggers modify the freed memory to reveal such hidden errors). You could simply test this by setting pyObject.ob_type = $80808080 after freeMem( pyObject). Then later, call py_finalize, and you will see the same problem (Access violation by trying to use ob_type) -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
