[issue21556] try to use hashtable in pickle

Antoine Pitrou added the comment: I'm saying attempt, because although it works correctly, some benchmarks are actually slower. I didn't profile it, so I don't know if it's due to the hashtable implementation, function call overheads, etc. It probably shows that Python dicts (which the

[issue21556] try to use hashtable in pickle

STINNER Victor added the comment: _Py_hashtable_hash_ptr is quite crude in that it doesn't even try to compensate for pointer alignment, so there will automatically be many collisions. Only one hash bucket every 8 or 16 will be used, at best. I chose to use _Py_HashPointer() to drop (shift)

[issue21556] try to use hashtable in pickle

Antoine Pitrou added the comment: _Py_hashtable_hash_ptr is quite crude in that it doesn't even try to compensate for pointer alignment, so there will automatically be many collisions. Only one hash bucket every 8 or 16 will be used, at best. I chose to use _Py_HashPointer() to drop

[issue21556] try to use hashtable in pickle

STINNER Victor added the comment: And the straightforward collision resolution in hashtable.c is much less efficient at mitigating collisions than a Python dict's. Modules/hashtable.c comes from http://sourceforge.net/projects/libcfu/ (cfuhash type). I adapted the code for my needs (the

[issue21556] try to use hashtable in pickle

Antoine Pitrou added the comment: I didn't want to use the Python dict type for tracemalloc because this type may use the Python memory allocator which would lead to reentrant calls to tracemalloc. Ah, so this means CF's patch will make the pickle memotable invisible to tracemalloc? It

[issue21556] try to use hashtable in pickle

STINNER Victor added the comment: Ah, so this means CF's patch will make the pickle memotable invisible to tracemalloc? Currently, _Py_hashtabe uses PyMem_RawMalloc and PyMem_RawFree by default (realloc is not needed, we need to keep the previous buckets on rehash). Using

[issue21556] try to use hashtable in pickle

New submission from Charles-François Natali: This patch is an attempt at making pickle use Modules/hashtable.{h,c} instead of its hash table ad-hoc implementation for its memoization table. I'm saying attempt, because although it works correctly, some benchmarks are actually slower. I didn't

[issue21556] try to use hashtable in pickle

Changes by Ned Deily n...@acm.org: -- nosy: +alexandre.vassalotti ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21556 ___ ___ Python-bugs-list