Changes by ebfe knabberknusperh...@yahoo.de:
Removed file: http://bugs.python.org/file12557/md5module_small_locks.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4751
New submission from ebfe knabberknusperh...@yahoo.de:
Here is another patch, this time for the fallback-md5-module. I know
that situations are rare where openssl is not present but threading is.
However they might occur out there and the md5module needed some love
anyway:
- The MD5 class from
ebfe knabberknusperh...@yahoo.de added the comment:
Haypo, we can probably reduce overhead by defining ENTER_HASHLIB like this:
#define ENTER_HASHLIB(obj) \
if ((obj)-lock) { \
if (!PyThread_acquire_lock((obj)-lock, 0)) { \
Py_BEGIN_ALLOW_THREADS
Changes by ebfe knabberknusperh...@yahoo.de:
Removed file: http://bugs.python.org/file12466/zlib_threads-2.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4738
ebfe knabberknusperh...@yahoo.de added the comment:
Here is a small test-script with concurrent access to a single
compressosbj. The original patch will immediately deadlock.
The patch attached releases the GIL before trying to get the zlib-lock.
This allows the other thread to release the zlib
ebfe knabberknusperh...@yahoo.de added the comment:
test-script
Added file: http://bugs.python.org/file12532/zlibtest2.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4738
ebfe knabberknusperh...@yahoo.de added the comment:
Releasing the GIL is somewhat expensive and should be avoided if
possible. I've moved LEAVE_HASHLIB in EVP_update so the object gets
unlocked before we call Py_END_ALLOW_THREADS. This is *only* possible
because EVP_update does not use
ebfe knabberknusperh...@yahoo.de added the comment:
test-script
Added file: http://bugs.python.org/file12534/hashlibtest2.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4751
Changes by ebfe knabberknusperh...@yahoo.de:
Removed file: http://bugs.python.org/file12461/hashopenssl_threads-3.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4751
ebfe knabberknusperh...@yahoo.de added the comment:
gnarf, actually it should be 'threads.append(Hasher(md))' in the script :-\
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4751
ebfe knabberknusperh...@yahoo.de added the comment:
I don't think this is actually worth the trouble. You run into situation
where one thread might decide that it needs a lock now with other
threads being in the to-be-locked-area at that time.
___
Python
ebfe knabberknusperh...@yahoo.de added the comment:
I don't think so.
The interface should stay simple - python has very few such magic knobs.
People will optimize for their own box as you said - and that code will
run worse on all the others...
Besides, we've lived so long with single
ebfe knabberknusperh...@yahoo.de added the comment:
haypo, the patch will not compile when WITH_THREADS is not defined. The
'lock'-member in the object structure is not present without
WITH_THREADS however the line 'if (self-lock == NULL view.len =
HASHLIB_GIL_MINSIZE)' will always refer
ebfe knabberknusperh...@yahoo.de added the comment:
Here is another patch, this time for the fallback-md5-module. I know
that situations are rare where openssl is not present but threading is.
However they might occur out there and the md5module needed some love
anyway:
- The MD5 class from
ebfe knabberknusperh...@yahoo.de added the comment:
Whenever the documentation says you must not it really says don't do
that or your application *will* crash, burn and die... Of course I can
allocate storage for the string, copy it's content and then free or -
nothing will happen. How would
ebfe knabberknusperh...@yahoo.de added the comment:
I don't think Python 2.x should be changed - but 3.0 or 3.1 should be:
- Characters don't mean a thing in zlib-land, all operations are based
on bytes and their (implicit) default encoding. This behaviour is hidden
and somewhat violates
ebfe knabberknusperh...@yahoo.de added the comment:
I can't reproduce the problem here.
Python 2.5.2 running on Linux lueg-desktop 2.6.24-22-generic #1 SMP Mon
Nov 24 18:32:42 UTC 2008 i686 GNU/Linux
--
nosy: +ebfe
___
Python tracker rep
New submission from ebfe knabberknusperh...@yahoo.de:
The hashlib functions provided by _hashopenssl.c hold the GIL all the
time although the underlying openssl-library is basically thread-safe.
I've attached a patch (svn diff) which basically does four things:
* If python is compiled
Changes by ebfe knabberknusperh...@yahoo.de:
Removed file: http://bugs.python.org/file12453/hashopenssl_threads.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4751
ebfe knabberknusperh...@yahoo.de added the comment:
Thanks for the advices.
Antoine, maybe you could clarify the situation regarding buffer-locks
for me. In older versions of PEP 3118 the PyBUF_LOCK flag was still
present but it doesn't seem to have made it's way into the final draft
ebfe knabberknusperh...@yahoo.de added the comment:
new svn diff attached
- GIL is now released for adler32 and crc32 if the buffer is larger than
5kb (we don't want to risk burning cpu cycles by GIL-stuff)
- adler32 got it's param by s# but now does s* - why s# anyway?
- ENTER_ZLIB no longer
Changes by ebfe knabberknusperh...@yahoo.de:
Removed file: http://bugs.python.org/file12448/zlib_threads.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4738
ebfe knabberknusperh...@yahoo.de added the comment:
Here is another simple benchmarker. For me it shows almost perfect
scaling (2 cores = 196% performance) if the buffer put into .update() is
large enough.
I deliberately did not move Py_BEGIN_ALLOW_THREADS into EVP_hash as we
might call
ebfe knabberknusperh...@yahoo.de added the comment:
new svn diff attached
the indentation in this file is not my fault, it has tabs all over it...
The 5kb limits protects from the overhead of releasing the GIL. With
very small buffers the overall runtime in my benchmark tends to double.
I set
Changes by ebfe knabberknusperh...@yahoo.de:
Removed file: http://bugs.python.org/file12463/zlib_threads-2.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4738
New submission from ebfe knabberknusperh...@yahoo.de:
Quote from http://docs.python.org/3.0/c-api/arg.html, regarding the s
argument:
s (string or Unicode object) [const char *]
Convert a Python string or Unicode object to a C pointer to a
character string. You must not provide storage
New submission from ebfe knabberknusperh...@yahoo.de:
My application needs to pack and unpack workunits all day long and does
this using multiple threading.Threads. I've noticed that the zlib module
seems to use only one thread at a time when using [de]compressobj(). As
the comment
27 matches
Mail list logo
