Changes by Kristján Valur Jónsson :
--
resolution: -> duplicate
status: open -> closed
superseder: -> On Mac / BSD sockets returned by accept inherit the parent's FD
flags
___
Python tracker
<http://bugs.pyth
Kristján Valur Jónsson added the comment:
socket.defaulttimeout(None)
s = socket.socket()
s.settimeout(0) #nonblocking
s.bind()
s2, a = s.accept()
print s2.gettimeout() #prints ´none´, meaning blocking
s2.receive(10) #raises EWOULDBLOCK error, since internally it is non-blocking
I don't
Kristján Valur Jónsson added the comment:
Antoine, absolument. Please see attached file bug.py
As for a different patch, we should agree what behaviour should be expected. I
don't think it is possible to rely on some platform specific behaviour. This
is because it is in genera
Kristján Valur Jónsson added the comment:
I can't see how that patch has anything to do with it. The problem has been
present since at least 2.5. Your patch fixed it for timeout>0.0 but left the
0.0 case still broken.
It comes from these lines in init_sockobject:
{
s->s
Kristján Valur Jónsson added the comment:
Is this moving anywhere? I was asked to apply this patch locally and am
curious as to whether it is moving into the stdlib.
--
nosy: +krisvale
___
Python tracker
<http://bugs.python.org/issue2
Kristján Valur Jónsson added the comment:
Now, I'd be super happy to see this strange semantics of PyThread_set_key_value
go away. Its very un-standard and complicates the mapping from an native
implementation to the python one.
But I think I did once bring up this issue, and was told
Kristján Valur Jónsson added the comment:
Antoine, I wonder if we can restore PyThread_set_key_value to behave like a
canonical TLS api function (always setting) but fix the cases that want to "set
if it has not already been set" like the cases you mention.
It is very unorthodox to
Kristján Valur Jónsson added the comment:
Ah, using the fallback implementation of tls? Surely this isn't a problem with
the pthreads tls, I'd be surprised if it retains TLS values after fork.
--
___
Python tracker
<http://bu
Kristján Valur Jónsson added the comment:
Yes, exec is unholy:)
For embedding Python into a console game we have removed the python compiler.
exec and eval don't work. This saves space and is also a security feature. I
had to modify the collections module so that namedtuple() just re
Kristján Valur Jónsson added the comment:
Committed as revision 86510
--
resolution: -> accepted
status: open -> closed
___
Python tracker
<http://bugs.python.org/i
Kristján Valur Jónsson added the comment:
Something is definietly weird on the PS3. I´ll give more concrete data soon.
(and yes, I may have misread the code)
--
___
Python tracker
<http://bugs.python.org/issue10
Kristján Valur Jónsson added the comment:
Yes, in my original myopic observation I was mistaken in thinking that we were
reading the digest out of the 5 entry int32 "digest" field in the SHAobject.
I´ve already verified that the "Endianness" field is correctly set. Wh
Kristján Valur Jónsson added the comment:
Ah yes, spaces in filenames. One always forgets.
Fixed the make_buildinfo.c (quote whole string, not just part of it) and
committed in revision 86689
--
resolution: -> accepted
status: open ->
Kristján Valur Jónsson added the comment:
I did various tests and quoting parts of the path seemed fragile, especially if
the quote landed next to a backslash. This is dangerous ground :)
--
___
Python tracker
<http://bugs.python.org/issue10
Kristján Valur Jónsson added the comment:
Found the issue and it wasn't with sha1.
Turned out that the code was doing somethign like
sha1(buffer(unicode('str'))) which exposed the endianness of the unicode
representation.
Sorry for wasting your time.
--
resolution: -&
New submission from Kristján Valur Jónsson :
The new "s*" code for PyArg_ParseTuple is used to fill a Py_buffer object from
the arguments. This object must be relased using PyBuffer_Release() after use.
However, if the object in the tuple does not support the new buffer interface
Changes by Kristján Valur Jónsson :
--
versions: -Python 3.2
___
Python tracker
<http://bugs.python.org/issue10538>
___
___
Python-bugs-list mailing list
Unsub
Kristján Valur Jónsson added the comment:
This looks fairly complete.
I spotted the word "optimze" in there :)
As for the C api, I always cringe when I add an "Ex" function myself, because
it feels like I'm spending my last bullet. What to do if I need yet another
pa
New submission from Kristján Valur Jónsson :
As discussed here:
http://mail.python.org/pipermail/python-ideas/2010-November/008813.html:
Adding the ability to register callbacks to be invoked before and after garbage
collection runs. This can be used to gather run-time statistics such as
Kristján Valur Jónsson added the comment:
Hi, as I stated, the original patch was simply our original implementation.
Here is a new patch. It is simpler:
1) it exposes a gc.callbacks list where users can register themselves, in the
spirit of sys.meta_path
2) One can have multiple callbacks
3
Kristján Valur Jónsson added the comment:
You are right, Antoine.
How about a string and a dict? the string can be "start" and "stop" and we can
add interesting information to the dict as you suggest.
--
___
Python tracker
<
Kristján Valur Jónsson added the comment:
1) I'm not sure what you are asking. Does anyone think that it is simpler to
register two different callbacks than one? IMHO it complicates the interface
and creates so many oppertunities to do things incorrectly.
2)No, it is an oversight, l
Kristján Valur Jónsson added the comment:
Here is a third patch. The callback now gets two argument, phase and info.
I've added documentation and unittests.
--
Added file: http://bugs.python.org/file19946/gccallback3.patch
___
Python tracker
Kristján Valur Jónsson added the comment:
The patch looks fine to me. If it runs and everyone is happy, by all means
lets put it in.
(I agree that the zlib stuff _should_ be handled elsewhere, but for reasons of
expediency it was put into this module
Kristján Valur Jónsson added the comment:
Yes, I have VC2010 so I'll see what happens...
--
___
Python tracker
<http://bugs.python.org/issue10683>
___
___
Kristján Valur Jónsson added the comment:
Ok the problem is this line in the pre-link step, that must have gotten changed
during the conversion:
"$(SolutionDir)make_buildinfo.exe" Debug "$(IntDir)"
should be
"$(SolutionDir)make_buildinfo.exe" Debug "$(IntDi
Kristján Valur Jónsson added the comment:
Implemented the "trailing quote removal" defensive programming strategy in
revision 87205.
Others that autoconvert the solution will not be hit by this problem.
--
resolution: -> fixed
status: o
Kristján Valur Jónsson added the comment:
Well, I can submit a patch if anyone is interested.
I came across this when writing asynchronous network code. By hanging onto the
Py_buffer, I should have access to the data during the network call. But it
only worked for "true" Py_buff
Kristján Valur Jónsson added the comment:
Uh oh. I forgot about this and there now we have passed beta 2.
Didn't anyone want to review the patch?
--
___
Python tracker
<http://bugs.python.org/is
Kristján Valur Jónsson added the comment:
Well, the idea is good and it did cross my mind. Particularly it could be
useful for performance sensitive applications.
However it complicates things.
1) If a GC is rejected, when do we make the next attempt?
2) If a callback cancels GC, then what
Kristján Valur Jónsson added the comment:
1) what I mean is that if a callback rejects GC, the GC algorithm may find its
condition for invoking GC in the first place to be still valid immediately
afterwards, so doing a GC will be immediately retried. I have to check, but it
could mean that
Kristján Valur Jónsson added the comment:
I _think_ that in our old 2.5 python (which had a backported ctypes from 2.6 to
support 64 bits) we always got the JIT debugger i.e. with _ctypes.pyd and
_ctypes_d.pyd.
This api, "DebugBreak" always invokes the JIT debugger, however the p
Kristján Valur Jónsson added the comment:
Bizarre indeed.
I think more quotes is the answer, since it is simpler to implement.
But the question remains, why has it worked until now?
--
___
Python tracker
<http://bugs.python.org/issue11
New submission from Kristján Valur Jónsson <[EMAIL PROTECTED]>:
In _PyState_AddModule(), a list of (initially) 20 elements is created,
but there is no guarantee that all elements are initialized. In
particular, it appears that element 0 is always NULL. This is bad
since this l
New submission from Kristján Valur Jónsson <[EMAIL PROTECTED]>:
If a PyTokenizer_FromString() is called with an empty string, the
tokenizer's line_start member never gets initialized. Later, it is
compared with the token pointer 'a' in parsetok.c:193 and that behavior
ca
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
Added a patch for this issue. Please comment.
--
keywords: +easy, patch
Added file: http://bugs.python.org/file10899/tmp2.patch
___
Python tracker <[EMAIL PROTECTE
New submission from Kristján Valur Jónsson <[EMAIL PROTECTED]>:
Failure to deallocate a buffer returned by PyArg_ParseTuple. Patch is
attached.
Found using purify while running the testsuite.
--
components: Interpreter Core
files: tmp3.patch
keywords: easy, patch, patch
me
New submission from Kristján Valur Jónsson <[EMAIL PROTECTED]>:
When a nan or inf is generated in PyFloat_FromString(), a temporary
buffer may be left behind.
Patch attached.
Found using Purify
--
components: Interpreter Core
files: tmp4.patch
keywords: easy, patch, patch
me
Changes by Kristján Valur Jónsson <[EMAIL PROTECTED]>:
--
type: -> resource usage
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.py
Changes by Kristján Valur Jónsson <[EMAIL PROTECTED]>:
--
versions: +Python 3.0
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3367>
___
New submission from Kristján Valur Jónsson <[EMAIL PROTECTED]>:
Purify complained about reading uninitialized memory in ast.c:752 of
two bytes which corresponds to the type field. Looking into this, line
750 increments i without checking that there are in fact this many
children.
If y
New submission from Kristján Valur Jónsson <[EMAIL PROTECTED]>:
In some cases, an error string generated by parsetok.c is not cleared
by err_input(). A patch is provided.
--
components: Interpreter Core
files: tmp5.patch
keywords: patch, patch
messages: 69788
nosy: krisvale
se
New submission from Kristján Valur Jónsson <[EMAIL PROTECTED]>:
The CryptGenRandomBytes uses whatever data is already in the buffer as
seed for the output. So, the buffer is effectively an in/out buffer.
Now, since we are generating random data anyway, the fact that we are
us
New submission from Kristján Valur Jónsson <[EMAIL PROTECTED]>:
Here is a suggested update to thread_nt.c. It has two significant
changes:
1) it uses the new and safer _beginthreadex API to start a thread
2) it implements native TLS functions on NT, which are certain to be as
fast as po
New submission from Kristján Valur Jónsson <[EMAIL PROTECTED]>:
When executing a script from a UNC path, e.g. //myhost/exports/a.py,
r"\\myhost\exports" gets prepended to sys.path. But it doesn't work.
This means that in a.py, "import b" will fail even thou
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
Correction: The workaround is to _append_ a backslash.
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.pytho
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
Now that the 'easy' keyword is absent, I'm afraid this is out of my
depth.
I'll run purify again and try to find the exact repro case.
___
Python tracker
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
No, not really.
Again, I refer to defect 954115 by glchapman. And note that those
functions added there are actually not used.
I was hoping that there was someone here more familiar with importing
on PC.
I'll go and see
New submission from Kristján Valur Jónsson :
The code.InteractiveConsole() is useful to emulate a python console. However,
any code currently "push"ed to it must be single statements. This is because
it passes the ´single´ symbol mode to the underlying compile function.
This patch
Kristján Valur Jónsson added the comment:
Note, there are no regression tests for the code module.
--
___
Python tracker
<http://bugs.python.org/issue7
Kristján Valur Jónsson added the comment:
Here is how to test this manually:
from code import InteractiveConsole
c = InteractiveConsole()
s = "if True:\n print 1\nprint 2"
c.push(s) #fails
c.push(s, "exec") #succeeds
--
___
P
New submission from Kristján Valur Jónsson :
The tp_dealloc of a type can chose to resurrect an object. the
subtype_dealloc() in typeobject.c does this when it calls the tp_del() member
and it has increased the refcount.
The problem is, that if you subclass a custom C object, and that C
New submission from Kristján Valur Jónsson :
This patch does several things:
1) Creates a separate lock type PyThread_type_gil and locking functions for
that. This allows tweaking of the GIL without affecting regular lock behaviour.
2) Creates a uniform implementation of the GIL on windows
Kristján Valur Jónsson added the comment:
Martin: Well, this patch was originally conceived more as a demonstration of
the GIL problem and an alternative fix proposal.
However, it is possible to configure it so that there is no change from
existing functionality, simply by not including
Kristján Valur Jónsson added the comment:
Antoine: Please take a look, the change is really simple, particularly the
ROUNDROBIN_GIL variant which fixes the originally observed problem.
the GIL is still a lock, implemented using a mutex and a semaphore. It is
modified to work exactly as the
Kristján Valur Jónsson added the comment:
Sorry, what I meant with the "original problem" was the phenomenon observed by
Antoine (IIRC) that the same CPU thread tends to hog the gil, even when
releaseing it in ceval.c.
What I have been looking at up to now is chiefly IO perform
Kristján Valur Jónsson added the comment:
I just did some profiling. I´m using visual studio team edition which has some
fancy built in profiling. I decided to compare the performance of the
iotest.py script with two cpu threads, running for 10 seconds with processor
affinity enabled and
Kristján Valur Jónsson added the comment:
The counter is "stall cycles".
During the 10 second run on my 2.4Ghz cpu, we had instruction cache miss stalls
for 2 billion cycles (2000 samples of 100 cycles per sample). That does
account for around 10% of the availible cpu.
I&#
Kristján Valur Jónsson added the comment:
David, yes messing about with processor affinities is certainly not nice.
Especially since the issue is cross-platform.
The pthreads api doesn't offer much. There is pthreadd_setschedparam(), and
pthreads_setconcurrency(). Unfortunately I don&#
Kristján Valur Jónsson added the comment:
I looked at ccbench. It's a great tool. I've added two features to it (see
the attached patch)
-y option to turn off the "do_yield" option in throughput, and so measure
thread scheduling without assistance, and the throug
Kristján Valur Jónsson added the comment:
Fyi, here is the output using the unmodified Windows GIL, i.e. without my patch
being active:
C:\pydev\python\trunk\PCbuild>python.exe ..\Tools\ccbench\ccbench.py -t -y
== CPython 2.7a4+.0 (trunk) ==
== AMD64 Windows on 'Intel64 Family 6
Kristján Valur Jónsson added the comment:
David, I don't necessarily think it is reasonable to yield every 100 opcodes,
but that is the _intent_ of the current code base. Checkinterval is set to 100.
If you don't want that, then set it higher. Your statement is like saying:
&quo
Kristján Valur Jónsson added the comment:
Maybe the state of this discussion is my fault for not being clear enough.
Let's abandon terms such as "broken" and "roundrobin." CS theory has the
perfectly useful terms "fair" and "unfair." Th
Kristján Valur Jónsson added the comment:
Here is yet another point:
if _POSIX_SEMAPHORES is defined, thread_pthread.h is designed to use the (fair)
semaphore. If it is not present, or HAVE_BROKEN_POSIX_SEMAPHORES defined, the
semaphore is supposed to be emulated using a condition variable
Kristján Valur Jónsson added the comment:
Oh dear. I was assuming that the mutex+condition variable were the actual
implementation mostly in use on pthreads. This is because of David's GIL open
talk at pycon, where we were looking at the source and bickering about the
placeme
Kristján Valur Jónsson added the comment:
Yes, we put #error in both places (defining and undefining USE_SEMAPHORES).
The colleague in question is Christian Tismer, he is unlikely to have gotten it
wrong. I am also curious why David Beazley kept talking about the "binary
semaphore&quo
Kristján Valur Jónsson added the comment:
You do realize, that if we enable the USE_SEMAPHORE, we get the GIL behaviour
as seen on windows and with my ROUNDROBIN_GIL implementation, right?
Also, at the GIL open space talk on PyCon, David did show us the "emulation"
source code as
Kristján Valur Jónsson added the comment:
David, I urge you to reconsider:
The "emulated" semaphore is broken because it is unfair. It is clearly a
programming error, born out of naivete about how to implement such primitives.
Proper semaphores therefore cannot be implemented
Kristján Valur Jónsson added the comment:
Googling a bit gave me this:
http://lists.apple.com/archives/darwin-kernel/2005/Dec/msg00022.html
It would appear that mac os X was at least lacking full posix semaphore support
in 2005.
--
___
Python
New submission from Kristján Valur Jónsson :
On pthreads plaforms, if the posix_sem functions aren't available (when
_POSIX_SEMAPHORES isn't defined), the python lock is implemented with a mutex
and a condition variable. This appears to be the case on Mac OS X, for example.
The
New submission from Kristján Valur Jónsson :
As per Antoine's suggestion, here is a patch to improve condition variable
emulation on windows.
By using the windows Semaphore (which hasn't always been available) all of the
problems in emulating condition variables using Events disa
Kristján Valur Jónsson added the comment:
Martin, it isn't the condition variable which is unfair, but how the
constructed locking apparatus is unfair that is the problem. This lock is
written by a Tim, whom I assume to be Tim Peters. I quote his comment:
"In general, if the
Kristján Valur Jónsson added the comment:
What your fair.py is doing is demonstrating the superior behaviour of a
time-based GIL interrupt to a bytecode based one. I have no quibbles with that
and I agree that it is superior. But I also think that your example is a very
artificial one. On
Kristján Valur Jónsson added the comment:
>I'm not trying to be a pain here, but do you have any explanation as to >why,
>with fair scheduling, the observed execution time of multiple CPU->bound
>threads is substantially worse than with unfair scheduling?
Yes. This is b
Kristján Valur Jónsson added the comment:
Martin, I´ve explained it in my other dissue, issue 8411, with a step by step
example.
It is unfair because a thread can _bypass_ the condition variable. A thread
just woken up from the condition variable has to race to get the lock, and it
is a
Kristján Valur Jónsson added the comment:
Is it possible that unistd.h isn't included by Python on mac builds? perhaps
the config script is broken and HAVE_UNISTD_H doesn't get defined. I'll have a
look at the generated pyconfig.h file on my colleague
Kristján Valur Jónsson added the comment:
Also, _POSIX_SEMAPHORES must be defined to be greater than 200112L. If it
isn't, then it isn't supported.
--
___
Python tracker
<http://bugs.python.
Kristján Valur Jónsson added the comment:
In 2), B is indeed signaled and the OS makes it "runnable". But it doesn´t run
immediately. A is still running. There is no need for A to stop running until
it runs out of timeslice. Meanwhile the OS has to put B on a separate core
(w
Kristján Valur Jónsson added the comment:
Sorry Martin, I meant issue 8410.
I have so many of these going on :)
--
___
Python tracker
<http://bugs.python.org/issue8
Kristján Valur Jónsson added the comment:
It is interesting to see, David, the difference in the behaviour of the
semaphore based and condition variable based lock on linux. It is clear that
the semaphore and the condition varable have different queuing characteristics.
I wouldn'
Kristján Valur Jónsson added the comment:
David, trying to get some more realistic IO benchmarks I did some more tests.
The idea is to have a threaded socket server, serving requests that take
different amounts of time to process, and see how io response measures up for
two classes of
Kristján Valur Jónsson added the comment:
Sorry, all the benchmarks were missing from my last comment. Here they are:
total time avg time/request stddev
LEGACY_GIL
serial
((30, 500), (2.6908777512225717, 0.08968708313486773
New submission from Kristján Valur Jónsson <[EMAIL PROTECTED]>:
This adds a special case for min() and max() when iterating over lists.
For simple lists of floats, the improvement is some 15% on a windows
machine using release build (non pgo)
--
components: Interpreter Core
New submission from Kristján Valur Jónsson <[EMAIL PROTECTED]>:
Comparing _heapq with our own legacy C implementation, blue.heapq at
CCP, I noticed that ours was somewhat faster.
I discovered that a lot of effort is being spent to dynamically search
for a __lt__ operator, to provide bac
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
I am sorry for not doing my research about the age of the compatibility
fix.
However, modifying the test slightly to work with tuples of
(random.random(), random.random())
shows a performance increase from:
heapify 0.366187
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
Okay, i add a new patch that includes tests.
I am unsure where I should add documentation for the Py_AddPendingCall
() function. Any suggestions?
Added file: http://bugs.python.org/file11978/pendingalls
Changes by Kristján Valur Jónsson <[EMAIL PROTECTED]>:
Removed file: http://bugs.python.org/file11974/pendingalls.patch
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.pytho
New submission from Kristján Valur Jónsson <[EMAIL PROTECTED]>:
At CCP We have started using the Py_AddPendingCall() mechanism to
signal python about a completed IO operation.
However, we noticed that the existing mechanism was hoplelessly un-
thread safe. This is bad, since on Wind
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
Good point. I'll make a test using ctypes and _testcapimodule.c to try
to make it as non-platform-specific as possible.
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
Right.
Isn't the best way to avoid it to block signals just while we pop the
queue? I'll see if python has a portable sigaction thing to do that.
Otherwise we'd have to start to mess with a "volatile busy
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
Here is a revised version.
Since there is no portable way to block signals, and no way at all on
windows (apparently) the simplest way is to simply use NOWAIT_LOCK when
adding a new pending call. While this does not guarante
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
bump.
Is there any interest in this fix?
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.pytho
New submission from Kristján Valur Jónsson <[EMAIL PROTECTED]>:
There are two performance problems in xmlrpclib.py:
POST requests use two send() calls, one to send the headers and one to
send the data. This can invoke the Nagle/Delayed ACK performance
problem. On many socket implement
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
Small refinement: Deadlock avoidance is only needed on the main thread
since we only install signal handlers for the main thread.
Added file: http://bugs.python.org/file12031/pendingalls
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
I had forgotten that locks aren't initialized until PyEval_InitThreads
() is called. Now we check for this case. Also reinitialize when fork
() is called.
Is there any suggested place where I should add documentation to t
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
I turns out that for SIGINT, windows creates a new thread. So, the
assumption that signals are delivered on the main thread are not valid
in general. Therefore, we must initialize the lock pending_lock
independen
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
Just a thought here:
Maybe it would be better just to change socket._fileobject to always
use a minimum of 8k readbuffer for readline() just as read() already
does. The read() documentation states that recv(1) is very
ineff
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
I have addressed some issues mentioned:
1) I have retained the _send_output() method.
2) the endheaders() method now takes an optional argument, send_data
that defaults to True. It also returns any unsent data as a string.
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
Sorry, I meant : "you may have concerns regarding point 2) above"
___
Python tracker <[EMAIL PROTECTED]>
<http://
Kristján Valur Jónsson <[EMAIL PROTECTED]> added the comment:
I like the suggestion of having endheaders() accept any extra data.
I have uploaded a new patch which implements this idea. It simplifies
things a lot.
The issue with the read buffer size is still open. I have sent an
em
101 - 200 of 950 matches
Mail list logo
