Re: [Python-Dev] Signals, threads, blocking C functions
Adam Olsen wrote: That brings you back to how you access the flags variable. The existing signal handler sets a flag, doesn't it? So it couldn't be any more broken than the current implementation. If we get too paranoid about this, we'll just end up deciding that signals can't be used for anything, at all, ever. That doesn't seem very helpful, although techically I suppose it would solve the problem. :-) My own conclusion from all this is that if you can't rely on writing to a variable in one part of your program and reading it back in another, then computer architectures have become far too clever for their own good. :-( -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
On 9/12/06, Greg Ewing [EMAIL PROTECTED] wrote: Adam Olsen wrote: That brings you back to how you access the flags variable. The existing signal handler sets a flag, doesn't it? So it couldn't be any more broken than the current implementation. If we get too paranoid about this, we'll just end up deciding that signals can't be used for anything, at all, ever. That doesn't seem very helpful, although techically I suppose it would solve the problem. :-) My own conclusion from all this is that if you can't rely on writing to a variable in one part of your program and reading it back in another, then computer architectures have become far too clever for their own good. :-( They've been that way for a long, long time. The irony is that x86 is immensely stupid in this regard, and as a result most programmers remain unaware of it. Other architectures have much more interesting read/write and cache reordering semantics, and the code is certainly broken there. C leaves it undefined with good reason. My previous mention of using a *single* flag may survive corruption simply because we can tolerate false positives. Signal handlers would write 0x, the poll loop would check if *any* bit is set. If so, write 0x0, read off the fd, then loop around and check it again. If the start of the read() acts as a write-barrier it SHOULD guarantee we don't miss any positive writes. Hmm, if that works we should be able to generalize it for all the other flags too. Something to think about anyway... -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
On 9/12/06, Adam Olsen [EMAIL PROTECTED] wrote:
On 9/12/06, Greg Ewing [EMAIL PROTECTED] wrote:
Adam Olsen wrote:
That brings you back to how you access the flags variable.
The existing signal handler sets a flag, doesn't it?
So it couldn't be any more broken than the current
implementation.
If we get too paranoid about this, we'll just end
up deciding that signals can't be used for anything,
at all, ever. That doesn't seem very helpful,
although techically I suppose it would solve
the problem. :-)
My own conclusion from all this is that if you
can't rely on writing to a variable in one part
of your program and reading it back in another,
then computer architectures have become far
too clever for their own good. :-(
They've been that way for a long, long time. The irony is that x86 is
immensely stupid in this regard, and as a result most programmers
remain unaware of it.
Other architectures have much more interesting read/write and cache
reordering semantics, and the code is certainly broken there. C
leaves it undefined with good reason.
My previous mention of using a *single* flag may survive corruption
simply because we can tolerate false positives. Signal handlers would
write 0x, the poll loop would check if *any* bit is set. If
so, write 0x0, read off the fd, then loop around and check it again.
If the start of the read() acts as a write-barrier it SHOULD guarantee
we don't miss any positive writes.
Why write 0x? Why can't the variable be of a volatile
char type? Assuming sizeof(char) == 1, please don't tell me
architecture XPTO will write the value 4 bits at a time! :P
I see your point of using a flag to avoid the read() syscall most of
the time. Slightly more complex, but possibly worth it.
I was going to describe a possible race condition, then wrote the
code below to help explain it, modified it slightly, and now I think
the race is gone. In any case, the code might be helpful to check if
we are in sync. Let me know if you spot any race condition I missed.
static volatile char signal_flag;
static int signal_pipe_r, signal_pipe_w;
PyErr_CheckSignals()
{
if (signal_flag) {
char signum;
signal_flag = 0;
while (read(signal_pipe_r, signum, 1) == 1)
process_signal(signum);
}
}
static void
signal_handler(int signum)
{
char signum_c = signum;
signal_flag = 1;
write(signal_pipe_w, signum_c, 1);
}
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] datetime's strftime implementation: by design or bug
Anthony Baxter wrote: Please log a bug - this is probably something suitable for fixing in 2.5.1. At the very least, if it's going to be limited to 127 characters, it should check that and raise a more suitable exception. [First time sent from wrong address, sorry if this is a dupe.] Done. The patch is at http://python.org/sf/1557390. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
On 9/12/06, Gustavo Carneiro [EMAIL PROTECTED] wrote:
On 9/12/06, Adam Olsen [EMAIL PROTECTED] wrote:
My previous mention of using a *single* flag may survive corruption
simply because we can tolerate false positives. Signal handlers would
write 0x, the poll loop would check if *any* bit is set. If
so, write 0x0, read off the fd, then loop around and check it again.
If the start of the read() acts as a write-barrier it SHOULD guarantee
we don't miss any positive writes.
Why write 0x? Why can't the variable be of a volatile
char type? Assuming sizeof(char) == 1, please don't tell me
architecture XPTO will write the value 4 bits at a time! :P
Nope. It'll write 32 bits, then break that up into 8 bits :)
Although, at the moment I can't fathom what harm that would cause...
For the record, all volatile does is prevent compiler reordering
across sequence points.
Interestingly, it seems volatile sig_atomic_t is the correct way to
declare a variable for (single-threaded) signal handling. Odd that
volatile didn't show up in any of the previous documentation I read..
I see your point of using a flag to avoid the read() syscall most of
the time. Slightly more complex, but possibly worth it.
I was going to describe a possible race condition, then wrote the
code below to help explain it, modified it slightly, and now I think
the race is gone. In any case, the code might be helpful to check if
we are in sync. Let me know if you spot any race condition I missed.
static volatile char signal_flag;
static int signal_pipe_r, signal_pipe_w;
PyErr_CheckSignals()
{
if (signal_flag) {
char signum;
signal_flag = 0;
while (read(signal_pipe_r, signum, 1) == 1)
process_signal(signum);
}
}
I'd prefer this to be a while (signal_flag) instead, although it
should technically work either way.
static void
signal_handler(int signum)
{
char signum_c = signum;
signal_flag = 1;
write(signal_pipe_w, signum_c, 1);
}
This is wrong. PyErr_CheckSignals could check and clear signal_flag
before you reach the write() call. signal_flag = 1 should come
after.
--
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Subversion 1.4
As many of you probably know: Subversion 1.4 has been released. It is safe to upgrade to this version, even if the repository server (for us svn.python.org) stays at an older version: they can interoperate just fine. There is one major pitfall: Subversion 1.4 changes the format of the working copy file structure (.svn/format goes from 4 to 8). This new format is more efficient: for a Python checkout, it saves about 15MiB (out of 125 MiB). Also, several operations (e.g. svn status) are faster. Subversion performs a silent upgrade of the existing repository on the first operation (I believe on the first modifying operation). However, this new format is not compatible with older clients; you need 1.4 clients to access an upgraded working copy. So if you use the same working copy with different clients (e.g. command line and turtoise, or from different systems through NFS), you either need to upgrade all clients, or else you should stay away from 1.4. Alternatively, you can have different checkouts for 1.3 and 1.4 clients, of course. Just in case you didn't know. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
On 9/12/06, Gustavo Carneiro [EMAIL PROTECTED] wrote:
On 9/12/06, Adam Olsen [EMAIL PROTECTED] wrote:
My previous mention of using a *single* flag may survive corruption
simply because we can tolerate false positives. Signal handlers would
write 0x, the poll loop would check if *any* bit is set. If
so, write 0x0, read off the fd, then loop around and check it again.
If the start of the read() acts as a write-barrier it SHOULD guarantee
we don't miss any positive writes.
PyErr_CheckSignals()
{
if (signal_flag) {
char signum;
signal_flag = 0;
while (read(signal_pipe_r, signum, 1) == 1)
process_signal(signum);
}
}
The more I think about this the less I like relying on read() imposing
a hardware write barrier. Unless somebody can say otherwise, I think
we'd be better of putting dummy
PyThread_aquire_lock/PyThread_release_lock calls in there.
--
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
Nick Maclaren schrieb: (coment by Arjan van de Ven): | afaik the kernel only sends signals to threads that don't have them blocked. | If python doesn't want anyone but the main thread to get signals, it should just | block signals on all but the main thread and then by nature, all signals will go | to the main thread Well, THAT'S wrong, I am afraid! Things ain't that simple :-( Yes, POSIX implies that things work that way, but there are so many get-out clauses and problems with trying to implement that specification that such behaviour can't be relied on. Can you please give one example for each (one get-out clause, and one problem with trying to implement that). I fail to see why it isn't desirable to make all signals occur in the main thread, on systems where this is possible. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
Michael Hudson schrieb: According to [1], all python needs to do to avoid this problem is block all signals in all but the main thread; Argh, no: then people who call system() from non-main threads end up running subprocesses with all signals masked, which breaks other things in very mysterious ways. Been there... Python should register a pthread_atfork handler then, which clears the signal mask. Would that not work? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
