[Python-Dev] _PyGILState_NoteThreadState should be static or not?
Michael, In Python/pystate.c, you made this checkin: """ r39044 | mwh | 2005-06-20 12:52:57 -0400 (Mon, 20 Jun 2005) | 8 lines Fix bug: [ 1163563 ] Sub threads execute in restricted mode basically by fixing bug 1010677 in a non-broken way. """ _PyGILState_NoteThreadState is declared as static on line 54, but the definition on line 508 is not static. (HP's cc is complaining.) I don't see this referenced in any header file, it seems like this should be static? $ grep _PyGILState_NoteThreadState */*.ch] Python/pystate.c:static void _PyGILState_NoteThreadState(PyThreadState* tstate); Python/pystate.c: _PyGILState_NoteThreadState(tstate); Python/pystate.c: _PyGILState_NoteThreadState(t); Python/pystate.c:_PyGILState_NoteThreadState(PyThreadState* tstate) n ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] _PyGILState_NoteThreadState should be static or not?
On 11 Sep 2006, at 09:34, Neal Norwitz wrote: > Michael, > > In Python/pystate.c, you made this checkin: > > """ > r39044 | mwh | 2005-06-20 12:52:57 -0400 (Mon, 20 Jun 2005) | 8 lines > > Fix bug: [ 1163563 ] Sub threads execute in restricted mode > basically by fixing bug 1010677 in a non-broken way. > """ > > _PyGILState_NoteThreadState is declared as static on line 54, but the > definition on line 508 is not static. (HP's cc is complaining.) I > don't see this referenced in any header file, it seems like this > should be static? It seems very likely, yes. I think at one point (in my working copy) there was a call in Modules/threadmodule.c, which may partially account for my confusion. Seems we have lots of HP users tracking SVN HEAD, then... Cheers, mwh ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] BRANCH FREEZE: release25-maint, 00:00UTC 12 September 2006
Ok, I haven't heard back from Martin, but I'm going to hope he's OK with tomorrow as a release date for 2.5rc2. If he's not, we'll try for the day after. In any case, I'm going to declare the release25-maint branch FROZEN as at 00:00 UTC on the 12th. That's about 12 hours from now. This is for 2.5rc2. Once this is out, I'd like to see as close to zero changes as possible for the next week or so until 2.5 final is released. My god, it's getting so close... Anthony -- Anthony Baxter <[EMAIL PROTECTED]> It's never too late to have a happy childhood. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
On 9/11/06, Adam Olsen <[EMAIL PROTECTED]> wrote: > Now on to my new proposal. I do still use write(). If you can't > accept that I think we should rip signals out entirely, just let them > kill the process. Not a reliable feature of any OS. > > We create a single pipe and use it for all signals. We never release > it, instead letting the OS do it when the process gets cleaned up. We > write the signal number to it as a byte (assuming there's at most 256 > unique signals). > > This much would allow a GUI's poll loop to wake up when there is a > signal, and give control back to the python main loop, which could > then read off the signals and queue up their handler functions. I like this approach. Not only we would get a poll-able file descriptor to notify a GUI main loop when signals arrive, we'd also avoid the lack of async safety in Py_AddPendingCall / Py_MakePendingCalls which affects _current_ Python code. Note that the file descriptor of the read end of the pipe has to become a public Python API so that 3rd party extensions may poll it. This is crucial. > > The only problem is when there is no GUI poll loop. We don't want > python to have to poll the fd, we'd rather it just check a variable. > Is it possible to set/clear a flag in a sufficiently portable > (reentrant-safe, non-blocking, thread-safe) fashion? It's simple. That pipe file descriptor has to be changed to non-blocking mode in both ends of the pipe, obviously, with fcntl. Then, to find out whether a signal happened or not we modify PyErr_CheckSignals() to try to read from the pipe. If it reads bytes from the pipe, we process the corresponding python signal handlers or raise KeyboardInterrupt. If the read() syscall returns zero bytes read, we know no signal was delivered and move on. The only potential problem left is that, by changing the pipe file descriptor to non-blocking mode we can only write as many bytes to it without reading from the other side as the pipe buffer allows. If a large number of signals arrive very quickly, that buffer may fill and we lose signals. But I think the default buffer should be more than enough. And normally programs don't receive lots of signals in a small time window. If it happens we may lose signals, but that's very rare, and who cares anyway. Regards. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] datetime's strftime implementation: by design or bug
[I hope this belongs on python-dev, since it's about the design of
something. But if not, let me know and I'll post to c.l.py.]
I'm willing to file a bug report and patch on this, but I'd like to know
if it's by design or not.
In datetimemodule.c, the function wrap_strftime() insists that the
length of a format string be <= 127 chars, by forcing the length into a
char. This seems like a bug to me. wrap_strftime() calls time's
strftime(), which doesn't have this limitation because it uses size_t.
>>> import datetime
>>> datetime.datetime.now().strftime('x'*128)
Traceback (most recent call last):
File "", line 1, in ?
MemoryError
>>> import datetime
>>> datetime.datetime.now().strftime('x'*256)
in wrap_strftime
totalnew=1
Traceback (most recent call last):
File "", line 1, in
SystemError: Objects/stringobject.c:4077: bad argument to internal function
>>> import time
>>> time.strftime('x'*128)
''
But before I write this up, I'd like to know if anyone knows if this is
by design or not.
This is reproducible on Windows 2.4.3, and Linux 2.3.3 and 2.5c1.
Thanks.
Eric.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] datetime's strftime implementation: by design or bug
[Eric V. Smith]
> [I hope this belongs on python-dev, since it's about the design of
> something. But if not, let me know and I'll post to c.l.py.]
>
> I'm willing to file a bug report and patch on this, but I'd like to know
> if it's by design or not.
>
> In datetimemodule.c, the function wrap_strftime() insists that the
> length of a format string be <= 127 chars, by forcing the length into a
> char. This seems like a bug to me. wrap_strftime() calls time's
> strftime(), which doesn't have this limitation because it uses size_t.
Yawn ;-) I'm very surprised the code doesn't verify that the format
size fits in a C char, but there's nothing deep about the assumption.
I expect it would work fine to just change the declarations of
`totalnew` and `usednew` from `char` to `Py_ssize_t` (for 2.5.1 and
2.6; to something else for 2.4.4 (I don't recall which C type
PyString_Size returned then -- probably `int`)), and /also/ change the
resize-and-overflow check. The current:
int bigger = totalnew << 1;
if ((bigger >> 1) != totalnew) { /* overflow */
PyErr_NoMemory();
goto Done;
}
doesn't actually make sense even if it's certain than sizeof(int) is
strictly larger than sizeof(totalnew) (which C guarantees for type
`char`, but is plain false on some boxes if changed to Py_ssize_t).
Someone must have been on heavy drugs when writing that endlessly
tedious wrapper ;-)
> ...
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] datetime's strftime implementation: by design or bug
Please log a bug - this is probably something suitable for fixing in 2.5.1. At the very least, if it's going to be limited to 127 characters, it should check that and raise a more suitable exception. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_BuildValue and decref
On Sun, Sep 10, 2006 at 07:35:59PM +1200, Greg Ewing wrote: > Barry Warsaw wrote: > > I just want to point out that the C API documentation is pretty > > silent about the refcounting side-effects in error conditions (and > > often in success conditions too) of most Python functions. For > > example, what is the refcounting side-effects of PyDict_SetItem() on > > val? What about if that function fails? Has val been incref'd or > > not? What about the side-effects on any value the new one replaces, > > both in success and failure? > > The usual principle is that the refcounting behaviour > is (or should be) independent of whether the function > succeeded or failed. In the absence of any statement > to the contrary in the docs, you should be able to > assume that. > > The words used to describe the refcount behaviour of > some functions can be rather confusing, but it always > boils down to one of two cases: either the function > "borrows" a reference (and does its own incref if > needed, the caller doesn't need to care) or it "steals" > a reference (so the caller is always responsible for > doing an incref if needed before calling). > > What that rather convoluted comment about PyTuple_SetItem > is trying to say is just that it *always* steals a reference, > regardless of whether it succeeds or fails. I expect the > same is true of Py_BuildValue. Given that it doesn't seem to be the case, and my quick look at the code indicates that even internally python is inconsistent, should I file a low-severity bug so we don't lose track of this? Misa ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
On 9/11/06, Gustavo Carneiro <[EMAIL PROTECTED]> wrote: > On 9/11/06, Adam Olsen <[EMAIL PROTECTED]> wrote: > > This much would allow a GUI's poll loop to wake up when there is a > > signal, and give control back to the python main loop, which could > > then read off the signals and queue up their handler functions. > > I like this approach. Not only we would get a poll-able file > descriptor to notify a GUI main loop when signals arrive, we'd also > avoid the lack of async safety in Py_AddPendingCall / > Py_MakePendingCalls which affects _current_ Python code. > > Note that the file descriptor of the read end of the pipe has to > become a public Python API so that 3rd party extensions may poll it. > This is crucial. Yeah, so long as Python still does the actual reading. > > The only problem is when there is no GUI poll loop. We don't want > > python to have to poll the fd, we'd rather it just check a variable. > > Is it possible to set/clear a flag in a sufficiently portable > > (reentrant-safe, non-blocking, thread-safe) fashion? > > It's simple. That pipe file descriptor has to be changed to > non-blocking mode in both ends of the pipe, obviously, with fcntl. > Then, to find out whether a signal happened or not we modify > PyErr_CheckSignals() to try to read from the pipe. If it reads bytes > from the pipe, we process the corresponding python signal handlers or > raise KeyboardInterrupt. If the read() syscall returns zero bytes > read, we know no signal was delivered and move on. Aye, but my point was that a syscall is costly, and we'd like to avoid it if possible. We'll probably have to benchmark it though, to find out if it's worth the hassle. > The only potential problem left is that, by changing the pipe file > descriptor to non-blocking mode we can only write as many bytes to it > without reading from the other side as the pipe buffer allows. If a > large number of signals arrive very quickly, that buffer may fill and > we lose signals. But I think the default buffer should be more than > enough. And normally programs don't receive lots of signals in a > small time window. If it happens we may lose signals, but that's very > rare, and who cares anyway. Indeed, we need to document very clearly that: * Signals may be dropped if there is a burst * Signals may be delayed for a very long time, and if you replace a previous handler your new handler may get signals intended for the old handler -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_BuildValue and decref
Mihai Ibanescu wrote: > Given that it doesn't seem to be the case, and my quick look at the code > indicates that even internally python is inconsistent, should I file a > low-severity bug so we don't lose track of this? I'd say so, yes. A function whose refcount behaviour differs when it fails is awkward to use safely at best, impossible at worst (if there's no way of finding out what needs to be decrefed in order to clean up properly).] -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
On 9/11/06, Greg Ewing <[EMAIL PROTECTED]> wrote: > Gustavo Carneiro wrote: > > The only potential problem left is that, by changing the pipe file > > descriptor to non-blocking mode we can only write as many bytes to it > > without reading from the other side as the pipe buffer allows. If a > > large number of signals arrive very quickly, that buffer may fill and > > we lose signals. > > That might be an argument for *not* trying to > communicate the signal number by the value > written to the pipe, but keep a separate set > of signal-pending flags, and just use the pipe > as a way of indicating that *something* has > happened. That brings you back to how you access the flags variable. At best it is very difficult, requiring unique assembly code for every supported platform. At worst, some platforms may not have any way to do it from an interrupt context.. A possible alternative is to keep a set of flags for every thread, but that requires the threads poll their variable regularly, and possibly a wake-up pipe for each thread.. -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
Gustavo Carneiro wrote: > The only potential problem left is that, by changing the pipe file > descriptor to non-blocking mode we can only write as many bytes to it > without reading from the other side as the pipe buffer allows. If a > large number of signals arrive very quickly, that buffer may fill and > we lose signals. That might be an argument for *not* trying to communicate the signal number by the value written to the pipe, but keep a separate set of signal-pending flags, and just use the pipe as a way of indicating that *something* has happened. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
Adam Olsen wrote: > That brings you back to how you access the flags variable. The existing signal handler sets a flag, doesn't it? So it couldn't be any more broken than the current implementation. If we get too paranoid about this, we'll just end up deciding that signals can't be used for anything, at all, ever. That doesn't seem very helpful, although techically I suppose it would solve the problem. :-) My own conclusion from all this is that if you can't rely on writing to a variable in one part of your program and reading it back in another, then computer architectures have become far too clever for their own good. :-( -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
