[issue7249] Consider allowing io.BytesIO sizes to be passed as 'long' in 2.6
New submission from Ryan Leslie ryle...@gmail.com: py StringIO.StringIO(foo).read(long(1)) 'f' py io.BytesIO(foo).read(long(1)) Traceback (most recent call last): File stdin, line 1, in module TypeError: integer argument expected, got 'long' This is known to cause problems when reading zip data from a BytesIO object with zipfile. See this recent thread on comp.lang.python: http://groups.google.com/group/comp.lang.python/browse_thread/thread/337c1b8a48e8acae/ -- components: Library (Lib) messages: 94818 nosy: ryles severity: normal status: open title: Consider allowing io.BytesIO sizes to be passed as 'long' in 2.6 type: behavior versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7249 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7169] zipfile leaves a file handle open if file is zero size
Ryan Leslie ryle...@gmail.com added the comment: Yes, I think this fix should have been included in the 2.6 branch. I subscribed Amaury to look into that when I last updated. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7169 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7169] zipfile leaves a file handle open if file is zero size
Ryan Leslie ryle...@gmail.com added the comment: I expect this should already be fixed by the commit in http://bugs.python.org/issue6511 BadZipFile will now be raised for empty files rather than IOError, and so ZipFile._GetContents() should now also close the file. The fix was committed to trunk, but I don't see it merged into 2.6. -- nosy: +amaury.forgeotdarc, ryles ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7169 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7157] Fix Download Current Documentation link
New submission from Ryan Leslie ryle...@gmail.com: http://docs.python.org/download.html shows this: Download Python 2.6.4c1 Documentation We don't package the documentation for development releases for download. Downloads will be available for the final release. This is not really acceptable for someone navigating through python.org to the Documentation page. The latest 2.6 reference, tutorial, etc, are all available, and so the archives with the latest PDF versions should be too. -- assignee: georg.brandl components: Documentation messages: 94158 nosy: georg.brandl, ryles severity: normal status: open title: Fix Download Current Documentation link versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7157 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7052] from logging import * causes an error under Ubuntu Karmic
Ryan Leslie ryle...@gmail.com added the comment: Looks like a merge has gone bad. NullHandler has existed for a while on trunk but is not present in the 2.6.3 tag (__all__ was updated to include it, however): /python/tags/r263/Lib/logging/__init__.py -- nosy: +ryles ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7052 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6759] zipfile.ZipExtFile.read() is missing universal newline support
Ryan Leslie ryle...@gmail.com added the comment: Hi Art, Thanks for working on this. I've taken a look at the patch. The fix to read_test looks correct. Of course, I would consider a more descriptive variable name than 'b'. The changes to read() are an improvement, but I think we need to be careful when we replace \r\n with \n. Basically, we've turned two characters into one and are now potentially one character short of 'size' bytes. This doesn't match the behavior of file.read(). Another thing to work out is the lack of the 'newlines' attribute, discussed in PEP 278. I've noted that bz2 seems to do a pretty good job with universal newline handling: python/trunk/Modules/bz2module.c. It's in C, however, and I don't think there's actually anything in the library doing UL in pure Python. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6759 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6759] zipfile.ZipExtFile.read() is missing universal newline support
New submission from Ryan Leslie ryle...@gmail.com: The zipfile.ZipFile.open() behavior with mode 'U' or 'rU' is not quite as advertised in http://docs.python.org/library/zipfile.html#zipfile.ZipFile.open Here is an example: $ echo -ne This is an example\r\nWhich demonstrates a problem\r\nwith ZipFile.open(..., 'U')\r\n foo.txt $ cat -v foo.txt This is an example^M Which demonstrates a problem^M with ZipFile.open(..., 'U')^M $ zip foo.zip foo.txt adding: foo.txt (deflated 1%) $ python Python 2.6.2 (r262:71600, Aug 21 2009, 17:52:12) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type help, copyright, credits or license for more information. open(foo.txt, 'U').read() This is an example\nWhich demonstrates a problem\nwith ZipFile.open(..., 'U')\n from zipfile import ZipFile ZipFile(foo.zip).open(foo.txt, 'U').read() This is an example\r\nWhich demonstrates a problem\r\nwith ZipFile.open(..., 'U')\r\n The open() method was added here: http://bugs.python.org/issue1121142 The cause is that the universal newline implementation is specific to readline(), which also implements readlines() and next() as well. Support was never added for read(), which is independent. Note that test_zipfile.UniversalNewlineTests.readTest() passes. This is suspect because it's explicitly coded to *not* expect translation of new line sequences. -- components: Library (Lib) messages: 91854 nosy: ryles severity: normal status: open title: zipfile.ZipExtFile.read() is missing universal newline support versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6759 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6362] multiprocessing: handling of errno after signals in sem_acquire()
New submission from Ryan Leslie ryle...@gmail.com:
While developing an application, an inconsistency was noted where,
depending on the particular signal handler in use,
multiprocessing.Queue.put() may (or may not) raise OSError() after
sys.exit() was called by the handler. The following example, which was
tested with Python 2.6.1 on Linux, demonstrates this.
#!/usr/bin/env python
import multiprocessing
import signal
import sys
def handleKill(signum, frame):
#sys.stdout.write(Exit requested by signal.\n)
print Exit requested by signal.
sys.exit(1)
signal.signal(signal.SIGTERM, handleKill)
queue = multiprocessing.Queue(maxsize=1)
queue.put(None)
queue.put(None)
When the script is run, the process will block (as expected) on the
second queue.put(). If (from another terminal) I send the process
SIGTERM, I consistently see:
$ ./q.py
Exit requested by signal.
$
Now, if I modify the above program by commenting out the 'print', and
uncommenting the 'sys.stdout' (a very subtle change), I would expect
the result to be the same when killing the process. Instead, I
consistently see:
$ ./q.py
Exit requested by signal.
Traceback (most recent call last):
File ./q.py, line 15, in module
queue.put(None)
File python2.6/multiprocessing/queues.py, line 75, in put
if not self._sem.acquire(block, timeout):
OSError: [Errno 0] Error
$
After debugging this further, the issue appears to be in
semlock_acquire() or semaphore.c in Modules/_multiprocessing:
http://svn.python.org/view/python/trunk/Modules/_multiprocessing/semaphore.c?revision=71009view=markup
The relevant code from (the Unix version of) semlock_acquire() is:
do {
Py_BEGIN_ALLOW_THREADS
if (blocking timeout_obj == Py_None)
res = sem_wait(self-handle);
else if (!blocking)
res = sem_trywait(self-handle);
else
res = sem_timedwait(self-handle, deadline);
Py_END_ALLOW_THREADS
if (res == MP_EXCEPTION_HAS_BEEN_SET)
break;
} while (res 0 errno == EINTR !PyErr_CheckSignals());
if (res 0) {
if (errno == EAGAIN || errno == ETIMEDOUT)
Py_RETURN_FALSE;
else if (errno == EINTR)
return NULL;
else
return PyErr_SetFromErrno(PyExc_OSError);
}
In both versions of the program (print vs. sys.stdout), sem_wait() is
being interrupted and is returning -1 with errno set to EINTR. This is
what I expected. Also, in both cases it seems that the loop is
(correctly) terminating with PyErr_CheckSignals() returning non-zero.
This makes sense too; the call is executing our signal handler, and then
returning -1 since our particular handler raises SystemExit.
However, I suspect that depending on the exact code executed
for the signal handler, errno may or may not wind up being reset in
some nested call of PyErr_CheckSignals(). I believe that the
error checking code below the do-while (where sem_wait() is called),
needed errno to have the value set by sem_wait(), and the author
wasn't expecting anything else to have changed it. In the print
version, errno apparently winds up unchanged with EINTR, resulting in
the `return NULL' statement. In the sys.stdout version (and probably
many others), errno winds up being reset to 0, and the error handling
results in the `return PyErr_SetFromErrno(PyExc_OSError)' statement.
To patch this up, we can probably just save errno as, say, `wait_errno'
at the end of the loop body, and then use it within the error handling
block that follows. However, the rest of the code should probably be
checked for this type of issue.
--
components: Library (Lib)
messages: 89804
nosy: ryles
severity: normal
status: open
title: multiprocessing: handling of errno after signals in sem_acquire()
type: behavior
versions: Python 2.6, Python 2.7, Python 3.0, Python 3.1, Python 3.2
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6362
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6362] multiprocessing: handling of errno after signals in sem_acquire()
Changes by Ryan Leslie ryle...@gmail.com: -- nosy: +jnoller ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6362 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6082] os.path.sameopenfile reports that standard streams are the same
New submission from Ryan Leslie ryle...@gmail.com: Python 2.6.1 (r261:67515, Apr 2 2009, 18:25:55) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type help, copyright, credits or license for more information. import sys, os os.path.sameopenfile(sys.stdin.fileno(), sys.stdout.fileno()) True os.path.sameopenfile(sys.stdout.fileno(), sys.stderr.fileno()) True null = open(os.devnull) os.path.sameopenfile(sys.stdin.fileno(), null.fileno()) False # That worked. -- components: Library (Lib) messages: 88174 nosy: ryles severity: normal status: open title: os.path.sameopenfile reports that standard streams are the same type: behavior versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6082 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6082] os.path.sameopenfile reports that standard streams are the same
Ryan Leslie ryle...@gmail.com added the comment:
Thanks for the quick response, Philip. Makes even more sense now that I see:
Python 2.6.1 (r261:67515, Apr 2 2009, 18:25:55)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
Type help, copyright, credits or license for more information.
open('/dev/stdout').readline()
hello
'hello\n'
open('/dev/stdin').readline()
hello
'hello\n'
--
status: closed - open
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6082
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6082] os.path.sameopenfile reports that standard streams are the same
Changes by Ryan Leslie ryle...@gmail.com: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6082 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager
New submission from Ryan Leslie ryle...@gmail.com: Terminal 1: Python 2.6.1 (r261:67515, Apr 2 2009, 18:25:55) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type help, copyright, credits or license for more information. from multiprocessing.managers import SyncManager manager = SyncManager(authkey=mykey) manager.start() queue = manager.Queue() import pickle pickle.dump(queue, open(myqueue.pkl, w)) Terminal 2: Python 2.6.1 (r261:67515, Apr 2 2009, 18:25:55) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type help, copyright, credits or license for more information. import socket socket.setdefaulttimeout(30) import multiprocessing, pickle multiprocessing.current_process().authkey = mykey queue = pickle.load(open(myqueue.pkl)) Traceback (most recent call last): File stdin, line 1, in module File python2.6/pickle.py, line 1370, in load return Unpickler(file).load() File python2.6/pickle.py, line 858, in load dispatch[key](self) File python2.6/pickle.py, line 1133, in load_reduce value = func(*args) File python2.6/multiprocessing/managers.py, line 845, in RebuildProxy return func(token, serializer, incref=incref, **kwds) File python2.6/multiprocessing/managers.py, line 894, in AutoProxy incref=incref) File python2.6/multiprocessing/managers.py, line 700, in __init__ self._incref() File python2.6/multiprocessing/managers.py, line 749, in _incref conn = self._Client(self._token.address, authkey=self._authkey) File python2.6/multiprocessing/connection.py, line 140, in Client answer_challenge(c, authkey) File python2.6/multiprocessing/connection.py, line 376, in answer_challenge response = connection.recv_bytes(256)# reject large message IOError: [Errno 11] Resource temporarily unavailable This works as expected without socket.setdefaulttimeout(). However, the timeout is useful since if the listening process on terminal 1 goes to sleep, e.g. ^Z, it would avoid blocking. I suspect the cause is similar to http://bugs.python.org/issue976613 -- components: Library (Lib) messages: 88040 nosy: ryles severity: normal status: open title: socket.setdefaulttimeout affecting multiprocessing Manager versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6056 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager
Ryan Leslie ryle...@gmail.com added the comment: Yeah, storing pickled queues in the file system makes for some easy IPC :) It wasn't a very original idea, I took the pickling comments in the documentation at face value: http://docs.python.org/library/multiprocessing.html#proxy-objects So, from what I can tell this issue is related to the mixing of standard python socket I/O with multiprocessing socket I/O, with state not being carried from the former to the latter. In multiprocessing/connection.py, SocketClient() creates a familiar python socket object which, when a default timeout has been set in the module, will be made non-blocking. In addition, the timeout is remembered in the socket object, and when calling socket.recv(), the function internal_select() will use this to perform the expected poll() or select(). However, after a connection is established, SocketClient() will not use python's socket implementation any further, and instead pass its low-level socket descriptor to a multiprocessing Connection object. This object has its own specialized socket I/O implementation, which is not at all aware of the timeout previously associated with the socket. As a result no select/poll occurs and, due to the socket's non-blocking status, recv() may return EAGAIN immediately. I suspect this is what's happening. There might be a number of ways to make SocketClient() more timeout friendly, but possibility could be to simply check if the python socket has a timeout associated, and if so, use connection.poll() in addition to connection.recv(). There may be other places in the code where similar changes would occur. You obviously have more experience with this code base so I'll be curious to see the outcome. -- type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6056 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5979] strptime() gives inconsistent exceptions
New submission from Ryan Leslie ryle...@gmail.com: e.g. from datetime import datetime datetime.strptime(19951001, %Y%m%d) datetime.datetime(1995, 10, 1, 0, 0) datetime.strptime(19951000, %Y%m%d) # day = 0, month 11 ... ValueError: time data '19951000' does not match format '%Y%m%d' datetime.strptime(19951100, %Y%m%d) # day = 0, month = 11 ... ValueError: unconverted data remains: 0 The exception messages are not really a serious issue, but note that the latter one can be confusing for users. However, there seems to be some underlying issues with the choice to recognize single digit months with double digit days, which can make date strings ambiguous: Consider 19951100 from above with the last '0' removed. datetime.strptime(1995110, %Y%m%d) datetime.datetime(1995, 1, 10, 0, 0) In this case, strptime has treated the middle '1' as the month, resulting in 1995-01-10. This hints at why the second exception from above gives a strange message: with the extra '0' the day portion of 19951100 (00) is invalid, and strptime falls back on parsing the first 7 characters as above, and then failing due to the remaining '0'. This seems a little dangerous. For instance: timestamp = 19951101-23:20:18 datestamp=timestamp[:7] # Oops, needed to use :8 to get up to index 7. reallyImportantButWrongDate = datetime.strptime(datestamp, %Y%m%d) Note that in this case strptime() from glibc would instead result in an error, which IMHO is the right thing to do. I do realize, though, that changing the behavior of strptime() could devastate some existing code. -- components: Library (Lib) messages: 87505 nosy: ryles severity: normal status: open title: strptime() gives inconsistent exceptions type: behavior versions: Python 2.4, Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5979 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5971] logging.Handler.handlerError() may raise IOError in traceback.print_exception()
New submission from Ryan Leslie ryle...@gmail.com: When using the logging package, if a StreamHandler is configured with stderr and stderr is redirected to a pipe which no longer has readers, then StreamHandler.emit() will result in an IOError for Broken pipe. The exception will be handled in logging.Handler.handleError(), which by default will call traceback.print_exception() with file=sys.stderr. This will cause in a second IOError exception which will not be caught within the logging code. Unless the user placed their log calls such as logging.info() in a try/except block (unlikely), the end result is termination of the process. While the logging code itself is certainly not the cause of the underlying problem, it does seem that the intent of the default handleError() was to eat exceptions, and possibly print them, without disturbing the application code. As the docstring correctly points out, the application can probably survive without the logging. To work around this issue without writing a custom handler, raiseExceptions can be set to false. But then the user would miss log trace when other types of errors not affecting stderr occurred. That is, I think handleError() does the right thing in trying to print the error, but suggest that if the print results in an IOError (or certain types of IOError), then it should be ignored, or optionally ignorable. -- components: Library (Lib) messages: 87469 nosy: ryles severity: normal status: open title: logging.Handler.handlerError() may raise IOError in traceback.print_exception() type: behavior versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5971 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
