[issue10956] file.write and file.read don't handle EINTR
New submission from Mark Florisson markflorisso...@gmail.com:
In both Python versions EINTR is not handled properly in the file.write and
file.read methods.
- file.write -
In Python 2, file.write can write a short amount of
bytes, and when it is interrupted there is no way to tell how many bytes it
actually wrote. In Python 2 it raises an IOError with EINTR, whereas in Python
3 it
simply stops writing and returns the amount of bytes written.
Here is the output of fwrite with Python 2.7 (see attached files). Note also
how inconsistent the IOError vs OSError difference is:
python2.7 fwrite.py
Writing 10 bytes, interrupt me with SIGQUIT (^\)
^\^\(3, frame object at 0x9535ab4)
Traceback (most recent call last):
File fwrite.py, line 16, in module
print(write_file.write(b'a' * 10))
IOError: [Errno 4] Interrupted system call
read 65536 bytes
^\(3, frame object at 0x9535ab4)
Traceback (most recent call last):
File fwrite.py, line 21, in module
print('read %d bytes' % len(os.read(r, 10)))
OSError: [Errno 4] Interrupted system call
Because os.read blocks on the second call to read, we know that only 65536 of
the 10 bytes were written.
- file.read -
When interrupting file.read in Python 3, it may have read bytes that are
inaccessible.
In Python 2 it returns the bytes, whereas in Python 3 it
raises an IOError with EINTR.
A demonstration:
$ python3.2 fread.py
Writing 7 bytes
Reading 20 bytes... interrupt me with SIGQUIT (^\)
^\(3, frame object at 0x8e1d2d4)
Traceback (most recent call last):
File fread.py, line 18, in module
print('Read %d bytes using file.read' % len(read_file.read(20)))
IOError: [Errno 4] Interrupted system call
Reading any remaining bytes...
^\(3, frame object at 0x8e1d2d4)
Traceback (most recent call last):
File fread.py, line 23, in module
print('reading: %r' % os.read(r, 4096))
OSError: [Errno 4] Interrupted system call
Note how in Python 2 it stops reading when interrupted and it returns our
bytes, but in Python 3 it raises IOError while there is no way to access the
bytes that it read.
So basically, this behaviour is just plain wrong as EINTR is not an error, and
this behaviour makes it impossible for the caller to handle the situation
correctly.
Here is how I think Python should behave. I think that it should be possible to
interrupt both read and write calls, however, it should also be possible for
the user to handle these cases.
file.write, on EINTR, could decide to continue writing if no Python signal
handler raised an exception.
Analogously, file.read could decide to keep on reading on EINTR if no Python
signal handler raised an exception.
This way, it is possible for the programmer to write interruptable code while
at the same time having proper file.write and file.read behaviour in case code
should not be interrupted.
KeyboardInterrupt would still interrupt read and write calls, because it
raises an exception. If the programmer decided that writes should finish
before allowing such an exception, the programmer could replace the default
signal handler for SIGINT.
So, in pseudo-code:
bytes_written = 0
while bytes_written len(buf):
result = write(buf)
if result 0:
if errno == EINTR
if PyErr_CheckSignals() 0:
/* Propagate exception from signal handler */
return NULL
continue
else:
PyErr_SetFromErrno(PyExc_IOError)
return NULL
buf += result
bytes_written += result
return bytes_written
Similar code could be used for file.read with the obvious adjustments.
However, in case of an error (either from the write call or from a Python
signal handler),
it would still be unclear how many bytes were actually written. Maybe (I think
this part would be bonus points) we could put the number of bytes written on
the exception object in this case, or make it retrievable in some other
thread-safe way.
For files with file descriptors in nonblocking mode (and maybe other cases) it
will still return a short amount of bytes.
--
components: IO
files: fwrite.py
messages: 126616
nosy: eggy
priority: normal
severity: normal
status: open
title: file.write and file.read don't handle EINTR
type: behavior
versions: Python 2.5, Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3
Added file: http://bugs.python.org/file20462/fwrite.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10956
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10956] file.write and file.read don't handle EINTR
Mark Florisson markflorisso...@gmail.com added the comment: Here is fread.py (why can you only attach one file at a time? :P) -- Added file: http://bugs.python.org/file20463/fread.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10956 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10956] file.write and file.read don't handle EINTR
Mark Florisson markflorisso...@gmail.com added the comment: I think this sums it up: file.write, on EINTR, could decide to continue writing if no Python signal handler raised an exception. Analogously, file.read could decide to keep on reading on EINTR if no Python signal handler raised an exception. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10956 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10956] file.write and file.read don't handle EINTR
Mark Florisson markflorisso...@gmail.com added the comment: Ok. This would only be done in buffered mode, though, so your fwrite.py example would have to be changed slightly (drop the ,0 in fdopen()). Indeed, good catch. So apparently file.write (in buffered mode) is also incorrect in Python 3. -- Added file: http://bugs.python.org/file20464/fwrite.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10956 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10566] gdb debugging support additions (Tools/gdb/libpython.py)
Mark Florisson markflorisso...@gmail.com added the comment: I forgot to remove a tempfile and reverted the Cython note at the top. A diff is provided (that should be applied upon the previously submitted patch). It's a diff because I don't have commit rights and svn does not support local commits. -- Added file: http://bugs.python.org/file20075/libpython.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10566 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10566] gdb debugging support additions (Tools/gdb/libpython.py)
Changes by Mark Florisson markflorisso...@gmail.com: Removed file: http://bugs.python.org/file19857/libpython_patch.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10566 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10566] gdb debugging support additions (Tools/gdb/libpython.py)
Mark Florisson markflorisso...@gmail.com added the comment:
Ok I attached a new patch that solves the things you mentioned. It can debug
Python inferiors with versions 2.6+. Execution control commands (py-{run, cont,
finish, step, next}) and py-exec need gdb 7.2+, py-break works with 7.1+.
It now also supports exceptions, which means that if there is a pending
exception when control is transferred to the debugger, the exception is printed
(safely).
Also stepping and stepping over should be a lot faster as it now uses hardware
watchpoints that watch the instruction pointer (f-f_lasti).
--
Added file: http://bugs.python.org/file20070/libpython.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10566
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10566] gdb debugging support additions (Tools/gdb/libpython.py)
Mark Florisson markflorisso...@gmail.com added the comment: Indeed, I'm trying to make the code Python 2 and Python 3 (for the inferior) compatible, it's not really hard but indeed, the 'u' (Python 2) and 'b' (Python 3) stuff need special casing. Python 2 compatibility was also the reason why the PyIntObjectPtr class was merged back. I will make a patch that's compatible with both Python 2 and 3 (and without any colorful code :)) as this would be most preferably I think (even if it's shipped with Python 3, users might still want to use it for Python 2, and it's also easier for the Cython debugger which wants to be compatible with 2.5+). As for the gdb version, I have tested with 7.1 (in which case the introduced commands won't work as they use stuff from the 7.2 API, in which case test_gdb.py also skips those tests) and 7.2. I agree that the functionality should be left out if it cannot work properly. Indeed, the _LoggingState redirects logging to a file and then reads the output after the command returns, I've been using it successfully with big amounts of output and I don't think there should be a problem as redirection and pagination should be unrelated. The good thing about _LoggingState is that it actually captures *all* output (and it's fully reentrant), which the 'to_string' argument was not taking care of properly in 7.2 (it's fixed in the archer branch). Indeed, gdb.get_selected_thread() is entirely broken in 7.2 (again, fixed in archer) but as you can see, it's not used (gdb.inferiors()[0] works). I'm currently looking into making stepping over faster and by extension stepping which we discussed earlier (I also discussed it with Tom Tromey previously). As you know, currently stepping over (and stepping) works with a step-over loop which might be turned into a set a conditional breakpoint or watchpoint + continue solution, which would mean a lot less context switches. I've not looked too serious into this matter, but I'll hope to get around to that soonish and I'll provide a new patch with all the corrections and improvements. Another issue I'm fixing is the determination of the type of an arbitrary Python object, which was previously done with the Py_TPFLAGS_INT_SUBCLASS flags and friends. This is because they are new in 2.6 and I'd prefer to be 2.5 compatible (again, because I'm trying to keep the Cython debugger 2.5 compatible). As for porting the gdb API to Python 3, I'm quite convinced that the API can be written in Cython, in which case it would mostly be a change in the build process rather than a serious code-refactoring issue. But I'll get around to that later... Anyway, should I diff against the original libpython or against the original libpython + my previous diff? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10566 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10566] gdb debugging support additions (Tools/gdb/libpython.py)
Mark Florisson markflorisso...@gmail.com added the comment: I forgot to mention, this patch works with gdb 7.2 or higher, but it does not prevent using other libpython functionality with gdb 7.1 or running the tests with gdb 7.1. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10566 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10566] gdb debugging support additions (Tools/gdb/libpython.py)
New submission from Mark Florisson markflorisso...@gmail.com: Attached is a patch that adds a few features to the Python debugging support for gdb: - listing of globals - python breakpoints - python stepping/stepping over/finishing of frames/running/continuing - python code execution in the most recent Python frame (relative to the selected frame) Unit tests are included in Lib/test/test_gdb.py. It should be compatible with both python 3 and python 2 (this is important because libpython.py is also part of the Cython debugger branch, see https://github.com/markflorisson88/cython/blob/master/Cython/Debugger/libpython.py ). Python 2 tests are in that Cython branch. It would be great if libpython.py could be installed as an actual Python module instead of a tool, where 'python-gdb.py' would be the actual tool that imports libpython.py. This may remove the need in the future to duplicate files in the Python source distribution and in future versions of Cython. Another good thing about that is that if users don't have python-gdb.py installed properly, or would like to use functionality without having loaded the interpreter in gdb (i.e. 'file python'), they can just do 'python import libpython' in gdb to load the Python support. I can't assign this issue, but Dave Malcolm (dmalcolm) asked me to assign the issue to him, so I kindly request someone with these capabilities to assign this issue accordingly. -- components: Demos and Tools files: libpython_patch.diff keywords: patch messages: 122681 nosy: eggy priority: normal severity: normal status: open title: gdb debugging support additions (Tools/gdb/libpython.py) type: feature request versions: Python 3.2 Added file: http://bugs.python.org/file19857/libpython_patch.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10566 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7548] If a generator raises TypeError when being unpacked, an unrelated error message is shown
New submission from Mark Florisson markflorisso...@gmail.com:
list(*('boo' for x in [1]))
['b', 'o', 'o']
list(*(range('error') for x in [1])) # notice the erroneous error
message
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: type object argument after * must be a sequence, not
generator
list(*[range('error') for x in [1]])
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: range() integer end argument expected, got str.
list(*(int('error') for x in [1])) # does work correctly for
ValueError
Traceback (most recent call last):
File stdin, line 1, in module
File stdin, line 1, in genexpr
ValueError: invalid literal for int() with base 10: 'error'
--
components: Interpreter Core
messages: 96642
nosy: eggy
severity: normal
status: open
title: If a generator raises TypeError when being unpacked, an unrelated error
message is shown
versions: Python 2.4, Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python
3.1, Python 3.2
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7548
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7238] frame.f_lineno doesn't get updated after local trace function assigned to it
New submission from Mark Florisson markflorisso...@gmail.com: As you can see, when a local trace function sets f_lineno, f_lineno doesn't get updated on subsequent lines. Otherwise it works fine. $ python tracer_testcase.py 12 12 12 13 14 15 The reference manual (for python 2.6) states the following: f_lineno is the current line number of the frame — writing to this from within a trace function jumps to the given line (only for the bottom-most frame). A debugger can implement a Jump command (aka Set Next Statement) by writing to f_lineno.. This is contradictory with the shown results, because apparently it doesn't always represent the current line number. -- components: Interpreter Core files: tracer_testcase.py messages: 94681 nosy: eggy severity: normal status: open title: frame.f_lineno doesn't get updated after local trace function assigned to it type: behavior versions: Python 2.4, Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 3.1, Python 3.2 Added file: http://bugs.python.org/file15229/tracer_testcase.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7238 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6268] Seeking to the beginning of a text file a second time will return the BOM as first character
New submission from Mark Florisson markflorisso...@gmail.com:
f = open('foo', 'wt+', encoding='UTF-16')
f.write('spam ham eggs')
13
f.seek(0)
0
f.read()
'spam ham eggs'
f.seek(0)
0
f.read()
'\ufeffspam ham eggs'
Although the BOM character is a ZERO WIDTH NO-BREAK SPACE, and should
therefore not impose many problems, the behavior is inconsistent and
unexpected.
codecs.open in 2.x suffers from this same behavior.
--
components: Unicode
messages: 89257
nosy: eggy
severity: normal
status: open
title: Seeking to the beginning of a text file a second time will return the
BOM as first character
type: behavior
versions: Python 2.4, Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python
3.1, Python 3.2
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6268
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6162] What should happen to signals when the main thread dies?
New submission from Mark Florisson markflorisso...@gmail.com: As signals are only delivered to the main thread, what should happen when the main thread dies? Currently, the signal mask is not unset in any other thread, so when the main thread dies, all signals set in the mask are simply ignored. Perhaps an other thread could be selected as the main thread? The accompanied file demonstrates this behavior. -- components: Interpreter Core files: ignore_signals.py messages: 88650 nosy: eggy severity: normal status: open title: What should happen to signals when the main thread dies? type: feature request versions: Python 2.4, Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 3.1, Python 3.2 Added file: http://bugs.python.org/file14143/ignore_signals.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6162 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6089] str.format raises SystemError
New submission from Mark Florisson markflorisso...@gmail.com:
'{0[0](10)}'.format([None])
Traceback (most recent call last):
File stdin, line 1, in module
SystemError: error return without exception set
--
components: Interpreter Core
messages: 88198
nosy: eggy
severity: normal
status: open
title: str.format raises SystemError
type: behavior
versions: Python 2.6, Python 2.7, Python 3.0, Python 3.1, Python 3.2
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6089
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5275] BaseCookie.load doesn't create Morsel objects for mappings
New submission from Mark Florisson markflorisso...@gmail.com: The load method, which is also called from the initializer if input is provided, doesn't create Morsel objects for things other than strs, because it calls self.update(rawdata), which does not invoke the custom __setitem__. The documentation states that when rawdata is not a string, it must be a dictionary and providing that will be equivalent to doing for k, v in rawdata.items(): cookie[k] = v So that is what it should do :) -- messages: 82177 nosy: eggy severity: normal status: open title: BaseCookie.load doesn't create Morsel objects for mappings type: behavior versions: Python 2.4, Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5275 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4684] sys.exit() exits program when non-daemonic threads are still running
New submission from Mark Florisson markflorisso...@gmail.com: sys.exit() exits the program when non-daemonic threads are still running, in versions = 2.5. Test included. A demonstration here: http://paste.pocoo.org/show/95766/ (On debian GNU/Linux) -- components: None files: foo.py messages: 77978 nosy: eggy severity: normal status: open title: sys.exit() exits program when non-daemonic threads are still running type: behavior versions: Python 2.5, Python 2.6, Python 3.0 Added file: http://bugs.python.org/file12381/foo.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4684 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4579] .read() and .readline() differ in failing
New submission from Mark Florisson [EMAIL PROTECTED]:
f = os.fdopen(os.open('spam!', os.O_TRUNC|os.O_CREAT|os.O_RDWR), 'w')
f.read()
''
f.readline()
Traceback (most recent call last):
File stdin, line 1, in module
IOError: [Errno 9] Bad file descriptor
f.write(spamspamhihi)
f.read()
''
f.seek(0)
f.read()
Traceback (most recent call last):
File stdin, line 1, in module
IOError: [Errno 9] Bad file descriptor
This is very strange behaviour. First, .read() succeeds, and .readline()
fails, but after writing and seeking, .read() also fails.
In python3, both read and readline fail, but with different exceptions:
f = os.fdopen(os.open('spam!', os.O_TRUNC|os.O_CREAT|os.O_RDWR), 'w')
f.read()
Traceback (most recent call last):
File stdin, line 1, in module
File /home/mark/source/code/_python-3.0/Lib/io.py, line 1718, in read
decoder.decode(self.buffer.read(), final=True))
File /home/mark/source/code/_python-3.0/Lib/io.py, line 668, in read
self._unsupported(read)
File /home/mark/source/code/_python-3.0/Lib/io.py, line 327, in
_unsupported
(self.__class__.__name__, name))
io.UnsupportedOperation: BufferedWriter.read() not supported
f.readline()
Traceback (most recent call last):
File stdin, line 1, in module
File /home/mark/source/code/_python-3.0/Lib/io.py, line 1807, in
readline
while self._read_chunk():
File /home/mark/source/code/_python-3.0/Lib/io.py, line 1554, in
_read_chunk
input_chunk = self.buffer.read1(self._CHUNK_SIZE)
AttributeError: 'BufferedWriter' object has no attribute 'read1'
In my opinion, all operations, in all python versions, should fail like
readline in the first example: IOError: [Errno 9] Bad file descriptor
--
messages: 77242
nosy: eggy
severity: normal
status: open
title: .read() and .readline() differ in failing
type: behavior
versions: Python 2.4, Python 2.5, Python 2.6, Python 3.0
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue4579
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4579] .read() and .readline() differ in failing
Mark Florisson [EMAIL PROTECTED] added the comment: Actually, I wouldn't expect it to fail like that, because it's not a bad file descriptor, it's an actual that doesn't agree with the (userspace) file mode. What I'd expect is probably a TypeError. Although an IOError, with a different message, wouldn't be totally inappropriate either. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4579 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4579] .read() and .readline() differ in failing
Mark Florisson [EMAIL PROTECTED] added the comment: s/actual/operation/ ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4579 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4579] .read() and .readline() differ in failing
Mark Florisson [EMAIL PROTECTED] added the comment: Perhaps it's linux specific then. I'm on debian lenny (2.6.26-1-amd64). ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4579 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4452] Incorrect docstring of os.setpgrp
New submission from Mark Florisson [EMAIL PROTECTED]: The docstring of os.setpgrp says 'Make this process a session leader.', but that's not what setpgrp does. setpgrp() is the same as setpgid(0, 0), which sets the pgid of the calling process to the pid of the calling process, thus making it a process group leader, not a session leader (it will still be in the same session). It might say instead 'Make this process a process group leader.'. -- messages: 76529 nosy: eggy severity: normal status: open title: Incorrect docstring of os.setpgrp versions: Python 2.4, Python 2.5, Python 2.6, Python 3.0 ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4452 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
