[issue12268] file readline, readlines & readall methods can lose data on EINTR

[Martijn Pieters]

Martijn Pieters added the comment:

Follow-up bug, readahead was missed: http://bugs.python.org/issue1633941

--
nosy: +mjpieters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Roundup Robot]

Roundup Robot added the comment:

New changeset a5e7b38caee2 by Gregory P. Smith in branch '2.7':
Additional fix for Issue #12268: The io module file object writelines() methods
http://hg.python.org/cpython/rev/a5e7b38caee2

New changeset 2fd669aa4abc by Gregory P. Smith in branch '3.2':
Additional fix for Issue #12268: The io module file object writelines() methods 
no longer abort early when one of its write system calls is interrupted (EINTR).
http://hg.python.org/cpython/rev/2fd669aa4abc

New changeset 30fc620e240e by Gregory P. Smith in branch '3.3':
Additional fix for issue #12268: The io module file object write methods no
http://hg.python.org/cpython/rev/30fc620e240e

New changeset 8f72519fd0e9 by Gregory P. Smith in branch 'default':
Additional fix for issue #12268: The io module file object write methods no
http://hg.python.org/cpython/rev/8f72519fd0e9

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[STINNER Victor]

STINNER Victor added the comment:

Oh, so we can now implement a version of writelines() using writev()!

2013/2/1 Roundup Robot rep...@bugs.python.org:

 Roundup Robot added the comment:

 New changeset a5e7b38caee2 by Gregory P. Smith in branch '2.7':
 Additional fix for Issue #12268: The io module file object writelines() 
 methods
 http://hg.python.org/cpython/rev/a5e7b38caee2

 New changeset 2fd669aa4abc by Gregory P. Smith in branch '3.2':
 Additional fix for Issue #12268: The io module file object writelines() 
 methods no longer abort early when one of its write system calls is 
 interrupted (EINTR).
 http://hg.python.org/cpython/rev/2fd669aa4abc

 New changeset 30fc620e240e by Gregory P. Smith in branch '3.3':
 Additional fix for issue #12268: The io module file object write methods no
 http://hg.python.org/cpython/rev/30fc620e240e

 New changeset 8f72519fd0e9 by Gregory P. Smith in branch 'default':
 Additional fix for issue #12268: The io module file object write methods no
 http://hg.python.org/cpython/rev/8f72519fd0e9

 --

 ___
 Python tracker rep...@bugs.python.org
 http://bugs.python.org/issue12268
 ___

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Gregory P. Smith added the comment:

it was easier to just take care of auditing the write calls as part of this 
given the code change was directly related to it.

On Python 2.7 most of the write calls in the builtin file object 
(Objects/fileobject.c) rather than the new io module use the libc fwrite() call 
which, in linux man pages at least, is non-specific about what happens on EINTR 
(does it retry internally or does it return the number of bytes written so 
far?).  Those could well abort leading to an error.

Setting up a testcase fo to confirm that with is painful (time consuming) so I 
can't claim the non io module based write's do not still have an EINTR issue on 
2.7.

Workaround: Use the io module instead of the builtin open() or file() calls in 
Python 2.7.

If someone can confirm that with a test case, it'd make another good issue to 
open.

As for the writev comment... go ahead. :)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Changes by Gregory P. Smith g...@krypto.org:


--
resolution:  - fixed
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Andrew Svetlov]

Changes by Andrew Svetlov andrew.svet...@gmail.com:


--
nosy: +asvetlov

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Antoine Pitrou]

Antoine Pitrou added the comment:

Is there anything left to do here?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Gregory P. Smith added the comment:

Yes. See my comment from June. The write paths need to be taken care of.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Antoine Pitrou]

Antoine Pitrou added the comment:

Well, this issue is about readline, readlines  readall. It would be easier 
to follow if you opened a separate issue.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gary Miguel]

Changes by Gary Miguel gar...@google.com:


--
nosy: +Gary.Miguel

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Mark Dickinson]

Changes by Mark Dickinson dicki...@gmail.com:


--
nosy: +mark.dickinson

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Roundup Robot]

Roundup Robot added the comment:

New changeset 751a91e332d9 by Gregory P. Smith in branch '2.7':
Fixes Issue #12268 for the io module - File readline, readlines and
http://hg.python.org/cpython/rev/751a91e332d9

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[STINNER Victor]

Changes by STINNER Victor victor.stin...@gmail.com:


--
nosy: +haypo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Changes by Gregory P. Smith g...@krypto.org:


--
priority: normal - high

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Gregory P. Smith g...@krypto.org added the comment:

I'm attaching an updated patch for 2.7.  It fixes read, readline, readlines and 
readinto and includes tests.

More code auditing for other methods to fix is still needed.

--
versions:  -Python 3.1
Added file: http://bugs.python.org/file26142/fix-signal-eintr-read-27-gps02.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Gregory P. Smith g...@krypto.org added the comment:

The 3.* ubuntu arm buildbot hanging in test_io is very odd.

I'm going to undo my supposedly straight forward signal.alarm(...) to 
signal.setitimer(...) change first to see if that is related.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Changes by Gregory P. Smith g...@krypto.org:


Removed file: http://bugs.python.org/file22258/file-signal-eintr-27.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 95b071194ddd by Gregory P. Smith in branch '3.2':
Backout change e8f44ebacda7052267318cecf5b6f128d35add17.  Reverting the test
http://hg.python.org/cpython/rev/95b071194ddd

New changeset b4ae7aa21b46 by Gregory P. Smith in branch 'default':
Backout change e8f44ebacda7052267318cecf5b6f128d35add17.  Reverting the test
http://hg.python.org/cpython/rev/b4ae7aa21b46

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Changes by Gregory P. Smith g...@krypto.org:


Removed file: 
http://bugs.python.org/file22262/test_and_fix_readers_3.2-gps02.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

I don't think setitimer() is the culprit, rather the fact that the timeout is 
too short. You could try setting it to e.g. 0.4.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 67dc99a989cd by Gregory P. Smith in branch '2.7':
Fixes issue #12268 for file readline, readlines and read() and readinto methods.
http://hg.python.org/cpython/rev/67dc99a989cd

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Gregory P. Smith g...@krypto.org added the comment:

The uses of fwrite() and fflush() also need this EINTR treatment in 2.7.  I 
haven't checked the write paths in 3.2 yet.

Also, the fix change to 3.2's _io module needs backporting to 2.7's _io module 
for people using that.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 781b95159954 by Gregory P. Smith in branch '3.2':
Fixes issue #12268: File readline, readlines and read() or readall() methods
http://hg.python.org/cpython/rev/781b95159954

New changeset 19a6bef57490 by Gregory P. Smith in branch 'default':
Fixes issue #12268: File readline, readlines and read() or readall() methods
http://hg.python.org/cpython/rev/19a6bef57490

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Gregory P. Smith g...@krypto.org added the comment:

I'm leaving this open as I still need to audit the write methods and commit the 
fix(es) for 2.7.

I tried to merge the test into test_io's signals tests but I could not get that 
to actually work to reproduce the original problem so I kept my process based 
test_file_eintr one which easily reproduces it prior to these patches being 
applied instead.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

For the record, there was a crash on the ARM buildbot:


[196/368/1] test_io
Timeout (1:00:00)!
Thread 0x2aac5300:
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/test/test_io.py, 
line 3051 in check_interrupted_write_retry
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/test/test_io.py, 
line 3073 in test_interrupted_write_retry_text
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/unittest/case.py, 
line 385 in _executeTestPart
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/unittest/case.py, 
line 440 in run
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/unittest/case.py, 
line 492 in __call__
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/unittest/suite.py,
 line 105 in run
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/unittest/suite.py,
 line 67 in __call__
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/unittest/suite.py,
 line 105 in run
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/unittest/suite.py,
 line 67 in __call__
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/unittest/runner.py,
 line 168 in run
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/test/support.py, 
line 1383 in _run_suite
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/test/support.py, 
line 1417 in run_unittest
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/test/test_io.py, 
line 3121 in test_main
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/test/regrtest.py, 
line 1194 in runtest_inner
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/test/regrtest.py, 
line 905 in runtest
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/test/regrtest.py, 
line 708 in main
  File 
/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/test/__main__.py, 
line 13 in module
  File /var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/runpy.py, 
line 75 in _run_code
  File /var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/runpy.py, 
line 162 in _run_module_as_main
make: *** [buildbottest] Error 1

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

I'm not sure why you're creating a separate test file. There are already 
signals-related tests in test_io. Also, perhaps you can reuse the idioms used 
there, rather than spawn subprocesses.

--
stage:  - patch review

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Gregory P. Smith g...@krypto.org added the comment:

I haven't looked beyond the reading methods it is possible that some of the 
write implementations have a similar issue.  Patch gps02 for 3.2 attached.

I'll use that as the basis for a stand alone test_file_eintr.py targeted at 2.7.

--
Added file: http://bugs.python.org/file22262/test_and_fix_readers_3.2-gps02.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Changes by Gregory P. Smith g...@krypto.org:


Removed file: 
http://bugs.python.org/file22261/test_fileio_readers_3.2-gps01.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Changes by Gregory P. Smith g...@krypto.org:


--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Gregory P. Smith g...@krypto.org added the comment:

.readall() and the equivalent unbounded .read() also have this problem.

--
title: file readline  readlines methods can lose data on EINTR - file 
readline, readlines  readall methods can lose data on EINTR

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[John O'Connor]

Changes by John O'Connor tehj...@gmail.com:


--
nosy: +jcon

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12268] file readline, readlines readall methods can lose data on EINTR

[Gregory P. Smith]

Changes by Gregory P. Smith g...@krypto.org:


Added file: http://bugs.python.org/file22261/test_fileio_readers_3.2-gps01.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12268
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com