[issue15068] fileinput requires two EOF when reading stdin

[Roundup Robot]

Roundup Robot added the comment:

New changeset 88d6742aa99a by Serhiy Storchaka in branch '2.7':
Issue #15068: Avoid creating a reference loop in fileinput.
https://hg.python.org/cpython/rev/88d6742aa99a

New changeset a0de41b46aa6 by Serhiy Storchaka in branch '3.5':
Issue #15068: Avoid creating a reference loop in fileinput.
https://hg.python.org/cpython/rev/a0de41b46aa6

New changeset 27c9849ba5f3 by Serhiy Storchaka in branch 'default':
Issue #15068: Avoid creating a reference loop in fileinput.
https://hg.python.org/cpython/rev/27c9849ba5f3

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Ah, thanks Martin. I forgot that assigning an attribute to a bound method 
creates a reference loop.

This can be fixed without performance lost by using a clever trick.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Martin Panter]

Martin Panter added the comment:

It seems this change is causing some (intermittent?) buildbot failures in 2.7:

http://buildbot.python.org/all/builders/s390x%20RHEL%202.7/builds/273/steps/test/logs/stdio

==
FAIL: test_saveall (test.test_gc.GCTests)
--
Traceback (most recent call last):
  File 
"/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_gc.py", 
line 199, in test_saveall
self.assertEqual(gc.garbage, [])
AssertionError: Lists differ: [

Diff is 1461 characters long. Set self.maxDiff to None to see it.

==
FAIL: test_create_read (test.test_csv.TestLeaks)
--
Traceback (most recent call last):
  File 
"/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_csv.py", 
line 1103, in test_create_read
self.assertEqual(gc.garbage, [])
AssertionError: Lists differ: [

Diff is 1461 characters long. Set self.maxDiff to None to see it.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Committed in changesets 5fbd16326353 (2.7), 9ead3a6c5f81 (3.5), and 
fefedbaac640 (default). Due to SMTP failure there is no Roundup report.

Warnings are not emitted in maintained releases.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Ping.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> It's more a question of whether the entire change is appropriate for 2.7.

What is your answer? To me there is a bug and we can fix it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Benjamin Peterson]

Benjamin Peterson added the comment:

That individually is probably okay. It's more a question of whether the
entire change is appropriate for 2.7.

Note PendingDeprecationWarning is fairly useless, since it's rarely
enabled.

On Sat, Dec 19, 2015, at 00:34, Serhiy Storchaka wrote:
> 
> Serhiy Storchaka added the comment:
> 
> Benjamin, is it good to add PendingDeprecationWarning in 2.7?
> 
> --
> nosy: +benjamin.peterson
> versions:  -Python 3.4
> 
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Benjamin, is it good to add PendingDeprecationWarning in 2.7?

--
nosy: +benjamin.peterson
versions:  -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Kristján Valur Jónsson]

Changes by Kristján Valur Jónsson :


--
nosy:  -kristjan.jonsson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
versions: +Python 2.7
Added file: http://bugs.python.org/file41226/fileinput_no_buffer-2.7.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Removed file: http://bugs.python.org/file41224/fileinput_no_buffer.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Added file: http://bugs.python.org/file41225/fileinput_no_buffer.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Using readlines() instead of readline() was added in 4dbbf322a9df for 
performance. But it looks that now this is not needed. Naive implementation 
with readline() is about 2 times slower, but with careful optimization we can 
achieve the same performance (or better).

Here are results of benchmarks.

Unpatched:

$ mkdir testdir
$ for i in `seq 10`; do for j in `seq 1000`; do echo "$j"; done 
>"testdir/file$i"; done
$ ./python -m timeit -s "import fileinput, glob; files = 
glob.glob('testdir/*')" -- "f = fileinput.input(files)" "while f.readline(): 
pass"
10 loops, best of 3: 56.4 msec per loop
$ ./python -m timeit -s "import fileinput, glob; files = 
glob.glob('testdir/*')" -- "list(fileinput.input(files))"10 loops, best of 3: 
68.4 msec per loop

Patched:

$ ./python -m timeit -s "import fileinput, glob; files = 
glob.glob('testdir/*')" -- "f = fileinput.input(files)" "while f.readline(): 
pass"
10 loops, best of 3: 47.4 msec per loop
$ ./python -m timeit -s "import fileinput, glob; files = 
glob.glob('testdir/*')" -- "list(fileinput.input(files))"
10 loops, best of 3: 63.1 msec per loop

The patch also fixes original issue.

It also fixes yet one issue. Currently lines are buffered and you need to enter 
many lines first then get first line:

>>> import fileinput
>>> fi = fileinput.input()
>>> line = fi.readline()
qwerty
asdfgh
zxcvbn
^D
>>> line
'qwerty\n'

With the patch you get the line just as it entered.

--
assignee: docs@python -> serhiy.storchaka
stage: needs patch -> patch review
versions: +Python 3.4, Python 3.5, Python 3.6 -Python 2.7, Python 3.2, Python 
3.3
Added file: http://bugs.python.org/file41224/fileinput_no_buffer.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Kristján Valur Jónsson]

Kristján Valur Jónsson krist...@ccpgames.com added the comment:

Thanks Antoine.  I tested this in my virtualbox so something new must have 
happened...  Anyway, the GIL code should not have changed from before, only 
moved about slightly.  I´ll figure out what happened.

--
nosy: +kristjan.jonsson

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Kristján Valur Jónsson]

Changes by Kristján Valur Jónsson krist...@ccpgames.com:


--
Removed message: http://bugs.python.org/msg163150

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Florent Xicluna]

Changes by Florent Xicluna florent.xicl...@gmail.com:


--
nosy: +flox

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Joey Geralnik]

Joey Geralnik jgeral...@gmail.com added the comment:

First off, I'm a complete noob looking at the python source code for the first 
time so forgive me if I've done something wrong.

What if the length of the chunk is checked as well? The following code works 
fine:

import sys
while True:
chunk = sys.stdin.read(1000)
if not chunk:
break
# process
if len(chunk)  1000:
break

Something similar could be done in the fileinput class. The patch I've attached 
checks if the number of bytes read from the file is less than the size of the 
buffer (which means that the file has ended). If so, the next time the file is 
to be read it skips to the next file instead.

joey@j-Laptop:~/cpython$ ./python 
Python 3.3.0a3+ (default:befd56673c80+, Jun 15 2012, 17:14:12) 
[GCC 4.6.3] on linux
Type help, copyright, credits or license for more information.
 import fileinput
[73732 refs]
 lines = list(fileinput.input())
foo
bar
^D
[73774 refs]
 lines
['foo\n', 'bar\n']
[73780 refs]

--
keywords: +patch
nosy: +jgeralnik
Added file: http://bugs.python.org/file26018/fileinput.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Serhiy Storchaka storch...@gmail.com added the comment:

  The patch I've attached checks if the number of bytes read from the file is 
 less than the size of the buffer (which means that the file has ended).

From io.RawIOBase.read docs:


Read up to n bytes from the object and return them. As a convenience, if
n is unspecified or -1, readall() is called. Otherwise, only one system
call is ever made. Fewer than n bytes may be returned if the operating
system call returns fewer than n bytes.

If 0 bytes are returned, and n was not 0, this indicates end of file.


This is not an arbitrary assumption. In particular, when reading from a
terminal with line buffering (you can edit the line until you press
Enter) on C level you read only a whole line (if line length is not
greater than buffer length) and 0 bytes you will receive only by
pressing ^D or ^Z at the beginning of the line. Same for pipes and
sockets. On Python level there are many third-party implementations of
file-like objects which rely on this behavior, you cannot rewrite all of
them.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Joey Geralnik]

Joey Geralnik jgeral...@gmail.com added the comment:

But this is calling the readlines function, which continually reads from the 
file until more bytes have been read than the specified argument.

From bz2.readlines:
size can be specified to control the number of lines read: no further lines 
will be read once the total size of the lines read so far equals or exceeds 
size.

Do other file-like objects interpret this parameter differently?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Joey Geralnik]

Joey Geralnik jgeral...@gmail.com added the comment:

Forget other filelike objects. The FileInput class only works with actual 
files, so the readlines function should always return at least as many bytes as 
its first parameter. Is this assumption wrong?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

fileinput should work (for some definition of work) for anything that can be 
opened by name using the open syscall on unix.  That includes many more things 
than files.  Named pipes are a particularly interesting example in this context.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

So the real question is: does readlines block until the byte count is 
satisified?  It might, but the docs for io.IOBase.readlines leave open the 
possibility that fewer lines will be read, and do not limit that to the EOF 
case.  It's not clear, however, if that is because the non-EOF-short-read case 
is specifically being allowed for, or if the documenter just didn't consider 
that case.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

The _pyio.py version of readlines does read until the count is equaled or 
exceeded.  This could, however, be an implementation detail and not part of the 
spec.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Le vendredi 15 juin 2012 à 14:41 +, Serhiy Storchaka a écrit :
 From io.RawIOBase.read docs:
 
 
 Read up to n bytes from the object and return them. As a convenience, if
 n is unspecified or -1, readall() is called. Otherwise, only one system
 call is ever made. Fewer than n bytes may be returned if the operating
 system call returns fewer than n bytes.

But sys.stdin does not implement RawIOBase, it implements TextIOBase.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Serhiy Storchaka storch...@gmail.com added the comment:

 Forget other filelike objects. The FileInput class only works with actual 
 files,

No. sys.stdin can be reassigned before using FileInput. And FileInput
has openhook parameter (for read compressed files or get files from Web,
for example).

  so the readlines function should always return at least as many bytes as its 
 first parameter. Is this assumption wrong?

qwert
'qwert\n'

You type five characters qwert end press Enter. Python immediately
receives these six characters, and returns a result of
sys.stdin.readline(1000). Only six characters, and no one symbol more,
because more characters you have not entered yet.

I believe that for such questions will be more appropriate to use a
mailing list (python-l...@python.org, or newsgroup
gmane.comp.python.general on news://news.gmane.org), and not bugtracker.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

   so the readlines function should always return at least as many bytes as 
  its first parameter. Is this assumption wrong?
 
 qwert
 'qwert\n'
 
 You type five characters qwert end press Enter. Python immediately
 receives these six characters, and returns a result of
 sys.stdin.readline(1000).

Well, did you try readline() or readlines()?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Serhiy Storchaka storch...@gmail.com added the comment:

 But sys.stdin does not implement RawIOBase, it implements TextIOBase.

sys.stdin.buffer.raw implements RawIOBase.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Serhiy Storchaka storch...@gmail.com added the comment:

  
  qwert
  'qwert\n'

Oh, it seems that the mail server again ate some lines of my examples.

 Well, did you try readline() or readlines()?

Yes, it's my mistake, I used readline().

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

 Oh, it seems that the mail server again ate some lines of my examples.

This is a bug in the e-mail gateway. You can lobby for a fix at
http://psf.upfronthosting.co.za/roundup/meta/issue264

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Jason R. Coombs]

New submission from Jason R. Coombs jar...@jaraco.com:

I found that fileinput.input() requires two EOF characters to stop reading 
input on Python 2.7.3 on Windows and Ubuntu:

PS C:\Users\jaraco python
Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)] on 
win32
 import fileinput
 lines = list(fileinput.input())
foo
bar
^Z
^Z
 lines
['foo\n', 'bar\n']

I don't see anything in the documentation that suggests that two EOF characters 
would be required, and I can't think of any reason why that should be the case.

--
components: Library (Lib)
messages: 162798
nosy: jason.coombs
priority: normal
severity: normal
status: open
title: fileinput requires two EOF when reading stdin
versions: Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Jason R. Coombs]

Jason R. Coombs jar...@jaraco.com added the comment:

I observed if I send EOF as the first character, it honors it immediately and 
doesn't require a second EOF.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Frankly I'm surprised it works at all, since fileinput.input() will by default 
read from stdin, and stdin is in turn being read by the python prompt.

I just checked 2.5 on linux, and the same situation exists there (two ^Ds are 
required to end the input()).  I suspect we'll find the explanation in the 
interaction between the default behavior of fileinput.input() and the 
interactive prompt.

--
nosy: +r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Jason R. Coombs]

Jason R. Coombs jar...@jaraco.com added the comment:

FWIW, I encountered the double-EOF behavior when invoking fileinput.input from 
a script running non-interactively (except of course for the input() call).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Zachary Ware]

Zachary Ware zachary.w...@gmail.com added the comment:

I just tested on Python 3.2, and found something interesting; it seems a ^Z 
character on a line that has other input read in as a character.  Also, other 
input after an EOF on its own means you still have to do two more EOFs to end.

Python 3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit (Intel)] on win
32
Type help, copyright, credits or license for more information.
 import fileinput
 lines = list(fileinput.input())
test
testing
^Z
^Z
 lines
['test\n', 'testing\n']
 lines = list(fileinput.input())
test
testing^Z
^Z
^Z
 lines
['test\n', 'testing\x1a\n']
 lines = list(fileinput.input())
testing^Z
test
^Z
testing
^Z
^Z
 lines
['testing\x1a\n', 'test\n', 'testing\n']

Also, the documentation for fileinput doesn't mention EOF at all.

--
nosy: +zach.ware
versions: +Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

I don't know how the EOF character works, but I wouldn't be surprised if it had 
to be on a line by itself to mean EOF.

If the double EOF is required when not at the interactive prompt, then there 
could be a long standing bug in fileinput's logic where it is doing another 
read after the last file is closed.  Normally this wouldn't even be visible 
since it would just get EOF again, but when the file is an interactive STDIN, 
closing it doesn't really close it...

--
stage:  - needs patch
type:  - behavior

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Serhiy Storchaka storch...@gmail.com added the comment:

It is not only the fileinput. The same effect can be achieved by simple 
idiomatic code:

import sys
while True:
chunk = sys.stdin.read(1000)
if not chunk:
break
# process

--
nosy: +storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

That makes sense.  It is a consequence of (a) buffered input and (b) the fact 
that EOF on stdin doesn't really close it.  (And by interactive here I don't 
just mean Python's interactive prompt, but also the shell).

By default fileinput uses readlines with a buffer size, so it suffers from the 
same issue.  It is only the second time that you close stdin that it gets an 
empty buffer, and so terminates.

Anyone want to try to come up with a doc footnote to explain this?

--
assignee:  - docs@python
components: +Documentation
nosy: +docs@python

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Serhiy Storchaka]

Serhiy Storchaka storch...@gmail.com added the comment:

Note that in the rare cases, when stdio ends immediately on the limit of the 
read buffer, just one EOF is sufficient. In particular for read(1) one EOF is 
sufficient always, and for read(2) it is sufficient in about half of the cases.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

It is unlikely to be solvable at the Python level. Witness the raw stream's 
behaviour (in Python 3):

 sys.stdin.buffer.raw.read(1000)

If you type a letter followed by ^D (Linux) or ^Z (Windows), this returns 
immediately:

 sys.stdin.buffer.raw.read(1000)
x^Db'x'

But since the result is non-empty, the buffering layer will not detect the EOF 
and will call read() on the raw stream again (as the 1000 bytes are not 
satisfied). To signal EOF to the buffered stream, you have to type ^D or ^Z 
*without preceding it with another character*. Try the following:

 sys.stdin.buffer.read(1000)

You'll see that as long as you type a letter before ^D or ^Z, the read() will 
not return (until you type more than 1000 characters, that is):
- ^D alone: returns!
- a letter followed by ^D: doesn't return
- a letter followed by ^D followed by ^D: returns!
- a letter followed by ^D followed by a letter followed by ^D: doesn't return

This is all caused by the fact that a C read() on stdin doesn't return until 
either the end of line or EOF (or the requested bytes number is satisfied). 
Just experiment with:

 os.read(0, 1000)

That's why I say this is not solvable at the Python level (except perhaps with 
bizarre ioctl hackery).

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15068] fileinput requires two EOF when reading stdin

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15068
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com